Data persistence method, device and storage medium

Abstract

The present application relates to the field of data storage technology and specifically discloses a data persistence method, device, and storage medium. The method comprises: obtaining a first module identifier when a persistence period has not yet ended, the first module identifier being used to indicate a first module whose module data has changed; determining a first pointer based on the first module identifier, the first pointer pointing to target data, the target data being module data in use in the first module; writing the first pointer to a first lock-free queue so that a serialization thread reads the first pointer from the first lock-free queue, performs a serialization operation on the target data based on the first pointer, and writes the obtained serialized data to a second lock-free queue; reading the serialized data in the second lock-free queue and storing the serialized data in a persistent storage medium. The present application can reduce serialization time consumption, improve the efficiency of data persistence on the server, and reduce the risk of data loss.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a data persistence method, device, and storage medium. Background Art

[0002] Modular storage means storing data in the form of modules, that is, one piece of data is composed of module data from multiple modules, and the module data from different modules is stored in the data table as an independent data type (such as BLOB).

[0003] However, currently, when a server transfers module data from a non-persistent storage medium (e.g., memory) to a persistent storage medium (e.g., a database), it must undergo a very time-consuming serialization process. This process increases with the size of the module data, leading to lower data persistence efficiency. Furthermore, due to the time-consuming serialization process, if a server exception occurs, module data for some modules may be lost due to lack of timely persistence, resulting in an unrecoverable disaster. Summary of the Invention

[0004] The present application provides a data persistence method, device and storage medium, which can reduce serialization time, improve the efficiency of server data persistence, and reduce the risk of data loss.

[0005] In one aspect, the present application provides a data persistence method, the method comprising:

[0006] When the persistence period has not ended, obtaining a first module identifier, where the first module identifier is used to indicate a first module in use whose module data has changed;

[0007] Determine a first pointer according to the first module identifier, where the first pointer points to target data, and the target data is module data in use in the first module;

[0008] Writing the first pointer into a first lock-free queue, so that a serialization thread reads the first pointer from the first lock-free queue, performs a serialization operation on the target data based on the first pointer, and writes the obtained serialized data into a second lock-free queue;

[0009] Read the serialized data in the second lock-free queue, and store the serialized data in a persistent storage medium.

[0010] Another aspect provides a data persistence device, the device comprising:

[0011] a tag acquisition module, configured to acquire a first module identifier when the persistence period has not ended, the first module identifier being used to indicate a first module in which module data in use has changed;

[0012] a data pointer determining module, configured to determine a first pointer according to the first module identifier, wherein the first pointer points to target data, and the target data is module data in use in the first module;

[0013] a data pointer operation module, configured to write the first pointer into a first lock-free queue, so that a serialization thread reads the first pointer from the first lock-free queue, performs a serialization operation on the target data based on the first pointer, and writes the obtained serialized data into a second lock-free queue;

[0014] The persistent storage module is used to read the serialized data in the second lock-free queue and store the serialized data in a persistent storage medium.

[0015] On the other hand, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored. The at least one instruction or the at least one program is loaded and executed by a processor to implement the data persistence method as described above.

[0016] This application determines the first pointer pointing to the module data in use in the first module according to the first module identifier obtained during the persistence cycle, and writes the first pointer to the first lock-free queue, so that the serialization thread performs serialization operations on the target data based on the first pointer read in the first lock-free queue, and writes the obtained serialized data to the second lock-free queue. By placing the time-consuming serialization operations in the data persistence process into an independent serialization thread, the processing pressure of the server is greatly reduced, thereby improving the efficiency of server persistence and reducing the risk of data loss caused by failure to perform persistence processing in a timely manner due to server abnormalities; the first lock-free queue and the second lock-free queue form a circular lock-free queue, which does not require additional locking logic processing, can reduce the performance overhead of the server and reduce the delay in data persistence. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1It is a schematic diagram of the implementation environment provided by the embodiment of the present application.

[0019] Figure 2 This is a flow chart of a data persistence method provided in an embodiment of the present application.

[0020] Figure 3 This is an example diagram of a circular lock-free queue provided in an embodiment of the present application.

[0021] Figure 4 This is an example diagram of data competition provided in an embodiment of the present application.

[0022] Figure 5 This is a flow chart of another data persistence method provided in an embodiment of the present application.

[0023] Figure 6 This is a flow chart of another data persistence method provided in an embodiment of the present application.

[0024] Figure 7 This is an example diagram of the processing flow after receiving a write request provided in an embodiment of the present application.

[0025] Figure 8 This is a flow chart of the data persistence method provided in the embodiment of the present application.

[0026] Figure 9 This is a flowchart of detecting whether the second target data has changed, provided in an embodiment of the present application.

[0027] Figure 10 This is an example diagram of changes in module data in use in the module provided in the embodiment of the present application.

[0028] Figure 11 This is a flow chart of the data persistence method provided in the embodiment of the present application.

[0029] Figure 12 It is a structural diagram of the data persistence device provided in an embodiment of the present application.

[0030] Figure 13 This is a hardware structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. It is clear that the embodiments described are only some of the embodiments of this application, and not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0033] First, the relevant terms involved in the embodiments of this application are explained as follows:

[0034] BLOB (Binary Large Object): A binary file container. In computers, BLOB is often used as a field type in databases to store binary files.

[0035] protobuf: is a protocol for serializing data structures, which is very useful for developing programs that communicate through pipes or store data.

[0036] Persistence: Storing data (such as objects in memory) to a permanent storage medium (such as a disk) is a mechanism for transitioning program data between a persistent state and a transient state. The primary application of persistence is storing objects in memory in a relational database, though it can also be stored in disk files or XML data files.

[0037] Lock-free queues: This means that multi-threaded synchronous operations do not require locking, because locking will slow down efficiency and cause delays. The most typical example is the producer and consumer model, which requires producers to consume while producing. This requires sufficiently low latency, and lock-free queues can be used at this time.

[0038] See also Figure 1 , which shows a schematic diagram of the implementation environment provided by the embodiment of the present application, which can implement the data persistence method of the present application. Figure 1 As shown, the implementation environment may include a terminal 01, a server 02, and a persistent storage medium 03. The terminal 01 and the server 02 may communicate data via a communication network. Optionally, the communication network may be a wired network or a wireless network, and the communication network may be at least one of a local area network, a metropolitan area network, and a wide area network.

[0039] The terminal 01 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited thereto. The terminal 01 may run a target client, which may be an operation web page provided by a service provider to a user, or a target application provided by the service provider to a user, and this embodiment of the application does not specifically limit this.

[0040] Server 02 can be a standalone server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. In the embodiment of the present application, server 02 is the backend server of the target client on terminal 01. Alternatively, server 02 can be a server that modularly stores data during the operation of the target client, such as a game server or a web server.

[0041] Schematically, in a game server, player data is composed of various modules, such as backpack, hero, technology, task and other modules. Different modules are stored as independent BLOBs in the player data table.

[0042] In the embodiment of the present application, each module includes at least two types of module data: one for the active state and the other for the inactive state. The active state module data refers to the data currently being used by the module, while the inactive state module data refers to the data not currently being used by the module, such as backup data or invalid data.

[0043] In a possible application scenario, such as Figure 1 As shown, the server is provided with a main business thread, a serialization thread, a first lock-free queue and a second lock-free queue. The main business thread is used to determine, during the persistence period, a first pointer pointing to the target data according to the first module identifier of the first module whose module data indicating the usage status has changed, and the target data is the module data in the usage status of the first module, and write the first pointer into the first lock-free queue; the serialization thread reads the first pointer from the first lock-free queue according to the serialization interval, performs a serialization operation on the target data based on the first pointer, and writes the serialized data obtained by the serialization operation into the second lock-free queue; if the main business thread reads the serialized data from the second lock-free queue, the serialized data is stored in the persistent storage medium.

[0044] From the above application scenarios, it can be seen that the first and second lock-free queues both adopt a lock-free approach. The server does not need additional locking logic, which can reduce the server's performance overhead and reduce the latency of data persistence. The main business thread executes the main business logic, and the time-consuming serialization operations are handled by a separate serialization thread. The multi-threaded asynchronous approach can reduce the server's serialization time. Without affecting the performance of the main business thread, the module data can be persisted in near real time, improving the real-time performance of data persistence and reducing the risk of data loss caused by failure to persist in time due to server exceptions.

[0045] In some embodiments, the terminal, server, and persistent storage medium may be a node in a distributed system, which may be a blockchain system. The blockchain system may be a distributed system formed by connecting multiple nodes through network communication. The nodes may form a peer-to-peer (P2P) network, and any computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0046] For ease of description, the data persistence method of this application is explained below using the server as an example execution entity.

[0047] Figure 2 It is a flowchart of a data persistence method provided by an exemplary embodiment of the present application. The present application provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 2 As shown, the method may include:

[0048] S201 : When the persistence period has not ended, obtain a first module identifier, where the first module identifier is used to indicate a first module in use whose module data has changed.

[0049] In this embodiment, the persistence period refers to the time period during which the server performs a persistence operation on the module data. The first module identifier represents the module identifier of the first module, and the module identifier is used to uniquely identify the module.

[0050] Optionally, the server may obtain the first module identifier from a preset identifier list, the preset identifier list being used to store module identifiers of modules with change flags set. The change flags are used to indicate that module data in use within the marked modules has changed within a preset time period. When a change is detected in module data in use within the preset time period, a change flag may be set for the module. The preset time period may be within a previous persistence period or between a previous persistence period and a current persistence period.

[0051] The server persists module data stored in non-persistent storage media (such as cache or content) according to a timed persistence strategy. In a timed persistence strategy, after the current persistence cycle ends, the server waits for the persistence interval before entering the next persistence cycle. The main business thread processes the persistence business logic within each persistence cycle. The server can detect whether a persistence cycle has ended based on the remaining time of the persistence cycle; or, it can detect whether a persistence cycle has ended based on the remaining time of the persistence cycle and the persistence interval.

[0052] In some embodiments, the server may compare the remaining time of the persistence period with the persistence interval; if the remaining time of the persistence period is less than the persistence interval, it may be determined that the persistence period has ended; if the remaining time of the persistence period is greater than or equal to the persistence interval, it may be determined that the persistence period has not ended.

[0053] In other embodiments, the server may also compare the remaining time of the persistence period with the preset end time; if the remaining time of the persistence period is less than the preset end time, it can be determined that the persistence period has ended; if the remaining time of the persistence period is greater than or equal to the preset end time, it can be determined that the persistence period has not ended.

[0054] The server is provided with a main business thread and a serialization thread. When it is determined that the persistence period has not ended, the main business thread is triggered to execute the step of obtaining the first module identifier. It should be noted that the persistence interval and the preset end time in the above embodiment can be set according to business needs. For example, the persistence interval or the preset end time can be set to 10ms, and this application does not make specific restrictions.

[0055] S203: Determine a first pointer according to the first module identifier, where the first pointer points to target data, and the target data is module data in use in the first module.

[0056] After modularizing data storage, the server configures at least two pointers for each module: one pointer pointing to the module data in the active state, and the other pointer pointing to the module data in the inactive state. The active state indicates that the module is currently in use, while the inactive state indicates that the module is not in use. Each module has only one active module data.

[0057] S205 , writing the first pointer into the first lock-free queue, so that the serialization thread reads the first pointer from the first lock-free queue, performs a serialization operation on the target data based on the first pointer, and writes the obtained serialized data into the second lock-free queue.

[0058] When the persistence period has not ended, the main business thread reads each first module identifier in the preset identifier list, and writes each first pointer determined according to each first module identifier into the first lock-free queue.

[0059] In practice, servers can mitigate the impact on server performance by increasing the persistence interval. However, the longer the persistence interval, the more data will be lost when a server exception occurs, and the greater the impact. However, the shorter the persistence interval, the more serialization overhead caused by frequent module data persistence will seriously affect server performance. Furthermore, as module data grows in size, the serialization overhead will also increase. Therefore, achieving near-real-time persistence while ensuring server performance is a challenge.

[0060] For modules, the persistence process goes through the steps of data collection, serialization, and storage. Data collection converts in-memory data into target objects, serialization serializes the target objects, and storage stores the serialized data in persistent storage. Target objects are objects that can be directly serialized, such as protobuf objects or objects in object-oriented programming languages (e.g., Java). Data collection and serialization can be synchronous, while storage can be asynchronous. Therefore, the time required for data collection and serialization directly affects persistence efficiency.

[0061] In one possible implementation, to reduce data collection time, the server can directly use the target object to store the module data when modularizing data storage. This means that the target data is stored in a non-persistent storage medium, such as a protobuf object, using the target object. When the persistence thread serializes the target data, it can avoid the data collection step, allowing the persistence thread to focus solely on data serialization, improving persistence efficiency. Optionally, the first pointer is a pointer to the target object, such as a protobuf object.

[0062] S207: Read the serialized data in the second lock-free queue, and store the serialized data in a persistent storage medium.

[0063] In this embodiment, the persistent storage medium can be a relational database, a disk, or the like. Optionally, if the persistent storage medium is a relational database, the main service thread in the server can store the serialized data in the relational database using the BLOB type. Throughout the persistence process, the server adopts a single-producer, single-consumer model, introducing a circular lock-free queue. Producers only need to generate content and place it in the circular lock-free queue, while consumers only need to retrieve content from the circular lock-free queue. Producers and consumers do not need to perform any operations related to locking logic.

[0064] Indicative, such as Figure 3 As shown, the main business thread writes the first pointer to the first lock-free queue, waiting for the serialization thread to serialize it. The serialization thread reads the first pointer from the first lock-free queue, serializes the target data pointed to by the first pointer, and writes the resulting serialized data to the second lock-free queue. The main business thread can read the serialized data in the second lock-free queue and then store it in a persistent storage medium. The first and second lock-free queues form a circular lock-free queue. Since there is no locking logic, the server performance overhead caused by unnecessary locking logic can be avoided.

[0065] In the above embodiment, the target data is stored in a non-persistent storage medium as a target object, which can avoid the time consumption of data collection; the more time-consuming serialization operations are placed in an independent serialization thread for processing, which greatly reduces the processing pressure of the main business thread, thereby improving the efficiency of server persistence and reducing the risk of data loss caused by failure to perform persistence processing in a timely manner due to server abnormalities; the first lock-free queue and the second lock-free queue do not require additional locking logic, which can reduce the performance overhead of the server.

[0066] In a possible implementation, after the main business thread writes the first pointer into the first lock-free queue, the main business thread may perform a write operation on the target data. At this time, a read-write competition occurs between two threads on the target data.

[0067] Indicative, such as Figure 4As shown, for the same module, during the first persistence cycle, the main business thread writes the first pointer to the first lock-free queue and obtains the serialized data during the fourth persistence cycle. If a write request for the target data is received during the second persistence cycle or the third persistence cycle (that is, between writing to the first lock-free queue and obtaining the result from the second lock-free queue), the main business thread needs to write the target data, while the serialization thread needs to read the target data, resulting in data contention and data conflict.

[0068] In order to resolve the above data conflicts, the main business thread can use the write-time copy method. Figure 5 As shown, after step S205 is implemented, the method may further include:

[0069] S2063: Upon receiving a write request for target data, detect whether the first module is in a serialization state.

[0070] In one possible implementation, after the main business thread writes the first pointer to the first lock-free queue, it can also mark the first module with a status mark indicating serialization. If the main business thread receives a write request for the target data in the first module, it can detect whether the module is in a serialized state based on the status mark.

[0071] Please refer to the following for details: Figure 6 As shown, before step S2063 is implemented, the method may further include:

[0072] S2061: Set a status flag for the first module.

[0073] In this embodiment, the status mark of each module is used to indicate that the marked module is in a serialized state. The module is in a serialized state, that is, the module is in serialization. The status mark can be expressed in digital form or string form. For example, the status mark is set to 1 or "serialization", which is not specifically limited here.

[0074] Accordingly, step S2063 may include, in specific implementation:

[0075] S20631: Receive a write request for target data, and detect whether the first module is in a serialization state according to a status flag of the first module.

[0076] Specifically, the main business thread may compare the status flag with a preset flag; if the two are consistent, it can be determined that the first module is in a serialized state; if the two are inconsistent, it can be determined that the first module is not in a serialized state. The preset flag is the value of the status flag when it is set or a fixed value, which is not specifically limited here.

[0077] Accordingly, after step S207 is implemented, the method may further include:

[0078] S209: Clear the status flag of the first module.

[0079] If the main business thread reads the serialized data corresponding to the module from the second lock-free queue, it can clear the module's status flag. By detecting whether the module is in the serialized state, it can determine whether there is a data race and thus whether to enable the copy-on-write technology. If the module is in the serialized state, the copy-on-write technology is enabled; if the module is not in the serialized state, the copy-on-write technology is not enabled.

[0080] S2065, when the first module is in a serialized state, back up the target data to obtain backup data; mark the status of the backup data as a used state, and mark the status of the target data as a non-used state; update the target data according to the mark of the data status in the module.

[0081] The main business thread updates the module data marked as in use to the target data based on the mark of the data status in the module. In one example, before receiving a write request, the target data in the memory is module data 1; after receiving the write request, the main business thread copies a copy of module data 1, and then modifies the obtained module data 2, marking the status of module data 2 as in use and the status of module data 1 as unused; then the target data is updated, and the updated target data is module data 2, thereby ensuring that the target data is always the module data in use in the module. It can be understood that there is only one module data in use in each module, but there can be one or more module data in an unused state.

[0082] S2067: When the first module is not in a serialized state, set a change flag for the first module.

[0083] The change flag is used to indicate that the module data in use in the marked module has changed. If the main business thread detects that the first module is not being serialized, it needs to set the change flag so that the first pointer pointing to the target data in the first module is added to the first lock-free queue at the end of the current persistence cycle, allowing the serialization thread to perform the serialization operation. If the main business thread detects that the first module is being serialized, it does not need to set the change flag, but waits until the serialized data of the first module is obtained before setting it.

[0084] S2069: Feedback the target data to the requester that issued the write request.

[0085] Schematically, as Figure 7As shown, after the main business thread receives the write request 701, it checks whether the module is being serialized 702; if the module is being serialized, the target data is copied and the obtained backup data is set as currently in use 704; if the module is not being serialized, a change mark is set for the module 703; and then the module data in use 705 is returned to the sender of the write request.

[0086] By adopting the write-time copy technology, the contention conflict of writing and reading the same module data can be avoided, the waiting time of another thread can be saved, and the efficiency of the server in data persistence can be improved.

[0087] In one possible implementation, in order to ensure that the serialized data stored in the persistent storage medium corresponds to the module data in use in the first module, after the main business thread obtains the serialized data from the second lock-free queue, it needs to determine the processing strategy for the serialized data based on the current usage of the second module corresponding to the serialized data.

[0088] See also Figure 8 , which shows a flow chart of the data persistence method provided by the embodiment of the present application. Figure 8 As shown, step S207 may include:

[0089] S2071, reading serialized data in the second lock-free queue, and determining a second module corresponding to the serialized data;

[0090] S2073, detecting whether the module data in use in the second module has changed;

[0091] S2075, if the module data in use in the second module has not changed, storing the serialized data in a persistent storage medium;

[0092] S2077: If the module data in use in the second module has changed, set a change mark for the second module and discard the serialized data.

[0093] In a possible implementation, if the target data in the second module has changed, the corresponding pointer pointing to the target data has also changed. The main business thread can determine whether the target data has changed based on the pointer pointing to the target data.

[0094] See also Figure 9 , which shows a flow chart of detecting whether the module data in use in the target module has changed according to an embodiment of the present application. Figure 9 As shown, step S2073 may include:

[0095] S20731, determining a second pointer, where the second pointer points to module data in use in the second module; and determining a third pointer, where the third pointer points to module data in the second module on which a serialization operation is performed.

[0096] Indicative, such as Figure 10 As shown, for the second module, before receiving a write request, module data 1 is the module data in use and will be serialized, that is, the third pointer points to module data 1; if no write request is received before obtaining the serialization result, the second pointer also points to module data 1; if a write request is received before obtaining the serialization result, the main business thread uses the write-time copy technology to back up module data 1 to obtain module data 2. Module data 2 is the module data in use, that is, the second pointer points to module data 2. Therefore, the second pointer can be directly compared with the third pointer to determine whether the module data in use has changed.

[0097] S20733, determine whether the second pointer is consistent with the third pointer.

[0098] If the second pointer is consistent with the third pointer, step S20735 is executed; if the second pointer is inconsistent with the third pointer, step S20737 is executed.

[0099] S20735: Determine that the module data in use in the second module has not changed.

[0100] S20737: Determine whether module data in use in the second module has changed.

[0101] In another possible implementation, the main business thread may set a conflict flag for the module when implementing step S2065. The conflict flag is used to indicate whether the module data in use in the module has changed. After the main business thread obtains the serialized data, it may detect whether a change has occurred based on the conflict flag of the target module corresponding to the serialized data.

[0102] The above embodiment can ensure that the serialized data stored in the persistent storage medium corresponds to the module data in use by determining the module data in use in the second module, thereby preventing invalid storage operations caused by data asynchrony.

[0103] As described in step S201 above, the server can obtain the first module identifier from the preset identifier list. In some embodiments, Figure 11 As shown, after step S205 is implemented, it may further include:

[0104] S1101: Remove the first module identifier from the preset identifier list, and clear the change mark of the first module.

[0105] Accordingly, the data persistence method in this embodiment may further include:

[0106] S1103: When the persistence period ends, the module identifier corresponding to the module with the change mark is determined as the first module identifier, and the first module identifier is stored in a preset identifier list.

[0107] In the above embodiment, at the end of the current persistence cycle, the module identifier of the module with the change mark is stored as the first module identifier in the preset identifier list, so that in the next persistence cycle, the first pointer of the first module indicated by all the first module identifiers in the preset identifier list is written to the first lock-free queue, rather than writing the first pointer to the first lock-free queue in the current persistence cycle. This ensures that when the module data in use is frequently changed, the resulting serialized data is invalid data, reducing the probability of the serialized data being discarded. In actual applications, the module data is not necessarily added to the first lock-free queue in the next persistence cycle after it is changed. The specific situation can be set according to the specific business.

[0108] In other embodiments, the serialization thread may also record the storage time when storing the module identifier of the module with the change mark in the preset identifier list. When reading the first module identifier from the preset identifier list, it may only process the module identifier whose storage time satisfies the preset storage condition. That is, the first pointer is written to the lock-free queue only when the storage time satisfies the preset storage condition. Among them, the preset storage condition represents the condition that the storage time is greater than the preset time threshold. The preset time threshold can be set only according to the persistence interval, or only according to the persistence period, or according to the persistence interval and the persistence period. This application does not make specific restrictions.

[0109] The data persistence method provided in the above embodiment is verified below by taking the target object as a protobuf object (PB for short) and the data as game data as an example.

[0110] First, we verified the data collection time required for different module data sizes. Table 1 shows the total time required to construct the PB for the corresponding module data 1000 times. As the module data size increases, the time required to construct the PB increases accordingly. This data collection overhead can be effectively avoided by directly storing module data in a non-persistent storage medium using the PB.

[0111] Table 1

[0112] Module data (bytes) 60 137 357 577 797 Time taken to construct PB (milliseconds) 3 7 14 22 31

[0113] Next, we verified the performance of copying and deserialization. Table 2 shows the total time taken for 1000 runs. Under the same conditions, copy-on-write outperforms direct serialization. Even in the most extreme case, where every write request results in a copy due to an access conflict, it still outperforms direct serialization in the main business thread.

[0114] Table 2

[0115] PB size (bytes) 105 807 5029 Copy-on-write time (milliseconds) 35 306 2463 Direct serialization time (milliseconds) 49 376 2994

[0116] Next, a set of stress tests was conducted to verify the serialization performance of multithreading and copy-on-write. This set of stress tests verified the server CPU utilization when using the main business thread for persistence, i.e., single-threaded (the first strategy), when using protobuf object storage module data (the second strategy), and when using multithreading and copy-on-write for persistence (the third strategy). Multithreading refers to the main business thread and serialization thread in the above embodiment.

[0117] During the verification process, 200 players per second (800 players total) triggered data changes in both modules, so the entire verification process lasted 4 seconds. The detection tool then captured the CPU utilization of the current process every 0.5 seconds. The results are shown in Table 3. Using protobuf objects to store module data nearly doubled server performance. Using protobuf objects and serializing them in a separate thread further doubled server performance compared to using only protobuf objects, significantly alleviating the computational pressure on the main business thread.

[0118] Table 3

[0119]

[0120]

[0121] It can be seen from the technical solutions provided by the above embodiments that the present application can avoid the time consumption caused by data collection in the persistence process by directly using the protobuf object storage module data to a non-persistent storage medium; by placing the time-consuming serialization operations in the data persistence process into an independent serialization thread, the processing pressure of the server is greatly reduced, thereby improving the efficiency of server persistence and reducing the risk of data loss caused by failure to perform persistence processing in a timely manner due to server abnormalities; the first lock-free queue and the second lock-free queue form a circular lock-free queue, which does not require additional locking logic processing, can reduce the performance overhead of the server and reduce the delay in data persistence.

[0122] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a data persistence device, which can implement the functions of the above method embodiment. Figure 12 As shown, the apparatus 1200 may include:

[0123] The tag acquisition module 1210 is used to acquire a first module identifier when the persistence period has not ended, where the first module identifier is used to indicate a first module in use whose module data has changed.

[0124] The data pointer determining module 1220 is configured to determine a first pointer according to the first module identifier, where the first pointer points to target data, and the target data is module data in use in the first module;

[0125] a data pointer operation module 1230 configured to write the first pointer into the first lock-free queue, so that the serialization thread reads the first pointer from the first lock-free queue, performs a serialization operation on the target data based on the first pointer, and writes the obtained serialized data into the second lock-free queue;

[0126] The persistent storage module 1240 is configured to read the serialized data in the second lock-free queue and store the serialized data in a persistent storage medium.

[0127] In a possible implementation, the apparatus 1200 may further include:

[0128] a write request receiving module, configured to receive a write request for the target data and detect whether the first module is in a serialization state;

[0129] The data backup module is used to back up the target data when the first module is in a serialized state to obtain backup data; mark the status of the backup data as a used state and the status of the target data as a non-used state; and update the target data according to the mark of the data status in the module;

[0130] a data marking module, configured to set a change mark for the first module when the first module is not in a serialized state, the change mark being used to indicate that module data in the marked module that is in use has changed;

[0131] The data feedback module is used to feed back the target data to the requester who issued the write request.

[0132] In a possible implementation, the apparatus 1200 may further include:

[0133] A serialization setting module is used to set a status mark for the first module, and the status mark is used to indicate that the marked module is in a serialization state;

[0134] Accordingly, the write request receiving module can also be used to:

[0135] A write request for target data is received, and whether the first module is in a serialization state is detected according to a state flag of the first module.

[0136] Accordingly, the apparatus 1200 may further include:

[0137] The serialization clearing module is used to clear the status mark of the first module.

[0138] In one possible implementation, the persistent storage module 1240 may include:

[0139] A serialized data reading unit, configured to read the serialized data in the second lock-free queue and determine the second module corresponding to the serialized data;

[0140] A data change detection unit, used to detect whether the module data in use in the second module has changed;

[0141] The first processing unit is configured to store the serialized data in a persistent storage medium when the module data in use in the second module has not changed.

[0142] In a possible implementation, the persistent storage module 1240 may further include:

[0143] The second processing unit is configured to set a change mark for the second module and discard the serialized data when module data in use in the second module has changed.

[0144] In a possible implementation, the data change detection unit may include:

[0145] a pointer determining unit, configured to determine a second pointer, the second pointer pointing to module data in use in the second module; and determine a third pointer, the third pointer pointing to module data in the second module on which a serialization operation is performed;

[0146] a pointer comparison unit, configured to determine whether the second pointer is consistent with the third pointer;

[0147] The first determining unit is configured to determine that the module data in use in the second module has not changed when the second pointer is consistent with the third pointer;

[0148] The second determining unit is configured to determine that module data in use in the second module has changed when the second pointer is inconsistent with the third pointer.

[0149] In a possible implementation, the tag acquisition module 1210 is further configured to acquire the first module identifier from a preset identifier list when the persistence period has not ended.

[0150] Accordingly, the apparatus 1200 may further include:

[0151] a mark clearing module, removing the first module identifier from the preset identifier list and clearing the change mark of the first module;

[0152] The to-be-processed data acquisition module is configured to, when the persistence period ends, determine the module identifier corresponding to the module with the change mark as the first module identifier, and store the first module identifier in a preset identifier list.

[0153] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0154] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded by the processor and executes the data persistence method provided by the above method embodiment.

[0155] Further, Figure 13 A schematic diagram of the hardware structure of a computer device for implementing the embodiment of the present application is shown. The device may participate in or include the apparatus or system provided in the embodiment of the present application. Figure 13 As shown, the device 13 may include one or more (illustrated as 1302a, 1302b, ..., 1302n in the figure) processors 1302 (the processor 1302 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1304 for storing data, and a transmission device 1306 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 13The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 13 More or fewer components than shown, or with Figure 13 Different configurations shown.

[0156] It should be noted that the one or more processors 1302 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the device 13 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0157] The memory 1304 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the methods described in the embodiments of the present application. The processor 1302 executes various functional applications and data processing by running the software programs and modules stored in the memory 1304, thereby realizing the above-mentioned fluency detection method. The memory 1304 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1304 may further include a memory remotely located relative to the processor 1302, and these remote memories may be connected to the device 13 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] Transmission device 1306 is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the communications provider of device 13. In one embodiment, transmission device 1306 may include a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 1306 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0159] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the device 13 (or mobile device).

[0160] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the data persistence method provided by the above method embodiment.

[0161] Optionally, in this embodiment, the storage medium may be located in at least one of a plurality of network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0162] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data persistence method provided in the above method embodiment.

[0163] It should be noted that the order of the embodiments of the present application described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0164] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device and electronic device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

[0165] The above description has fully disclosed the specific embodiments of this application. It should be noted that any changes made by those skilled in the art to the specific embodiments of this application do not depart from the scope of the claims of this application. Accordingly, the scope of the claims of this application is not limited to the above specific embodiments.

Claims

1. A data persistence method, characterized in that: The method comprises: When the persistence period has not ended, obtaining a first module identifier, where the first module identifier is used to indicate a first module in use whose module data has changed; Determine a first pointer according to the first module identifier, where the first pointer points to target data, and the target data is module data in use in the first module; Writing the first pointer into a first lock-free queue, so that a serialization thread reads the first pointer from the first lock-free queue, performs a serialization operation on the target data based on the first pointer, and writes the obtained serialized data into a second lock-free queue; Read the serialized data in the second lock-free queue, and store the serialized data in a persistent storage medium.

2. The method according to claim 1, characterized in that After writing the first pointer into the first lock-free queue, the method further includes: receiving a write request for the target data, and detecting whether the first module is in a serialization state; When the first module is in the serialized state, backing up the target data to obtain backup data; Marking the state of the backup data as the used state, and marking the state of the target data as the unused state; updating the target data according to the mark of the data state in the first module; The target data is fed back to the requester that issues the write request.

3. The method according to claim 2, characterized in that The method further comprises: In a case where the first module is not in the serialized state, a change mark is set for the first module, where the change mark is used to indicate that module data in the marked module that is in the used state has changed.

4. The method according to claim 2 or 3, characterized in that Before receiving the write request for the target data and detecting whether the first module is in a serialization state, the method further includes: Setting a status mark for the first module, wherein the status mark is used to indicate that the marked module is in the serialization state; Accordingly, upon receiving a write request for the target data, detecting whether the first module is in a serialization state includes: receiving a write request for the target data, and detecting, according to a status flag of the first module, whether the first module is in the serialization state; Accordingly, after reading the serialized data in the second lock-free queue and storing the serialized data in a persistent storage medium, the method further includes: Clear the status flag of the first module.

5. The method according to claim 1 or 2, characterized in that The reading of the serialized data in the second lock-free queue and storing the serialized data in a persistent storage medium includes: Reading serialized data in the second lock-free queue, and determining a second module corresponding to the serialized data; detecting whether module data in the second module that is in the use state has changed; When the module data in the second module that is in the use state has not changed, the serialized data is stored in the persistent storage medium.

6. The method according to claim 5, characterized in that The method further comprises: In the case that the module data in the second module that is in the use state has changed, a change mark is set for the second module, and the serialized data is discarded.

7. The method according to claim 5 or 6, characterized in that The detecting whether the module data in the second module in the use state has changed includes: determining a second pointer, the second pointer pointing to the module data in the second module that is in the used state; and determining a third pointer, the third pointer pointing to the module data in the second module that is subjected to the serialization operation; Determining whether the second pointer is consistent with the third pointer; If the second pointer is consistent with the third pointer, it is determined that the module data in the second module in the used state has not changed; If the second pointer is inconsistent with the third pointer, it is determined that the module data in the second module that is in the use state has changed.

8. The method according to claim 1, characterized in that The obtaining of the first module identifier when the persistence period has not ended includes: When the persistence period has not ended, obtaining the first module identifier from the preset identifier list; After writing the first pointer into the first lock-free queue, the method further includes: Remove the first module identifier from the preset identifier list, and clear the change mark of the first module; The method further comprises: When the persistence period ends, determining the module identifier corresponding to the module set with the change mark as the first module identifier; The first module identifier is stored in the preset identifier list.

9. A data persistence device, characterized in that: The device comprises: a tag acquisition module, configured to acquire a first module identifier when the persistence period has not ended, the first module identifier being used to indicate a first module in which module data in use has changed; a data pointer determining module, configured to determine a first pointer according to the first module identifier, wherein the first pointer points to target data, and the target data is module data in use in the first module; a data pointer operation module, configured to write the first pointer into a first lock-free queue, so that a serialization thread reads the first pointer from the first lock-free queue, performs a serialization operation on the target data based on the first pointer, and writes the obtained serialized data into a second lock-free queue; The persistent storage module is used to read the serialized data in the second lock-free queue and store the serialized data in a persistent storage medium.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the data persistence method according to any one of claims 1 to 8.

Images