Millisecond-level data disaster recovery method for game server

Abstract

The application discloses a kind of game server millisecond level data disaster recovery methods, comprising: main thread is delivered by lockless queue, journal thread is received by lockless queue and constantly raises once and once data, then this part of data is written to local disk;Automatically collect the landing data generated by server, and save in local disk, even if due to game server suddenly abnormal or database abnormal, previous data writing will not be lost;By automatic recovery method, data can be quickly recovered.The method of the application can recover millisecond level data, effectively improve the data security of game server, and ensure the stability of game server.The operation of the application occurs in the system, and no additional operation is required for game service, which greatly improves the game data security without reducing the efficiency of game development.

Description

Technical Field

[0001] This invention belongs to the field of software technology and relates to data disaster recovery technology, specifically a millisecond-level data disaster recovery method for game servers. Background Technology

[0002] In online games, players log in to their respective characters using usernames, and all data is stored on the game server. Most game servers on the market store player data in memory and periodically save it to a database. If the game server fails between these scheduled saves, player data is lost, causing a serious rollback and incalculable damage to the game. To solve this problem, we have developed a millisecond-level data disaster recovery solution that automatically records every change to player data in real time. In the event of a server failure, it automatically restores player data, improving the data security of the game's operations. Summary of the Invention

[0003] This invention discloses a millisecond-level data disaster recovery solution. This solution can automatically collect the data generated by the server and save it on the local disk. Even if the game server suddenly fails or the database fails, the previous data will not be lost. Through automated recovery methods, data can be recovered quickly.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows:

[0005] The millisecond-level data disaster recovery method for game servers includes the following steps:

[0006] Step 1: The main thread sends out data through a lock-free queue. The journal thread that receives the data continuously retrieves data from the lock-free queue and then writes this data to the local disk.

[0007] Step 2: Maintain a DataLine for each player. Each DataLine contains at least one BaseLine, which records all data changes for that player from logging in to logging out. Each BaseLine contains at least one VersionLine, which represents the data changes that occur between two saves. After logging in, a player initiates a base operation, which saves a snapshot of the saved data to disk. After successfully saving offline, a player initiates a baseEnd operation, which saves the save result to disk. Each time a player initiates a scheduled save, a save operation is initiated, which saves the action of the initiated operation to disk. Each time a scheduled save is successful, a saveAck operation is initiated, which saves the save result to disk. During the player's online period, any data operation will initiate an operation, which writes the specific data changes to the local disk.

[0008] Step 3: The data files generated in Step 2 include three types: cur files, pre files, and merge files. The cur file is the file that the journal thread is currently writing to. The pre file is the file that is split each time the cur file reaches a threshold size. The merge file is the result file obtained by merging the pre files after removing file redundancy. The journal thread first generates and writes to the cur file. When the cur file reaches a threshold size, it splits the cur file into pre files of the threshold size and a new cur file. The journal thread continues to generate data into the new cur file until it reaches the threshold again, and then splits it again. The merge program merges at least two pre files into a merge file and removes data redundancy.

[0009] Step 4: After processing each small file in Step 3, an automated program initiates a recovery process after the game server malfunctions. Player data is recovered using the remaining files and finally stored in the database.

[0010] Furthermore, the data generated in step 2 is stored in a file in a streaming manner.

[0011] Furthermore, in step 3, the three types of files share the same sequence number: the sequence number of the cur file is strictly incremented, the sequence number of the pre file corresponds to the cur sequence number of its split, and the sequence number of the merge file is equal to the maximum per sequence number it processes.

[0012] Furthermore, the recovery process in step 4 is as follows: check the journal file of the game process; if there are any residual files, restore the player data through the residual files and finally store it in the database; if there are no residual files, start the normal game process and start the timed merge task.

[0013] Furthermore, the files left over from step 4 include cur files and at least one of the following: pre files and merge files.

[0014] The beneficial effects of this invention are as follows:

[0015] It can automatically collect server-generated data and store it on the local disk, ensuring no data loss even in the event of sudden anomalies. Through automated recovery methods, it can automatically recover data at the millisecond level, effectively improving game server data security and ensuring server stability. All operations in this invention occur internally, requiring no additional processing from the game's operational staff, thus significantly enhancing game data security without reducing game development efficiency. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating data delivery to the main thread.

[0017] Figure 2 A diagram illustrating the process of saving player data.

[0018] Figure 3 A diagram illustrating the generation of journal data files.

[0019] Figure 4 This is a diagram illustrating data recovery. Detailed Implementation

[0020] The technical solutions provided by the present invention will be described in detail below with reference to specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0021] This embodiment uses a specific player data recovery process to simulate the implementation of the solution of the present invention.

[0022] The millisecond-level data disaster recovery method for game servers provided by this invention includes the following steps:

[0023] Step 1, as follows Figure 1As shown, this invention designs a lock-free queue. The main thread uses this lock-free queue to send data out, minimizing the overhead of the main thread. The journal thread that receives the data will continuously retrieve data from the lock-free queue and then write this data to the local disk.

[0024] Step 2: This invention maintains a DataLine for each player. Each DataLine contains at least one BaseLine, which records all data changes for the player from logging in to logging out. Each BaseLine also contains at least one VersionLine, representing the data changes that occur between two saves. Upon logging in, the player initiates a base operation, which saves a snapshot of the saved data to disk. After successfully saving offline, the player initiates a baseEnd operation, saving the save result to disk. Each time the player initiates a scheduled save, a save operation is initiated, saving the action of the initiated operation to disk. Each time a scheduled save is successful, a saveAck operation is initiated, saving the save result to disk. During the player's online period, any data operation will initiate an operation, which writes the specific data changes to the local disk.

[0025] like Figure 2As shown, after a player logs into the game server, they first initiate a `base` operation. This reads data from the database and generates a `base` with version 100. This operation saves a snapshot of the current data to the disk. Subsequently, the player performs three data modification operations on the game server. Each data modification operation triggers an `operation` operation, resulting in three `operation` operations that write the specific data changes to the local disk. Then, the scheduled save logic is triggered, and the player initiates a scheduled save, changing the version from 100 to 101. This inserts a `save101` operation, saving the initiated action to the disk. The player then performs three more data modifications, resulting in three more `operation` operations. Then, the timed save logic was triggered. The player initiated a `save102` operation, received a save result of `101`, inserted a `saveack101` operation, performed a data change, initiated an `operation` operation, received a save result of `102`, inserted a `saveack102` operation, and finally, the player logged off, generating a `baseend100` operation. After some time, the player logged back in, generating a `base102` operation. After three consecutive data operations, the game server experienced an anomaly. Because the journal system stored the three `operation` operations following `base102`, we were able to perform effective data recovery.

[0026] Step 3: The data generated in Step 2 is ultimately stored in files in a streaming manner. Since the data generated in Step 2 is redundant, we can further remove redundant data during file processing. We define files into three categories: `cur` files, which are currently being written by the journal thread; `pre` files (the size of `pre` files is greater than or equal to a threshold, mainly depending on the size of the last inserted data), which are split after the size of the `cur` file reaches the threshold; and `merge` files, which are the result files of merging `pre` files after redundancy removal. The basic principle of redundancy removal is that whenever the log receives a `saveackversion` message, it indicates that the database has been updated to the `version` version, and data operations before the `version` version in the log are redundant and can be deleted. All file types share a sequence number, which follows certain usage principles: the sequence number of a `cur` file is strictly incremental; the sequence number of a `pre` file corresponds to the sequence number of its split `cur` file; and the sequence number of a `merge` file is equal to the maximum `per` sequence number it processes. This sequence number principle enables the entire file processing process to be fault-tolerant; even if the file processing process is abnormally interrupted at any step, it can be self-corrected by restarting.

[0027] like Figure 3 As shown, journal data is continuously generated throughout the game server's operation, and the entire file generation process is as follows:

[0028] 1. The Journal thread first generates the cur.1 file. After a period of time, the cur.1 file reaches its maximum size and is then split into the pre.1 and cur.2 files.

[0029] 2. The Journal thread continues to generate data into the cur.2 file. After a period of time, when cur.2 reaches its limit, it is split into the pre.2 file and the cur.3 file.

[0030] 3. The Merge program finds the pre.1 and pre.2 files, merges them into a merge.2 file, and removes data redundancy. The merge program will start periodically to merge the pre files.

[0031] This process was repeated until the cur.4 and merge.3 files were generated.

[0032] Step 4: After processing each small file in Step 3, an automated program initiates a recovery process in the event of a game server failure. This involves checking the game process's journal file. If any legacy files are found, player data is recovered using these files and finally stored in the database. If no legacy files are found, the normal game process begins, and a scheduled merge task is started.

[0033] like Figure 4 As shown, when a server malfunctions, restarting the corresponding game server will execute the corresponding automated operation and maintenance process. This process will check for any remaining cur.4 and merge.3 files, and based on the contents of the files, restore the three data entries after base102 to the database, ultimately ensuring the safety of player data.

[0034] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A millisecond-level data disaster recovery method for game servers, characterized in that, Includes the following steps: Step 1: The main thread sends out data through a lock-free queue. The journal thread that receives the data continuously retrieves data from the lock-free queue and then writes this data to the local disk. Step 2: Maintain a DataLine for each player. Each DataLine contains at least one BaseLine, which records all data changes for that player from logging in to logging out. Each BaseLine contains at least one VersionLine, which represents the data changes that occur between two saves. After logging in, a player initiates a base operation, which saves a snapshot of the saved data to disk. After successfully saving offline, a player initiates a baseEnd operation, which saves the save result to disk. Each time a player initiates a scheduled save, a save operation is initiated, which saves the action of the initiated operation to disk. Each time a scheduled save is successful, a saveAck operation is initiated, which saves the save result to disk. During the player's online period, any data operation will initiate an operation, which writes the specific data changes to the local disk. Step 3: The data files generated in Step 2 include three types: cur files, pre files, and merge files. The cur file is the file that the journal thread is currently writing to. The pre file is the file that is split each time the cur file reaches a threshold size. The merge file is the result file obtained by merging the pre files after removing file redundancy. The journal thread first generates and writes to the cur file. When the cur file size reaches the threshold, it splits the cur file into pre files of the threshold size and a new cur file. The journal thread continues to generate data into the new cur file until it reaches the threshold again, and then splits it again. The merge program merges at least two pre files into a merge file and removes data redundancy. The merge file is the result file of merging the pre files after removing file redundancy. When the log receives a saveackversion message, it indicates that the database has been updated to version version. Data before version version is deleted, thus achieving redundancy removal. Step 4: After processing each small file in Step 3, an automated program initiates a recovery process after the game server malfunctions. Player data is recovered using the remaining files and finally stored in the database. Specifically, this includes: detecting any residual cur, per, or merge files, and restoring the data since the last save to the database based on the content of the residual files, ultimately ensuring the safety of player data.

2. The millisecond-level data disaster recovery method for game servers according to claim 1, characterized in that, The data generated in step 2 is stored in a file in a streaming manner.

3. The millisecond-level data disaster recovery method for game servers according to claim 1, characterized in that, In step 3, the three types of files share the same sequence number: the sequence number of the cur file is strictly incremented, the sequence number of the pre file corresponds to the cur sequence number of its split, and the sequence number of the merge file is equal to the maximum per sequence number it processes.

4. The millisecond-level data disaster recovery method for game servers according to claim 1, characterized in that, The recovery process in step 4 is as follows: check the journal file of the game process; if there are any residual files, recover the player data through the residual files and finally store it in the database. If no legacy files are found, the normal game process will begin, and a scheduled merge task will be started.

5. The millisecond-level data disaster recovery method for game servers according to claim 1, characterized in that, The files left over from step 4 include the .cur file and at least one of the following: the .pre file and the .merge file.

Images