Data synchronization methods, apparatus, electronic devices and computer-readable storage media

Abstract

This application discloses a data synchronization method, apparatus, electronic device, and computer-readable storage medium, applicable to various scenarios such as computer technology, cloud technology, artificial intelligence, intelligent transportation, and assisted driving. The method includes: receiving a data synchronization request from a server, the data synchronization request requesting data to be synchronized; obtaining configuration information of the data to be synchronized according to the data synchronization request, the configuration information including the data type of the data to be synchronized; performing a data synchronization operation according to a synchronization process corresponding to the configuration information, the synchronization process being used to synchronize the data to be synchronized of different data types to the server, and managing the data to be synchronized of different data types. This application embodiment can help servers efficiently and accurately synchronize different types of data files and effectively manage the synchronized data files.

Description

Technical Field

[0001] This application relates to the field of computer technology, and specifically to a data synchronization method, apparatus, computer device, and readable storage medium. Background Technology

[0002] There are two ways to synchronize data files on a server: one is to develop a data management module inside the server to synchronize and manage data files; the other is to use external tools, configure them for data synchronization, and then use these tools to help the server achieve data synchronization.

[0003] However, both of the above data synchronization methods are difficult to develop and require a large amount of development work. Furthermore, the synchronization tools for different types of data files vary significantly, making it difficult to achieve reliable and fast data synchronization and data management. Summary of the Invention

[0004] This application provides a data synchronization method, apparatus, electronic device, and computer-readable storage medium, which can help servers efficiently and accurately synchronize different types of data files and effectively manage the synchronized data files.

[0005] A first aspect of this application provides a data synchronization method, the method comprising:

[0006] Receive a data synchronization request from the server, the data synchronization request being used to request data to be synchronized;

[0007] According to the data synchronization request, obtain the configuration information of the data to be synchronized, the configuration information including the data type of the data to be synchronized;

[0008] Data synchronization is performed according to the synchronization process corresponding to the configuration information. The synchronization process is used to synchronize the data to be synchronized of different data types to the server and to manage the data to be synchronized of different data types.

[0009] Accordingly, a second aspect of the present application provides a data synchronization apparatus, the apparatus comprising:

[0010] A data synchronization request receiving module is used to receive data synchronization requests from the server, wherein the data synchronization requests are used to request data to be synchronized;

[0011] The configuration information acquisition module is used to acquire the configuration information of the data to be synchronized according to the data synchronization request, wherein the configuration information includes the data type of the data to be synchronized;

[0012] The data synchronization module is used to perform data synchronization operations according to the synchronization process corresponding to the configuration information. The synchronization process is used to synchronize the data to be synchronized of different data types to the server and to manage the data to be synchronized of different data types.

[0013] Optionally, the data synchronization module may further include:

[0014] The delivery task generation submodule is used to generate data delivery tasks, which are used to send the configuration information to the server.

[0015] The delivery task scheduling submodule is used to schedule the data delivery tasks;

[0016] The delivery task execution submodule is used to execute the scheduled data delivery task.

[0017] Optionally, the delivery task generation submodule is further specifically used for:

[0018] When the data type of the data to be synchronized is a global type, the data to be synchronized is stored in the first space to generate a delivery task for the data to be synchronized.

[0019] The delivery task execution submodule is also specifically used for:

[0020] A global synchronization process is performed on the data to be synchronized in the first space to execute the delivery task of the data to be synchronized.

[0021] Optionally, the data type includes a distributed type, and the delivery task generation submodule is further specifically used for:

[0022] When the data type of the data to be synchronized is distributed, the data to be synchronized is stored in the second space to generate the delivery task of the data to be synchronized;

[0023] The delivery task execution submodule is also specifically used for:

[0024] A distributed synchronization process is performed on the data to be synchronized in the second space to execute the delivery task of the data to be synchronized. Optionally, the delivery task scheduling submodule may also be specifically used for:

[0025] The global type of data to be synchronized is sent to the server in a first manner. The first manner includes sending the global type of data to be synchronized under the target list to each service node of the server. The target list is a list of data files where the data to be synchronized is located.

[0026] The distributed data to be synchronized is sent to the server in a second manner, wherein the second manner includes sending the distributed data to be synchronized corresponding to each service node in the target list to the corresponding service node.

[0027] Optionally, the delivery task generation submodule may further include:

[0028] The first detection result acquisition submodule is used to acquire the first detection result, which is used to characterize whether the first delivery task for the target configuration information has been generated.

[0029] The second detection result acquisition submodule is used to acquire the second detection result if the first detection result is yes and the first delivery task for the target configuration information has been generated. The second detection result is used to characterize whether the target configuration information is different from the stored configuration information.

[0030] The target configuration information update submodule is used to update the stored configuration information to the target configuration information if the second detection result is yes and the target configuration information is different from the stored configuration information.

[0031] Optionally, the first detection result acquisition submodule can also be specifically used for:

[0032] If the first detection result is negative, then the first delivery task is generated;

[0033] The first delivery task is sent to a target queue that contains at least one delivery task.

[0034] Optionally, the delivery task scheduling submodule can also be specifically used for:

[0035] Obtain a third detection result, which is used to characterize the running status of the data delivery task;

[0036] If the data delivery task is in the state of pending execution, the data delivery task is sent to the first thread, which is the thread used to execute the data delivery task.

[0037] While executing the data delivery task, obtain the third detection result corresponding to the next data delivery task.

[0038] Optionally, the delivery task execution submodule can also be specifically used for:

[0039] Obtain the fourth detection result, which is used to characterize the version of the data to be synchronized corresponding to the data delivery task;

[0040] If the fourth detection result indicates that the version of the data to be synchronized is higher than the corresponding data stored on the server, then the data to be synchronized is sent to the server; otherwise, the operation of sending the data to be synchronized is skipped.

[0041] After the data to be synchronized is sent to the server, a fifth detection result is obtained. The fifth detection result is used to characterize the integrity of the data to be synchronized.

[0042] If the fifth detection result indicates that the data to be synchronized is complete, then the data to be synchronized is determined to be synchronized to the server; otherwise, the scheduled data delivery task is re-executed.

[0043] Optionally, the configuration information acquisition module can also be specifically used for:

[0044] Obtain third-party configuration information;

[0045] Check if the third configuration information already exists;

[0046] If the third configuration information is not detected, then the third configuration information is received;

[0047] If the existence of the third configuration information is detected, it is further detected whether the third configuration information is different from the already stored configuration information;

[0048] If the third configuration information changes, the third configuration information is updated.

[0049] Optionally, the data synchronization module may also be specifically used for:

[0050] Acquire a data synchronization acceleration node, which is registered by the first service node of the server;

[0051] The data to be synchronized is sent to the second service node of the server through the data synchronization acceleration node.

[0052] A third aspect of this application also provides an electronic device, including a memory storing a plurality of instructions; the processor loads the instructions from the memory to execute steps in any of the data synchronization methods provided in this application.

[0053] A fourth aspect of this application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the data synchronization methods provided in this application.

[0054] A fifth aspect of this application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in any of the data synchronization methods provided in this application.

[0055] Therefore, the embodiments of this application can perform fast and effective data synchronization of data of different data types, and can also manage the data to be synchronized during the data synchronization process. This effectively solves the problem that data to be synchronized cannot be uniformly synchronized and managed due to different data types. Furthermore, this application can synchronize the data to be synchronized to the server through a fully managed approach, which avoids the waiting time for data synchronization operations when the server restarts, and also improves the effectiveness and efficiency of synchronizing data of different data types. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of a scenario illustrating the data synchronization method provided in an embodiment of this application;

[0058] Figure 2 This is a flowchart illustrating the data synchronization method provided in an embodiment of this application;

[0059] Figure 3 This is a flowchart illustrating another data synchronization method provided in an embodiment of this application;

[0060] Figure 4 This is a schematic diagram illustrating the configuration platform obtaining configuration information provided in this application embodiment;

[0061] Figure 5 This is a schematic diagram illustrating the synchronous update of configuration information provided in an embodiment of this application;

[0062] Figure 6 This is a schematic diagram of the process for synchronously updating configuration information provided in an embodiment of this application;

[0063] Figure 7 This is a schematic diagram of the data delivery task management provided in an embodiment of this application;

[0064] Figure 8 This is a schematic diagram of the update data delivery task provided in an embodiment of this application;

[0065] Figure 9This is a schematic diagram of a scheduling data delivery task provided in an embodiment of this application;

[0066] Figure 10 This is a flowchart illustrating the scheduling data delivery task provided in an embodiment of this application;

[0067] Figure 11 This is a schematic diagram illustrating the synchronization of global type data to be synchronized, provided in an embodiment of this application.

[0068] Figure 12 This is a schematic diagram illustrating the data synchronization of distributed data to be synchronized, provided in an embodiment of this application.

[0069] Figure 13 This is a schematic diagram illustrating the data synchronization acceleration node performing data synchronization as provided in this application embodiment;

[0070] Figure 14 This is a schematic diagram of the structure of the data synchronization device provided in the embodiments of this application;

[0071] Figure 15 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] Artificial intelligence (AI) is a technology that uses digital computers to simulate human perception of the environment, acquisition of knowledge, and use of that knowledge. This technology can enable machines to possess functions similar to human perception, reasoning, and decision-making. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0074] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, and intelligent transportation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0075] This application provides a data synchronization method, apparatus, electronic device, and storage medium.

[0076] Specifically, the data synchronization device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, desktop computer, smart TV, in-vehicle device, etc.; the server can be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0077] In some embodiments, the data synchronization device may also be integrated into multiple electronic devices, such as multiple servers, with multiple servers implementing the data synchronization method of this application.

[0078] For example, refer to Figure 1 The electronic device can be a server / terminal that applies the embodiments of this application, or it can be other terminals that can execute the embodiments of this application. Data on the storage end can be synchronized to the server or to the terminal, and the server and the terminal can also interact with each other.

[0079] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0080] like Figure 1 As shown, the storage end can be a remote file storage system storing the data to be synchronized. The server and terminal can run the target storage system or target database for synchronizing the data. The storage end can be integrated into the server / terminal or set up separately. When synchronizing data through electronic devices, the storage end can receive data synchronization requests from the server or terminal, which request the data to be synchronized. Based on the data synchronization request, it obtains the configuration information of the data to be synchronized, including the data type. Finally, it performs the data synchronization operation according to the synchronization process corresponding to the configuration information. The synchronization process is used to synchronize data of different data types to the server and manage the data of different data types to be synchronized.

[0081] Furthermore, in the embodiments of this application, "multiple" refers to two or more. The terms "first" and "second," etc., in the embodiments of this application are used for distinguishing descriptions and should not be construed as implying relative importance.

[0082] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0083] This data synchronization method can be executed by the terminal or the server alone, or by both the terminal and the server. In the embodiments of this application, the method is described by example when it is executed by the terminal.

[0084] In this embodiment, a data synchronization method is provided, such as... Figure 2 As shown, the specific process of this data synchronization method can be as follows:

[0085] 201. Receive data synchronization requests from the server.

[0086] Data synchronization requests can be used to request data to be synchronized. In some scenarios, updates to data on a local server or database rely on external storage systems or databases to synchronize the updated data to the server or database. For example... Figure 1 In this scenario, the server / terminal can synchronize data from a remote file storage system to the server to update the data on the server.

[0087] In some scenarios, servers or terminals can also utilize tools, programs, or software with data synchronization capabilities to execute the data synchronization method of this application. For ease of understanding, let's take the data synchronization scenario of a video recommendation system server as an example. Video recommendation systems typically use user viewing data to specify video recommendation strategies and ultimately recommend videos to users based on these strategies. Therefore, in this scenario, the data to be synchronized is viewing data. This viewing data can include information such as the video name, video category, video tags, viewing duration, and viewing time period. It is understood that this viewing data is constantly changing and time-sensitive; the video recommendation system should acquire the latest or most recent viewing data to enable more accurate and efficient video recommendations.

[0088] To obtain the latest or more recent viewing data, the video recommendation system can send a data synchronization request to the remote file storage system storing this data, and can also receive this data synchronization request synchronously. This allows the remote file storage system, upon receiving the data synchronization request, to send the requested data to be synchronized to the video recommendation system, thereby updating older versions and older-stored viewing data in the video recommendation system.

[0089] Step 202: Obtain the configuration information of the data to be synchronized according to the data synchronization request.

[0090] The configuration information may include the data type of the data to be synchronized. Optionally, the data type of the data to be synchronized may include global data of global type and distributed data of distributed type. That is, global data of global type is global data, and distributed data of distributed type is distributed data. Global data refers to data that all service nodes in the server need to share and update globally. Global data can be viewed as a whole, and each service node on the server needs to load this global data during application. Optionally, global data may include local data stored locally.

[0091] Distributed data refers to data stored and manipulated using a distributed infrastructure and computing methods. Distributed data can contain multiple data shards, which can be distributed across various service nodes of the server.

[0092] Please see Figure 4 , Figure 4 This is a schematic diagram of the configuration platform obtaining configuration information provided in the embodiments of this application.

[0093] The electronic device in this application embodiment may be equipped with, for example, Figure 4 The configuration shown allows the electronic device to be a terminal, server, or similar device, such as a personal computer or cloud server. It may include a configuration platform for performing data configuration, which can obtain basic information about the data to be synchronized. The storage path refers to the location path of the data to be synchronized in the remote storage system. The data format refers to whether the data to be synchronized is in directory or file format. Taking user viewing data as an example, viewing data can be stored in directory format, file format, or folder format.

[0094] The file identifier can be the identification information of the file corresponding to the data to be synchronized, such as the file's prefix, suffix, or other identifiers. Other identifiers can be functional identifiers; taking the video recommendation scenario as an example, other identifiers could be invalid file identifiers. By obtaining this invalid file identifier during the data configuration phase, data filtering can be performed during data synchronization, such as filtering out data with this invalid file identifier to retain valid data to be synchronized.

[0095] like Figure 4As shown, the configuration platform stores a data configuration list, which includes stored data configuration information. The number of data configuration information entries depends on the data type to be synchronized. As mentioned earlier, the data type to be synchronized can include global data and distributed data; therefore, the data configuration list can include both first and second configuration information. That is, during the process of obtaining the basic information of the data to be synchronized, the basic information of the obtained global data can be classified as the first configuration information in the data configuration list, and the basic information of the obtained distributed data can be classified as the second configuration information in the data configuration list.

[0096] It should be noted that the number and classification criteria of the data configuration information included in the data configuration list can be set according to requirements. For example, the number of configuration information is not limited to two, nor is it limited to including only the first and second configuration information. The classification criteria of the configuration information are not limited to the data type of the data to be synchronized. This application embodiment does not impose any restrictions.

[0097] Step 203: Perform data synchronization operation according to the synchronization process corresponding to the configuration information.

[0098] The synchronization process can be used to synchronize data of different data types to the server and manage the data of different data types to be synchronized. It can be understood that in the data configuration list, since the first configuration information corresponds to global data in the data to be synchronized and the second configuration information corresponds to distributed data in the data to be synchronized, this application can achieve unified data synchronization and data management for global data and distributed data. Optionally, step 203 may also include the following steps:

[0099] A data delivery task is generated, which is used to send the configuration information to the server;

[0100] Schedule the data delivery task;

[0101] Execute the scheduled data delivery task.

[0102] Optionally, the step "Generate Data Delivery Task" may include:

[0103] When the data type of the data to be synchronized is global, the data to be synchronized is stored in the first space to generate a delivery task for the data to be synchronized; when the data type of the data to be synchronized is distributed, the data to be synchronized is stored in the second space to generate a delivery task for the data to be synchronized.

[0104] Optionally, the step "executing the scheduled data delivery task" may include:

[0105] A global synchronization process is performed on the data to be synchronized in the first space to execute the delivery task of the data to be synchronized; a distributed synchronization process is performed on the data to be synchronized in the second space to execute the delivery task of the data to be synchronized.

[0106] According to the description in step 202, the data to be synchronized may include global data and distributed data, i.e., global data and distributed data. Figure 4 As shown, the configuration platform can be configured with multiple spaces for storing configuration information, such as the first space, the second space, ..., the Nth space, etc. Each space can store different data types of data to be synchronized, for example... Figure 4 As shown, the first space can be used to store the first configuration information corresponding to the global type of data to be synchronized, and the second space can store the second configuration information corresponding to the distributed type of data to be synchronized.

[0107] Therefore, the embodiments of this application use a global synchronization process to handle data synchronization operations of global data to be synchronized and a distributed synchronization process to handle data synchronization operations of distributed data to be synchronized. This can be compatible with synchronizing and managing global data and distributed data, effectively solving the problem that data to be synchronized cannot be uniformly synchronized and managed due to different data types.

[0108] Please see Figure 7 , Figure 7 This is a schematic diagram of a data delivery task provided in an embodiment of this application. It can be understood that after obtaining the configuration information of the data to be synchronized, the specific data to be synchronized can be quickly and accurately determined from the remote file storage system based on the basic information contained in the configuration information. Therefore, a data delivery task for delivering the data to be synchronized can be generated, so that the data to be synchronized can be subsequently synchronized to the server according to the data delivery task.

[0109] like Figure 7 As shown, to facilitate unified management of the data to be synchronized, the server and terminal of this application can maintain a data delivery task management module. This module internally maintains a global data delivery task queue. The data configuration management module can obtain the configuration information of the data to be synchronized from the configuration platform, and then send a request message to the data delivery task management module to generate a data delivery task. Each time the data delivery task management module receives a request message, it will generate a corresponding and unique data delivery task for the data to be synchronized contained in the request message, and then send the data delivery task to the data delivery task queue to complete the subsequent delivery.

[0110] Please see Figure 9The figure is a schematic diagram of a scheduling data delivery task provided in an embodiment of this application. Figure 9 As shown, the data delivery task management module is responsible for scheduling all data delivery tasks. Internally, this module maintains a global, background-running task execution thread pool, which can include multiple first threads, i.e., the task execution threads. In addition, it also maintains a global, background-running task scheduling thread. Specifically, after a data configuration task is generated, the task scheduling thread scans the data delivery tasks in the target queue and distributes qualified data delivery tasks to the first threads in the task execution thread pool, thereby synchronizing the data to be synchronized from the remote file storage system to the server.

[0111] Optionally, the step "execute the scheduled data delivery task" may include:

[0112] The global data is sent to the server in a first manner, wherein the first manner includes sending the global data under the target list to each service node of the server, and the target list is a list of data files where the data to be synchronized is located;

[0113] The distributed data is sent to the server in a second manner, wherein the second manner includes sending the distributed data corresponding to each service node under the target list to the corresponding service node.

[0114] Please see Figure 11 and Figure 12 , Figure 11 This is a schematic diagram illustrating global data synchronization provided in an embodiment of this application. Figure 12 This is a schematic diagram of data synchronization for distributed data provided in an embodiment of this application.

[0115] Specifically, the data delivery task management module of the configuration platform maintains a global task execution thread pool. Whenever the task execution thread pool receives a data delivery task, the first idle thread in the task execution thread pool will actually execute the data delivery task. During execution, different data file delivery processes can be executed according to the data type of the data file delivery task (e.g., global data or distributed data).

[0116] Optionally, the step "execute the scheduled data delivery task" may include:

[0117] Obtain the fourth detection result, which is used to characterize the version of the data to be synchronized corresponding to the data delivery task;

[0118] If the fourth detection result indicates that the version of the data to be synchronized is higher than the corresponding data stored on the server, then the data to be synchronized is sent to the server; otherwise, the operation of sending the data to be synchronized is skipped.

[0119] After the data to be synchronized is sent to the server, a fifth detection result is obtained. The fifth detection result is used to characterize the integrity of the data to be synchronized.

[0120] If the fifth detection result indicates that the data to be synchronized is complete, then the data to be synchronized is determined to be synchronized to the server; otherwise, the scheduled data delivery task is re-executed.

[0121] like Figure 11 The diagram illustrates the global data file distribution process. It retrieves a list of data file directories under the corresponding storage path of a remote file storage system, parses and obtains a list of global data file versions, sorts them according to version time, and then checks each version in ascending order, obtaining a fourth check result. If the fourth check result indicates that the version directory of the currently checked data to be synchronized has been completely generated and is newer than the version already saved on the server, then all data in the version directory corresponding to the currently checked data to be synchronized will be downloaded to the server. Otherwise, the current version of the data will be skipped, and the process will continue to check the version of the next data to be synchronized in sequence.

[0122] Once all data files in the corresponding version directory of the data to be synchronized have been downloaded to the server, the configuration platform will check the integrity of the data file versions and obtain the fifth check result. If the fifth check result passes, indicating that the data to be synchronized is complete, an official version of the data to be synchronized will be generated on the server, the data version information maintained internally by the configuration platform will be updated, and the data will be output to the local storage node of the server. Otherwise, the download of the current version of the data to be synchronized is considered to have failed, and the configuration platform can directly terminate the download process, clean up the directory and data files corresponding to the current data version on the server, and then return the current task execution thread to the task execution thread pool.

[0123] like Figure 12 The diagram illustrates the distributed data delivery process. The configuration platform can obtain the topology of the distributed data from an external device, such as an intelligent dynamic distributed routing management device. When executing a distributed data delivery task, the logical list of distributed data fragments that the current server node needs to download can be determined, and then the data delivery task for that distributed data is executed according to the same data delivery process as for global data.

[0124] It should be noted that the difference between global data and distributed data in the execution of data delivery tasks is that when configuring the platform to synchronize global data, it needs to download all global data files in the corresponding data version directory; while when configuring the platform to synchronize distributed data, it needs to download the distributed data files corresponding to the logical shard list in the corresponding data version directory.

[0125] Optionally, the step "Generate Data Delivery Task" may include:

[0126] Obtain a first detection result, which is used to characterize whether a first delivery task for the target configuration information has been generated;

[0127] If the first detection result is yes, the first delivery task for the target configuration information has been generated, then the second detection result is obtained. The second detection result is used to characterize whether the target configuration information is different from the stored configuration information.

[0128] If the second detection result is yes, the target configuration information is different from the stored configuration information, then the stored configuration information is updated to the target configuration information.

[0129] Optionally, the step "Obtain the first detection result" may also include:

[0130] If the first detection result is negative, then the first delivery task is generated;

[0131] The first delivery task is sent to a target queue that contains at least one delivery task.

[0132] Please see Figure 8 , Figure 8 This is a schematic diagram of an update data delivery task provided in an embodiment of this application. Optionally, an embodiment of this application may further include the following steps:

[0133] Step 501: Synchronize messages in the data configuration management module;

[0134] Step 502: Obtain configuration information;

[0135] Step 503: Check if the delivery task in the configuration information exists; if yes, proceed to step 504; otherwise, proceed to step 505.

[0136] Step 504: Are the configuration information different? If yes, proceed to step 507; otherwise, proceed to step 508.

[0137] Step 505: Create the associated data delivery task;

[0138] Step 506: Send the data delivery task to the target queue;

[0139] Step 507: Update the configuration information for this data delivery task;

[0140] Step 508: End operation.

[0141] like Figure 8 As shown, the data configuration management module of the configuration platform can generate a delivery request message for the delivery target configuration information and send the delivery request information to the data delivery task management module. After receiving the delivery request message, the data delivery task management module can first obtain the associated target configuration information from the delivery request message, and then obtain the first detection result. The first detection result is used to characterize whether the first delivery task for the target configuration information has been generated.

[0142] If the first detection result is negative, it means that the first delivery task of the target configuration information has not been generated. In this case, the configuration platform can create a data delivery task for the target configuration information and put it into the target queue. If the first detection result is positive, it means that the first delivery task of the target configuration information has been generated.

[0143] Furthermore, if the first detection result is yes, a second detection result can be obtained. The second detection result is used to characterize whether the target configuration information is different from the stored configuration information. If the second detection result is yes, it means that the target configuration information is different from the configuration information stored in the configuration platform. In this case, the stored configuration information can be updated to the target configuration information. For example, the target configuration information can overwrite the older version of the stored configuration information to update the older version of the stored data configuration information. Correspondingly, if the second detection result is no, the above update process is not performed.

[0144] It is understood that this application can verify whether a delivery task for the delivery target configuration information has been generated by obtaining the first detection result, thereby ensuring that the object of the delivery task is the target configuration information. Furthermore, after determining that a delivery task for the target configuration information has been generated, it can further detect whether the configuration information stored on the configuration platform is the target configuration information, thereby ensuring that the data to be delivered is configured with the latest version of the target configuration information. This significantly improves the accuracy of data synchronization through the above methods.

[0145] Optionally, the step of "scheduling the data delivery task" may include:

[0146] Obtain a third detection result, which is used to characterize the running status of the data delivery task;

[0147] If the data delivery task is in the state of pending execution, the data delivery task is sent to the first thread, which is the thread in the configuration platform used to execute the data delivery task;

[0148] While executing the data delivery task, obtain the third detection result corresponding to the next data delivery task.

[0149] Please see Figure 10 , Figure 10 This is a schematic flowchart of a data delivery task scheduling method provided in an embodiment of this application. Optionally, an embodiment of this application may include the following steps:

[0150] Step 601: The task scheduling thread scans the target queue;

[0151] Step 602: Is the data delivery task status "pending"?

[0152] If not, proceed to step 603 and skip the current task to scan the next target queue;

[0153] If so, proceed to step 604: send the data delivery task to the task execution thread pool;

[0154] Step 605: The task execution thread pool executes the data delivery task;

[0155] Step 606: Reset the data delivery task completion status;

[0156] Step 607: End operation.

[0157] like Figure 10 As shown, in practical applications, the task scheduling thread periodically scans the data delivery task queue. Each time a data delivery task is scanned, its status is determined. For example, the status can be either running or pending. If the data delivery task is running, the current scan is skipped, and the next data delivery task in the queue is scanned directly. If the data delivery task is pending, the data delivery task is sent to the task execution thread pool for processing.

[0158] As can be seen from the above, the embodiments of this application can quickly and effectively synchronize data of different data types through a configuration platform. Furthermore, it can manage the data to be synchronized during the synchronization process, thus effectively solving the problem that data of different data types cannot be uniformly synchronized and managed. Moreover, the configuration platform of this application can synchronize the data to be synchronized to the server in a fully managed manner, avoiding the waiting time for data synchronization operations during server restarts and improving the effectiveness and efficiency of synchronizing data of different data types.

[0159] Optionally, step 202 may also include the following steps:

[0160] Obtain third-party configuration information;

[0161] Check if the third configuration information already exists;

[0162] If the third configuration information is not detected, then the third configuration information is received;

[0163] If the existence of the third configuration information is detected, it is further detected whether the third configuration information is different from the already stored configuration information;

[0164] If the third configuration information changes, the third configuration information is updated.

[0165] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the process of synchronously updating configuration information provided in an embodiment of this application. Optionally, an embodiment of this application may include the following steps:

[0166] Step 401: Obtain configuration information;

[0167] Step 402: Does the configuration information exist? If yes, proceed to step 404; otherwise, proceed to step 403.

[0168] Step 403: Insert configuration information;

[0169] Step 404: Are the configuration information different?

[0170] Step 405: Update configuration information;

[0171] Step 406: End operation.

[0172] like Figure 6 As shown, when the data configuration management module obtains data configuration information from the configuration platform, it first determines whether the data configuration information already exists in the configuration list of the configuration platform. For example, if the newly obtained data configuration information is the third configuration information, if the third configuration information does not exist, it is considered that the third configuration information is newly added, and the newly added third configuration information will be inserted into the data configuration list stored in the configuration platform.

[0173] Accordingly, if the third configuration information exists, the basic information in the newly acquired third configuration information will be compared with the basic information corresponding to the data configuration information already saved in the configuration platform. If the basic information in the data configuration information has changed, the basic content in the newly acquired data configuration information will be used to overwrite the basic content of the data configuration information already saved in the configuration platform. If the basic information in the data configuration information has not changed, no processing will be performed.

[0174] Optionally, step 203 may also include the following steps:

[0175] Acquire a data synchronization acceleration node, which is registered by the first service node of the server;

[0176] The data to be synchronized is sent to the second service node of the server through the data synchronization acceleration node.

[0177] Please see Figure 13 , Figure 13 This is a schematic diagram illustrating the data synchronization acceleration node performing data synchronization according to an embodiment of this application. Optionally, in the process of distributing the data to be synchronized to various service nodes of the server, for use scenarios involving servers with multiple service nodes (i.e., service clusters), this embodiment of the application introduces P2P distribution to improve the distribution efficiency of the data to be synchronized, thereby reducing the bandwidth pressure on the remote file storage system and accelerating the speed of data file distribution.

[0178] like Figure 13 As shown, this application introduces a central coordination device outside the configuration platform. This central coordination device can be responsible for authorization management and coordination control of a cluster composed of multiple configuration platforms, such as... Figure 13 As shown, after each configuration platform node synchronously downloads the data to be synchronized to the local server, it registers its corresponding first service node with the central coordination device. This first service node then becomes a data synchronization acceleration node for the corresponding data. When a configuration platform on a second service node begins downloading a data file, it communicates with the central coordination device to obtain the acceleration node information corresponding to the data to be synchronized. It prioritizes downloading the data file from the acceleration node, thus fully utilizing the network bandwidth and low network transmission latency between multiple nodes in the service cluster to improve the efficiency of data file delivery. Conversely, if there is no corresponding data acceleration node in the service cluster, the data file is then downloaded from the remote file storage system.

[0179] In addition, the configuration platform can manage the versions of multiple data files downloaded to the local server and output the version management information to the local server for access by external devices or servers.

[0180] Optionally, the configuration platform can also periodically clean up old version data in the data to be synchronized, retaining only a specified number of version data (e.g., controlled by data configuration). It also supports version rollback, allowing the data to be synchronized on the local server to be rolled back to the specified version data through data configuration.

[0181] As can be seen from the above, the embodiments of this application can register service nodes as data synchronization acceleration nodes, and other service nodes can obtain the data to be synchronized from the data synchronization acceleration nodes. Since the transmission between each service node is faster and has less latency than the transmission between the service node and the remote file storage system, the method of the embodiments of this application can further improve the efficiency of data synchronization.

[0182] The method described in the above embodiments will be further described in detail below.

[0183] In this embodiment, data synchronization will be used as an example to describe the method of this application embodiment in detail.

[0184] like Figure 3 As shown, the specific process of a data synchronization method is as follows:

[0185] Step 301: Receive the server's data synchronization request.

[0186] While the server sends a data synchronization request to the remote file storage system, the configuration platform can also simultaneously obtain the data synchronization request, so as to send the data to be synchronized from the remote storage system to the server in the future.

[0187] Step 302: Obtain the configuration information of the data to be synchronized according to the data synchronization request.

[0188] The configuration information may include basic information about the data to be synchronized, such as the data type, storage path, data format, and file identifier.

[0189] Step 303: Periodically synchronize and update the configuration information.

[0190] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the synchronous update of configuration information provided in the embodiments of this application.

[0191] like Figure 5 As shown, it can be understood that in order to strengthen the management of the data to be synchronized, the configuration information of the data to be synchronized can be updated periodically. For example, the frequency of periodic updates can reach the millisecond level, so as to achieve real-time update of the configuration information to the greatest extent.

[0192] Step 304: Generate data delivery task.

[0193] Specifically, after obtaining the data configuration information, the configuration platform can generate a data delivery task based on the data configuration information to deliver the data to be synchronized in the remote file storage system to the target server.

[0194] Step 305: Check if the data delivery task exists and if the configuration information of the data delivery task has changed, and perform the corresponding operations.

[0195] It is understandable that during data synchronization, if the configuration platform detects that a data delivery task already exists, it will continue to check for the next data delivery task. Conversely, if it detects that a data delivery task does not exist, it can add the data delivery task to the target queue.

[0196] If the configuration platform detects a change in the data configuration information of the data delivery task, it will update the data configuration information in the data delivery task to the latest version. Conversely, if the configuration platform detects that the data configuration information of the data delivery task has not changed, it will end the process and continue executing the data delivery task.

[0197] Step 306: Initiate data delivery tasks.

[0198] Specifically, after a data delivery task is generated, it can be scheduled to the first thread that executes the data delivery task.

[0199] Step 307: Check whether the data delivery task already exists in the target queue and perform the corresponding operation.

[0200] Specifically, if the configuration platform detects that a data delivery task already exists in the target queue, it skips the current detection operation and continues to detect the next data delivery task. Conversely, if the configuration platform detects that a data delivery task does not exist in the target queue, it creates the corresponding data delivery task and adds it to the target queue.

[0201] Step 308: Through the configuration platform, synchronize the data to be synchronized from the storage end to the first service node of the server.

[0202] Step 309: Using the first service node as the data synchronization acceleration node, synchronize the data to other second service nodes of the server.

[0203] Specifically, during data synchronization, the configuration platform can first send the data to be synchronized to one of the server's service nodes, such as the first service node. If the first service node obtains the data to be synchronized before other service nodes (such as the second service node), then the first service node can be registered as a data synchronization acceleration node. This allows other second service nodes to obtain the data to be synchronized directly from the data synchronization acceleration node without having to obtain it from the remote storage system, thereby reducing latency during the data synchronization process and improving the efficiency of data synchronization.

[0204] Therefore, the embodiments of this application can quickly and effectively synchronize data of different data types through a configuration platform. Furthermore, the platform can manage the data to be synchronized during the synchronization process, effectively solving the problem that data of different data types cannot be uniformly synchronized and managed. Moreover, the configuration platform of this application can synchronize the data to be synchronized to the server in a fully managed manner, avoiding the waiting time during server restarts and improving the effectiveness and efficiency of synchronizing data of different data types.

[0205] To better implement the above methods, this application also provides a data transmission device that can be integrated into a terminal or server.

[0206] For example, such as Figure 14 The diagram shown is a structural schematic of a data transmission device provided in an embodiment of this application. This data transmission device can be applied to a configuration platform. The device may include a data synchronization request receiving module 701, a configuration information acquisition module 702, and a data synchronization module 703, as detailed below:

[0207] The data synchronization request receiving module 701 is used to receive a data synchronization request from the server, wherein the data synchronization request is used to request data to be synchronized.

[0208] The configuration information acquisition module 702 is used to acquire the configuration information of the data to be synchronized according to the data synchronization request, wherein the configuration information includes the data type of the data to be synchronized;

[0209] The data synchronization module 703 is used to perform data synchronization operations according to the synchronization process corresponding to the configuration information. The synchronization process is used to synchronize the data to be synchronized of different data types to the server and to manage the data to be synchronized of different data types.

[0210] Optionally, the data synchronization module 703 may further include:

[0211] The delivery task generation submodule is used to generate data delivery tasks, which are used to send the configuration information to the server.

[0212] The delivery task scheduling submodule is used to schedule the data delivery tasks;

[0213] The delivery task execution submodule is used to execute the scheduled data delivery task.

[0214] Optionally, the delivery task generation submodule is further specifically used for:

[0215] When the data type of the data to be synchronized is a global type, the data to be synchronized is stored in the first space to generate a delivery task for the data to be synchronized.

[0216] The delivery task execution submodule is also specifically used for:

[0217] A global synchronization process is performed on the data to be synchronized in the first space to execute the delivery task of the data to be synchronized.

[0218] Optionally, the data type includes a distributed type, and the delivery task generation submodule is further specifically used for:

[0219] When the data type of the data to be synchronized is distributed, the data to be synchronized is stored in the second space to generate the delivery task of the data to be synchronized;

[0220] The delivery task execution submodule is also specifically used for:

[0221] A distributed synchronization process is performed on the data to be synchronized in the second space to execute the delivery task of the data to be synchronized.

[0222] Optionally, the basic information classification submodule may further include:

[0223] The delivery task generation submodule is used to generate data delivery tasks, which are used to send the configuration information to the server.

[0224] The delivery task scheduling submodule is used to schedule the data delivery tasks;

[0225] The delivery task execution submodule is used to execute the scheduled data delivery task.

[0226] Optionally, the delivery task scheduling submodule can also be specifically used for:

[0227] The global data is sent to the server in a first manner, wherein the first manner includes sending the global data under the target list to each service node of the server, and the target list is a list of data files where the data to be synchronized is located;

[0228] The distributed data is sent to the server in a second manner, wherein the second manner includes sending the distributed data corresponding to each service node under the target list to the corresponding service node.

[0229] Optionally, the delivery task generation submodule may further include:

[0230] The first detection result acquisition submodule is used to acquire the first detection result, which is used to characterize whether the first delivery task for the target configuration information has been generated.

[0231] The second detection result acquisition submodule is used to acquire the second detection result if the first detection result is yes and the first delivery task for the target configuration information has been generated. The second detection result is used to characterize whether the target configuration information is different from the stored configuration information.

[0232] The target configuration information update submodule is used to update the stored configuration information to the target configuration information if the second detection result is yes and the target configuration information is different from the stored configuration information.

[0233] Optionally, the first detection result acquisition submodule can also be specifically used for:

[0234] If the first detection result is negative, then the first delivery task is generated;

[0235] The first delivery task is sent to a target queue that contains at least one delivery task.

[0236] Optionally, the delivery task scheduling submodule can also be specifically used for:

[0237] Obtain a third detection result, which is used to characterize the running status of the data delivery task;

[0238] If the data delivery task is in the state of pending execution, the data delivery task is sent to the first thread, which is the thread in the configuration platform used to execute the data delivery task;

[0239] While executing the data delivery task, obtain the third detection result corresponding to the next data delivery task.

[0240] Optionally, the delivery task execution submodule can also be specifically used for:

[0241] Obtain the fourth detection result, which is used to characterize the version of the data to be synchronized corresponding to the data delivery task;

[0242] If the fourth detection result indicates that the version of the data to be synchronized is higher than the corresponding data stored on the server, then the data to be synchronized is sent to the server; otherwise, the operation of sending the data to be synchronized is skipped.

[0243] After the data to be synchronized is sent to the server, a fifth detection result is obtained. The fifth detection result is used to characterize the integrity of the data to be synchronized.

[0244] If the fifth detection result indicates that the data to be synchronized is complete, then the data to be synchronized is determined to be synchronized to the server; otherwise, the scheduled data delivery task is re-executed.

[0245] Optionally, the configuration information acquisition module 702 can also be specifically used for:

[0246] Obtain third-party configuration information;

[0247] Check if the third configuration information already exists;

[0248] If the third configuration information is not detected, then the third configuration information is received;

[0249] If the existence of the third configuration information is detected, it is further detected whether the third configuration information is different from the already stored configuration information;

[0250] If the third configuration information changes, the third configuration information is updated.

[0251] Optionally, the data synchronization module 703 may also be specifically used for:

[0252] Acquire a data synchronization acceleration node, which is registered by the first service node of the server;

[0253] The data to be synchronized is sent to the second service node of the server through the data synchronization acceleration node.

[0254] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0255] As described above, the data transmission device provided in this application embodiment receives a data synchronization request from a server through a data synchronization request receiving module 701. The data synchronization request is used to request data to be synchronized. The configuration information acquisition module 702 acquires the configuration information of the data to be synchronized according to the data synchronization request. The configuration information includes the data type of the data to be synchronized. The data synchronization module 703 is used to perform data synchronization operations according to the synchronization process corresponding to the configuration information. The synchronization process is used to synchronize data of different data types to the server and manage the data of different data types to be synchronized.

[0256] Therefore, the embodiments of this application can quickly and effectively synchronize data of different data types, and can also manage the data to be synchronized during the data synchronization process. This effectively solves the problem that data to be synchronized cannot be uniformly synchronized and managed due to different data types. Furthermore, this application can synchronize the data to be synchronized to the server in a fully managed manner, which can avoid the waiting time of data synchronization operation when the server restarts, and also improve the effect and efficiency of synchronizing data of different data types.

[0257] This application also provides an electronic device, which can be a terminal, server, or other similar device. Figure 15 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0258] The electronic device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, an input module 804, and a communication module 805. Those skilled in the art will understand that... Figure 15 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0259] The processor 801 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802. In some embodiments, the processor 801 may include one or more processing cores; in some embodiments, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 801.

[0260] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0261] The electronic device also includes a power supply 803 that supplies power to the various components. In some embodiments, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0262] The electronic device may also include an input module 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0263] The electronic device may also include a communication module 805. In some embodiments, the communication module 805 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 805 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0264] Although not shown, the electronic device may also include a display module, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 runs the application programs stored in the memory 802 to realize various functions, as follows:

[0265] Receive a data synchronization request from the server, the data synchronization request being used to request data to be synchronized;

[0266] According to the data synchronization request, obtain the configuration information of the data to be synchronized, the configuration information including the data type of the data to be synchronized;

[0267] Data synchronization is performed according to the synchronization process corresponding to the configuration information. The synchronization process is used to synchronize the data to be synchronized of different data types to the server and to manage the data to be synchronized of different data types.

[0268] In some embodiments, a computer program product is also provided, comprising a computer program or instructions that, when executed by a processor, implement the steps in any of the above-described data synchronization methods.

[0269] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0270] As can be seen from the above, the embodiments of this application can quickly and effectively synchronize data of different data types, and can also manage the data to be synchronized during the data synchronization process. This effectively solves the problem that data to be synchronized cannot be uniformly synchronized and managed due to different data types. Furthermore, this application can synchronize the data to be synchronized to the server through a fully managed approach, which avoids the waiting time for data synchronization operations when the server restarts, and also improves the effectiveness and efficiency of synchronizing data of different data types.

[0271] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0272] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the data synchronization methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0273] Receive a data synchronization request from the server, the data synchronization request being used to request data to be synchronized;

[0274] According to the data synchronization request, obtain the configuration information of the data to be synchronized, the configuration information including the data type of the data to be synchronized;

[0275] Data synchronization is performed according to the synchronization process corresponding to the configuration information. The synchronization process is used to synchronize the data to be synchronized of different data types to the server and to manage the data to be synchronized of different data types.

[0276] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0277] According to one aspect of this application, a computer program product is provided, which includes a computer program or instructions that, when executed by a processor, implement the steps in the data synchronization method described above.

[0278] Therefore, the embodiments of this application can perform fast and effective data synchronization of data of different data types, and can also manage the data to be synchronized during the data synchronization process. This effectively solves the problem that data to be synchronized cannot be uniformly synchronized and managed due to different data types. Furthermore, this application can synchronize the data to be synchronized to the server through a fully managed approach, which avoids the waiting time for data synchronization operations when the server restarts, and also improves the effectiveness and efficiency of synchronizing data of different data types.

[0279] Since the instructions stored in the storage medium can execute the steps of any of the data synchronization methods provided in the embodiments of this application, the beneficial effects that any of the data synchronization methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0280] The foregoing has provided a detailed description of a data synchronization method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data synchronization method, characterized in that, The method includes: Receive a data synchronization request from the server, the data synchronization request being used to request the data to be synchronized; According to the data synchronization request, the configuration information of the data to be synchronized is obtained. The configuration information includes the data type of the data to be synchronized. The data type includes global type and distributed type. The global type of data to be synchronized refers to data that needs to be shared and updated globally by all service nodes in the server. The distributed type of data to be synchronized refers to data that is stored and operated using a distributed infrastructure and computing method. The data synchronization operation is performed according to the synchronization process corresponding to the configuration information, including: when the data type of the data to be synchronized is global, storing the data to be synchronized in the first space to generate a delivery task for the data to be synchronized; performing global synchronization process processing on the data to be synchronized in the first space to execute the delivery task for the data to be synchronized; when the data type of the data to be synchronized is distributed, storing the data to be synchronized in the second space to generate a delivery task for the data to be synchronized; performing distributed synchronization process processing on the data to be synchronized in the second space to execute the delivery task for the data to be synchronized; the synchronization process is used to synchronize the data to be synchronized of different data types to each service node of the server or the service node corresponding to the distributed data to be synchronized, and to manage the data to be synchronized of different data types; wherein, when executing the delivery task, when synchronizing global data, it is necessary to download all global data files in the corresponding data version directory; when synchronizing distributed data, it is necessary to download the distributed data files corresponding to the logical shard list in the corresponding data version directory.

2. The data synchronization method according to claim 1, further comprising: The data delivery task is scheduled.

3. The data synchronization method according to claim 1, characterized in that, Performing delivery tasks, including: The global type of data to be synchronized is sent to the server in a first manner. The first manner includes sending the global type of data to be synchronized under the target list to each service node of the server. The target list is a list of data files where the global type of data to be synchronized is located. The distributed data to be synchronized is sent to the server in a second manner, wherein the second manner includes sending the distributed data to be synchronized corresponding to each service node in the target list to the corresponding service node.

4. The data synchronization method according to claim 1, characterized in that, Generating delivery tasks also includes: Obtain a first detection result, which is used to characterize whether a first delivery task for the target configuration information has been generated; If the first detection result is yes, the first delivery task for the target configuration information has been generated, then the second detection result is obtained. The second detection result is used to characterize whether the target configuration information is different from the stored configuration information. If the second detection result is yes, and the target configuration information is different from the stored configuration information, then the stored configuration information is updated to the target configuration information.

5. The data synchronization method according to claim 4, characterized in that, The process of obtaining the first detection result also includes: If the first detection result is negative, then the first delivery task is generated; The first delivery task is sent to a target queue that contains at least one delivery task.

6. The data synchronization method according to claim 2, characterized in that, The process of scheduling the data delivery task includes: Obtain a third detection result, which is used to characterize the running status of the data delivery task; If the data delivery task is in the state of pending execution, the data delivery task is sent to the first thread, which is the thread used to execute the data delivery task. While executing the data delivery task, obtain the third detection result corresponding to the next data delivery task.

7. The data synchronization method according to claim 1, characterized in that, Performing delivery tasks also includes: Obtain the fourth detection result, which is used to characterize the version of the data to be synchronized corresponding to the data delivery task; If the fourth detection result indicates that the version of the data to be synchronized is higher than the corresponding data stored on the server, then the data to be synchronized is sent to the server; otherwise, the operation of sending the data to be synchronized is skipped. After the data to be synchronized is sent to the server, a fifth detection result is obtained. The fifth detection result is used to characterize the integrity of the data to be synchronized. If the fifth detection result indicates that the data to be synchronized is complete, then the data to be synchronized is determined to be synchronized to the server; otherwise, the data delivery task after scheduling is re-executed.

8. The data synchronization method according to claim 1, characterized in that, The configuration information for obtaining the data to be synchronized includes: Obtain third-party configuration information; Check if the third configuration information already exists; If the third configuration information is not detected, then the third configuration information is received; If the existence of the third configuration information is detected, it is further detected whether the third configuration information is different from the already stored configuration information; If the third configuration information changes, the third configuration information is updated.

9. The data synchronization method according to any one of claims 1-8, characterized in that, The data synchronization operation based on the synchronization process corresponding to the configuration information further includes: Acquire a data synchronization acceleration node, which is registered by the first service node of the server; The data to be synchronized is sent to the second service node of the server through the data synchronization acceleration node.

10. A data synchronization device, characterized in that, The device includes: A data synchronization request receiving module is used to receive data synchronization requests from the server, wherein the data synchronization requests are used to request data to be synchronized; The configuration information acquisition module is used to acquire the configuration information of the data to be synchronized according to the data synchronization request. The configuration information includes the data type of the data to be synchronized. The data type includes global type and distributed type. The global type of data to be synchronized refers to data that needs to be shared and updated globally by all service nodes in the server. The distributed type of data to be synchronized refers to data that is stored and operated using a distributed infrastructure and computing method. The data synchronization module is used to perform data synchronization operations according to the synchronization process corresponding to the configuration information, including: when the data type of the data to be synchronized is global, storing the data to be synchronized in a first space to generate a delivery task for the data to be synchronized; performing global synchronization process processing on the data to be synchronized in the first space to execute the delivery task for the data to be synchronized; when the data type of the data to be synchronized is distributed, storing the data to be synchronized in a second space to generate a delivery task for the data to be synchronized; performing distributed synchronization process processing on the data to be synchronized in the second space to execute the delivery task for the data to be synchronized; the synchronization process is used to synchronize the data to be synchronized of different data types to each service node of the server or the service node corresponding to the distributed data to be synchronized, and to manage the data to be synchronized of different data types; wherein, when executing the delivery task, when synchronizing global data, it is necessary to download all global data files in the corresponding data version directory; when synchronizing distributed data, it is necessary to download the distributed data files corresponding to the logical shard list in the corresponding data version directory.

11. An electronic device, characterized in that, The system includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the data synchronization method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the data synchronization method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the steps of the data synchronization method according to any one of claims 1 to 9.

Images