Data batch synchronization method and device and computer equipment

Abstract

The invention relates to a data batch synchronization method and device and computer equipment. The method comprises the following steps: in a source database, acquiring to-be-synchronized data needing to be synchronized in a timestamp query mode, and performing conflict detection processing on each piece of to-be-synchronized data based on timestamp information corresponding to each piece of to-be-synchronized data to obtain data conflict information among the to-be-synchronized data; based on the data conflict information between the to-be-synchronized data, conflict adjustment processing is carried out on the to-be-synchronized data through a conflict resolution strategy to obtain target to-be-synchronized data, and data synchronization processing is carried out on the target to-be-synchronized data through a data synchronization program to obtain synchronized data; and storing the synchronized data in the source database through an encryption transmission strategy, and updating the timestamps of the synchronized data in the source database. By adopting the method, the multi-data synchronization efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data batch synchronization, and in particular to a data batch synchronization method, apparatus and computer equipment. Background Art

[0002] Today, as all industries are undergoing digital transformation, a large amount of data is generated during business operations. When ODS extracts, cleans, and aggregates data from various isolated databases, there is a problem of real-time, efficient, and accurate synchronization of incremental data and stock change data. Therefore, how to improve the synchronization efficiency and accuracy of incremental data and stock change data is the current research focus.

[0003] The existing technology often adds a timestamp to the data when the current incremental data is generated. The changing timestamp can provide a basis for the scope of data synchronization. However, this method can only achieve partial data batch synchronization or full data stream synchronization. In the actual synchronization process, the synchronization efficiency cannot be guaranteed, resulting in low synchronization efficiency for multiple data. Summary of the invention

[0004] Based on this, it is necessary to provide a data batch synchronization method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical issues.

[0005] In a first aspect, the present application provides a data batch synchronization method, comprising:

[0006] In the source database, the data to be synchronized is obtained by querying the timestamp, and based on the timestamp information corresponding to each data to be synchronized, conflict detection is performed on each data to be synchronized to obtain data conflict information between the data to be synchronized;

[0007] Based on the data conflict information between the data to be synchronized, the conflict adjustment processing is performed on the data to be synchronized through the conflict resolution strategy to obtain the target data to be synchronized, and the data synchronization processing is performed on the target data to be synchronized through the data synchronization program to obtain the synchronized data;

[0008] Each of the synchronized data is stored in the source database through an encrypted transmission strategy, and a timestamp of each of the synchronized data is updated in the source database.

[0009] Optionally, obtaining the data to be synchronized in the source database by querying the timestamp includes:

[0010] Obtain a historical timestamp after the last data synchronization, and query the target data information corresponding to the timestamp after the historical timestamp in each data information of the source database;

[0011] All target data information is regarded as data to be synchronized.

[0012] Optionally, performing conflict detection processing on each of the data to be synchronized based on the timestamp information corresponding to each of the data to be synchronized to obtain data conflict information between the data to be synchronized includes:

[0013] Based on the timestamp information corresponding to each of the data to be synchronized, identifying the content of each conflicting data having data conflicts in each of the data to be synchronized through a conflict detection program;

[0014] The data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are identified, and the data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are used as the data conflict information between each of the data to be synchronized.

[0015] Optionally, based on the data conflict information between the data to be synchronized, the conflict adjustment processing is performed on the data to be synchronized by using a conflict resolution strategy to obtain each target data to be synchronized, including:

[0016] For each data conflict type, query the conflict resolution strategy corresponding to the data conflict type in the strategy processing storage area;

[0017] Based on the conflict contents of each data of the data conflict type, conflict adjustment processing is performed on each data to be synchronized by using a conflict resolution strategy corresponding to the data conflict type to obtain each target data to be synchronized.

[0018] Optionally, performing data synchronization processing on each of the target data to be synchronized by a data synchronization program to obtain each synchronized data includes:

[0019] Performing data synchronization processing on each of the target data to be synchronized to obtain a data synchronization result for each of the target data to be synchronized;

[0020] By using the data atomicity identification strategy, it is determined whether the data synchronization result meets the data atomicity condition, and when the data synchronization result does not meet the data atomicity condition, it is returned to execute the step of performing data synchronization processing on each of the target data to be synchronized to obtain the data synchronization result of each of the target data to be synchronized, until the data synchronization result meets the data atomicity condition;

[0021] Based on the data synchronization result obtained in the last iteration, each synchronized data is determined.

[0022] Optionally, storing each synchronized data in the source database through an encrypted transmission strategy includes:

[0023] Acquire a data security transmission policy of the source database, and based on the data security transmission policy, identify a current data encryption method of the source database and a current data transmission method of the source database;

[0024] Based on the current data encryption method, data encryption processing is performed on each of the synchronized data to obtain an encrypted data packet, and the encrypted data packet is transmitted to the source database through the current data transmission method.

[0025] In a second aspect, the present application also provides a data batch synchronization device, including:

[0026] The acquisition module is used to acquire the data to be synchronized in the source database by querying the timestamp, and to perform conflict detection on each data to be synchronized based on the timestamp information corresponding to each data to be synchronized, so as to obtain the data conflict information between the data to be synchronized;

[0027] A synchronization module, configured to perform conflict adjustment processing on each of the data to be synchronized based on the data conflict information between the data to be synchronized and through a conflict resolution strategy to obtain each target data to be synchronized, and perform data synchronization processing on each of the target data to be synchronized through a data synchronization program to obtain each synchronized data;

[0028] The updating module is used to store each synchronized data in the source database through an encrypted transmission strategy, and to update the timestamp of each synchronized data in the source database.

[0029] Optionally, the acquisition module is specifically used to:

[0030] Obtain a historical timestamp after the last data synchronization, and query the target data information corresponding to the timestamp after the historical timestamp in each data information of the source database;

[0031] All target data information is regarded as data to be synchronized.

[0032] Optionally, the acquisition module is specifically used to:

[0033] Based on the timestamp information corresponding to each of the data to be synchronized, identifying the content of each conflicting data having data conflicts in each of the data to be synchronized through a conflict detection program;

[0034] The data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are identified, and the data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are used as the data conflict information between each of the data to be synchronized.

[0035] Optionally, the synchronization module is specifically used to:

[0036] For each data conflict type, query the conflict resolution strategy corresponding to the data conflict type in the strategy processing storage area;

[0037] Based on the conflict contents of each data of the data conflict type, conflict adjustment processing is performed on each data to be synchronized by using a conflict resolution strategy corresponding to the data conflict type to obtain each target data to be synchronized.

[0038] Optionally, the synchronization module is specifically used to:

[0039] Performing data synchronization processing on each of the target data to be synchronized to obtain a data synchronization result for each of the target data to be synchronized;

[0040] By using the data atomicity identification strategy, it is determined whether the data synchronization result meets the data atomicity condition, and when the data synchronization result does not meet the data atomicity condition, it is returned to execute the step of performing data synchronization processing on each of the target data to be synchronized to obtain the data synchronization result of each of the target data to be synchronized, until the data synchronization result meets the data atomicity condition;

[0041] Based on the data synchronization result obtained in the last iteration, each synchronized data is determined.

[0042] Optionally, the update module is specifically used to:

[0043] Acquire a data security transmission policy of the source database, and based on the data security transmission policy, identify a current data encryption method of the source database and a current data transmission method of the source database;

[0044] Based on the current data encryption method, data encryption processing is performed on each of the synchronized data to obtain an encrypted data packet, and the encrypted data packet is transmitted to the source database through the current data transmission method.

[0045] In a third aspect, the present application provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the methods in the first aspect are implemented.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0047] In a fifth aspect, the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of any one of the methods in the first aspect are implemented.

[0048] The above-mentioned data batch synchronization method, device and computer equipment obtain the data to be synchronized by querying the timestamp in the source database, and perform conflict detection processing on each of the data to be synchronized based on the timestamp information corresponding to each of the data to be synchronized, so as to obtain the data conflict information between each of the data to be synchronized; based on the data conflict information between each of the data to be synchronized, perform conflict adjustment processing on each of the data to be synchronized through the conflict resolution strategy to obtain each target data to be synchronized, and perform data synchronization processing on each of the target data to be synchronized through the data synchronization program to obtain each synchronized data; each of the synchronized data is stored in the source database through the encrypted transmission strategy, and the timestamp of each synchronized data is updated in the source database. This scheme uses the timestamp to identify the last modification time of the data, and only synchronizes the data that has changed since the last synchronization, thereby significantly reducing the amount of data transmission, reducing the consumption of network bandwidth and storage resources, and improving the synchronization efficiency and performance. Then, this scheme sets the timestamp corresponding to each data to ensure the accurate identification of whether each data is the data to be synchronized, and avoids the problem of missing or missing data to be synchronized. Then, before data synchronization, this solution improves the conflict prevention effect of data information by identifying and resolving data conflicts, avoiding data synchronization anomalies caused by data conflicts and the risk of data synchronization failure. Finally, after data synchronization, this solution ensures the security of synchronized data through encrypted transmission, thereby comprehensively improving the synchronization efficiency and accuracy of multiple data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A schematic diagram of a flow chart of a data batch synchronization method in one embodiment;

[0051] Figure 2 A flowchart of an example of data batch synchronization in one embodiment;

[0052] Figure 3 is a structural block diagram of a data batch synchronization device in one embodiment;

[0053] Figure 4FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The data batch synchronization method provided in the embodiment of the present application can be applied to the application environment of data batch synchronization. Among them, the method can be applied to a terminal, a server, or a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be, but is not limited to, various personal computers, laptops, medium-sized computers, etc. Among them, the terminal uses a timestamp to identify the last modification time of the data, and only synchronizes the data that has changed since the last synchronization, thereby significantly reducing the amount of data transmission, reducing the consumption of network bandwidth and storage resources, and improving synchronization efficiency and performance. Then, this solution ensures accurate identification of whether each data is the data to be synchronized by setting the timestamp corresponding to each data, avoiding the problem of missing or missing data to be synchronized. After that, before data synchronization, this solution identifies and resolves data conflicts, thereby improving the conflict prevention effect of data information, avoiding data synchronization anomalies caused by data conflicts, and the risk of data synchronization failure. Finally, after data synchronization, this solution ensures the security of synchronized data through encrypted transmission, thereby comprehensively improving the synchronization efficiency and accuracy of multiple data.

[0056] In an exemplary embodiment, Figure 1 As shown, a data batch synchronization method is provided, which is described by taking the method applied to a terminal as an example, and includes the following steps S101 to S103. Among them:

[0057] Step S101, in the source database, the data to be synchronized is obtained by timestamp query, and based on the timestamp information corresponding to each data to be synchronized, conflict detection processing is performed on each data to be synchronized to obtain data conflict information between the data to be synchronized.

[0058] In this implementation, the terminal uses a timestamp to identify the last modification time of the data, and stores the identified data in the source database, and then queries the data corresponding to each timestamp after the timestamp of the last synchronized data as the data to be synchronized. Then, based on the timestamp information corresponding to each data to be synchronized, the terminal performs conflict detection processing on each data to be synchronized, and obtains data conflict information between each data to be synchronized. Among them, the data conflict information includes but is not limited to conflict information such as data version overlap and data duplication.

[0059] Step S102, based on the data conflict information between the data to be synchronized, the conflict adjustment processing is performed on the data to be synchronized through the conflict resolution strategy to obtain the target data to be synchronized, and the data synchronization processing is performed on the target data to be synchronized through the data synchronization program to obtain the synchronized data.

[0060] In this implementation, based on the data conflict information between the data to be synchronized, the terminal uses the conflict resolution strategy to adjust the conflict of the data to be synchronized, obtains the target data to be synchronized, and performs data synchronization on the target data to be synchronized through the data synchronization program to obtain the synchronized data. The conflict resolution strategy includes but is not limited to the latest timestamp priority strategy and the merge strategy. The specific conflict resolution process will be described in detail later.

[0061] Step S103: Each synchronized data is stored in the source database through an encrypted transmission strategy, and the timestamp of each synchronized data is updated in the source database.

[0062] In this implementation, the terminal stores each synchronized data in the source database through an encrypted transmission strategy, and updates the timestamp of each synchronized data in the source database. After the update, the timestamp of each synchronized data is the timestamp corresponding to the time point after the synchronization.

[0063] Based on the above scheme, the last modification time of the data is identified by the timestamp, and only the data that has changed since the last synchronization is synchronized, thereby significantly reducing the amount of data transmission, reducing the consumption of network bandwidth and storage resources, and improving synchronization efficiency and performance. Then, this scheme ensures the accurate identification of whether each data is the data to be synchronized by setting the timestamp corresponding to each data, thereby avoiding the problem of omission or missing of the data to be synchronized. After that, before data synchronization, this scheme improves the conflict prevention effect of data information by identifying and resolving data conflicts, avoiding data synchronization anomalies caused by data conflicts, and the risk of data synchronization failure. Finally, after data synchronization, this scheme ensures the security of synchronized data through encrypted transmission, thereby comprehensively improving the synchronization efficiency and accuracy of multiple data.

[0064] Optionally, in the source database, the data to be synchronized is obtained through timestamp query, including: obtaining the historical timestamp after the last data synchronization, and querying the target data information corresponding to the timestamp after the historical timestamp in each data information of the source database; all target data information is regarded as the data to be synchronized.

[0065] In this implementation, the terminal obtains the historical timestamp after the last data synchronization, and searches for target data information corresponding to the timestamp after the historical timestamp in each data information of the source database. Finally, the terminal regards all target data information as data to be synchronized.

[0066] Based on the above solution, by synchronizing only the data that has changed since the last synchronization (incremental data), the amount of data transmission can be significantly reduced, and the consumption of network bandwidth and storage resources can be reduced.

[0067] Optionally, based on the timestamp information corresponding to each data to be synchronized, conflict detection processing is performed on each data to be synchronized to obtain data conflict information between each data to be synchronized, including: based on the timestamp information corresponding to each data to be synchronized, through a conflict detection program, identifying each conflicting data content in which a data conflict exists in each data to be synchronized; identifying the data conflict type between each conflicting data content, and the data conflict content between each conflicting data, and using the data conflict type between each conflicting data content, and the data conflict content between each conflicting data as the data conflict information between each data to be synchronized.

[0068] In this implementation, the terminal identifies the conflicting data contents in the data to be synchronized based on the timestamp information corresponding to each data to be synchronized through a conflict detection program. Then, the terminal identifies the data conflict type between the conflicting data contents and the data conflict content between the conflicting data, and uses the data conflict type between the conflicting data contents and the data conflict content between the conflicting data as the data conflict information between the data to be synchronized. The data conflict type includes, but is not limited to, overlapping data types, same version data types, etc.

[0069] Based on the above solution, in a multi-source synchronization or multi-write scenario, the conflict type and conflict information between data are determined by timestamps, thereby improving the efficiency of identifying data conflicts.

[0070] Optionally, based on the data conflict information between each data to be synchronized, conflict adjustment processing is performed on each data to be synchronized through a conflict resolution strategy to obtain each target data to be synchronized, including: for each data conflict type, querying the conflict resolution strategy corresponding to the data conflict type in the policy processing storage area; based on the data conflict content of each data conflict type, conflict adjustment processing is performed on each data to be synchronized through the conflict resolution strategy corresponding to the data conflict type to obtain each target data to be synchronized.

[0071] In this implementation, the terminal queries the conflict resolution strategy corresponding to each data conflict type in the policy processing storage area. Different conflict types correspond to different conflict resolution strategies. The conflict resolution strategy for overlapping data types is the merge strategy, while the conflict resolution strategy for the same version data type is the latest timestamp priority strategy.

[0072] Then, the terminal performs conflict adjustment processing on each to-be-synchronized data based on the conflict content of each data of the data conflict type and through a conflict resolution strategy corresponding to the data conflict type to obtain each target to-be-synchronized data.

[0073] Based on the above solution, different conflict resolution methods are applied to conflicting data of different data conflict types to ensure data consistency and integrity and reduce the negative impact of data conflicts.

[0074] Optionally, through the data synchronization program, data synchronization processing is performed on each target data to be synchronized to obtain each synchronized data, including: performing data synchronization processing on each target data to be synchronized to obtain a data synchronization result of each target data to be synchronized; judging whether the data synchronization result meets the data atomicity condition through a data atomicity identification strategy, and when the data synchronization result does not meet the data atomicity condition, returning to execute the step of performing data synchronization processing on each target data to be synchronized to obtain a data synchronization result of each target data to be synchronized, until the data synchronization result meets the data atomicity condition; and determining each synchronized data based on the data synchronization result obtained in the last iteration.

[0075] In this implementation, the terminal performs data synchronization processing on each target data to be synchronized, and obtains the data synchronization result of each target data to be synchronized. Among them, the data synchronization result includes the result information of whether the synchronization of each target data to be synchronized is successful or not, and the result information only includes two types, one when the synchronization is successful and the other when the synchronization fails. Then, the terminal determines whether the data synchronization result meets the data atomicity condition through the data atomicity identification strategy, and when the data synchronization result does not meet the data atomicity condition, it returns to execute the data synchronization processing on each target data to be synchronized, and obtains the data synchronization result step of each target data to be synchronized, until the data synchronization result meets the data atomicity condition. Among them, the data atomicity condition is that all target data to be synchronized are synchronized successfully.

[0076] Finally, the terminal determines each synchronized data based on the data synchronization result obtained in the last iteration, that is, the terminal takes the data of all target data to be synchronized after the data synchronization is completed as each synchronized data.

[0077] Based on the above solution, the atomicity of each synchronization operation is ensured by using database transactions or application logic, thereby improving the consistency of the synchronized data.

[0078] Optionally, each synchronized data is stored in the source database through an encrypted transmission strategy, including: obtaining the data security transmission strategy of the source database, and based on the data security transmission strategy, identifying the current data encryption method of the source database and the current data transmission method of the source database; based on the current data encryption method, performing data encryption processing on each synchronized data to obtain an encrypted data packet, and transmitting the encrypted data packet to the source database through the current data transmission method.

[0079] In this implementation, the terminal obtains the data security transmission policy of the source database, and based on the data security transmission policy, identifies the current data encryption method of the source database and the current data transmission method of the source database. The current data encryption method includes but is not limited to encryption technology, secure transmission protocol (such as HTTPS, TLS), etc.

[0080] Then, the terminal performs data encryption processing on each synchronized data based on the current data encryption method to obtain an encrypted data packet, and transmits the encrypted data packet to the source database through the current data transmission method.

[0081] Based on the above solution, by combining encryption technology and secure transmission protocols (such as HTTPS and TLS), the security of data during transmission is protected, the risk of data being intercepted or tampered with is reduced, and the security of data transmission is improved.

[0082] This application also provides an example of data batch synchronization, such as Figure 2 As shown, the specific processing process includes the following steps:

[0083] Step S201, obtaining the historical timestamp after the last data synchronization, and searching the target data information corresponding to the timestamp after the historical timestamp in each data information of the source database.

[0084] Step S202: All target data information is taken as data to be synchronized.

[0085] Step S203 : Based on the timestamp information corresponding to each data to be synchronized, through a conflict detection program, the conflicting data contents in each data to be synchronized are identified.

[0086] Step S204, identifying the data conflict type between the conflicting data contents and the data conflict content between the conflicting data, and using the data conflict type between the conflicting data contents and the data conflict content between the conflicting data as the data conflict information between the data to be synchronized.

[0087] Step S205: for each data conflict type, query the conflict resolution strategy corresponding to the data conflict type in the strategy processing storage area.

[0088] Step S206 , based on the conflict contents of each data of the data conflict type, and using the conflict resolution strategy corresponding to the data conflict type, conflict adjustment processing is performed on each data to be synchronized, so as to obtain each target data to be synchronized.

[0089] Step S207: performing data synchronization processing on each target data to be synchronized, and obtaining a data synchronization result of each target data to be synchronized.

[0090] Step S208, through the data atomicity identification strategy, determine whether the data synchronization result meets the data atomicity condition, and when the data synchronization result does not meet the data atomicity condition, return to execute the data synchronization processing for each target data to be synchronized, and obtain the data synchronization result step of each target data to be synchronized, until the data synchronization result meets the data atomicity condition.

[0091] Step S209: determining each synchronized data based on the data synchronization result obtained in the last iteration.

[0092] Step S210: acquiring the data security transmission policy of the source database, and identifying the current data encryption mode of the source database and the current data transmission mode of the source database based on the data security transmission policy.

[0093] Step S211, based on the current data encryption method, data encryption processing is performed on each synchronized data to obtain an encrypted data packet, and the encrypted data packet is transmitted to the source database through the current data transmission method.

[0094] Step S212: updating the timestamp of each synchronized data in the source database.

[0095] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0096] Based on the same inventive concept, the embodiment of the present application also provides a data batch synchronization device for implementing the data batch synchronization method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more data batch synchronization device embodiments provided below can refer to the limitations of the data batch synchronization method above, and will not be repeated here.

[0097] In an exemplary embodiment, Figure 3 As shown, a data batch synchronization device is provided, including: an acquisition module 310, a synchronization module 320 and an update module 330, wherein:

[0098] The acquisition module 310 is used to acquire the data to be synchronized in the source database by querying the timestamp, and perform conflict detection processing on each data to be synchronized based on the timestamp information corresponding to each data to be synchronized, so as to obtain data conflict information between each data to be synchronized;

[0099] The synchronization module 320 is used to perform conflict adjustment processing on each of the to-be-synchronized data based on the data conflict information between the to-be-synchronized data through a conflict resolution strategy to obtain each target to-be-synchronized data, and perform data synchronization processing on each of the target to-be-synchronized data through a data synchronization program to obtain each synchronized data;

[0100] The updating module 330 is used to store each of the synchronized data in the source database through an encrypted transmission strategy, and to update the timestamp of each of the synchronized data in the source database.

[0101] Optionally, the acquisition module 310 is specifically configured to:

[0102] Obtain a historical timestamp after the last data synchronization, and query the target data information corresponding to the timestamp after the historical timestamp in each data information of the source database;

[0103] All target data information is regarded as data to be synchronized.

[0104] Optionally, the acquisition module 310 is specifically configured to:

[0105] Based on the timestamp information corresponding to each of the data to be synchronized, identifying the content of each conflicting data having data conflicts in each of the data to be synchronized through a conflict detection program;

[0106] The data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are identified, and the data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are used as the data conflict information between each of the data to be synchronized.

[0107] Optionally, the synchronization module 320 is specifically configured to:

[0108] For each data conflict type, query the conflict resolution strategy corresponding to the data conflict type in the strategy processing storage area;

[0109] Based on the conflict contents of each data of the data conflict type, conflict adjustment processing is performed on each data to be synchronized by using a conflict resolution strategy corresponding to the data conflict type to obtain each target data to be synchronized.

[0110] Optionally, the synchronization module 320 is specifically used for:

[0111] Performing data synchronization processing on each of the target data to be synchronized to obtain a data synchronization result for each of the target data to be synchronized;

[0112] By using the data atomicity identification strategy, it is determined whether the data synchronization result meets the data atomicity condition, and when the data synchronization result does not meet the data atomicity condition, it is returned to execute the step of performing data synchronization processing on each of the target data to be synchronized to obtain the data synchronization result of each of the target data to be synchronized, until the data synchronization result meets the data atomicity condition;

[0113] Based on the data synchronization result obtained in the last iteration, each synchronized data is determined.

[0114] Optionally, the updating module 330 is specifically configured to:

[0115] Acquire a data security transmission policy of the source database, and based on the data security transmission policy, identify a current data encryption method of the source database and a current data transmission method of the source database;

[0116] Based on the current data encryption method, data encryption processing is performed on each of the synchronized data to obtain an encrypted data packet, and the encrypted data packet is transmitted to the source database through the current data transmission method.

[0117] Each module in the above-mentioned data batch synchronization device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0118] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data batch synchronization method is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0119] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the data batch synchronization method when executing the computer program.

[0121] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data batch synchronization method are implemented.

[0122] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the data batch synchronization method when executed by a processor.

[0123] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0124] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0125] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data batch synchronization method, characterized in that: The method comprises: In the source database, the data to be synchronized is obtained by querying the timestamp, and based on the timestamp information corresponding to each data to be synchronized, conflict detection is performed on each data to be synchronized to obtain data conflict information between the data to be synchronized; Based on the data conflict information between the data to be synchronized, the conflict adjustment processing is performed on the data to be synchronized through the conflict resolution strategy to obtain the target data to be synchronized, and the data synchronization processing is performed on the target data to be synchronized through the data synchronization program to obtain the synchronized data; Each of the synchronized data is stored in the source database through an encrypted transmission strategy, and a timestamp of each of the synchronized data is updated in the source database.

2. The method according to claim 1, characterized in that The method of obtaining the data to be synchronized in the source database by querying the timestamp includes: Obtain a historical timestamp after the last data synchronization, and query the target data information corresponding to the timestamp after the historical timestamp in each data information of the source database; All target data information is regarded as data to be synchronized.

3. The method according to claim 1, characterized in that The performing conflict detection processing on each of the data to be synchronized based on the timestamp information corresponding to each of the data to be synchronized to obtain data conflict information between the data to be synchronized includes: Based on the timestamp information corresponding to each of the data to be synchronized, identifying the content of each conflicting data having data conflicts in each of the data to be synchronized through a conflict detection program; The data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are identified, and the data conflict type between each of the conflicting data contents and the data conflict content between each of the conflicting data are used as the data conflict information between each of the data to be synchronized.

4. The method according to claim 1, characterized in that: Based on the data conflict information between the data to be synchronized, the conflict adjustment processing is performed on the data to be synchronized by using a conflict resolution strategy to obtain each target data to be synchronized, including: For each data conflict type, query the conflict resolution strategy corresponding to the data conflict type in the strategy processing storage area; Based on the conflict contents of each data of the data conflict type, conflict adjustment processing is performed on each data to be synchronized by using a conflict resolution strategy corresponding to the data conflict type to obtain each target data to be synchronized.

5. The method according to claim 1, characterized in that The data synchronization program is used to perform data synchronization processing on each target data to be synchronized to obtain each synchronized data, including: Performing data synchronization processing on each of the target data to be synchronized to obtain a data synchronization result for each of the target data to be synchronized; By using the data atomicity identification strategy, it is determined whether the data synchronization result meets the data atomicity condition, and when the data synchronization result does not meet the data atomicity condition, it is returned to execute the step of performing data synchronization processing on each of the target data to be synchronized to obtain the data synchronization result of each of the target data to be synchronized, until the data synchronization result meets the data atomicity condition; Based on the data synchronization result obtained in the last iteration, each synchronized data is determined.

6. The method according to claim 5, characterized in that The step of storing each synchronized data in the source database through an encrypted transmission strategy includes: Acquire a data security transmission policy of the source database, and based on the data security transmission policy, identify a current data encryption method of the source database and a current data transmission method of the source database; Based on the current data encryption method, data encryption processing is performed on each of the synchronized data to obtain an encrypted data packet, and the encrypted data packet is transmitted to the source database through the current data transmission method.

7. A data batch synchronization device, characterized in that: The device comprises: The acquisition module is used to acquire the data to be synchronized in the source database by querying the timestamp, and to perform conflict detection on each data to be synchronized based on the timestamp information corresponding to each data to be synchronized, so as to obtain the data conflict information between the data to be synchronized; A synchronization module, configured to perform conflict adjustment processing on each of the data to be synchronized based on the data conflict information between the data to be synchronized and through a conflict resolution strategy to obtain each target data to be synchronized, and perform data synchronization processing on each of the target data to be synchronized through a data synchronization program to obtain each synchronized data; The updating module is used to store each synchronized data in the source database through an encrypted transmission strategy, and to update the timestamp of each synchronized data in the source database.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images