A cross-system data synchronization method and device

By verifying data timestamps and converting and processing according to business needs, the problems of high difficulty and low efficiency of cross-system data synchronization are solved, and flexibility setting and efficiency improvement are achieved.

CN113761001BActive Publication Date: 2024-12-06BEIJING WODONG TIANJUN INFORMATION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010935432.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-08
Publication Date
2024-12-06
Estimated Expiration
2040-09-08

AI Technical Summary

Technical Problem

The existing cross-system data synchronization methods have problems such as high data synchronization difficulty coefficient, inability to set flexibly, and low data synchronization efficiency.

Method used

By receiving the data number of data to be synchronized, call the data query interface to verify the timestamp, ensure that the data is the latest updated data, and convert it according to the data table structure type and business needs. After judging that the synchronization is completed, the data that failed is sent to Retry the queue for processing again.

Benefits of technology

The difficulty coefficient of data synchronization is reduced, and the flexibility setting of cross-system data synchronization is realized, while improving data synchronization efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113761001B_ABST
    Figure CN113761001B_ABST
Patent Text Reader

Abstract

The present invention discloses a cross-system data synchronization method and device, and relates to the field of computer technology. A specific implementation of the method includes: receiving data to be synchronized sent by a first system, calling a data query interface of the first system according to the data number of the data to be synchronized, and checking the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data; converting and processing the data to be synchronized according to the data table structure type and business requirements corresponding to the second system; judging whether the data to be synchronized is successfully converted, and if so, determining that the cross-system data synchronization is completed; if not, sending the data to be synchronized that failed to be converted to a retry queue, and converting and processing the data to be synchronized in the retry queue again to achieve cross-system data synchronization. This implementation reduces the difficulty coefficient of data synchronization, improves the flexibility of cross-system data synchronization, and can also effectively improve the efficiency of data synchronization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cross-system data synchronization method and device. Background Art

[0002] In the process of technological development, new and old systems often coexist for a period of time after a new system is developed. At this time, the old and new systems will share some data, and the new system needs to synchronize the corresponding data in the old system. In the existing technology, a shared database is usually used or data synchronization is achieved directly through a database change log.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] Existing cross-system data synchronization methods have the following problems: high difficulty in data synchronization, inability to flexibly set data synchronization according to business needs, and low data synchronization efficiency. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a cross-system data synchronization method and device, which can significantly reduce the difficulty coefficient of data synchronization, realize flexible setting of cross-system data synchronization, and effectively improve data synchronization efficiency.

[0006] To achieve the above object, according to a first aspect of an embodiment of the present invention, a cross-system data synchronization method is provided, comprising:

[0007] Receive the data to be synchronized sent by the first system, call the data query interface of the first system according to the data number of the data to be synchronized, and check the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data;

[0008] Convert the data to be synchronized according to the data table structure type and business requirements corresponding to the second system;

[0009] Determine whether the data to be synchronized is converted successfully. If so, determine that the cross-system data synchronization is completed; if not, send the data to be synchronized that failed to be converted to the retry queue, and convert the data to be synchronized in the retry queue again to achieve cross-system data synchronization.

[0010] Furthermore, before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the method further includes:

[0011] The business type of the data to be synchronized is determined, and the data to be synchronized is filtered according to the business type and filtering conditions of the data to be synchronized, wherein the filtering conditions are associated with the business requirements.

[0012] Furthermore, before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the method further includes:

[0013] The business type of the data to be synchronized is determined, and according to the business type of the data to be synchronized, a conversion processing sequence and a conversion processing cycle corresponding to the data to be synchronized of different business types are determined.

[0014] Furthermore, the step of performing data conversion processing on the data to be synchronized according to the data table structure type and business requirements corresponding to the second system includes:

[0015] The conversion type corresponding to the data to be synchronized is determined according to business requirements, and the data to be synchronized is converted according to the conversion type and the data table structure type corresponding to the second system, wherein the conversion processing includes adding processing and / or modifying processing.

[0016] Furthermore, after the step of determining the conversion type corresponding to the data to be synchronized according to the business requirements, the method further includes: performing a lock operation on the data to be synchronized according to the conversion type corresponding to the data to be synchronized.

[0017] Furthermore, the steps of constructing retry queues according to the conversion types and sending the to-be-synchronized data that failed to be converted to the retry queues include:

[0018] The conversion type corresponding to the to-be-synchronized data that failed to be converted is determined, and the to-be-synchronized data that failed to be converted is sent to a retry queue of the corresponding conversion type.

[0019] Furthermore, if the data to be synchronized is successfully converted, the status of the successfully converted data to be synchronized is updated to synchronized; the method further includes:

[0020] Set a proofreading cycle and / or proofreading parameters, call the data query interface according to the proofreading cycle and / or proofreading parameters to determine the amount of data to be synchronized in the first system within a specific period, and check whether the amount of data to be synchronized in the first system is consistent with the amount of synchronized data in the second system; if inconsistent, convert the inconsistent data to be synchronized.

[0021] According to a second aspect of an embodiment of the present invention, a cross-system data synchronization device is provided, including:

[0022] A data receiving module is used to receive the data to be synchronized sent by the first system, call the data query interface of the first system according to the data number of the data to be synchronized, and check the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data;

[0023] A data conversion module, used to convert the data to be synchronized according to the data table structure type and business requirements corresponding to the second system;

[0024] The retry module is used to determine whether the data to be synchronized is converted successfully. If so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted again to achieve cross-system data synchronization.

[0025] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0026] one or more processors;

[0027] a storage device for storing one or more programs,

[0028] When one or more programs are executed by one or more processors, the one or more processors implement any of the above-mentioned cross-system data synchronization methods.

[0029] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, any of the above-mentioned cross-system data synchronization methods is implemented.

[0030] An embodiment of the above invention has the following advantages or beneficial effects: because the data to be synchronized sent by the first system is received, the data query interface of the first system is called according to the data number of the data to be synchronized, and the timestamp of the data to be synchronized is checked to determine that the data to be synchronized corresponding to the data number is the most recently updated data; the data to be synchronized is converted and processed according to the data table structure type and business needs corresponding to the second system; it is determined whether the conversion of the data to be synchronized is successful, and if so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted and processed again to realize the technical means of cross-system data synchronization, so the technical problems of high difficulty coefficient, inability to flexibly set data synchronization according to business needs, and low data synchronization efficiency in the existing cross-system data synchronization methods are overcome, thereby achieving the technical effect of reducing the difficulty coefficient of data synchronization, realizing flexible setting of cross-system data synchronization, and effectively improving the efficiency of data synchronization.

[0031] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0033] Figure 1 is a schematic diagram of the main process of the cross-system data synchronization method provided according to the first embodiment of the present invention;

[0034] Figure 2 is a schematic diagram of the main process of the cross-system data synchronization method provided according to the second embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of main modules of a cross-system data synchronization device provided according to an embodiment of the present invention;

[0036] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0037] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0039] Figure 1 is a schematic diagram of the main process of the cross-system data synchronization method provided according to the first embodiment of the present invention; Figure 1 As shown, the cross-system data synchronization method provided by the embodiment of the present invention mainly includes:

[0040] Step S101, receiving the data to be synchronized sent by the first system, calling the data query interface of the first system according to the data number of the data to be synchronized, and checking the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data.

[0041] Specifically, according to an embodiment of the present invention, the first system sends the data to be synchronized to the second system. According to a specific implementation of the embodiment of the present invention, the first system is an old system and the second system is a new system (or the first system is a new system and the second system is an old system). It should be noted that the above example is not intended to limit the embodiment of the present invention. The first system and the second system in the present invention may refer to any system that needs to perform data synchronization operations, and the number of systems corresponding to the second system is greater than or equal to one.

[0042] In the existing cross-system synchronization methods, new systems often adopt new designs, resulting in differences in the database table structures between the new and old systems. The use of a shared database requires compatibility in the table structure, making data synchronization difficult. The database change log has strict requirements on the time sequence of data and cannot be flexibly set according to the business needs of the new system.

[0043] During the actual execution of the conversion processing, due to the problem of processing efficiency, certain data to be synchronized is sent to the second system. While waiting for the conversion processing to be executed, the first system may have updated the data to be synchronized, which resulted in the data to be synchronized waiting to be processed in the second system not being the most recently updated data. Through the above setting, before executing the conversion processing step, the timestamp of the data to be synchronized is checked to ensure that the data to be processed is the most recently updated data (if the timestamp shows that the data to be synchronized with the data number is not the most recently updated data, it is necessary to call the data acquisition interface to obtain the most recently updated data corresponding to the data number), which can avoid unnecessary synchronization operations and significantly improve data synchronization efficiency.

[0044] Step S102: converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system.

[0045] One difficulty in achieving data synchronization between different systems is that different systems have corresponding data table interface types of different types. Through the above settings, according to the data table structure type and business needs of the second system, data synchronization can be achieved quickly and flexibly according to business needs.

[0046] Exemplarily, according to an embodiment of the present invention, before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the method further includes:

[0047] The business type of the data to be synchronized is determined, and the data to be synchronized is filtered according to the business type and filtering conditions of the data to be synchronized, wherein the filtering conditions are associated with the business requirements.

[0048] In the actual processing process, the second system does not need to synchronize all the data sent by the first system, but can flexibly select data of a specified business type for processing according to the business needs of the second system itself. Through the above settings, filter conditions are formulated according to business needs, and the data to be synchronized is filtered according to the filter conditions and the business type of the data to be synchronized, which can not only improve the efficiency of data synchronization, but also realize the flexibility of data synchronization settings.

[0049] Further, according to an embodiment of the present invention, before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the method further includes:

[0050] The business type of the data to be synchronized is determined, and according to the business type of the data to be synchronized, a conversion processing sequence and a conversion processing cycle corresponding to the data to be synchronized of different business types are determined.

[0051] Through the above settings, the conversion processing order and conversion processing cycle corresponding to the data to be synchronized are determined according to the business type, which is conducive to batch synchronization of the data to be synchronized of the same business type, further improving the data synchronization efficiency.

[0052] According to a specific implementation of the embodiment of the present invention, in response to the service requirements of the second system, a synchronization priority may be set for the service type of the data to be synchronized, so as to implement a priority synchronization conversion of the data to be synchronized of the specified service type.

[0053] Preferably, according to an embodiment of the present invention, the step of performing data conversion processing on the data to be synchronized according to the data table structure type and business requirements corresponding to the second system includes:

[0054] The conversion type corresponding to the data to be synchronized is determined according to business needs, and the data to be synchronized is converted according to the conversion type and the data table structure type corresponding to the second system, wherein the conversion processing includes adding processing and / or modifying and deleting processing.

[0055] On the one hand, the business requirements of the second system can specify the business type of the data to be synchronized that needs to be converted and processed by the second system, so as to filter the data to be synchronized and flexibly set the business type for synchronization; on the other hand, the conversion type of the data to be synchronized can be determined according to each business requirement. Specifically, the business requirements can specify that certain fields in a certain business type of data need to be converted and synchronized, and certain fields do not need to be converted and synchronized, and then it can be determined whether the conversion type of the data to be synchronized is to be added or modified (modification includes deletion).

[0056] According to an embodiment of the present invention, the data to be synchronized of different conversion types can be divided into two conversion processing queues, a new processing queue and a modification processing queue (including deletion, this part can also separate modification and deletion for conversion processing), and then the data to be synchronized in the same processing queue can perform the same processing operation, thereby realizing batch synchronization processing and improving data synchronization efficiency.

[0057] Addition modification refers to adding new data to the data table of the current system; modification processing refers to updating the updated data (changed data) in the first system to the data table of the current system.

[0058] Further, according to an embodiment of the present invention, after the step of determining the conversion type corresponding to the data to be synchronized according to business requirements, the method further includes: performing a lock operation on the data to be synchronized according to the conversion type corresponding to the data to be synchronized.

[0059] Since the above statement mentions building different message processing queues, the above locking operation can avoid the situation where multiple conversion processors perform conversion processing on the same data to be synchronized, thereby causing processing exceptions.

[0060] Step S103, determine whether the data to be synchronized is converted successfully, if so, determine that the cross-system data synchronization is completed; if not, send the data to be synchronized that failed to be converted to the retry queue, and convert the data to be synchronized in the retry queue again to achieve cross-system data synchronization.

[0061] In the existing methods, retrying the data that failed to synchronize often leads to the squeezing of the data to be synchronized, and the data synchronization efficiency is low. That is, in the existing methods, it is often required to re-acquire the data from the outside (first system) before performing synchronization, which often leads to message backlog, infinite retries, and affects the synchronization efficiency. Through the above settings, the embodiment of the present invention sends the data that failed to be converted to the retry queue, and realizes resynchronization processing from the inside (second system). The outside will not perceive the internal retry operation, and the internal can also handle various abnormal data (data that failed to synchronize) through manual intervention. For example, when the number of retries reaches a certain threshold, manual intervention can be performed to avoid infinite retries affecting the synchronization efficiency.

[0062] It is understandable that before the retry queue performs conversion processing again, it also includes: calling the data query interface of the first system according to the data number of the data to be synchronized to check the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data.

[0063] Further, according to an embodiment of the present invention, the step of respectively constructing retry queues according to conversion types and sending the to-be-synchronized data that failed to be converted to the retry queues includes:

[0064] The conversion type corresponding to the to-be-synchronized data that failed to be converted is determined, and the to-be-synchronized data that failed to be converted is sent to a retry queue of the corresponding conversion type.

[0065] Through the above settings, for data that fails to be converted, multiple retry queues can be set according to their corresponding conversion types to improve data synchronization efficiency.

[0066] Preferably, according to an embodiment of the present invention, if the to-be-synchronized data is successfully converted, the status of the successfully converted to-be-synchronized data is updated to synchronized; the above cross-system data synchronization method further includes:

[0067] Set a proofreading cycle and / or proofreading parameters, and call the data query interface according to the proofreading cycle and / or proofreading parameters to determine the amount of data to be converted in the first system within a specific period, and check whether the amount of data to be synchronized in the first system is consistent with the amount of converted data in the second system; if not, convert the inconsistent data to be synchronized.

[0068] The above settings can be used as a fallback strategy for the cross-system data synchronization method provided by the embodiment of the present invention. By setting the proofreading cycle and / or proofreading parameters, the synchronized data can be proofread regularly. The proofreading parameters can be in the form of key fields, etc., which are used to detect whether all the data corresponding to the key fields have been synchronized, thereby improving the accuracy of cross-system data synchronization.

[0069] According to the technical solution of the embodiment of the present invention, because the data to be synchronized sent by the first system is received, the data query interface of the first system is called according to the data number of the data to be synchronized, and the timestamp of the data to be synchronized is checked to determine that the data to be synchronized corresponding to the data number is the most recently updated data; the data to be synchronized is converted and processed according to the data table structure type and business needs corresponding to the second system; it is determined whether the conversion of the data to be synchronized is successful, and if so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted and processed again to realize the technical means of cross-system data synchronization, so the technical problems of high difficulty coefficient, inability to flexibly set data synchronization according to business needs, and low data synchronization efficiency in the existing cross-system data synchronization methods are overcome, thereby achieving the technical effect of reducing the difficulty coefficient of data synchronization, realizing flexible setting of cross-system data synchronization, and effectively improving the efficiency of data synchronization.

[0070] Figure 2 is a schematic diagram of the main process of the cross-system data synchronization method provided according to the second embodiment of the present invention; an application scenario of the embodiment of the present invention is that a new system synchronizes data in an old system, such as Figure 2 As shown, the cross-system data synchronization method provided by the embodiment of the present invention mainly includes:

[0071] Step S201: receiving the data to be synchronized sent by the first system, determining the service type of the data to be synchronized, and filtering the data to be synchronized according to the service type and filtering conditions of the data to be synchronized.

[0072] Specifically, the filtering conditions are associated with business requirements. In the actual processing process, the second system does not need to synchronize all the data sent by the first system, but can flexibly select data of a specified business type for processing according to the business requirements of the second system itself. Through the above settings, filtering conditions are formulated according to business requirements, and the data to be synchronized is filtered according to the filtering conditions and the business type of the data to be synchronized, which can not only improve the efficiency of data synchronization, but also realize the flexible setting of data synchronization.

[0073] Step S202: determining the conversion processing sequence and conversion processing cycle corresponding to the data to be synchronized of different service types according to the service type of the data to be synchronized.

[0074] Through the above settings, determining the conversion processing order and conversion processing cycle corresponding to the data to be synchronized according to the business type is conducive to batch synchronization of the data to be synchronized of the same business type, further improving the efficiency of data synchronization. For example, in the old system, the business type of department A is orders, and the business type corresponding to department B is customer service logs. The business requirements of the new system only require synchronization of the data corresponding to the business type of department A, and do not need to synchronize the data of the business type of department B. This can be achieved by setting filtering conditions.

[0075] Furthermore, according to an embodiment of the present invention, it also includes: in response to the business needs of the second system, a synchronization priority can be set for the business type of the data to be synchronized, so as to implement a priority synchronization conversion of the data to be synchronized of the specified business type.

[0076] Step S203, determine the target synchronization data according to the conversion processing sequence and the conversion processing cycle, call the data query interface of the first system according to the data number of the target synchronization data, and check the timestamp of the target synchronization data to determine that the target synchronization data corresponding to the data number is the most recently updated data.

[0077] Through the above settings, in view of the business needs of the new system, the target synchronization data is obtained from the data to be synchronized according to the conversion processing order as the data to be currently converted.

[0078] Step S204: determine the conversion type corresponding to the target synchronization data according to the business requirements, and perform a lock operation on the target synchronization data according to the conversion type corresponding to the target synchronization data.

[0079] Through the above settings, a lock operation is performed on the data to be converted, which can avoid the situation where multiple conversion processors perform conversion processing on the same data to be synchronized, thereby causing processing abnormalities.

[0080] Step S205 , converting the target synchronization data according to the conversion type and the data table structure type corresponding to the second system.

[0081] Specifically, the conversion process includes adding process and / or modifying and deleting process. Adding and modifying refers to adding new data to the data table of the current system; modifying refers to updating the updated data (changed data) in the first system to the data table of the current system.

[0082] On the one hand, the business requirements of the new system can specify the business type of the data to be synchronized that needs to be converted and processed by the second system, so as to filter the data to be synchronized and flexibly set the business type to be synchronized; on the other hand, the conversion type of the data to be synchronized can also be determined according to each business requirement. Specifically, the business requirements can specify that certain fields in a certain business type of data need to be converted and synchronized, and certain fields do not need to be converted and synchronized, and then it can be determined whether the conversion type of the data to be synchronized is new addition or modification (modification includes deletion). According to an embodiment of the present invention, the data to be synchronized with different conversion types can be divided into two conversion processing queues, a new addition processing queue and a modification processing queue (including deletion, this part can also separate modification and deletion for conversion processing), and then the data to be synchronized in the same processing queue can perform the same processing operation, thereby realizing batch synchronization processing and improving data synchronization efficiency.

[0083] Step S206, judging whether the target synchronization data is successfully converted. If yes, that is, the conversion of the target synchronization data is successful, then executing step S208; if no, that is, the conversion of the target synchronization data fails, then going to step S207.

[0084] Step S207, construct retry queues according to the conversion types, determine the conversion type corresponding to the target synchronization data that failed to be converted, and send the target synchronization data to the retry queue of the corresponding conversion type. Then go to step S203.

[0085] In the existing methods, for data that fails to be synchronized, it is often required to re-acquire data from the outside (old system) before performing synchronization, which often leads to message backlog, infinite retries, and affects synchronization efficiency. Through the above settings, the embodiments of the present invention can set multiple retry queues for data that fails to be converted according to its corresponding conversion type to improve data synchronization efficiency.

[0086] The embodiment of the present invention realizes resynchronization processing from the inside (new system), and the internal retry operation will not be perceived by the outside. Various abnormal data (data that failed to synchronize) can also be processed internally through manual intervention. For example, when the number of retries reaches a certain threshold, manual intervention can be performed to avoid infinite retries affecting synchronization efficiency.

[0087] It is understandable that before the retry queue performs conversion processing again, it also includes: calling the data query interface of the first system according to the data number of the data to be synchronized to check the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data.

[0088] Step S208, determining that the cross-system data synchronization is completed, and updating the status of the successfully converted target synchronization data to synchronized.

[0089] Step S209, set the proofreading cycle and / or proofreading parameters, call the data query interface according to the proofreading cycle and / or proofreading parameters to determine the number of data to be synchronized in the first system within a specific period, and check whether the number of data to be synchronized in the first system is consistent with the number of synchronized data in the second system; if inconsistent, convert the inconsistent data to be synchronized.

[0090] The above settings can be used as a fallback strategy for the cross-system data synchronization method provided by the embodiment of the present invention. By setting the proofreading cycle and / or proofreading parameters, the synchronized data can be proofread regularly. The proofreading parameters can be in the form of key fields, etc., which are used to detect whether all the data corresponding to the key fields have been synchronized, thereby improving the accuracy of cross-system data synchronization.

[0091] According to an embodiment of the present invention, the proofreading parameter "Keyword: store" can be set, and the specific period is "past week". By querying the number of data to be synchronized with "keyword is store" in the old system in the past week, and then comparing it with the number of synchronized data in the new system in the past week, it is possible to effectively avoid incomplete data synchronization due to abnormal operations in the aforementioned links.

[0092] According to the technical solution of the embodiment of the present invention, because the data to be synchronized sent by the first system is received, the data query interface of the first system is called according to the data number of the data to be synchronized, and the timestamp of the data to be synchronized is checked to determine that the data to be synchronized corresponding to the data number is the most recently updated data; the data to be synchronized is converted and processed according to the data table structure type and business needs corresponding to the second system; it is determined whether the conversion of the data to be synchronized is successful, and if so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted and processed again to realize the technical means of cross-system data synchronization, so the technical problems of high difficulty coefficient, inability to flexibly set data synchronization according to business needs, and low data synchronization efficiency in the existing cross-system data synchronization methods are overcome, thereby achieving the technical effect of reducing the difficulty coefficient of data synchronization, realizing flexible setting of cross-system data synchronization, and effectively improving the efficiency of data synchronization.

[0093] Figure 3is a schematic diagram of the main modules of the cross-system data synchronization device provided according to an embodiment of the present invention; Figure 3 As shown, the cross-system data synchronization device 300 provided by the embodiment of the present invention mainly includes:

[0094] The data receiving module 301 is used to receive the data to be synchronized sent by the first system, call the data query interface of the first system according to the data number of the data to be synchronized, and check the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data.

[0095] During the actual execution of the conversion processing, due to the problem of processing efficiency, certain data to be synchronized is sent to the second system. While waiting for the conversion processing to be executed, the first system may have updated the data to be synchronized, which resulted in the data to be synchronized waiting to be processed in the second system not being the most recently updated data. Through the above setting, before executing the conversion processing step, the timestamp of the data to be synchronized is checked to ensure that the data to be processed is the most recently updated data (if the timestamp shows that the data to be synchronized with the data number is not the most recently updated data, it is necessary to call the data acquisition interface to obtain the most recently updated data corresponding to the data number), which can avoid unnecessary synchronization operations and significantly improve data synchronization efficiency.

[0096] The data conversion module 302 is used to convert the data to be synchronized according to the data table structure type and business requirements corresponding to the second system.

[0097] Exemplarily, according to an embodiment of the present invention, the cross-system data synchronization device 300 further includes a filtering module. Before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the filtering module is used to:

[0098] The business type of the data to be synchronized is determined, and the data to be synchronized is filtered according to the business type and filtering conditions of the data to be synchronized, wherein the filtering conditions are associated with the business requirements.

[0099] In the actual processing process, the second system does not need to synchronize all the data sent by the first system, but can flexibly select data of a specified business type for processing according to the business needs of the second system itself. Through the above settings, filter conditions are formulated according to business needs, and the data to be synchronized is filtered according to the filter conditions and the business type of the data to be synchronized, which can not only improve the efficiency of data synchronization, but also realize the flexibility of data synchronization settings.

[0100] Further, according to an embodiment of the present invention, the cross-system data synchronization device 300 further includes a business type determination module. Before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the business type determination module is used to:

[0101] The business type of the data to be synchronized is determined, and the conversion processing sequence and conversion processing cycle corresponding to the data to be synchronized of different business types are determined according to the business type of the data to be synchronized.

[0102] Through the above settings, determining the conversion processing order and conversion processing cycle corresponding to the data to be synchronized according to the business type is conducive to batch synchronization of the data to be synchronized of the same business type, further improving data synchronization efficiency.

[0103] According to a specific implementation of the embodiment of the present invention, in response to the service requirements of the second system, a synchronization priority may be set for the service type of the data to be synchronized, so as to implement a priority synchronization conversion of the data to be synchronized of the specified service type.

[0104] Preferably, according to an embodiment of the present invention, the above-mentioned conversion module is also used to: determine the conversion type corresponding to the data to be synchronized according to business needs, and perform conversion processing on the data to be synchronized according to the conversion type and the data table structure type corresponding to the second system, wherein the conversion processing includes new addition processing and / or modification and deletion processing.

[0105] On the one hand, the business requirements of the second system can specify the business type of the data to be synchronized that needs to be converted and processed by the second system, so as to filter the data to be synchronized and flexibly set the business type for synchronization; on the other hand, the conversion type of the data to be synchronized can be determined according to each business requirement. Specifically, the business requirements can specify that certain fields in a certain business type of data need to be converted and synchronized, and certain fields do not need to be converted and synchronized, and then it can be determined whether the conversion type of the data to be synchronized is to be added or modified (modification includes deletion).

[0106] According to an embodiment of the present invention, the data to be synchronized of different conversion types can be divided into two conversion processing queues, a new processing queue and a modification processing queue (including deletion, this part can also separate modification and deletion for conversion processing), and then the data to be synchronized in the same processing queue can perform the same processing operation, thereby realizing batch synchronization processing and improving data synchronization efficiency.

[0107] Addition modification refers to adding new data to the data table of the current system; modification processing refers to updating the updated data (changed data) in the first system to the data table of the current system.

[0108] Further, according to an embodiment of the present invention, the above-mentioned cross-system data synchronization device 300 also includes a lock operation module. After the step of determining the conversion type corresponding to the data to be synchronized according to business needs, the lock operation module is used to: perform a lock operation on the data to be synchronized according to the conversion type corresponding to the data to be synchronized.

[0109] Since the above statement mentions building different message processing queues, the above locking operation can avoid the situation where multiple conversion processors perform conversion processing on the same data to be synchronized, thereby causing processing exceptions.

[0110] The retry module 303 is used to determine whether the data to be synchronized is converted successfully. If so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted again to achieve cross-system data synchronization.

[0111] Through the above-mentioned setting, the embodiment of the present invention sends the data that fails to be converted to the retry queue, thereby realizing resynchronization processing from the inside (the second system). The internal retry operation will not be perceived by the outside, and various abnormal data (data that fails to synchronize) can also be processed internally through manual intervention. For example, when the number of retries reaches a certain threshold, manual intervention can be performed to avoid infinite retries affecting the synchronization efficiency.

[0112] It can be understood that before the retry queue performs conversion processing again, the retry module 303 is used to call the data query interface of the first system according to the data number of the data to be synchronized, so as to verify the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data.

[0113] Further, according to an embodiment of the present invention, the retry module 303 is also used for:

[0114] The conversion type corresponding to the to-be-synchronized data that failed to be converted is determined, and the to-be-synchronized data that failed to be converted is sent to a retry queue of the corresponding conversion type.

[0115] Through the above settings, for data that fails to be converted, multiple retry queues can be set according to their corresponding conversion types to improve data synchronization efficiency.

[0116] Preferably, according to an embodiment of the present invention, if the to-be-synchronized data is successfully converted, the status of the successfully converted to-be-synchronized data is updated to synchronized. The cross-system data synchronization device 300 further includes a proofreading module for:

[0117] Set a proofreading cycle and / or proofreading parameters, and call the data query interface according to the proofreading cycle and / or proofreading parameters to determine the amount of data to be converted in the first system within a specific period, and check whether the amount of data to be synchronized in the first system is consistent with the amount of converted data in the second system; if not, convert the inconsistent data to be synchronized.

[0118] By setting the proofreading cycle and / or proofreading parameters, the synchronized data can be proofread regularly. The proofreading parameters can be in the form of key fields, etc., which are used to detect whether all the data corresponding to the key fields have been synchronized, thereby improving the accuracy of cross-system data synchronization.

[0119] According to the technical solution of the embodiment of the present invention, because the data to be synchronized sent by the first system is received, the data query interface of the first system is called according to the data number of the data to be synchronized, and the timestamp of the data to be synchronized is checked to determine that the data to be synchronized corresponding to the data number is the most recently updated data; the data to be synchronized is converted and processed according to the data table structure type and business needs corresponding to the second system; it is determined whether the conversion of the data to be synchronized is successful, and if so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted and processed again to realize the technical means of cross-system data synchronization, so the technical problems of high difficulty coefficient, inability to flexibly set data synchronization according to business needs, and low data synchronization efficiency in the existing cross-system data synchronization methods are overcome, thereby achieving the technical effect of reducing the difficulty coefficient of data synchronization, realizing flexible setting of cross-system data synchronization, and effectively improving the efficiency of data synchronization.

[0120] Figure 4 An exemplary system architecture 400 is shown to which a cross-system data synchronization method or a cross-system data synchronization apparatus according to an embodiment of the present invention can be applied.

[0121] like Figure 4 As shown, the system architecture 400 may include terminal devices 401, 402, 403, a network 404 and a server 405 (this architecture is only an example, and the components included in the specific architecture may be adjusted according to the specific application). The network 404 is used to provide a medium for a communication link between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0122] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0123] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0124] The server 405 may be a server that provides various services, such as a background management server (only for example) that provides support for shopping websites browsed by users using the terminal devices 401, 402, and 403. The background management server may analyze and process the received data such as the data to be synchronized, and feed back the processing results (such as the converted data - only for example) to the terminal device.

[0125] It should be noted that the cross-system data synchronization method provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the cross-system data synchronization device is generally disposed in the server 405 .

[0126] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0127] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 5 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0128] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0129] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0130] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0131] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0132] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0133] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be arranged in a processor, for example, may be described as: a processor including a data receiving module, a data conversion module and a retry module. The names of these modules do not constitute limitations on the modules themselves in certain cases. For example, the data receiving module may also be described as "a module for receiving data to be synchronized sent by the first system, and calling the data query interface of the first system according to the data number of the data to be synchronized to check the timestamp of the data to be synchronized, so as to determine that the data to be synchronized corresponding to the data number is the most recently updated data".

[0134] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: receiving the data to be synchronized sent by the first system, calling the data query interface of the first system according to the data number of the data to be synchronized to check the timestamp of the data to be synchronized, so as to determine that the data to be synchronized corresponding to the data number is the most recently updated data; converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system; judging whether the data to be synchronized is successfully converted, and if so, determining that the cross-system data synchronization is completed; if not, sending the data to be synchronized that failed to be converted to the retry queue, and converting the data to be synchronized in the retry queue again to achieve cross-system data synchronization.

[0135] According to the technical solution of the embodiment of the present invention, because the data to be synchronized sent by the first system is received, the data query interface of the first system is called according to the data number of the data to be synchronized, and the timestamp of the data to be synchronized is checked to determine that the data to be synchronized corresponding to the data number is the most recently updated data; the data to be synchronized is converted and processed according to the data table structure type and business needs corresponding to the second system; it is determined whether the conversion of the data to be synchronized is successful, and if so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted and processed again to realize the technical means of cross-system data synchronization, so the technical problems of high difficulty coefficient, inability to flexibly set data synchronization according to business needs, and low data synchronization efficiency in the existing cross-system data synchronization methods are overcome, thereby achieving the technical effect of reducing the difficulty coefficient of data synchronization, improving the flexibility of cross-system data synchronization settings, and effectively improving data synchronization efficiency.

[0136] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A cross-system data synchronization method, characterized in that: include: Receiving the data to be synchronized sent by the first system, calling the data query interface of the first system according to the data number of the data to be synchronized, and checking the timestamp of the data to be synchronized to determine that the data to be synchronized corresponding to the data number is the most recently updated data; If the timestamp shows that the data to be synchronized with the data number is not the most recently updated data, it is necessary to call the data acquisition interface to acquire the most recently updated data corresponding to the data number; Convert the data to be synchronized according to the data table structure type and business requirements corresponding to the second system; Determine whether the data to be synchronized is converted successfully. If so, determine that the cross-system data synchronization is completed; if not, send the data to be synchronized that failed to be converted to the retry queue, and convert the data to be synchronized in the retry queue again to achieve cross-system data synchronization.

2. The cross-system data synchronization method according to claim 1, characterized in that: Before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the method further includes: The service type of the data to be synchronized is determined, and the data to be synchronized is filtered according to the service type and a filtering condition of the data to be synchronized, wherein the filtering condition is associated with the service requirement.

3. The cross-system data synchronization method according to claim 1, characterized in that: Before the step of converting the data to be synchronized according to the data table structure type and business requirements corresponding to the second system, the method further includes: The service type of the data to be synchronized is determined, and according to the service type of the data to be synchronized, a conversion processing sequence and a conversion processing cycle corresponding to the data to be synchronized of different service types are determined.

4. The cross-system data synchronization method according to claim 1, characterized in that: The step of performing data conversion processing on the data to be synchronized according to the data table structure type and business requirements corresponding to the second system includes: The conversion type corresponding to the data to be synchronized is determined according to the business requirements, and the data to be synchronized is converted according to the conversion type and the data table structure type corresponding to the second system, wherein the conversion processing includes adding processing and / or modifying processing.

5. The cross-system data synchronization method according to claim 4, characterized in that: After the step of determining the conversion type corresponding to the data to be synchronized according to the business requirements, the method further includes: performing a lock operation on the data to be synchronized according to the conversion type corresponding to the data to be synchronized.

6. The cross-system data synchronization method according to claim 4, characterized in that: Retry queues are constructed respectively according to the conversion types, and the step of sending the to-be-synchronized data that failed to be converted to the retry queues includes: The conversion type corresponding to the to-be-synchronized data that has failed to be converted is determined, and the to-be-synchronized data that has failed to be converted is sent to a retry queue of the corresponding conversion type.

7. The cross-system data synchronization method according to claim 1, characterized in that: If the data to be synchronized is successfully converted, the status of the successfully converted data to be synchronized is updated to synchronized; The method further comprises: Set a proofreading cycle and / or proofreading parameters, call the data query interface according to the proofreading cycle and / or proofreading parameters to determine the amount of data to be synchronized in the first system within a specific period, and check whether the amount of data to be synchronized in the first system is consistent with the amount of synchronized data in the second system; if inconsistent, convert the inconsistent data to be synchronized.

8. A cross-system data synchronization device, characterized in that: include: The data receiving module is used to receive the data to be synchronized sent by the first system, call the data query interface of the first system according to the data number of the data to be synchronized, and check the timestamp of the data to be synchronized to determine whether the data to be synchronized corresponding to the data number is the most recently updated data; and is also used to: if the timestamp shows that the data to be synchronized with the data number is not the most recently updated data, call the data acquisition interface to obtain the most recently updated data corresponding to the data number; A data conversion module, used to convert the data to be synchronized according to the data table structure type and business requirements corresponding to the second system; The retry module is used to determine whether the data to be synchronized is converted successfully. If so, it is determined that the cross-system data synchronization is completed; if not, the data to be synchronized that failed to be converted is sent to the retry queue, and the data to be synchronized in the retry queue is converted again to achieve cross-system data synchronization.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.