Data synchronization method and device, equipment and medium

Abstract

The embodiment of the invention relates to a data synchronization method and device, equipment and a medium, and the method is applied to a second system comprising a service instance, and comprises the steps: obtaining a data updating message of a first system through the service instance, and obtaining a data identifier based on the data updating message; acquiring a distributed lock corresponding to the data identifier through the service instance; obtaining target to-be-synchronized data corresponding to the data identifier from the first system through the service instance based on preset business logic; wherein the target to-be-synchronized data is one of a plurality of to-be-synchronized data formed by updating at least one data table in the first system; and synchronizing the second system based on the target to-be-synchronized data through the service instance, and releasing the distributed lock after the synchronization is finished. According to the embodiment of the invention, the distributed lock mode is adopted to ensure that the data consistency is always kept in the data synchronization process, and the data reliability is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a data synchronization method, device, equipment and medium. Background Art

[0002] Currently, in software development, data between different systems needs to be consistent. Especially in large enterprises, the databases of various systems often need to synchronize data with each other.

[0003] In the related art, data change messages of the target system are obtained by subscribing to binary log files and sent to the message queue, and other systems update their own databases by consuming the contents of the message queue. However, since the arrival order of messages cannot be guaranteed, if they are written directly to the database, data inconsistency may occur, and multiple service instances in the system may process messages of the same user at the same time, which will eventually lead to data being out of sync between the two systems. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a data synchronization method, apparatus, device and medium.

[0005] An embodiment of the present disclosure provides a data synchronization method, which is applied to a second system including a service instance, the method comprising: obtaining a data update message of a first system through the service instance, and obtaining a data identifier based on the data update information; obtaining a distributed lock corresponding to the data identifier through the service instance; obtaining target data to be synchronized corresponding to the data identifier from the first system through the service instance based on a preset business logic; wherein the target data to be synchronized is one of a plurality of data to be synchronized formed by updating at least one data table in the first system; synchronizing the second system based on the target data to be synchronized through the service instance, and releasing the distributed lock after the synchronization is completed.

[0006] Optionally, obtaining the distributed lock corresponding to the data identifier through the service instance includes: sending a distributed lock request including the data identifier to the distributed lock system through the service instance at a preset request frequency; and receiving the distributed lock sent by the distributed lock system through the service instance.

[0007] Optionally, the method further includes: obtaining an expiration time corresponding to the distributed lock through the service instance; and releasing the distributed lock when the service instance determines that the expiration time is met.

[0008] Optionally, obtaining the target data to be synchronized corresponding to the data identifier from the first system through the service instance based on preset business logic includes: determining the first field corresponding to the data identifier based on the business logic through the service instance; and querying the first database of the first system through a target interface through the service instance to obtain the field value of the first field as the target data to be synchronized.

[0009] Optionally, synchronizing the second system based on the target data to be synchronized through the service instance includes: obtaining a processing method that matches the business logic through the service instance, and processing the field value of the first field based on the processing method to obtain a processing result; synchronizing the field value of the second field in the second database of the second system based on the processing result through the service instance.

[0010] Optionally, the method also includes: obtaining data synchronization failure information through the service instance, querying preset business rules to obtain retry instructions and / or recording instructions; re-executing the synchronization of the second system based on the target data to be synchronized based on the retry instructions through the service instance, and / or recording the data synchronization failure information in the target storage location based on the recording instructions.

[0011] Optionally, after all the service instances are synchronized, the method further includes: obtaining first data information of a first database of the first system through the service instance; obtaining second data information of a second database of the second system through the service instance; comparing the first data information and the second data information for consistency through the service instance; when the first data information and the second data information are inconsistent, generating a prompt message and sending it to the target device for display.

[0012] The disclosed embodiment also provides a data synchronization device, which is applied to a second system including a service instance, and the device includes: a first acquisition module, which is used to obtain a data update message of the first system through the service instance, and obtain a data identifier based on the data update information; a second acquisition module, which is used to obtain a distributed lock corresponding to the data identifier through the service instance; a third acquisition module, which is used to obtain target data to be synchronized corresponding to the data identifier from the first system based on a preset business logic through the service instance; wherein the target data to be synchronized is one of multiple data to be synchronized formed by updating at least one data table in the first system; a synchronization module, which is used to synchronize the second system based on the target data to be synchronized through the service instance; and a release module, which is used to release the distributed lock after the synchronization is completed.

[0013] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the data synchronization method provided by the embodiment of the present disclosure.

[0014] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the data synchronization method provided by the embodiment of the present disclosure.

[0015] The embodiment of the present disclosure also provides a computer program product, including a computer program, wherein the computer program is executed by a processor as the data synchronization method provided in the embodiment of the present application.

[0016] The above technical solution provided by the embodiment of the present disclosure ensures that data update messages with the same data identifier are processed by only one service instance at the same time through distributed locks, which can effectively avoid data coverage and inconsistency problems caused by concurrent processing and ensure the accuracy and consistency of the data; in addition, the target data to be synchronized can be obtained according to the pre-set business logic for synchronization processing, further meeting the flexibility and diversity of business scenarios, and ensuring data synchronization consistency while meeting the needs of diverse business scenarios.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1A A data synchronization process diagram provided for related technologies;

[0021] Figure 1B A data synchronization process diagram provided for related technologies;

[0022] Figure 1C A data synchronization process diagram provided for related technologies;

[0023] Figure 2 A flowchart of a data synchronization method provided by an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of a data synchronization process provided by an embodiment of the present disclosure;

[0025] Figure 4 A flowchart of a data synchronization method provided by an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of a data synchronization process provided by an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of a data synchronization device provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0031] In existing software development scenarios, data between different systems needs to be consistent. For example, changes to certain fields in system A also need to be kept up to date in another system B. At this time, by monitoring changes to certain fields in the database of system A and reflecting these changes in the database of system B, the aforementioned synchronization scenario is a common requirement.

[0032] For easier understanding, please refer to Figure 1A A data synchronization process diagram provided by the related technology shown in FIG. Figure 1A As shown in the figure, field 2 in a table of system A data changes, and sends a change message to the message queue. After receiving the message, system B updates its own data related fields, such as Figure 1A The field 3.

[0033] However, in the related art, since the arrival order of messages cannot be guaranteed, if they are written directly into the database, data inconsistency may occur, and multiple instances in system B may process data update messages of the same user at the same time, further increasing the complexity of data processing.

[0034] For easier understanding, please refer to Figure 1B A data synchronization process diagram provided by the related technology shown in FIG. Figure 1B As shown in the figure, the data tables of the two systems use the user ID (Identifier) ​​as the unique key. A business field of user 123 of system A is changed twice in a short period of time. Figure 1B In the message queue, the business value of user 123 of the first message is 9, and the business value of user 123 of the second message is 10. Since the order of message arrival cannot be guaranteed, for example, the system first writes 10 into the business value of user 123 and then writes 9 into the business value of user 123. In other words, a value of the same data in system A is updated multiple times in a short period of time. After the message is sent, system B listens to the message for processing. Due to the disorder of messages and multiple instances of consumers, the values ​​of the two systems are finally different.

[0035] Specifically, Figure 1B There is no version number or other fields in the message to distinguish the order of messages, so the above Figure 1B The scenario in the above example can be solved by using a version number. That is, every time the data of system A is changed, the version number is increased by one, and the version number information is also sent to the message. When system B receives the message for processing, it records the version number and only processes messages with a larger version number each time to avoid the above problem.

[0036] However, for slightly more complex situations, version numbers cannot solve this problem. For easier understanding, please refer to Figure 1C A data synchronization process diagram provided by the related technology shown in FIG. Figure 1C As shown in the figure, system A uses user ID plus membership type as the unique key, and system B uses user ID as the unique key. A business field of user 123 of system A is changed twice in a short period of time, such as Figure 1CThe business value of user 123 in the first message in the message queue is 10, and the business value of user 123 in the second message is 20. Since the order of message arrival cannot be guaranteed, for example, the system first writes 20 into the business value of user 123 and then writes 10 into the business value of user 123. In other words, a business value of multiple membership records of the same user in system A may change at the same time in a short period of time. System B only cares about the largest business value among these multiple membership data. If the message with a larger business value is processed first and the message with a smaller business value is processed later, the final value in the database will be incorrect. Figure 1C The data of each membership type of users in system A only changes once, so using the version number cannot solve the problem.

[0037] In response to the data consistency problem that occurs during database synchronization, that is, due to the disordered arrival of messages and the concurrent processing of multiple service instances, multiple data records of the same user may be confused when being written, thereby affecting the accuracy of the final data. To this end, the present disclosure proposes a data synchronization method that implements database monitoring conversion and synchronization of data consistency through distributed locks. That is, when this method processes multiple data changes of users of the above-mentioned example system A, after system B receives the message, it first obtains the distributed lock to ensure that only one thread operates at the same time, and then, for example, queries the relevant data of the user in the current database of system A through an interface, determines the target data to be synchronized, updates the database, and ensures data consistency and accuracy. The following is a detailed explanation:

[0038] Figure 2 The flowchart of a data synchronization method provided by the embodiment of the present disclosure is as follows. The method can be applied to a second system including a service instance, and the second system can be, for example, a computer, a distributed server cluster, etc., which is not limited here. Figure 2 As shown, the method mainly includes the following steps S202 to S208:

[0039] Step S202: obtaining a data update message of the first system through the service instance, and obtaining a data identifier based on the data update information.

[0040] In the embodiment of the present disclosure, the second system may be a distributed system including multiple service instances, and there may be multiple data update messages at the same time. Therefore, multiple service instances may process data update messages with the same data identifier at the same time, that is, the number of service instances that obtain data update messages of the first system may be one or more; a service instance may be understood as a server node in the second system.

[0041] In the embodiments of the present disclosure, the first system refers to one or more data tables in the database of the system, and the field values ​​of one or more first fields in each data table change; the second system refers to the system to be synchronized, that is, the system that needs to be synchronized and updated based on the data of the first system; the data identifier can uniquely identify a data.

[0042] In the disclosed embodiment, the first system may be monitored, and after data update occurs in the first system, the service instance obtains data update information; alternatively, after data update occurs, the first system may generate data update information and send it to one or more service instances of the second system.

[0043] In the embodiment of the present disclosure, the data update message includes a data identifier such as a user identifier used to uniquely identify a piece of data. The data identifier obtained by the data update information can be understood as a unique identifier of an updated data. It can also include field values ​​of other fields such as business type, business value, etc. The specific settings are selected according to the actual application. After the service instance obtains the data update information, it can parse the data update information to obtain the data identifier.

[0044] Step S204: Obtain the distributed lock corresponding to the data identifier through the service instance.

[0045] In the disclosed embodiment, different data identifiers can be pre-set to correspond to each distributed lock, that is, each data identifier has only one distributed lock stored in the distributed lock system, and a mapping relationship between the data identifier and the distributed lock can be established, that is, a one-to-one correspondence, so that a distributed lock request including the data identifier can be sent to the distributed lock system through the service instance, and the distributed lock sent by the distributed lock system can be received through the service instance.

[0046] Step S206: acquiring the target data to be synchronized corresponding to the data identifier from the first system based on the preset business logic through the service instance.

[0047] In the embodiment of the present disclosure, the second system may include one or more service instances, but only one service instance obtains the distributed lock. The service instance that obtains the distributed lock can obtain the target data to be synchronized corresponding to the data identifier from the first system based on the business logic; wherein the business logic is set in advance according to the application scenario requirements, such as calculating the maximum or minimum field value corresponding to a certain field.

[0048] It can be understood that one or more data tables in the database of the first system and the field values ​​of one or more first fields in each data table change to form data to be synchronized. Therefore, there may be one or more data tables updated in the first system to form multiple data to be synchronized, and the target data to be synchronized is one of the multiple data to be synchronized.

[0049] In the embodiments of the present disclosure, different business logics obtain different target data to be synchronized corresponding to data identifiers. For example, one or more first fields corresponding to the data identifier are determined based on the business logic through a service instance, and the first database of the first system is queried through the target interface to obtain the field value of the first field. For another example, one or more first fields corresponding to the data identifier are determined based on the business logic through a service instance, the first field is sent to the first system, and the field value corresponding to the first field sent by the first system is received.

[0050] Step S208: synchronize the second system based on the target data to be synchronized through the service instance, and release the distributed lock after the synchronization is completed.

[0051] In an embodiment of the present disclosure, after acquiring the target data to be synchronized, the second system is synchronized based on the target data to be synchronized through a service instance. For example, a processing method matching the business logic is acquired through the service instance to process the field value based on the first field to obtain a processing result, and the field value of the second field in the second database of the second system is synchronized based on the processing result through the service instance; for another example, the field value of the second field in the second database of the second system is synchronized directly based on the field value of the first field.

[0052] In the embodiment of the present disclosure, the distributed lock is released after the synchronization is completed, that is, data synchronization processing is performed after the service instance obtains the distributed lock, and only the service instance that obtains the distributed lock synchronizes the data of the data identifier, and other service instances that have not obtained the distributed lock cannot synchronize the data of the data identifier, and the distributed lock is released after the synchronization is completed. Then the service instance or other service instances can re-acquire the distributed lock of the data identifier, and the service instance that successfully obtains the distributed lock updates the data of the data identifier, ensuring that the distributed lock is released after all operations to avoid occupying the distributed lock resources for a long time, further improving the stability of data synchronization.

[0053] In the above embodiment, a distributed lock is used to ensure that data update messages with the same data identifier are processed by only one service instance at the same time, which can effectively avoid data coverage and inconsistency problems caused by concurrent processing and ensure the accuracy and consistency of the data; in addition, the target data to be synchronized can be obtained according to the pre-set business logic for synchronization processing, further meeting the flexibility and diversity of business scenarios, and ensuring data synchronization consistency while meeting the needs of diverse business scenarios.

[0054] In some embodiments, obtaining a distributed lock corresponding to a data identifier through a service instance includes: sending a distributed lock request including a data identifier to a distributed lock system through the service instance at a preset request frequency, and receiving a distributed lock sent by the distributed lock system through the service instance.

[0055] In the embodiment of the present disclosure, there are multiple data update messages. Therefore, after multiple service instances in the second system obtain the data update message of the first system and obtain the data identifier from the data update message, there are multiple service instances that simultaneously send distributed lock requests including the data identifier to the distributed lock system, but only one service instance receives the distributed lock sent by the distributed lock system to ensure that only one data update message can be processed at the same time for the same data identifier, thereby avoiding data conflicts and inconsistencies caused by concurrent processing, and further ensuring the accuracy and consistency of the data.

[0056] In the embodiment of the present disclosure, the distributed lock system may be, for example, Redisson (a Java in-memory data grid based on Redis (RemoteDictionary Server)), and the distributed locks corresponding to each data identifier are pre-stored in the distributed system, and each data identifier includes only one one-to-one distributed lock. For example, the data identifier and the distributed lock may be stored in the form of a mapping table, so as to match the data identifier in the mapping table to obtain the corresponding distributed lock.

[0057] In the disclosed embodiment, the request frequency is preset, such as a number of requests per second, so that the acquisition method of the distributed lock adopts a retry mechanism to ensure that the distributed lock can be successfully acquired under high concurrency conditions, thereby further improving the efficiency and effect of data synchronization.

[0058] In the above embodiment, the role of distributed locks is used to ensure that data update messages with the same data identifier are processed by only one service instance at the same time, and the distributed locks are obtained according to a certain request frequency to ensure that they can be successfully acquired under high concurrency conditions, thereby ensuring the efficiency and effectiveness of data synchronization.

[0059] In some embodiments, the method further includes: obtaining an expiration time corresponding to the distributed lock through a service instance, and releasing the distributed lock when the service instance determines that the expiration time is met.

[0060] In an embodiment of the present disclosure, the expiration time of the distributed lock corresponding to each data identifier can be pre-set. The expiration time refers to the survival time of the distributed lock after the service instance acquires the distributed lock, such as one minute. The timing starts from the time when the service instance acquires the distributed lock. When the expiration time, such as one minute, is met, the distributed lock needs to be released regardless of whether the synchronization is completed or not.

[0061] In the above embodiment, by setting the expiration time of the distributed lock, it is avoided that the distributed lock is locked due to anomalies, and the reliability of data synchronization is further guaranteed.

[0062] In some embodiments, the target data to be synchronized corresponding to the data identifier is obtained from the first system based on preset business logic through a service instance, including: determining the first field corresponding to the data identifier based on the business logic through the service instance, and querying the first database of the first system through the target interface through the service instance to obtain the field value of the first field.

[0063] In some embodiments, the second system is synchronized based on the target data to be synchronized through a service instance, including: obtaining a processing method that matches the business logic through the service instance, and processing the field value of the first field based on the processing method to obtain a processing result, and synchronizing the field value of the second field in the second database of the second system based on the processing result through the service instance.

[0064] In the embodiment of the present disclosure, the data table of the first system includes one or more first fields such as member type, business value 1 and business value 2, etc. The corresponding business logic can be set in advance according to the business scenario, so that when synchronizing data, it is determined according to the business logic which first field to obtain. For example, the business logic is the maximum value of business value 1, so that the first field corresponding to the data identifier is determined to be business value 1, and the first database of the first system is queried through the service instance through the target interface to obtain the field value of business value 1.

[0065] In the embodiment of the present disclosure, the service instance determines a processing method based on business logic, such as obtaining a maximum value, obtaining a minimum value, etc., and processes the field value of the first field based on the processing method to obtain a processing result. The service instance synchronizes the field value of the second field in the second database of the second system based on the processing result.

[0066] In the embodiment of the present disclosure, the data table of the second system includes one or more second fields such as member type, business value 3, etc. For example, all field values ​​corresponding to the first field are processed to obtain the maximum value, and the maximum value is synchronized to the field value of business value 3.

[0067] In the above embodiments, the business logic calculation rules can be flexibly adjusted according to specific business needs to support a variety of data processing scenarios, so that data synchronization can be widely applied to different business scenarios to meet diverse needs.

[0068] In some embodiments, the method further includes: obtaining data synchronization failure information through a service instance, querying preset business rules to obtain retry instructions and / or recording instructions, re-executing synchronization of the second system based on the target data to be synchronized based on the retry instructions through the service instance, and / or recording the data synchronization failure information in the target storage location based on the recording instructions.

[0069] In the embodiments of the present disclosure, data synchronization failure information refers to information about data synchronization failure of a service instance. Business rules are pre-set, such as re-updating when data synchronization fails, recording data synchronization failure information when data synchronization fails, etc. Business rules are set specifically according to the needs of the application scenario.

[0070] In an embodiment of the present disclosure, a retry instruction is obtained after the service instance obtains data synchronization failure information, and synchronization of the second system based on the target data to be synchronized is re-executed based on the retry instruction. A recording instruction can also be obtained after the service instance obtains data synchronization failure information, and the data synchronization failure information is recorded in a target storage location based on the recording instruction; wherein the target storage location can be selected and set in advance according to needs for subsequent viewing and analysis.

[0071] In the above embodiment, after the data synchronization update fails, it can be selected whether to retry or record the error according to the business rules, so as to further meet the data synchronization requirements.

[0072] In some embodiments, after all service instances have completed synchronization, the method further includes: obtaining first data information of a first database of a first system through a service instance, obtaining second data information of a second database of a second system through a service instance, comparing the first data information and the second data information for consistency through the service instance, and when the first data information and the second data information are inconsistent, generating a prompt message and sending it to the target device for display.

[0073] In the disclosed embodiment, after data synchronization is completed, in order to further improve the reliability of the data, a consistency comparison can be performed on the first data information of the first database and the second data information of the second database. In the event of inconsistency, a prompt message is generated and sent to the target device for display to prompt the user; wherein the target device can be a mobile phone, a computer, a visual monitoring panel, etc.

[0074] In the disclosed embodiments, performance indicators such as the response time of the aforementioned target interface query, data processing status such as data consistency, etc. can also be displayed in real time on the visual monitoring panel. An automatic alarm mechanism can also be set up, such as automatic alarm when the response time is greater than a preset time threshold, so as to respond to potential problems and failures in a timely manner.

[0075] The disclosed embodiment utilizes a distributed lock mechanism to ensure data consistency during the data synchronization process. That is, when a change message is received from the first system database, a distributed lock is first used to ensure that messages of the same user are processed by only one thread at the same time. This lock mechanism can effectively avoid data coverage and inconsistency problems caused by concurrent processing. After the second system acquires the distributed lock, the second system will query all relevant records of the user in the current database, and perform calculations and conversions according to predefined business logic, such as taking the maximum value, minimum value or other business rules. In this way, no matter how the messages arrive, the second system can always calculate the correct final value and update the calculation results to the database, thereby ensuring the accuracy and consistency of the data.

[0076] For ease of understanding, reference can also be made to the embodiments of the present disclosure. Figure 3 A data synchronization process diagram shown in FIG. Figure 3 In this paper, system A and system B are used as examples. Figure 1A-Figure 1C The disclosed embodiment ensures that data consistency is always maintained during the data synchronization process through distributed locks and logical computing mechanisms, especially when data of multiple services need to be synchronized in real time, and can effectively avoid errors caused by data conflicts.

[0077] Specifically, use message middleware (such as RocketMQ) to configure the listener of database change messages of system A, determine the database tables and fields to be monitored, create a message handler, set the topic or queue to receive messages, etc. Figure 3 As shown, monitor a table and field in the system A database, obtain the field value when a business value changes, then create a message handler and set the message queue as follows Figure 3 As shown, the first message data identifier is user ID 123, the field is a certain business value and the field value is 10, and the second message data identifier is user ID 123, the field is a certain business value and the field value is 20.

[0078] Specifically, when the service instance of system B captures the data change message, it parses the message content and extracts the user ID and other related information. Figure 3 As shown, for the service instance of system B, when obtaining a message message (data update message of the first system), it can be understood that the messages are out of order and the service instance processes the messages at different speeds. The message message can be Figure 3The first message in the message queue can also be the second message, and then the user identifier is obtained based on the message message, thereby obtaining the distributed lock corresponding to the user identifier. It can be understood that, for example, the service instance B1 in system B first obtains the first message to request a distributed lock from the distributed system, and then the service instance B2 obtains the second message to request a distributed lock from the distributed system. Finally, the service instance B2 obtains the distributed lock, and then the service instance B2 performs subsequent data synchronization, that is, update processing; that is, data synchronization is performed by obtaining the service instance of the distributed lock, so that the service instance queries the maximum business value of all member types of the user identifier (uid) and updates the database of system B, and finally releases the distributed lock.

[0079] Among them, necessary data conversion or calculation can be performed according to the business scenario. For example, calculation involving multiple fields can be performed to ensure that the logic is correct, such as Figure 3 As shown, the new value calculated by the interface query is updated in the system B database, and when the database update is completed, the distributed lock is released to ensure that the lock is released after all operations to avoid occupying the lock resources for a long time. If the update fails, whether to retry or record the error is selected according to the business rules.

[0080] Figure 4 A schematic diagram of a data synchronization method provided in an embodiment of the present disclosure is provided. The method mainly includes the following steps S402 to S408:

[0081] Step S402: obtaining a data update message of the first system through the service instance, and obtaining a data identifier based on the data update information.

[0082] Step S404: Send a distributed lock request including a data identifier to the distributed lock system through the service instance according to a preset request frequency, and receive a distributed lock sent by the distributed lock system through the service instance.

[0083] Step S406: Determine the first field corresponding to the data identifier based on the business logic through the service instance, and query the first database of the first system through the target interface through the service instance to obtain the field value of the first field.

[0084] Step S408, obtain a processing method that matches the business logic through the service instance, and process the field value of the first field based on the processing method to obtain a processing result, synchronize the field value of the second field in the second database of the second system based on the processing result through the service instance, and release the distributed lock after the synchronization is completed.

[0085] The disclosed embodiment adopts a distributed lock mechanism to ensure that only one thread processes the message of a specific user at the same time, effectively avoiding the data overwrite problem caused by concurrent processing and improving the reliability of data consistency protection. In addition, the logical calculation rules can be flexibly adjusted according to specific business needs, supporting a variety of data processing scenarios, and can be widely used in different business scenarios to meet diverse needs.

[0086] With the above Figure 3 For example, if the data of two membership types of user 123 in system A is changed, two messages will be sent. Due to the distributed lock, the service instance of system B can only process one message at a time.

[0087] Specifically, Figure 5 As shown, as an example, if the database of system A has updated the data before processing the first data change message (i.e. the aforementioned data update information), then the service instance of system B will use accurate values ​​each time it processes the message, because the maximum business value of all members queried each time is the same.

[0088] As another example, if only the business value of gold members has changed when processing the message, and there is no data for platinum members, then the data in system B is updated to 10. When the platinum data is stored, system B receives the message and queries the system A interface to obtain the maximum business value of user 123. It finds that it is 20 and stores it in the database of system A. In this scenario, after updating system B twice, the data in the databases of the two systems are consistent.

[0089] As another example, if only the business value of Platinum members has changed when processing the message, and there is no data for Gold members, then the data of System B is updated to 20. When the Gold data is stored, System B receives the message and queries System A interface to obtain the maximum business value of user 123. It finds that it is still 20, so it is stored in the database of System B. In this scenario, after System B is updated twice, the data in the databases of the two systems are consistent.

[0090] To sum up, the data synchronization method provided by the embodiment of the present disclosure ensures that data consistency is always maintained during the data synchronization process through distributed locks and logical computing mechanisms. Especially when the data of multiple departments or services need to be synchronized in real time, errors caused by data conflicts can be effectively avoided, and data consistency can be improved. The data processing logic can be flexibly adjusted according to specific needs, supporting diverse business scenarios, meeting the flexibility of data processing, and promoting rapid business response.

[0091] Corresponding to the aforementioned data synchronization method, the embodiment of the present disclosure further provides a data synchronization device, Figure 6This is a schematic diagram of the structure of a data synchronization device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and is applied to a second system including a service instance. The device includes:

[0092] A first acquisition module 602, configured to acquire a data update message of the first system through the service instance, and acquire a data identifier based on the data update information;

[0093] A second acquisition module 604 is used to acquire the distributed lock corresponding to the data identifier through the service instance;

[0094] A third acquisition module 606 is used to acquire the target data to be synchronized corresponding to the data identifier from the first system based on the preset business logic through the service instance; wherein the target data to be synchronized is one of the multiple data to be synchronized formed by updating at least one data table in the first system;

[0095] A synchronization module 608, configured to synchronize the second system based on the target data to be synchronized through the service instance;

[0096] The release module 610 is used to release the distributed lock after the synchronization is completed.

[0097] The above-mentioned device provided by the embodiment of the present disclosure ensures that data update messages with the same data identifier are processed by only one service instance at the same time through a distributed lock, which can effectively avoid data coverage and inconsistency problems caused by concurrent processing and ensure the accuracy and consistency of the data; in addition, the target data to be synchronized can be obtained according to the pre-set business logic for synchronization processing, further meeting the flexibility and diversity of business scenarios, and ensuring data synchronization consistency while meeting the needs of diverse business scenarios.

[0098] In some implementations, the second acquisition module 604 is specifically used to: send a distributed lock request including the data identifier to the distributed lock system through the service instance at a preset request frequency; and receive the distributed lock sent by the distributed lock system through the service instance.

[0099] In some embodiments, the device further includes: a fourth acquisition module, used to obtain the expiration time corresponding to the distributed lock through the service instance; a release module 610, also used to release the distributed lock when it is determined through the service instance that the expiration time is met.

[0100] In some embodiments, the third acquisition module 606 is specifically used to: determine the first field corresponding to the data identifier based on the business logic through the service instance; and obtain the field value of the first field by querying the first database of the first system through the target interface through the service instance.

[0101] In some embodiments, the synchronization module 610 is specifically the same as: obtaining a processing method that matches the business logic through the service instance, and processing the field value of the first field based on the processing method to obtain a processing result; synchronizing the field value of the second field in the second database of the second system based on the processing result through the service instance.

[0102] In some embodiments, the device also includes: an acquisition query module, used to obtain data synchronization failure information through the service instance, and query preset business rules to obtain retry instructions and / or recording instructions; a processing module, used to re-execute the synchronization of the second system based on the target data to be synchronized based on the retry instructions through the service instance, and / or record the data synchronization failure information in the target storage location based on the recording instructions.

[0103] In some embodiments, after all the service instances are synchronized, the device further includes: a fifth acquisition module, used to acquire first data information of the first database of the first system through the service instance; a sixth acquisition module, used to acquire second data information of the second database of the second system through the service instance; a comparison module, used to compare the first data information and the second data information for consistency through the service instance; and a prompt module, used to generate a prompt message and send it to the target device for display when the first data information and the second data information are inconsistent.

[0104] The data synchronization device provided in the embodiments of the present disclosure can execute the data synchronization method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0106] An embodiment of the present disclosure provides an electronic device, which includes: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of any method in the present disclosure.

[0107] Reference below Figure 7, which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0108] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0109] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0110] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0111] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to perform the image processing method provided by the embodiments of the present disclosure. The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present disclosure, the programming languages ​​including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0112] In addition, the embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the data synchronization method provided by the embodiment of the present disclosure.

[0113] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0114] The embodiment of the present disclosure also provides a computer program product, including a computer program / instruction, which implements the data synchronization method in the embodiment of the present disclosure when executed by a processor.

[0115] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0116] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0117] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0118] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0119] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0120] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data synchronization method, characterized in that: Applied to a second system including a service instance, the method comprises: Obtaining a data update message of the first system through the service instance, and obtaining a data identifier based on the data update information; Obtaining the distributed lock corresponding to the data identifier through the service instance; Acquire the target data to be synchronized corresponding to the data identifier from the first system through the service instance based on the preset business logic; wherein the target data to be synchronized is one of the multiple data to be synchronized formed by updating at least one data table in the first system; The second system is synchronized based on the target data to be synchronized through the service instance, and the distributed lock is released after the synchronization is completed.

2. The method according to claim 1, characterized in that The acquiring the distributed lock corresponding to the data identifier through the service instance includes: Sending a distributed lock request including the data identifier to the distributed lock system through the service instance according to a preset request frequency; The distributed lock sent by the distributed lock system is received through the service instance.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining the expiration time corresponding to the distributed lock through the service instance; When determining that the expiration time is met, the service instance releases the distributed lock.

4. The method according to claim 1, characterized in that: The acquiring, through the service instance based on a preset business logic, the target data to be synchronized corresponding to the data identifier from the first system includes: Determine, by the service instance and based on the business logic, a first field corresponding to the data identifier; The service instance queries the first database of the first system through a target interface to obtain the field value of the first field as the target data to be synchronized.

5. The method according to claim 4, characterized in that The synchronizing the second system based on the target data to be synchronized by using the service instance includes: Acquire a processing method matching the business logic through the service instance, and process the field value of the first field based on the processing method to obtain a processing result; The field value of the second field in the second database of the second system is synchronized based on the processing result through the service instance.

6. The method according to claim 1, characterized in that The method further comprises: Obtain data synchronization failure information through the service instance, and query preset business rules to obtain retry instructions and / or record instructions; The service instance re-executes synchronization of the second system based on the target data to be synchronized based on the retry instruction, and / or records the data synchronization failure information in a target storage location based on the recording instruction.

7. The method according to claim 1, characterized in that After all the service instances are synchronized, the method further includes: Acquire first data information of a first database of the first system through the service instance; Acquire second data information of a second database of the second system through the service instance; Performing a consistency comparison between the first data information and the second data information through the service instance; When the first data information and the second data information are inconsistent, a prompt message is generated and sent to the target device for display.

8. A data synchronization device, characterized in that: Applied to a second system including a service instance, the apparatus comprises: A first acquisition module, configured to acquire a data update message of the first system through the service instance, and acquire a data identifier based on the data update information; A second acquisition module, used to acquire the distributed lock corresponding to the data identifier through the service instance; A third acquisition module is used to acquire the target data to be synchronized corresponding to the data identifier from the first system based on the preset business logic through the service instance; wherein the target data to be synchronized is one of the multiple data to be synchronized formed by updating at least one data table in the first system; A synchronization module, configured to synchronize the second system based on the target data to be synchronized through the service instance; The release module is used to release the distributed lock after the synchronization is completed.

9. An electronic device, characterized in that: The electronic device comprises: a storage device having a computer program stored thereon; A processing device, used to execute the computer program in the storage device to implement the steps of the data synchronization method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the data synchronization method described in any one of claims 1 to 7.

Images