End-to-end data alignment medical data eventual consistency synchronization method and apparatus

By introducing a coordinator system to perform heartbeat detection and data alignment processes, the problem of data inconsistency in medical big data synchronization is solved, end-to-end data alignment is achieved, data accuracy and consistency is ensured, and big data applications and artificial intelligence analysis are supported.

CN119652908BActive Publication Date: 2025-07-18HANGZHOU COOPER MEDICAL TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510176052.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-02-18
Publication Date
2025-07-18
Estimated Expiration
2045-02-18

AI Technical Summary

Technical Problem

In the field of medical big data, there are problems of data inconsistency and missing in the end-to-end data synchronization process, which affects data application and governance, increases the difficulty of data collection and manpower investment, and reduces data accuracy.

Method used

The coordinator system is introduced. Through the process of heartbeat detection, pre-checking and data alignment, the coordinator is responsible for receiving participants' heartbeat requests and task script information, and the participants conduct self-tests and data alignment to ensure the accuracy and consistency of data synchronization.

Benefits of technology

It realizes end-to-end data alignment of medical data in a cloud center environment, ensures data accuracy and consistency, supports big data applications and artificial intelligence analysis, and improves the stability and accuracy of data acquisition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119652908B_ABST
    Figure CN119652908B_ABST
Patent Text Reader

Abstract

The present invention discloses an end-to-end data alignment method and device for medical data final consistency synchronization, belonging to the technical field of medical data synchronization, including: the coordinator performs heartbeat detection on the participants in the data synchronization process and issues a task script request instruction; the participant submits data collection task script information to the coordinator according to the task script request instruction, and the coordinator finds tasks with differences in the amount of read and written data based on the data collection task script information and issues data alignment task script information, and the participant performs self-checking according to the data alignment task script information; the participant pushes the corresponding data alignment task script information request to the coordinator according to the self-check result, the coordinator filters out the corresponding data alignment task script information according to the data alignment task script information request and sends it to the participant, and the participant performs data alignment. The present invention can achieve end-to-end data alignment in the medical data synchronization process, ensuring the accuracy and consistency of the data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of medical data synchronization, and particularly relates to an end-to-end data alignment medical data eventual consistency synchronization method and device. Background Art

[0002] In the context of the rapid development of medical informatization today, the processing and analysis of medical big data have become the key to improving the quality and efficiency of medical services. For relevant enterprises in the field of medical big data processing, it is necessary to efficiently and accurately collect a large amount of medical data in the regional lower-level medical systems (such as HIS, PASS, SAAS systems, etc.) into the analysis libraries (such as databases like TiDB, MongoDB, Elasticsearch, etc. and file storages like Hdfs, Oss, s3, etc.) in the cloud center network environment.

[0003] Regional lower-level medical systems generally use data storage media such as Oracle and MySQL to store a large amount of medical data. These data cover key information such as patient medical records, examination results, and medication records, and are of extremely high value for subsequent data fusion, data analysis, data quality control, artificial intelligence applications, medical auxiliary diagnosis applications, etc. However, due to the diversity of data storage media and the complexity of system architectures, collecting these data from the source libraries of regional lower-level medical systems to the analysis libraries in the cloud center faces the challenge of ensuring data synchronization consistency in the end-to-end data collection synchronization process.

[0004] In multiple end-to-end data synchronization and collection scenarios, first, it is necessary to execute the data collection process from the source library in the hospital to the backup library. Data collection is carried out in a table-to-table manner or by real-time incremental data log change capture (CDC) to reduce the risk of locking tables in the source library, thereby preventing interruptions or suspensions in the daily use of lower-level medical systems and avoiding affecting daily operations such as outpatient clinics or admissions in the hospital. Then, execute the data synchronization process from the backup library to multiple analysis libraries at the cloud center nodes and finally integrate them into the combined analysis library. Based on the backup library, full-volume and real-time data collection are performed. The full-volume and real-time incremental synchronization of data can achieve table-to-table, view-to-table, and SQL statement-to-table, with strong plasticity and high scalability, and support the operation of fusing the data of multiple single tables into a large wide table (using views or SQL JOIN methods to collect data into a large wide table in the analysis library TiDB), which provides great convenience for subsequent data processing and applications.

[0005] However, in actual operation, occasional network interruptions may occur during the real-time data collection process, the connection pool of the database is full, the fields of the data table in the source database are modified during reading, and there are abnormal situations such as length limitations of the fields written into the source data table. These abnormalities often lead to problems such as end-to-end (source read database and target write database) data inconsistency and data loss in the data synchronization and collection tasks, seriously affecting subsequent data applications, processing, and governance work. As a result, data governance needs to be reprocessed, more manpower needs to be invested in troubleshooting, increasing the difficulty and challenges of data collection, and greatly reducing the data accuracy rate of data collection.

[0006] Currently, end-to-end data synchronization consistency is a common problem faced by enterprises related to data solutions. In the field of medical big data, the accuracy and consistency of data are directly related to doctors' diagnostic decisions and patients' treatment effects. Therefore, solving the problem of data synchronization consistency and improving the accuracy and stability of data collection are of great significance for promoting the in-depth application of medical big data, and it is necessary to continuously explore and innovate data synchronization solutions. Summary of the Invention

[0007] In view of the above, the purpose of the present invention is to provide a method and device for end-to-end data alignment of medical data final consistency synchronization. Regarding the data synchronization and collection system as a participant role and creating a coordination system as a coordinator, through the processes of heartbeat detection, pre-check, and data alignment executed by the participant and the coordinator, end-to-end data alignment during the medical data synchronization process can be achieved, ensuring the accuracy and consistency of the data.

[0008] To achieve the above invention purpose, the technical solutions provided by the present invention are as follows:

[0009] In the first aspect, a method for end-to-end data alignment of medical data final consistency synchronization provided by an embodiment of the present invention includes the following steps:

[0010] In the heartbeat detection stage, the coordinator is responsible for performing heartbeat detection on all participants in the data synchronization process and issuing a task script request instruction.

[0011] In the pre-check stage, all participants submit data collection task script information to the coordinator according to the task script request instruction. The coordinator finds tasks with differences in the amount of read and written data based on the data collection task script information and issues data alignment task script information. The participants perform self-checks according to the data alignment task script information.

[0012] In the data alignment phase, the participants push the corresponding data alignment task script information requests to the coordinator according to the self-check results. The coordinator filters out the corresponding data alignment task script information based on the data alignment task script information requests and sends it to the participants. The participants perform data alignment according to the filtered data alignment task script information.

[0013] Preferably, the coordinator is responsible for performing heartbeat detection on all participants during the data synchronization process and issuing task script upload request instructions, including:

[0014] All participants in the data synchronization process send heartbeat requests to the coordinator. The coordinator receives the heartbeat requests and issues the task script request instructions for the corresponding participants into the topic of the message queue Kafka for the participants to subscribe to.

[0015] Preferably, all the participants submit the data collection task script information to the coordinator according to the task script request instructions. The coordinator finds the tasks with differences in the amount of read and written data based on the data collection task script information and issues the data alignment task script information, including:

[0016] After all participants subscribe to the task script request instructions in the topic of the message queue kafka, they submit the data collection task script information to the coordinator. The coordinator constructs a data collection task list based on the data collection task script information. The coordinator starts a timed task through the time executor to query the data collection task list, and finds one by one the tasks with differences in the amount of data read from the source library and written to the target library in the data collection task list. At the same time, the coordinator issues the data alignment task script information corresponding to the tasks with differences into the corresponding topic of the message queue Kafka. The participants subscribe to the data alignment task script information in the topic of the message queue Kafka.

[0017] Preferably, the participants perform self-check according to the data alignment task script information, including:

[0018] The participants start a self-check process for self-check according to the obtained data alignment task script information. Through the script information of the read source and write source in the data alignment task script information, the corresponding SQL query statements are executed through the API interface of JDBC to obtain the total count of the read source data table and the total count of the write source data table. If the total count of the read source data table is equal to the total count of the write source data table, it means that the data collection task is normal. If the total count of the read source data table is not equal to the total count of the write source data table, it means that the data collection task is abnormal and enters the data alignment phase.

[0019] Preferably, the participant pushes the corresponding data alignment task script information request to the coordinator according to the self-check result, and the coordinator filters out the corresponding data alignment task script information according to the data alignment task script information request and sends it to the participant. The participant performs data alignment according to the filtered data alignment task script information, including:

[0020] The participant sends the data alignment task script information request for data alignment to the coordinator according to its self-check result. The coordinator fills the data alignment tasks into the task queue according to the data alignment task script information request, filters out the data alignment task script information corresponding to the data alignment tasks one by one by executing SQL statements, and maintains the data alignment task status information in the status table. The coordinator regularly sends the data alignment task script information corresponding to the filtered data alignment tasks to the topic Topic of the message queue Kafka. After the participant subscribes to the data alignment task script information corresponding to the data alignment tasks under the topic Topic of the message queue Kafka, the participant performs the data alignment operation.

[0021] Preferably, when the participant performs the data alignment operation, it includes:

[0022] The participant creates a thread pool, and each thread in the thread pool executes the corresponding data alignment task. The data alignment tasks correspond to the threads one by one. The running state variables of the tasks are managed independently in a thread isolation manner, and the data alignment tasks are executed in parallel by multiple threads to complete the end-to-end data alignment task from reading the source to writing to the source.

[0023] Preferably, the corresponding state variable state is modified by using the keywords synchronize and volatile in JAVA to ensure the atomicity and visibility of the state variable cannot be damaged in a multi-threaded environment.

[0024] Preferably, all participants execute the corresponding threads and, according to the data alignment task script information, reread the source table under the source library in the form of JDBC to obtain the corresponding data records for the missing data, and batch write them into the table of the write source for storage.

[0025] Preferably, when the participant performs the data alignment operation, it also includes:

[0026] The participant submits the task running state variable of each data collection alignment task to the coordinator, and the coordinator writes the value of the task state variable into the corresponding status table for real-time task monitoring.

[0027] Second aspect, embodiments of the present invention further provide an end-to-end data alignment medical data eventual consistency synchronization device, which is implemented by using the above-mentioned end-to-end data alignment medical data eventual consistency synchronization method, and includes: a heartbeat detection module, a pre-check module, and a data alignment module;

[0028] The heartbeat detection module is used to be responsible for performing heartbeat detection on all participants in the data synchronization process through the coordinator and issuing a task script request instruction;

[0029] The pre-check module is used to enable all participants to submit data collection task script information to the coordinator according to the task script request instruction. The coordinator finds tasks with differences in the amount of read and written data based on the data collection task script information and issues data alignment task script information. The participants perform self-checks according to the data alignment task script information;

[0030] The data alignment module is used to enable the participants to push requests for corresponding data alignment task script information to the coordinator according to the self-check results. The coordinator filters out the corresponding data alignment task script information according to the data alignment task script information request and sends it to the participants. The participants perform data alignment according to the filtered data alignment task script information.

[0031] Compared with the prior art, the beneficial effects of the present invention at least include:

[0032] In a medical system with regional deployment in a local area network environment, where data is stored in the intranet environment of each region, the present invention realizes end-to-end data alignment in the medical data synchronization process by introducing a coordinator identity system, combining the processes of heartbeat detection, pre-check, and data alignment, ensuring the accuracy and consistency of data, and being able to efficiently synchronize and collect medical data into a unified analysis library in the cloud center environment, thereby supporting various scenarios such as big data applications, artificial intelligence analysis, and big data assisting doctors in diagnosis and treatment, showing significant application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 It is a flowchart of the end-to-end data alignment medical data eventual consistency synchronization method provided by the embodiments of the present invention;

[0035] Figure 2It is a schematic diagram of multiple end-to-end data synchronization and transfer scenarios provided by an embodiment of the present invention;

[0036] Figure 3 It is a schematic diagram of the heartbeat detection stage provided by an embodiment of the present invention;

[0037] Figure 4 It is a schematic diagram of the pre-check stage and the data alignment stage provided by an embodiment of the present invention;

[0038] Figure 5 It is a schematic structural diagram of a medical data end-to-end data alignment final consistency synchronization device provided by an embodiment of the present invention. Detailed implementation manners

[0039] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation manners described herein are only used to explain the present invention and do not limit the protection scope of the present invention.

[0040] The inventive concept of the present invention is: aiming at the problem that end-to-end data synchronization in the prior art is prone to inconsistency, an embodiment of the present invention provides a medical data end-to-end data alignment final consistency synchronization method and device. The coordinator undertakes the functions of receiving heartbeat requests pushed by all participants at regular intervals, receiving task script information data of all participants, and maintaining a task queue, a time executor, and a status table. Combining the two stages of pre-check and data alignment realizes end-to-end data alignment in the medical data synchronization process, ensuring the accuracy and consistency of the data.

[0041] To ensure the integrity and real-time nature of the data, in the embodiment, for multiple end-to-end data synchronization and transfer scenarios as Figure 2 shown, a CDC-based data synchronization and acquisition system is used to realize data reading and writing from the source library to the backup library in the regional network environment, and data reading and writing from multiple libraries in the analysis library to the combined analysis library in the cloud center network environment. A DataWorks-based data synchronization and acquisition system or a data synchronization and acquisition system based on secondary development of the XTL tool using Datax is used to realize data reading and writing from multiple libraries in the backup library to the analysis library. However, in this way, there is a high probability of end-to-end data inconsistency problems.

[0042] Figure 1 It is a flowchart of a medical data end-to-end data alignment final consistency synchronization method provided by an embodiment of the present invention. As Figure 1 shown, the embodiment provides a medical data end-to-end data alignment final consistency synchronization method, including the following steps:

[0043] S1. In the heartbeat detection phase, the coordinator is responsible for performing heartbeat detection on all participants in the data synchronization process and issuing a task script request instruction.

[0044] In the embodiment, the data synchronization acquisition system is used as the coordinator, and a coordination system is constructed as the coordinator through a JAVA application. As Figure 3 shown, the heartbeat detection phase includes:

[0045] S1.1. All participants in the data synchronization process send heartbeat requests to the coordinator, which is equivalent to reporting numbers, and set to report numbers regularly every 5 seconds.

[0046] S1.2. The coordinator receives the heartbeat request and issues the task script request instruction corresponding to the participant into the topic of the message queue Kafka.

[0047] S1.3. All participants subscribe to and consume the task script request instructions in the topic of the message queue Kafka.

[0048] S2. In the pre-check phase, all participants submit data acquisition task script information to the coordinator according to the task script request instruction. The coordinator finds tasks with differences in the amount of read and written data based on the data acquisition task script information and issues data alignment task script information. The participants perform self-checks according to the data alignment task script information.

[0049] S2.1. As Figure 4 shown, all participants in the data synchronization submit data acquisition task script information to the coordinator according to the subscribed task script request instruction. The coordinator constructs a data acquisition task list based on the data acquisition task script information. The data acquisition task script information varies according to different participants and is formulated according to specific scenarios. For example, for a data synchronization acquisition system that is a secondary development of the XTL tool based on Datax, the task script information data may include read source information, database name, table name, connection method, connection information, driver package type, table name, view script, SQL, as well as write source information, database name, table name, connection method, connection information, driver package type, write table name, etc. For a data synchronization acquisition system based on CDC, the task script information data may include read source information, database name, table name, connection information, connection method, driver package type, incremental field identifier, as well as write source information, database name, table name, connection information, connection method, driver package type.

[0050] S2.2. The coordinator starts a timing task through a Timer to query the data collection task list, and finds one by one the tasks in the data collection task list where there is a difference (the difference amount is greater than 0) in the data volume read from the source library and written to the target library, obtaining the data alignment task details as shown in Table 1. The data alignment task details include the source library read, the name of the source data table read, the source table data volume, the written data volume, the current difference, the name of the database written to, and the name of the data table written to.

[0051] Table 1 Data Alignment Task Details

[0052]

[0053] S2.3. The coordinator sends the data alignment task script information corresponding to the tasks with differences to the topic of the corresponding message queue Kafka.

[0054] S2.4. All participants subscribe to the topic of the message queue Kafka to obtain the data alignment task script information.

[0055] S2.5. The participant starts a self-checking process for self-checking according to the obtained data alignment task script information. Through the script information of the read source and write source in the data alignment task script information, the corresponding SQL query statement is executed through the API interface of JDBC (JAVA Database Connectivity). According to the executed query and statistical statement, the total count of the source data table read and the total count of the source data table written are obtained. If the total count of the source data table read is equal to the total count of the source data table written, it means that the data collection task is normal. If the total count of the source data table read is not equal to the total count of the source data table written, it means that the data collection task is abnormal and enters the data alignment stage.

[0056] S3. In the data alignment stage, the participant pushes the corresponding data alignment task script information request to the coordinator according to the self-check result. The coordinator filters out the corresponding data alignment task script information according to the data alignment task script information request and sends it to the participant. The participant performs data alignment according to the filtered data alignment task script information.

[0057] S3.1. As Figure 4 described above, for the self-check result that the data collection task is abnormal, it means that data alignment is required. The participant sends the data alignment task script information request corresponding to the data alignment required to the coordinator. The coordinator fills the data alignment task into the task queue according to the data alignment task script information request.

[0058] S3.2. The coordinator requests to execute the corresponding SQL statement according to the data alignment task script information, filters out the data alignment task script information corresponding to the data alignment task, and maintains the data alignment task status information in the status table. The status table includes the running status of the data alignment task, and the running status includes to-be-run, running, and completed, etc. The initialized data alignment task status is filled as the to-be-run status.

[0059] S3.3. The coordinator regularly sends the data alignment task script information corresponding to the data alignment task to the topic of the corresponding message queue Kafka through the time executor.

[0060] S3.4. After all participants subscribe to the data alignment task script information corresponding to the data alignment task under the topic of the message queue Kafka, they perform data alignment operations.

[0061] S3.5. The participant creates a thread pool, and each thread in the thread pool executes a corresponding data alignment task. The data alignment tasks correspond to the threads one by one. The running status variables of the tasks are managed independently through thread isolation. The corresponding status variables are modified with the keywords synchronize and volatile in JAVA to ensure the atomicity and visibility of the status variables cannot be damaged in a multi-threaded environment.

[0062] All participants execute the corresponding threads and according to the data alignment task script information, reread the source table under the source library in the form of JDBC for the missing time range and obtain the data records in the corresponding time interval, and batch write them into the table of the write source for storage. The data alignment tasks from the read source to the write source are completed in batches through multi-threaded parallel execution of the data alignment tasks.

[0063] S3.6. All participants submit the task status (state) variables of each data alignment task to the coordinator, and the coordinator writes the values of the task status variables (including running, completed, etc.) into the corresponding status table for real-time task monitoring.

[0064] Based on the same inventive concept, as Figure 5 shown, the embodiment of the present invention further provides an end-to-end data alignment medical data eventual consistency synchronization device 500, including: a heartbeat detection module 510, a pre-check module 520, and a data alignment module 530.

[0065] The heartbeat detection module 510 is responsible for performing heartbeat detection on all participants in the data synchronization process through the coordinator and issuing task script request instructions.

[0066] The pre-check module 520 is used to request all participants to submit data collection task script information to the coordinator according to the task script. The coordinator finds tasks with different read and write data volumes based on the data collection task script information and issues data alignment task script information. The participants perform self-checks according to the data alignment task script information.

[0067] The data alignment module 530 is used to request the corresponding data alignment task script information to be pushed to the coordinator by the participants according to the self-check results. The coordinator filters out the corresponding data alignment task script information according to the data alignment task script information request and sends it to the participants. The participants perform data alignment according to the filtered data alignment task script information.

[0068] It should be noted that the medical data end-to-end data alignment final consistency synchronization device provided in the above embodiment and a medical data end-to-end data alignment final consistency synchronization method belong to the same inventive concept. The specific implementation process can be seen in the embodiment of the medical data end-to-end data alignment final consistency synchronization method, which will not be elaborated here.

[0069] The above specific implementation manners have detailed the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, supplements, equivalent replacements, etc. made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An end-to-end data alignment method for eventually consistent synchronization of medical data, characterized in that It includes the following steps: In the heartbeat detection stage, the coordinator is responsible for performing heartbeat detection on all participants in the data synchronization process and issuing a task script request instruction; In the pre-check stage, all participants submit data collection task script information to the coordinator according to the task script request instruction. The coordinator finds tasks with different read and write data volumes based on the data collection task script information and issues data alignment task script information. The participants perform self-checks according to the data alignment task script information; In the data alignment stage, the participants push the data alignment task script information requests corresponding to the data to be aligned to the coordinator according to the self-check results. The coordinator filters out the corresponding data alignment task script information according to the data alignment task script information requests and sends it to the participants. The participants perform data alignment according to the filtered data alignment task script information.

2. The end-to-end data alignment-based medical data eventual consistency synchronization method according to claim 1, wherein The coordinator is responsible for performing heartbeat detection on all participants in the data synchronization process and issuing a task script upload request instruction, including: All participants in the data synchronization process send heartbeat requests to the coordinator. The coordinator receives the heartbeat requests and issues the task script request instructions for the corresponding participants into the topic of the message queue Kafka for the participants to subscribe.

3. The end-to-end data alignment medical data eventual consistency synchronization method according to claim 2, characterized in that, All the participants submit data collection task script information to the coordinator according to the task script request instruction. The coordinator finds tasks with different read and write data volumes based on the data collection task script information and issues data alignment task script information, including: After all participants subscribe to the task script request instructions in the topic of the message queue kafka, they submit data collection task script information to the coordinator. The coordinator constructs a data collection task list according to the data collection task script information. The coordinator starts a timing task through the time executor to query the data collection task list, and finds one by one the tasks with different data volumes read from the source library and written to the target library in the data collection task list. At the same time, the coordinator issues the data alignment task script information corresponding to the tasks with differences into the corresponding topic of the message queue Kafka. The participants subscribe to the data alignment task script information in the topic of the message queue Kafka.

4. The end-to-end data alignment-based medical data eventual consistency synchronization method according to claim 1, characterized in that The participants perform self-checks according to the data alignment task script information, including: The participants start a self-check process for self-checking according to the obtained data alignment task script information. Through the script information of the read source and write source in the data alignment task script information, they execute the corresponding SQL query statements through the API interface of JDBC to obtain the total count of the read source data table and the total count of the write source data table. If the total count of the read source data table is equal to the total count of the write source data table, it means that the data collection task is normal. If the total count of the read source data table is not equal to the total count of the write source data table, it means that the data collection task is abnormal and enters the data alignment stage.

5. The end-to-end data alignment-based medical data eventual consistency synchronization method according to claim 1 or 4, characterized in that The participant pushes the data alignment task script information request corresponding to the data alignment that needs to be performed to the coordinator according to the self-check result. The coordinator filters out the corresponding data alignment task script information according to the data alignment task script information request and sends it to the participant. The participant performs data alignment according to the filtered data alignment task script information, including: The participant sends the data alignment task script information request for the data alignment that needs to be performed to the coordinator according to its self-check result. The coordinator fills the data alignment tasks into the task queue according to the data alignment task script information request, filters out the data alignment task script information corresponding to the data alignment tasks one by one by executing SQL statements, and maintains the data alignment task status information in the status table. The coordinator regularly sends the data alignment task script information corresponding to the filtered data alignment tasks to the topic Topic of the message queue Kafka through the time executor. After subscribing to the data alignment task script information corresponding to the data alignment tasks under the topic Topic of the message queue Kafka, the participant performs the data alignment operation.

6. The end-to-end data alignment-based medical data eventual consistency synchronization method according to claim 1, wherein When the participant performs the data alignment operation, it includes: The participant creates a thread pool, and each thread in the thread pool executes the corresponding data alignment task. The data alignment tasks correspond to the threads one by one. The running state variables of the tasks are managed independently through thread isolation, and the data alignment tasks are executed in parallel through multiple threads to complete the end-to-end data alignment task from reading the source to writing the source.

7. The end-to-end data alignment-based method for eventually consistent synchronization of medical data according to claim 1, wherein The keywords synchronize and volatile in JAVA are used to modify the corresponding state variable state to ensure the atomicity and visibility of the state variable cannot be destroyed in a multi-threaded environment.

8. The end-to-end data alignment-based medical data eventual consistency synchronization method according to claim 1, wherein All participants execute the corresponding threads and, according to the data alignment task script information, reread the source table under the source library in the form of JDBC to obtain the corresponding data records for the missing data, and batch write them into the table of the writing source for storage.

9. The end-to-end data alignment based medical data eventual consistency synchronization method according to claim 1, characterized in that, When the participant performs the data alignment operation, it also includes: The participant submits the task running state variable of each data collection alignment task to the coordinator, and the coordinator writes the value of the task state variable into the corresponding status table for real-time task monitoring.

10. An end-to-end data alignment medical data eventual consistency synchronization device, implemented by using the end-to-end data alignment medical data eventual consistency synchronization method according to any one of claims 1-7, characterized in that, It includes: a heartbeat detection module, a pre-check module, and a data alignment module; The heartbeat detection module is responsible for performing heartbeat detection on all participants during the data synchronization process through the coordinator and issuing task script request instructions; The pre-check module is used for all participants to submit data collection task script information to the coordinator according to the task script request instructions. The coordinator finds the tasks with differences in the amount of data read and written based on the data collection task script information and issues data alignment task script information. The participant performs self-check according to the data alignment task script information. The data alignment module is used for the participant to push the data alignment task script information request corresponding to the data that needs to be aligned to the coordinator according to the self-check result. The coordinator filters out the corresponding data alignment task script information according to the data alignment task script information request and sends it to the participant. The participant performs data alignment according to the filtered data alignment task script information.

Citation Information

Patent Citations

  • Method and system for keep consistency of data under copy database environment

    CN101329670A

  • Transaction processing method, electronic equipment and computer readable storage medium

    CN115981800A