Medical data real-time synchronization method, system and device and medium

Abstract

The invention provides a medical data real-time synchronization method, system and device and a medium, and belongs to the technical field of medical data processing. A data source, a kafka module and a target database are configured; judging whether the test connection is normal; if normal, configuring a real-time synchronization task; judging whether the data source supports the capture of the CDC tool or not; if so, monitoring the change of the data source, and sending change information to a Kafka module; judging whether a follow-up processing flow is self-defined or not; and if a subsequent processing flow is self-defined, the user monitors and processes the Kafka message by himself / herself. In various heterogeneous database environments, the most suitable synchronization strategy can be selected according to the change data capture capability of different databases. And it is ensured that the data can meet the requirements of the target database after being synchronized to the target database and meet the specific medical service requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical data processing, and in particular relates to a medical data real-time synchronization method, system, device and medium. Background Art

[0002] Medical data refers to data related to medical and health activities, covering basic information of patients, medical history information, diagnosis and treatment information, disease diagnosis, condition assessment, treatment plan, etc., providing a basis for doctors' treatment decisions and efficacy evaluation. It also involves medical imaging images such as X-rays and CT scans. It also includes hospital information, doctor information, etc.

[0003] In the processing of medical data by the existing technology, after configuring the data source, middleware and target database, the synchronization task is directly carried out, while ignoring whether the connection is normal, which will lead to interruptions, data loss or errors in the subsequent synchronization process due to connection problems. The existing technology is not flexible enough in the configuration of synchronization tasks. Either the synchronization can only be carried out in a fixed mode, or the configuration process is cumbersome and complicated. It is impossible to set the synchronization frequency, the data range of synchronization, etc. based on the actual medical data synchronization needs, so that the synchronization task cannot be used in specific business scenarios, resulting in instability and unreliability of the synchronization process. Summary of the invention

[0004] The present invention provides a real-time synchronization method for medical data, which can process data according to factors such as data formats and business rules of different databases. It ensures that the data can meet the requirements of the target database and meet specific medical business needs after being synchronized to the target database. Methods include: S101: Configure the data source, kafka module and target database; S102: Determine whether the test connection is normal; S103: If normal, configure the real-time synchronization task; S104: Determine whether the data source supports capture by the CDC tool; S105: If supported, monitor the data source changes and send the change information to the Kafka module; S106: Determine whether to customize the subsequent processing flow; S107: If you customize the subsequent processing flow, the user monitors and processes the Kafka messages by himself.

[0005] It should be further explained that, in step S104, if it is not supported, a scheduled task is configured to scan the data source regularly; Then send the change information to the Kafka module.

[0006] It should be further explained that step S101 also includes: the method of configuring the data source includes: selecting a data source type, and configuring a JDBC driver according to the data source type; Define connection parameters for medical data; Use the code snippet to verify that you can successfully connect to the data source; The ways to configure the Kafka module include: configuring the Kafka binary file; starting Zookeeper and Kafka servers; Create a Kafka topic and use the kafka-topics.sh script to create a topic about medical data; set configuration items; Set the user ID, auto-commit offset parameters, test message delivery, and send a test message to the Kafka module and write the user ID address to verify receipt.

[0007] Step S101 sets a middleware service module, which is used to read medical data from a data source, send the medical data to a Kafka module, and write the medical data into a target database based on the Kafka module.

[0008] It should be further explained that, in S103, configuring the real-time synchronization task includes: Determine the synchronization frequency based on the timeliness requirements of medical data; Analyze medical business needs and determine which data needs to be synchronized in real time.

[0009] In the configuration interface, a list can be provided, listing all data tables or data collections in the data source database, allowing users to select the data to be synchronized by checking the box.

[0010] It should be further explained that step S104 also includes: Retrieve text information from the target database; Scan text information, extract keywords, and determine whether the target database supports the capture function of the CDC tool and the supported capture methods; If the keywords match the capture function and supported capture methods of the CDC tool, check the target database configuration options and function enablement status; Execute the CDC tool to capture the target medical data in the target database; If the target medical data of the target database can be successfully obtained, it means that the data source supports the CDC tool.

[0011] It should be further explained that, in step S106, the method of determining whether to customize the subsequent processing flow includes: Retrieve the configuration file and set the custom parameters for the custom subsequent processing flow in the configuration file; Pass custom parameters through environment variables or command line parameters; Among them, customizing the subsequent processing flow also involves determining the processing operations that need to be performed based on the characteristics of medical data and business needs; Write a processing program to encrypt the medical data in the Kafka message and enter the subsequent processing flow.

[0012] It should be further explained that the customizing of the subsequent processing flow in step S107 includes: determining the processing operations to be performed by the subsequent processing flow according to the characteristics of the medical data and business requirements; Configure the handler and integrate it with the Kafka application; Use simulated Kafka medical data to test the processing program and check whether the processed results meet the preset requirements. If they do, the processing program integrated with the Kafka program is defined as the subsequent processing flow; The ways for users to monitor and process Kafka messages include: Create a Kafka consumer, use the consumer instance to get messages from the Kafka topic, and process them based on the handler integrated with the Kafka program; store the processed messages in the target database; Monitor the consumer status and if any errors occur, optimize the consumer's performance based on the actual situation.

[0013] The present application also provides a medical data real-time synchronization system, the system comprising: Configuration module, used to configure data source, kafka module and target database; The connection status judgment module is used to judge whether the test connection is normal; if the test connection is abnormal, reconfigure the data source, kafka module and target database until the test connection is normal; The data source capture module is used to configure the real-time synchronization task when the test connection is normal, and to determine whether the data source supports the capture of the CDC tool; The data monitoring module is used to monitor changes in data sources when supporting the capture of CDC tools and send the change information to the Kafka module; A custom judgment module is used to determine whether to customize the subsequent processing flow; The monitoring and processing module is used to execute customized subsequent processing flows. Users monitor and process Kafka messages by themselves.

[0014] According to another embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the real-time medical data synchronization method when executing the program.

[0015] According to another embodiment of the present application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the real-time synchronization method of medical data are implemented.

[0016] It can be seen from the above technical solutions that the present invention has the following advantages: The medical data real-time synchronization method provided by the present invention uses a judgment mechanism to configure a scheduled task to regularly scan the data source when the data source does not support CDC tool capture, and can also send change information to the Kafka module, thereby enhancing the compatibility with data sources with different characteristics and broadening the scope of applicable data sources. In this method, it is judged whether the test connection is normal, and the real-time synchronization task is configured only when the connection is normal, which effectively avoids interruptions, data loss or errors in the subsequent synchronization process due to connection problems, and ensures that the basic conditions of the entire synchronization process are stable and reliable in advance.

[0017] After confirming that the connection is normal, this method can perform real-time synchronization task configuration. According to the actual medical data synchronization needs, key elements such as synchronization frequency and synchronization data range can be flexibly set to make the synchronization task more suitable for specific business scenarios and improve the accuracy and practicality of data synchronization.

[0018] This method determines and allows users to customize the subsequent processing flow. Users can monitor Kafka messages and process them according to the rules they set, such as performing special conversions for certain special medical data formats, filtering or verifying data based on specific medical business logic, etc. This improves the adaptability of the entire data synchronization process to different business needs, allowing the data to be better integrated into subsequent medical application links after synchronization.

[0019] This method can select the most appropriate synchronization strategy for the change data capture capabilities of different databases in a variety of heterogeneous database environments. For example, for databases that support efficient CDC functions, its advantages can be directly utilized to reduce the delay and complexity of data synchronization; for databases that do not support it, other alternative methods can be used to ensure timely capture of data changes. Change information of different types of databases can be effectively buffered and transmitted.

[0020] This application implements the processing of data received from Kafka according to user-defined requirements, and then stores the processed data in the target database to complete data synchronization. This application can capture changes in the data source in a timely manner, so that medical data can be synchronized to the target database in real time or near real time. Compared with regular full data synchronization, change data capture only transmits changed data, greatly reducing the workload of data transmission. Through CDC processing, only the updated part of the report is synchronized, which can effectively reduce the burden on the system. Ensure that the target database can accurately update the changed data part, avoiding data overwrite or loss problems that may be caused by full updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1 A flowchart of a method for real-time synchronization of medical data; Figure 2 A flowchart of an embodiment of a method for real-time synchronization of medical data; Figure 3 A flowchart of a method for determining whether a data source supports the capture of change data in order to configure a real-time synchronization task; Figure 4 A flowchart of how users can monitor Kafka messages and process them; Figure 5 This is a schematic diagram of the medical data real-time synchronization system; Figure 6 Schematic diagram of an electronic device. DETAILED DESCRIPTION

[0023] The real-time synchronization method of medical data provided by this application is mainly to solve the problem that the existing medical data synchronization often only supports specific types of databases, which limits the scope of application, especially in the medical field, where data sources are diverse and complex. The current medical data synchronization system only synchronizes data to the target database and cannot build more functions downstream in an event-driven manner.

[0024] The real-time synchronization method for medical data provided in this application can process the data received from Kafka according to the user's customized requirements, and then store the processed data in the target database to complete data synchronization. In this way, users can flexibly process data according to factors such as data formats and business rules of different databases. Ensure that the data can meet the requirements of the target database and meet specific medical business needs after being synchronized to the target database, such as data integrity, accuracy and compliance.

[0025] The specific steps of the real-time synchronization method for medical data will be described in detail below. For the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are provided to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details.

[0026] It should be understood that when used in the present specification, the term "includes" indicates the presence of the described features, integral bodies, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integral bodies, steps, operations, elements, components and / or their collections. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] See also Figure 1 and Figure 2 The figure is a flowchart of a method for real-time synchronization of medical data in a specific embodiment, the method comprising: S101: Configure the data source, Kafka module, and target database.

[0029] In some embodiments, for multiple heterogeneous target databases, first identify the connection information of different types of databases, configure the database host address, port number, user name, password, database name, etc.; for non-relational databases, configure their corresponding connection strings, database names, etc. as data sources.

[0030] When configuring the Kafka module, you can determine the address and port number of the Kafka server, create topics for data transmission, and set configuration parameters, such as message serialization method, JSON, Avro, etc.

[0031] This embodiment sets the configuration parameters in a structured manner through a configuration file or a configuration interface. The configuration information can be read by subsequent program modules to establish corresponding connections.

[0032] In this way, the basic architecture of data synchronization is established, and the data source, Kafka module and target database are clearly defined. It can adapt to different types of medical databases, whether it is the basic information of patients and medical records stored in traditional relational databases, or metadata such as medical images stored in non-relational databases, which can be integrated as data sources to provide a basis for subsequent real-time synchronization.

[0033] As an implementation method of the present application, configuring the data source, Kafka module, and target database may involve the following methods.

[0034] First, configure the data source and select the data source type. The data source of this embodiment includes a MySQL database, a NoSQL database, or a related file system.

[0035] Configure the JDBC driver corresponding to the data source type. Create a configuration file to set the database driver class name.

[0036] For configuring the Kafka module, you can configure the Kafka binary file; start Zookeeper and Kafka servers; create a Kafka topic and use the kafka-topics.sh script to create a topic about medical data; set configuration items; set the user ID, automatically commit offset parameters, test message delivery, and send a test message to the Kafka module, and write the user ID address to verify the reception.

[0037] In order to facilitate the configuration of medical data, this embodiment also creates a database and table structure, creates a new database instance in the target database, and designs and creates the required tables and indexes.

[0038] This embodiment also integrates the data source, the Kafka module and the target database. Specifically, a middleware service module is set up, which is used to read medical data from the data source, send the medical data to the Kafka module, and write the medical data to the target database based on the Kafka module. The partitions and copies of Kafka are adjusted according to the actual situation based on the middleware service module. The database query and indexing strategies can be optimized. Batch processing is used to improve data transmission efficiency.

[0039] S102: Determine whether the test connection is normal; If the test connection is abnormal, reconfigure the data source, kafka module, and target database until the test connection is normal.

[0040] S103: If normal, configure the real-time synchronization task.

[0041] In some embodiments, for the method of testing the connection between the configured data source, Kafka module and target database, the corresponding connection test tool or API can be used to perform the connection test. For the data source, for example, JDBC (Java Database Connectivity) is used in Java to try to establish a connection and perform a simple query operation to check whether the connection is successful. For the Kafka module, the Kafka client is used to try to send and receive test messages to the configured topic; for the target database, a connection test method similar to that of the data source is also used.

[0042] In the specific execution process, the code logic of the test connection can be executed to capture possible exceptions that may occur during the connection process, such as network connection timeout, authentication failure, etc. In this way, it is ensured that the various components can communicate normally and avoid data loss or synchronization failure due to connection problems during the subsequent data synchronization process.

[0043] In this embodiment, according to the characteristics and requirements of medical data, the synchronization frequency, the synchronized data table or data set, and only the patient data of a specific department are synchronized are determined. For heterogeneous databases, data format conversion or field mapping configuration is performed considering the differences in data structures of different databases.

[0044] For example, mapping a "patient name" field in a relational database to a "name" field in another non-relational database.

[0045] In the configuration management module, the parameters of these synchronization tasks are recorded in the form of a visual interface or configuration file, and these parameters are passed to the data synchronization execution module. In this way, specific rules and plans are formulated for data synchronization operations, so that data can be transmitted from the data source to the target database in real time as expected.

[0046] In a variety of heterogeneous database environments, synchronization tasks can be flexibly configured according to the data content and purpose of different databases. For example, real-time test data in the clinical laboratory system database can be synchronized to the hospital's data warehouse according to certain rules, which is convenient for data analysis and decision support, while adapting to the differences in data structures of different databases.

[0047] S104: Determine whether the data source supports capture by the CDC tool; For this embodiment, for different types of data source databases, it is checked whether they have a built-in change data capture (CDC) function.

[0048] For example, in some advanced relational databases (such as Oracle's LogMiner and SQLServer's CDC function), you can directly use the tools or mechanisms provided by them to capture data changes; for databases that do not have built-in CDC functions, other methods need to be used, such as database triggers, log analysis, etc. For heterogeneous databases, each database has a different inspection method and needs to be judged based on its characteristics. In this way, by querying the database's documents and version information, or executing specific database system functions and commands, it is determined whether it supports the CDC function. The starting point and method of data synchronization can be determined so that the change information of the data source can be accurately obtained to achieve real-time synchronization.

[0049] S105: If supported, monitor the data source changes and send the change information to the Kafka module.

[0050] It should be noted that this embodiment uses the CDC function of the data source to capture data change events in real time. The captured data change information is encapsulated into a message and sent to the Topic specified by Kafka. Here, based on the background running of the listening program, data change events are captured and processed in real time, and the processed messages are sent to the Kafka module. Real-time data synchronization is achieved to ensure that the data in the target database is consistent with the data source.

[0051] For data sources, change data capture can be supported. Based on relational databases, monitoring can be set up using CDC tools or mechanisms. For example, when using Oracle LogMiner, configure the corresponding parameters so that data insertion, update, and deletion operations can be captured; for non-relational databases, monitor according to the change notification mechanism provided by them (such as MongoDB's Change Streams). Then the captured change information is serialized in a pre-set format (such as JSON format), and the message is sent to the corresponding topic of the Kafka module through the Kafka producer. In this way, the real-time change information of the data source is promptly delivered to the message middleware Kafka, and the high throughput and high reliability of Kafka are used to temporarily store and distribute data.

[0052] S106: Determine whether to customize the subsequent processing flow.

[0053] In some specific embodiments, a configuration option or interface is provided to allow users (such as system administrators, data engineers, etc.) to choose whether to perform custom processing on the data received from the Kafka module. For example, it can be determined by a Boolean variable in a configuration file or an option in a management interface. In this way, users can be given flexibility to meet different business needs and data processing requirements.

[0054] S107: If you customize the subsequent processing flow, the user monitors and processes the Kafka messages by himself.

[0055] In this embodiment, if the user chooses to customize the subsequent processing flow, the user can use the Kafka consumer API to write his own processing program. In this embodiment, the user subscribes to the relevant topics in the Kafka module, receives messages, and then processes them according to the customized logic. For example, the message is decrypted, the data format is converted, the data is aggregated, and the processed message is finally stored in the target database.

[0056] In this way, the processing program written by the user runs in the background, continuously obtaining messages from Kafka and processing them until the program is stopped or an exception occurs.

[0057] This embodiment processes the data received from the Kafka module according to user-defined requirements, and then stores the processed data in the target database, completing the last step of data synchronization.

[0058] On the basis of the above embodiments, in order to further improve the reliability of the medical data real-time synchronization method provided in the above embodiments, as an implementable method, the following provides a method for configuring real-time synchronization tasks in the medical data real-time synchronization process for multiple heterogeneous databases. This embodiment determines the synchronization frequency according to the timeliness requirements of the medical data.

[0059] For example, some key vital signs monitoring data (such as heart rate and blood pressure of patients in the intensive care unit) may need to be synchronized every second or every few seconds to ensure that medical staff can obtain the latest patient information in a timely manner. For some relatively stable data, such as the patient's basic information (name, gender, date of birth, etc.), it can be synchronized at regular intervals (such as every day or every week).

[0060] This embodiment takes into account the performance of the data source database and the target database. If the synchronization frequency is too high, it may cause greater query and transmission pressure on the data source database, and may also exceed the writing capacity of the target database. For example, in some old database systems, frequent data writing may cause performance degradation or even system crash.

[0061] In the synchronization task configuration interface or configuration file, set a time interval parameter. For example, in seconds, setting it to 30 means data synchronization every 30 seconds. This parameter will be read by the synchronization task scheduling module to control the cycle of data synchronization.

[0062] In this embodiment, you can choose to synchronize data tables or data sets. Here, we analyze the medical business needs and determine which data needs to be synchronized in real time. For example, in the scenario where the hospital information system (HIS) and the electronic medical record system (EMR) are integrated, it is necessary to synchronize the patient's medical record information, doctor's order information, examination and test report and other data tables. At the same time, the correlation between the data should be considered to avoid data inconsistency caused by synchronizing only part of the data.

[0063] For heterogeneous databases, understand the data structure and storage methods in different databases. For example, in a relational database, data may be stored in a normalized table structure, while in a non-relational database, data may be stored in the form of documents, key-value pairs, or graphs. You need to determine how to select appropriate data from these different storage structures for synchronization.

[0064] In the configuration interface, you can provide a list of all data tables or data collections in the data source database, allowing users to select the data to be synchronized by checking the box. Or in the configuration file, you can configure it in the form of a list of data table names or data collection identifiers. For example, in a JSON-based configuration file, there can be a "tables_to_sync" field whose value is an array containing data table names, such as ["patient_info", "medical_orders", "test_reports"].

[0065] Specifically, you can set filtering conditions based on medical business rules and data usage scenarios. For example, you can synchronize only the medical data of patients in a certain department (such as cardiology), or synchronize only the data generated within a specific time period (such as the past week). SQL is usually used for queries based on relational databases.

[0066] This embodiment also involves processing data format conversion and field mapping. This is because, considering that the data structures and field definitions between heterogeneous databases may be different, it is necessary to determine how to convert the data format in the data source database to a format acceptable to the target database. For example, the date field in the data source database may be in the "YYYY-MM-DD" format, while the target database requires a timestamp format, so date format conversion is required.

[0067] The implementation method of this method is that a field mapping setting area in the form of a table can be provided in the configuration interface. The columns of the table are "data source field", "target field" and "data type conversion rule". Users can fill in the corresponding information in the table, such as "patient_name", "name" and "keep string type". In the configuration file, field mapping and data format conversion rules can be recorded in the form of JSON objects.

[0068] Furthermore, as a refinement and expansion of the specific implementation methods of the above-mentioned embodiment, in order to fully illustrate the specific implementation process in this embodiment, in the real-time synchronization process of medical data for multiple heterogeneous databases, real-time synchronization tasks are configured and whether the data source supports the capture of changed data is determined. The specific execution steps for this method will be given below.

[0069] like Figure 3 As shown, S201: collecting target database information.

[0070] In this embodiment, it can be determined by a database connection string, a configuration file, or a database management tool. For example, if the connection string contains the word "mysql", then it is a MySQL database; if it contains "oracle" related content, then it is an Oracle database.

[0071] S202: Check database documents.

[0072] This example refers to the corresponding official documents according to the database type and version. The documents will clearly state whether the database supports CDC and how it supports it. For example, SQL Server has the Change Data Capture (CDC) function, which supports configuration and acquisition of change data through system stored procedures and functions from the beginning; Oracle provides the LogMiner tool for mining redo logs to obtain data change information, and the documents will explain its usage conditions and steps in detail.

[0073] S203: Check database configuration options and function activation status.

[0074] Taking MySQL as an example, check whether the binary log is enabled, because the binary log is one of the foundations for implementing CDC. You can check whether the relevant parameters (such as "log-bin") in the database configuration file (such as my.cnf) are set, or execute the "SHOW VARIABLES LIKE'log_bin';" command in the database management tool to determine.

[0075] S204: Use the CDC tool to perform simple tests.

[0076] Taking Oracle's LogMiner as an example, use the DBMS_LOGMNR package to add log files for analysis. For example, execute a query to obtain change data. If the change data of the table can be successfully obtained, it means that the data source supports CDC to a certain extent.

[0077] It can be seen that this embodiment can capture changes in the data source in a timely manner, so that medical data can be synchronized to the target database in real time or near real time. For example, when a doctor updates a patient's medication order in the hospital information system (HIS), the CDC function can quickly synchronize this change to other related systems (such as the pharmacy management system) to ensure the timeliness and consistency of the data.

[0078] Compared with regular full data synchronization, change data capture only transmits changed data, greatly reducing the workload of data transmission. Through CDC processing, only the updated part of the report (such as the modified diagnosis conclusion) is synchronized, which can effectively reduce the burden on the system.

[0079] In the real-time synchronization of medical data for multiple heterogeneous databases, determine whether to customize the subsequent processing flow. If the subsequent processing flow is customized, the user listens to the Kafka message and processes it by himself. The specific execution step can be to set the parameter of whether to enable the customized subsequent processing flow. The parameter can be a Boolean value (such as "true" or "false"). For example, in a configuration file based on JSON format, if its value is "true", it means that the subsequent processing flow needs to be customized. In addition, a check box or a drop-down menu option can be provided through a graphical configuration interface to allow users to choose whether to enable the customized processing flow.

[0080] Check system environment variables or command line parameters to pass configuration information through environment variables or command line parameters. In this way, you can check whether there are variables related to the custom processing flow in the system environment variables. Similarly, when starting a synchronization task from the command line, you can also check whether similar parameters are passed in.

[0081] For the specific execution steps of the user monitoring Kafka messages and processing them by himself in this embodiment, as follows Figure 4 An embodiment is shown.

[0082] S301: Create a consumer of the Kafka module.

[0083] Here, specify the topic name of the Kafka module to be monitored, the Kafka server address, the consumer group ID, and the message deserialization method.

[0084] S302: Use consumer messages to obtain messages from the Kafka topic and process them.

[0085] Specifically, the format of medical data is defined, processed according to custom processing logic, and the post-processing operations are performed, such as storing in the target database.

[0086] S303: Identify consumer status and error handling.

[0087] When a consumer fails or the network is interrupted, you can set the automatic commit offset to record the position of the message that the consumer has processed, so that the message can be processed from the correct position after the failure is recovered. At the same time, comprehensive handling of possible errors is performed, such as handling Kafka server connection failures, message parsing errors, etc. For example, in Python, you can set enable_auto_commit=True to automatically commit the offset, and handle various error situations in the try-except block.

[0088] S304: Adjust the number of batch messages obtained by the consumer to balance the timeliness of data processing and the occupation of system resources.

[0089] This embodiment can also optimize the performance of the code for processing messages, such as using multi-threading or asynchronous processing to improve the efficiency of message processing.

[0090] In this embodiment, users can process the medical data in Kafka messages according to their specific needs and business logic. The customized processing method can adapt to complex and changeable medical data synchronization scenarios, and users can also participate in and control every aspect of data processing, including data reception, processing and storage.

[0091] The following is an embodiment of a medical data real-time synchronization system provided by an embodiment of the present disclosure. This system and the medical data real-time synchronization method of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the medical data real-time synchronization system, please refer to the embodiment of the above-mentioned medical data real-time synchronization method.

[0092] like Figure 5 As shown in the figure, the system includes: Configuration module, used to configure data source, kafka module and target database; The connection status judgment module is used to judge whether the test connection is normal; if the test connection is abnormal, reconfigure the data source, kafka module and target database until the test connection is normal; The data source capture module is used to configure the real-time synchronization task when the test connection is normal, and to determine whether the data source supports the capture of the CDC tool; The data monitoring module is used to monitor changes in data sources when supporting the capture of CDC tools and send the change information to the Kafka module; A custom judgment module is used to determine whether to customize the subsequent processing flow; The monitoring and processing module is used to execute customized subsequent processing flows. Users monitor and process Kafka messages by themselves.

[0093] like Figure 6 As shown, the present application also provides an electronic device, including a display module 103, a memory 102, a processor 101, and a computer program stored in the memory and executable on the processor 101, wherein the processor 101 implements the steps of the real-time synchronization method of medical data when executing the program.

[0094] In embodiments of the present invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0095] In the embodiment of the present application, the processor 101 can be implemented by using at least one of a special purpose integrated circuit, a programmable logic device, a processor, a controller, a microcontroller, a microprocessor, and an electronic unit designed to perform the functions described herein. In some cases, such an implementation can be implemented in a controller. For software implementation, implementations such as processes or functions can be implemented with separate software modules that allow execution of at least one function or operation. The software code can be implemented by a software application (or program) written in any appropriate programming language, and the software code can be stored in a memory and executed by a controller.

[0096] The display module 103 is used to display information input by the user or information provided to the user. The display module 103 may include a display panel, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.

[0097] The memory 102 may be used to store software programs and various data. The memory 102 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0098] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0099] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0100] The present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the real-time medical data synchronization method are implemented.

[0101] The storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0102] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for real-time synchronization of medical data, characterized in that: Methods include: S101: Configure the data source, kafka module and target database; S102: Determine whether the test connection is normal; S103: If normal, configure the real-time synchronization task; S104: Determine whether the data source supports capture by the CDC tool; S105: If supported, monitor the data source changes and send the change information to the Kafka module; S106: Determine whether to customize the subsequent processing flow; S107: If you customize the subsequent processing flow, the user monitors and processes the Kafka messages by himself.

2. The method for real-time synchronization of medical data according to claim 1, characterized in that: In step S104, if it is not supported, a scheduled task is configured to scan the data source regularly; Then send the change information to the Kafka module.

3. The method for real-time synchronization of medical data according to claim 1, characterized in that: Step S101 also includes: the method of configuring the data source includes: selecting a data source type, and configuring a JDBC driver according to the data source type; Define connection parameters for medical data; Use the code snippet to verify that you can successfully connect to the data source; The ways to configure the Kafka module include: configuring the Kafka binary file; starting Zookeeper and Kafka servers; Create a Kafka topic and use the kafka-topics.sh script to create a topic about medical data; set configuration items; Set the user ID, automatically commit offset parameters, test message delivery, and send a test message to the Kafka module and write the user ID address to verify reception; Step S101 sets a middleware service module, which is used to read medical data from a data source, send the medical data to a Kafka module, and write the medical data into a target database based on the Kafka module.

4. The method for real-time synchronization of medical data according to claim 1, characterized in that: In S103, configuring the real-time synchronization task includes: Determine the synchronization frequency based on the timeliness requirements of medical data; Analyze medical business needs and determine which data needs to be synchronized in real time; In the configuration interface, a list can be provided, listing all data tables or data collections in the data source database, allowing users to select the data to be synchronized by checking the box.

5. The method for real-time synchronization of medical data according to claim 1, characterized in that: Step S104 also includes: Retrieve text information from the target database; Scan text information, extract keywords, and determine whether the target database supports the capture function of the CDC tool and the supported capture methods; If the keywords match the capture function and supported capture methods of the CDC tool, check the target database configuration options and function enablement status; Execute the CDC tool to capture the target medical data in the target database; If the target medical data of the target database can be successfully obtained, it means that the data source supports the CDC tool.

6. The method for real-time synchronization of medical data according to claim 1, characterized in that: In step S106, the method of determining whether to customize the subsequent processing flow includes: Retrieve the configuration file and set the custom parameters for the custom subsequent processing flow in the configuration file; Pass custom parameters through environment variables or command line parameters; Among them, customizing the subsequent processing flow also involves determining the processing operations that need to be performed based on the characteristics of medical data and business needs; Write a processing program to encrypt the medical data in the Kafka message and enter the subsequent processing flow.

7. The real-time synchronization method for medical data according to claim 1, characterized in that: Customizing the subsequent processing flow in step S107 includes: determining the processing operations to be performed by the subsequent processing flow according to the characteristics of the medical data and business requirements; Configure the handler and integrate it with the Kafka application; Use simulated Kafka medical data to test the processing program and check whether the processed results meet the preset requirements. If they do, the processing program integrated with the Kafka program is defined as the subsequent processing flow; The ways for users to monitor and process Kafka messages include: Create a Kafka consumer, use the consumer instance to get messages from the Kafka topic, and process them based on the handler integrated with the Kafka program; store the processed messages in the target database; Monitor the consumer status and if any errors occur, optimize the consumer's performance based on the actual situation.

8. A medical data real-time synchronization system, characterized in that: The system is used to implement the real-time synchronization method of medical data as described in any one of claims 1 to 7; The system includes: Configuration module, used to configure data source, kafka module and target database; The connection status judgment module is used to judge whether the test connection is normal; if the test connection is abnormal, reconfigure the data source, kafka module and target database until the test connection is normal; The data source capture module is used to configure the real-time synchronization task when the test connection is normal, and to determine whether the data source supports the capture of the CDC tool; The data monitoring module is used to monitor changes in data sources when supporting the capture of CDC tools and send the change information to the Kafka module; A custom judgment module is used to determine whether to customize the subsequent processing flow; The monitoring and processing module is used to execute customized subsequent processing flows. Users monitor and process Kafka messages by themselves.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the real-time medical data synchronization method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the real-time medical data synchronization method according to any one of claims 1 to 7 are implemented.

Images