Multi-source heterogeneous medical data collection method based on template design pattern
Through the multi-source heterogeneous medical data acquisition method based on template design pattern, the problem of poor interoperability of multi-source heterogeneous data is solved, the matching and processing of data sources and formats is realized, and the adaptability and scalability of data processing is improved. Ensures data quality and consistency.
Patent Information
- Application Number
- CN202011453610.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-12
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-12-12
AI Technical Summary
Existing medical data processing systems are difficult to communicate data due to multi-source heterogeneity, resulting in data quality problems and difficulty in using it.
Using a multi-source heterogeneous medical data acquisition method based on template design pattern, the data acquisition template, data cleaning template and data mapping template are established by establishing a data acquisition subsystem, data cleaning template and data mapping template based on the template management subsystem. Match and pull to realize data processing.
When facing multi-source and isomerized clinical medical data, template matching and processing can be performed according to the data source and data format, improving the adaptability and scalability of data processing and ensuring data quality and consistency.
Smart Images

Figure CN112507681B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of multi-source heterogeneous data integration, and in particular to a multi-source heterogeneous medical data acquisition method based on a template design pattern. Background Art
[0002] The rapid development of hospital informatization has led to an increasing scale of information system construction. These systems have different standards for collecting, processing, storing and exchanging data due to the different businesses they handle and the technical architectures they adopt. From the perspective of the structure of data, it is divided into structured data, semi-structured data and unstructured data: from the perspective of the information recording mode, the same entity may be recorded in multiple systems, but the specific attribute sets may be different. Even for the same attribute, its naming or data may be contradictory and conflicting due to errors in the system or manual recording during the interaction process. Obviously, the data in the medical industry presents a multi-source heterogeneity, which will cause serious data quality problems and have an extremely adverse impact on the subsequent use of medical big data.
[0003] Most of the existing hospitals or medical institutions have their own medical data processing systems and use independent processing methods to process data. This separate collection and processing method has the problem of data interoperability. When faced with data from different channels and sources, it is impossible to use the same processing method to process data. Summary of the invention
[0004] In view of the shortcomings existing in the above-mentioned technologies, the present invention provides a multi-source heterogeneous medical data acquisition method based on a template design pattern. When faced with a large number of multi-source and heterogeneous clinical medical data, the present invention can match and pull templates according to the type of data source and the data format type, and then match and pull data acquisition templates, data cleaning templates and data mapping templates to process the data, thereby establishing a data processing method based on the template design principle.
[0005] To achieve the above object, the present invention provides a multi-source heterogeneous medical data collection method based on a template design pattern, comprising:
[0006] Establish data acquisition subsystem, data cleaning subsystem and data mapping subsystem;
[0007] Establish different template implementation methods based on different clinical medical information formats;
[0008] Establish data acquisition templates, data cleaning templates and data mapping templates based on target information systems of different clinical medical information and store them in the template management subsystem;
[0009] According to the data acquisition template type, data cleaning template type and data mapping template type in the template management subsystem, corresponding pattern matching rules are established and stored in the pattern matching subsystem;
[0010] When executing the data acquisition subsystem, data cleaning subsystem and data mapping subsystem, the corresponding template is pulled from the template management subsystem through the pattern matching subsystem for data processing, and the data is stored in the target database after processing.
[0011] Specifically: in the step of pulling the corresponding template from the template management subsystem for data processing through the pattern matching subsystem, after the data acquisition subsystem processes the data, it is transmitted to the data cleaning subsystem for data cleaning, and finally transmitted to the data mapping subsystem for data mapping.
[0012] Specifically: the pattern matching rules include data source matching rules, data format matching rules and action behavior matching rules.
[0013] Specifically: the template management subsystem includes an action behavior type template management module, a data source type template management module and a data format template management module.
[0014] Preferably: before executing the steps of the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem, a step is also included, in which the action being executed is matched with the corresponding action behavior matching rule from the pattern matching subsystem, and the corresponding action behavior type template is pulled from the template management subsystem according to the matched action behavior matching rule and output for execution.
[0015] Preferably, the steps of executing the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem include a step of matching the acquired data source with the data source matching rule from the pattern matching subsystem, and pulling the corresponding data source type template from the template management subsystem according to the matched data source matching rule and outputting it for execution.
[0016] Preferably: after the step of pulling the corresponding data source type template from the template management subsystem according to the matched data source matching rule and outputting it for execution, a further step is included: matching the acquired data format with the data format matching rule from the pattern matching subsystem, pulling the corresponding data format type template from the template management subsystem according to the matched data format matching rule and outputting it for execution.
[0017] Specifically: the data source format type template includes one or more of an information system data template, a hardware data template and an electronic document data template.
[0018] Specifically: the data format type template includes one or more of a DB format template, a JSON format template, an XML format template and a CSV format template.
[0019] The beneficial effect of the present invention is that compared with the prior art, the present invention provides a multi-source heterogeneous medical data collection method based on a template design pattern, comprising:
[0020] Establish data acquisition subsystem, data cleaning subsystem and data mapping subsystem;
[0021] Establish different template implementation methods based on different clinical medical information formats;
[0022] Establish data acquisition templates, data cleaning templates and data mapping templates based on target information systems of different clinical medical information and store them in the template management subsystem;
[0023] According to the data acquisition template type, data cleaning template type and data mapping template type in the template management subsystem, corresponding pattern matching rules are established and stored in the pattern matching subsystem; when executing the data acquisition subsystem, data cleaning subsystem and data mapping subsystem, the corresponding template is pulled from the template management subsystem through the pattern matching subsystem for data processing, and the data is stored in the target database after processing; when facing a large number of multi-source and heterogeneous clinical medical data, this method can match and pull templates according to the type of data source and data format type, and then match and pull data acquisition templates, data cleaning templates and data mapping templates to process the data, thereby establishing a data processing method based on template design principles. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a schematic diagram of the method principle of the present invention;
[0025] Figure 2 It is a schematic diagram of the pattern matching subsystem of the present invention;
[0026] Figure 3 It is a schematic diagram of template distribution of the present invention;
[0027] Figure 4 It is a schematic diagram of the working principle of the data acquisition subsystem of the present invention;
[0028] Figure 5 This is a schematic diagram of the working principle of the data cleaning subsystem of the present invention;
[0029] Figure 6 It is a schematic diagram of the working principle of the data mapping subsystem of the present invention. DETAILED DESCRIPTION
[0030] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with the accompanying drawings.
[0031] With the rapid development of medical level and medical information technology, the establishment of clinical medical system has become an indispensable part of major hospitals and medical systems. For clinical medical system, the most important step is the collection and processing of clinical medical data. There are many sources and types of clinical medical data. The data formats and data types corresponding to data from different sources are different, and their data structures are also different. The existing clinical medical data is multi-source and heterogeneous.
[0032] The present invention takes the template design mode as the core, and realizes a data processing method based on the template design principle, which can match and pull templates according to the type of data source and data format type when facing a variety of multi-source and heterogeneous clinical medical data, and then match and pull data acquisition templates, data cleaning templates and data mapping templates to process the data; no matter how large the difference in data sources and structures is, this method can extract the corresponding processing template from the established template, and has the characteristics of wider range, stronger adaptability and strong expansibility.
[0033] The present invention specifically provides a multi-source heterogeneous medical data collection method based on a template design pattern, the method comprising the following steps:
[0034] Establish data acquisition subsystem, data cleaning subsystem and data mapping subsystem;
[0035] Establish different template implementation methods based on different clinical medical information formats;
[0036] Establish data acquisition templates, data cleaning templates and data mapping templates based on target information systems of different clinical medical information and store them in the template management subsystem;
[0037] According to the data acquisition template type, data cleaning template type and data mapping template type in the template management subsystem, corresponding pattern matching rules are established and stored in the pattern matching subsystem;
[0038] See also Figure 1-3, when executing the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem, the corresponding template is pulled from the template management subsystem through the pattern matching subsystem for data processing, and the data is stored in the target database after the processing is completed; the whole method is implemented by the data acquisition subsystem, the data cleaning subsystem, the data mapping subsystem, the template management subsystem and the template matching subsystem in collaboration, and the template management subsystem is provided with an action behavior type template management module, a data source type template management module and a data format template management module, and the corresponding action behavior type template management module stores data acquisition templates, data cleaning template types and data mapping templates; the data source type template management module stores common data source types of medical data such as information system data templates, hardware data templates and electronic document data templates, but not limited to the above-mentioned ones; the data format template management module stores common data format types of medical data such as DB format templates, JSON format templates, XML format templates and CSV format templates, but not limited to the above-mentioned ones;
[0039] In this embodiment: in the step of pulling the corresponding template from the template management subsystem for data processing through the pattern matching subsystem, after the data acquisition subsystem processes the data, it is transmitted to the data cleaning subsystem for data cleaning, and finally transmitted to the data mapping subsystem for data mapping; data acquisition, data cleaning and data mapping are indispensable and key steps in the whole process. Data acquisition is a preliminary step in identifying the data type. In this step, the data source type of the data is preliminarily determined, and matched with the corresponding template, and the template adapted to the data source is extracted and output. After that, the data format is determined, and the corresponding template is matched and output. After completing the above steps, the initial step; after completing the data acquisition step, the data is transmitted to the data cleaning subsystem for data cleaning. The purpose of data cleaning is to make the originally scattered and incomplete data complete, and to extract and derive valuable and meaningful data and data structures from a large amount of complex, disorganized and difficult to understand data; after cleaning, the truly valuable and organized data is saved to reduce the analysis obstacles for subsequent data analysis; after completing the data cleaning step, the cleaned data is transmitted to the data mapping subsystem for data mapping. The purpose of data mapping is to map the originally difficult to identify and multi-source heterogeneous data into the same and easy to identify data type and keep it the same, so as to facilitate the subsequent data identification and preservation.
[0040] In this embodiment: pattern matching rules include data source matching rules, data format matching rules and action behavior matching rules; specifying a variety of different matching rules is to quickly match the acquired data and the actions being executed with the corresponding templates. The rules point to a template in the template management subsystem, which helps to quickly match the corresponding template.
[0041] In this embodiment: the template management subsystem includes an action type template management module, a data source type template management module and a data format template management module; multiple management modules are established to perform partition management on different templates, so as to facilitate rapid matching to the corresponding template according to the matching rules when executing data processing;
[0042] In a preferred embodiment, refer to: Figure 4-6 Before executing the steps of the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem, a step is also included, in which the action being executed is matched with the corresponding action behavior matching rule from the pattern matching subsystem, and the corresponding action behavior type template is pulled from the template management subsystem according to the matched action behavior matching rule and output for execution; before the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem are executed, it is necessary to judge and match the type of action to be executed, so that the action to be performed by the data can be accurately identified, and the action behavior that has been executed by the data can be avoided from being repeated, resulting in repeated processing of the data in a subsystem, affecting the processing efficiency and processing timeliness.
[0043] In a preferred embodiment, refer to: Figure 4-6 , in the steps of executing the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem, a step is included: matching the acquired data source with the data source matching rule from the pattern matching subsystem, pulling the corresponding data source type template from the template management subsystem according to the matched data source matching rule and outputting it for execution; each subsystem performs template matching of the data source during execution, and in different subsystems, the corresponding processing templates for different data sources are different. Although the type matching of the data source has been performed in the previous subsystem, the templates corresponding to the data source in different subsystems are different due to different processing, so it is necessary to match the corresponding template processing of the data source in this step.
[0044] In a preferred embodiment, refer to: Figure 4-6, after the step of pulling the corresponding data source type template from the template management subsystem according to the matched data source matching rule and outputting it for execution, it also includes a step: matching the acquired data format with the data format matching rule from the pattern matching subsystem, pulling the corresponding data format type template from the template management subsystem according to the matched data format matching rule and outputting it for execution; similarly, each subsystem performs data format template matching during execution, and in different subsystems, the processing templates corresponding to different data formats are different. Although the data format type matching has been performed in the previous subsystem, the templates corresponding to the data formats in different subsystems are different due to different processing, so it is necessary to match the data format corresponding to the template processing in this step.
[0045] In this example: See Figure 3 The data source format type templates include common medical data format types such as information system data templates, hardware data templates, and electronic document data templates, but are not limited to the types mentioned above.
[0046] In this example: See Figure 3 The data format type templates include common medical data format types such as DB format template, JSON format template, XML format template and CSV format template, but are not limited to the above-mentioned ones.
[0047] In this embodiment, it is mentioned that:
[0048] For the entire system operation process, different data sources will bring different data structures, and different actions will be performed on the data at different stages, resulting in different pattern matching rule algorithms. This requires the pattern matching algorithm to be able to meet the needs of different scenarios and be universal, adaptive and scalable; the common template design pattern in application software engineering proposes the concept of adaptive templates.
[0049] To realize the above core, it is necessary to store templates, matching rules, etc. The present invention mainly performs storage based on a relational database.
[0050] First, by establishing the basic database tables, mainly including the "data source type table" (DATASOURCE_DEF_TABLE), "data format type table" (DATATYPE_DEF_TABLE), "action behavior type table" (OPERATIONTYPE_DEF_TABLE), used to save the basic information of the system, see Figure 3 .
[0051] Then, create the corresponding template information table, which mainly includes the "data source template table" (DATASOURCE_TEMP_TABLE), the "data format template table" (DATATYPE_TEMP_TABLE), and the "action behavior template table" (OPERATIONTYPE_TEMP_TABLE) to save various template information data.
[0052] Each data source uniquely corresponds to a record in the "data source template table", that is, it corresponds to a template for data source parsing;
[0053] Each data format uniquely corresponds to a record in the "data format template table", that is, it corresponds to a template for data format parsing;
[0054] Each system action behavior (mainly data collection, data cleaning, and data mapping) uniquely corresponds to a record in the "action behavior template table", that is, it corresponds to a template for action behavior analysis.
[0055] Finally, a "Template Adaptation Table" (TEMPLATE_MAPPING_TABLE) is established to save the mapping relationship between various basic data and template information data. Its type field is mainly divided into three types: "DATASOURCE", "DATATYPE" and "OPERATIONTYPE".
[0056] Among them, the "DATASOURCE" type mainly defines how to associate a record in the "data source type table" with which record in the "data source template table"; the "DATATYPE" type mainly defines how to associate a record in the "data format type table" with which record in the "data format template table"; the "OPERATIONTYPE" type mainly defines how to associate a record in the "action behavior type table" with which record in the "action behavior template table".
[0057] In one instance it is mentioned:
[0058] Let’s take a typical data collection as an example: a medical information system, the data format is JSON, and the overall behavior includes data acquisition, data cleaning and data mapping to illustrate the overall operation process.
[0059] Step 1: Determine the data source type as INFOSYS, the data format type as JSON, and the action behaviors as EXTRACT, CLEAN, and MAPPING.
[0060] Step 2: According to the action behavior types "EXTRACT", "CLEAN" and "MAPPING", find the execution sequence numbers of the above three behaviors from the "Action Behavior Template Table" (OPERATIONTYPE_TEMP_TABLE) to determine the execution step sequence of the above actions, usually data acquisition EXTRACT=1, data cleaning=2 and data mapping MAPPING=3.
[0061] Step 3: Follow the above steps in sequence. The steps for each action type are as follows:
[0062] Sub-step 3.1: According to the action type (here, take data acquisition EXTRACT as an example), query the corresponding action template record in the "action template table" (OPERATIONTYPE_TEMP_TABLE) and pass it to the "template matching subsystem" as a system parameter;
[0063] Sub-step 3.2: The "Template Matching Subsystem" searches for a mapping record of the type "OPERATIONTYPE" that matches the name "EXTRACT" according to the passed action behavior name from the "Template Adaptation Table" (TEMPLATE_MAPPING_TABLE), and passes it to the "Template Acquisition Subsystem" as a system parameter;
[0064] Sub-step 3.3: The "Data Acquisition Subsystem" queries the "Template Adaptation Table" (TEMPLATE_MAPPING_TABLE) for a mapping record that matches the name and is of type "DATASOURCE" based on the parameter "INFOSYS" passed in; then queries the corresponding data source template record from the "Data Source Template Table" (DATASOURCE_TEMP_TABLE).
[0065] Sub-step 3.4: The "Data Acquisition Subsystem" queries the "Template Adaptation Table" (TEMPLATE_MAPPING_TABLE) for a mapping record that matches the name and is of type "DATATYPE" based on the passed parameter "JSON"; then queries the corresponding data format template record from the "Data Format Template Table" (DATATYPE_TEMP_TABLE).
[0066] Sub-step 3.5: The "data acquisition subsystem" calls the "pattern matching rule interface" to perform specific data acquisition according to the data source template record information and the data format template record information of the above sub-steps 3.3 and 3.4.
[0067] Sub-step 4: Determine whether there is a next action behavior. If so, continue the operations of sub-steps 3.1 to 3.5 until there is no next action behavior, and the overall system operation process ends.
[0068] The advantages of the present invention are:
[0069] 1. Match and pull templates according to the type of data source and data format type, then match and pull data acquisition templates, data cleaning templates and data mapping templates to process the data and establish a data processing method based on template design principles.
[0070] 2. Before each subsystem is executed, the action behavior type is matched to identify the action type that the data is about to execute and extract the corresponding template for data processing.
[0071] 3. Each subsystem performs matching of data source type and data format type to ensure that different data sources and data formats can be accurately matched to the corresponding templates for processing in different subsystems.
[0072] The above disclosures are only several specific embodiments of the present invention, but the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.
Claims
1. A multi-source heterogeneous medical data collection method based on template design pattern, characterized in that: include: Establish data acquisition subsystem, data cleaning subsystem and data mapping subsystem; Establish different template implementation methods based on different clinical medical information formats; Establish data acquisition templates, data cleaning templates and data mapping templates based on target information systems of different clinical medical information and store them in the template management subsystem; Establish corresponding pattern matching rules according to the data acquisition template type, data cleaning template type and data mapping template type and store them in the pattern matching subsystem; When executing the data acquisition subsystem, data cleaning subsystem and data mapping subsystem, the corresponding template is pulled from the template management subsystem through the pattern matching subsystem for data processing, and the data is stored in the target database after processing; The pattern matching rules include data source matching rules, data format matching rules and action behavior matching rules; the template management subsystem includes an action behavior type template management module, a data source type template management module and a data format template management module; before executing the steps of the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem, a step is also included, matching the action being executed with the corresponding action behavior matching rules from the pattern matching subsystem, pulling the corresponding action behavior type template from the template management subsystem according to the matched action behavior matching rules and outputting it for execution; in the steps of executing the data acquisition subsystem, the data cleaning subsystem and the data mapping subsystem, a step is included: matching the acquired data source with the data source matching rules from the pattern matching subsystem, pulling the corresponding data source type template from the template management subsystem according to the matched data source matching rules and outputting it for execution; after the step of pulling the corresponding data source type template from the template management subsystem according to the matched data source matching rules and outputting it for execution, a step is also included: matching the acquired data format with the data format matching rules from the pattern matching subsystem, pulling the corresponding data format type template from the template management subsystem according to the matched data format matching rules and outputting it for execution.
2. The multi-source heterogeneous medical data collection method based on template design pattern according to claim 1 is characterized in that: In the step of pulling the corresponding template from the template management subsystem for data processing through the pattern matching subsystem, after the data acquisition subsystem processes the data, it is transmitted to the data cleaning subsystem for data cleaning, and finally transmitted to the data mapping subsystem for data mapping.
3. The multi-source heterogeneous medical data collection method based on template design pattern according to claim 1 is characterized in that: The data source type template includes one or more of an information system data template, a hardware data template and an electronic document data template.
4. The multi-source heterogeneous medical data collection method based on template design pattern according to claim 1 is characterized in that: The data format type template includes one or more of a DB format template, a JSON format template, an XML format template and a CSV format template.
