Data integration method applied to online management of medical data

Abstract

The application discloses a data integration method applied to online management of medical data and relates to the technical field of medical data processing, and comprises the following steps: reading and merging local standardized data sets, performing coding mapping on structured coding fields and unstructured text fields respectively, and generating a standard coding data set; parsing data records in the standard coding data set into semantic nodes and establishing a correlation relationship, constructing a semantic network, and detecting subsequent coding, and coding meeting conditions is included in the semantic network for dynamic updating; providing a unified data query interface based on the semantic network, receiving a query condition through the interface, and returning a result set in a unified format; and through construction of the semantic network, multiple-source heterogeneous medical data is uniformly associated and integrated at a semantic level, so that originally isolated data can be organized and accessed in a patient-centered association view, and the integration efficiency and query efficiency of the medical data are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of medical data processing technology, specifically a data integration method for online management of medical data. Background Technology

[0002] Medical data permeates the entire process of clinical diagnosis and treatment, scientific research and analysis, and hospital management. Its completeness and relevance directly affect the quality of treatment decisions and the credibility of research conclusions. However, different business systems in medical institutions are independent of each other, with different data storage formats and inconsistent coding standards. This results in the data of the same patient being scattered across hospital information systems, laboratory information systems, electronic medical record systems, and image archiving and communication systems, forming data silos. This leads to the fragmented storage of structured diagnosis and treatment data and textual description information for the same patient. Existing data integration methods mainly use data warehouse or data lake technology to aggregate data from various systems into a unified storage, and then use ETL tools for cleaning and transformation. However, existing integration methods usually only process data from various systems at the level of field mapping and format conversion. For key fields such as diagnostic codes, test item codes, and drug codes, mapping relationships are often established one by one by manual comparison, which requires a lot of manpower and is difficult to cover all codes. For unstructured text such as medical records and examination reports, existing methods often store them directly as text fields, which consumes a lot of storage resources and also affects subsequent dynamic data updates.

[0003] How to achieve automatic integration and dynamic updating of medical data is a problem we need to solve. To this end, we now provide a data integration method for online management of medical data. Summary of the Invention

[0004] The purpose of this invention is to provide a data integration method for online management of medical data.

[0005] The objective of this invention can be achieved through the following technical solution: a data integration method for online management of medical data, comprising:

[0006] S1: Collect raw medical data from various business systems, preprocess it, and generate a local standardized dataset;

[0007] S2: Read and merge local standardized datasets, construct encoding mapping rules, execute encoding mapping, and generate standard encoded datasets;

[0008] S3: Parse the data records in the standard coding dataset into semantic nodes, establish the relationships between nodes, and construct a semantic network; detect the codes in the subsequent standard coding dataset, include the codes that meet the conditions into the semantic network, and update the semantic network;

[0009] S4: Based on the semantic network, it provides a unified data query interface. The interface receives query conditions, locates the corresponding nodes and related data in the semantic network, and returns a result set in a unified format.

[0010] Preferably, the process of collecting raw medical data from various business systems, preprocessing it, and generating a local standardized dataset includes:

[0011] Multiple front-end servers are deployed in the hospital's network environment. Each front-end server establishes a network connection with the database server of each business system through the hospital's local area network. By using change data capture technology, the front-end server reads the log files of each business system's database in real time. When a new or changed data record appears in the log file, the front-end server immediately captures the data record. The captured data record is then subjected to null value verification, format normalization, and text cleaning. The preprocessed data record is organized into a table format according to a preset data structure and stored in the local hard drive of the front-end server to generate a local standardized dataset.

[0012] Preferably, the process of reading and merging local standardized datasets includes:

[0013] Read the local standardized datasets stored on the local hard drives of each front-end machine, use patient identifiers and visit identifiers as association keys to associate data records from different business systems to the same visit event, and arrange the merged data records in ascending order of occurrence time to form a dataset to be encoded.

[0014] Preferably, the process of constructing encoding mapping rules and executing encoding mapping includes:

[0015] For structured coded fields, obtain a pre-defined lookup table of local codes and standard codes for each business system, extract the local code field from the data record, and accurately search and replace it with the standard code in the lookup table;

[0016] For unstructured text fields, key entities are extracted from the text, the entities are matched with a preset term thesaurus and converted into standard terms, and then the standard terms are mapped to the corresponding standard codes.

[0017] All data records after mapping are organized into a table format according to a unified data structure to generate a standard coded dataset.

[0018] Preferably, the process of constructing a semantic network includes:

[0019] Traverse each data record in the standard coded dataset, extract key information from the data records, create different types of nodes based on the extracted information, and fill the information into the node attributes; take the consultation node as the center, connect the patient node, diagnosis node, test node, medication node and document node under the same consultation event to the consultation node respectively; all nodes and their associations constitute a semantic network with consultation events as the unit.

[0020] Preferably, the semantic network update process includes:

[0021] When a new standard coding dataset is stored, the standard codes and standard terms of each data record are extracted and compared with existing nodes in the existing semantic network. For standard coding fields, exact string matching is used to determine whether they already exist. For standard term fields, semantic similarity of attribute feature sets is calculated to determine whether they already exist. If they already exist, the new data record is associated with the existing node. If they do not exist, the inclusion condition is met, a new node is created and an association is established with the corresponding medical treatment node. The new node is then connected to the semantic network.

[0022] Preferably, the process of providing a unified data query interface and receiving query conditions includes:

[0023] Build a unified data query interface to receive query requests submitted by external systems or users, extract query conditions from the received requests, and validate the query conditions.

[0024] Preferably, the process of returning the result set includes:

[0025] The system receives the verified query conditions, determines the type of node to be matched based on the type of query conditions, searches for the target node in the semantic network whose attribute value matches the query value, obtains all related nodes connected to the target node and their attribute information, assembles the obtained information according to a preset unified format, generates a query result set, and returns it.

[0026] Compared with existing technologies, the beneficial effects of this invention are as follows: By collecting and preprocessing the original medical data from various business systems, multi-source heterogeneous data is transformed into a unified format local standardized dataset. Through encoding mapping, the local codes and terms of various business systems are uniformly converted into standard codes and terms, eliminating semantic differences in the data. Furthermore, by constructing a semantic network, records in the standard coded dataset are parsed into semantic nodes and relationships are established, enabling the originally isolated data to form a unified relational view centered on the patient. By detecting and dynamically updating subsequent codes, the semantic network is continuously expanded. Finally, a unified format result set is returned through a unified data query interface. This achieves unified integration of multi-source heterogeneous medical data at the semantic level, greatly improving the integration and query efficiency of cross-system medical data. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0028] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0029] like Figure 1 As shown, the data integration method applied to online medical data management includes:

[0030] S1: Collect raw medical data from various business systems, preprocess it, and generate a local standardized dataset;

[0031] S2: Read and merge local standardized datasets, construct encoding mapping rules, execute encoding mapping, and generate standard encoded datasets;

[0032] S3: Parse the data records in the standard coding dataset into semantic nodes, establish the relationships between nodes, and construct a semantic network; detect the codes in the subsequent standard coding dataset, include the codes that meet the conditions into the semantic network, and update the semantic network;

[0033] S4: Based on the semantic network, it provides a unified data query interface. The interface receives query conditions, locates the corresponding nodes and related data in the semantic network, and returns a result set in a unified format.

[0034] Furthermore, the process of generating a locally standardized dataset includes:

[0035] Multiple front-end servers are deployed in the hospital's intranet environment. Each front-end server establishes a network connection with the database servers of various business systems through the intranet. The front-end server reads the log files of each business system's database in real time through change data capture technology. When a new or changed data record appears in the log file, the front-end server immediately captures the data record. The front-end server performs null value verification, format normalization, and text cleaning on the captured data record, and organizes the preprocessed data record into a table format according to a preset data structure, and stores it on the front-end server's local hard drive to generate a local standardized dataset.

[0036] The data record is a piece of business information in the business system database, which includes patient identifier, visit identifier, occurrence time and specific business data content;

[0037] It should be further explained that the front-end machine is an independently configured physical server or virtual machine. The front-end machine's local hard drive pre-stores connection configuration files, which record the network IP address, port number, database name, and access credentials of each business system's database server. The front-end machine reads the parameters in the connection configuration file and initiates connection requests to each business system's database server via the hospital's local area network. After the database server verifies the access credentials, the two parties establish a connection.

[0038] Each business system includes a hospital information system, a laboratory information system, an electronic medical record system, and an image archiving and communication system, each running on its own database server. The raw medical data consists of unprocessed source data stored in the databases of each business system, including both structured and unstructured data. Structured data includes patient registration information, diagnostic codes, test values, and medication records, which are stored in relational tables in the business system database in the form of two-dimensional tables. Unstructured data includes medical record text, examination report text, and discharge summary text, which are stored in tables in the business system database as text fields.

[0039] During preprocessing, the front-end machine performs null value validation, format normalization, and text cleaning on each captured data record. Null value validation is used to detect whether key fields in the data record are missing. Key fields include patient identification, consultation time, and business type. When a key field is missing, the data record is pushed to the manual review queue. Format normalization is used to unify the date format to YYYY-MM-DD and the number format to decimal format. Text cleaning is used to remove line breaks, tabs, extra spaces, and special symbols from unstructured text. The preset data structure is in tabular form, with each row corresponding to one data record and each column corresponding to one data dimension. The data dimension includes the record source system identifier, business type identifier, patient identifier, consultation identifier, occurrence time, and data content. The record source system identifier is the name or number of each business system. The business type identifier includes diagnosis, testing, medication, and document. The patient identifier and consultation identifier are the patient number and consultation number recorded in the database of each business system. The occurrence time is the business occurrence time corresponding to the data record. The data content is the specific business data carried by the data record.

[0040] Furthermore, the process of generating a standard coded dataset includes:

[0041] The system reads the local standardized datasets stored on the local hard drives of each front-end machine, merges data records from different business systems to form a dataset to be encoded, constructs encoding mapping rules, and performs mapping processing on the structured encoded fields and unstructured text fields in the dataset to be encoded. After mapping, the structured encoded fields are mapped to obtain standard codes, and the unstructured text fields are mapped to obtain standard terms. All data records after mapping are processed according to the same data structure as the local standardized dataset to generate a standard encoded dataset.

[0042] It should be further explained that during the merging process, the local standardized datasets stored on the local hard drives of each front-end machine are traversed and the data records are read. Using the patient identifier and the visit identifier as the association key, the data records from the hospital information system, laboratory information system, electronic medical record system, and image archiving and communication system are associated with the same visit event. The merged data records are arranged in ascending order according to the occurrence time to form the dataset to be encoded.

[0043] The process of constructing coding mapping rules is as follows: For structured coding fields such as diagnostic codes, test item codes, and drug codes, a lookup table mapping rule is adopted; a pre-set lookup table of local codes and standard codes of each business system is obtained. The lookup table records the mapping relationship between each local code and the corresponding standard code. For each data record in the dataset to be coded, its local code field is extracted and accurately searched in the lookup table. If a corresponding record exists, it is directly replaced with the standard code. If it does not exist, it is marked as unmapped and pushed to manual processing.

[0044] For unstructured text fields such as disease descriptions, examination findings, and medical records, a three-level mapping rule is used to extract key entities from the unstructured text. The text is then input into a pre-trained Chinese medical named entity recognition model. The model annotates the text character by character with entity type, recognizing key entities of predefined types such as disease names, symptom descriptions, examination findings, and body parts. The model uses a bidirectional long short-term memory network to encode the contextual features of the input text, outputting the emission score of each character corresponding to each entity type label. A conditional random field layer then applies global constraints to the label sequence, and the Viterbi algorithm is used to solve for the optimal label sequence, yielding the final entity recognition result. The formula for solving the optimal label sequence is:

[0045] ,

[0046] in, To obtain the optimal label sequence, The t-th character is predicted as the label. The score is output by the bidirectional long short-term memory network layer. For tags Transfer to label The constraint score is learned from the training data by the conditional random field layer, where n is the total number of characters in the input text; by jointly modeling the label score of each character and the transition constraints between adjacent labels, the globally optimal one is selected from all possible label sequences as the entity recognition result;

[0047] The extracted entities are uniformly converted into standard terms. Each identified entity is then matched against a pre-defined term thesaurus. Each standard term and its known synonyms in the thesaurus are stored with corresponding word vector representations. During matching, the cosine similarity between the entity's word vector and the word vectors of each candidate term is calculated.

[0048] ,

[0049] in, The word vector of the entity to be matched. The word vectors of candidate terms in the term thesaurus. and These are the magnitudes of the two vectors. The similarity value ranges from [−1, 1], with values ​​closer to 1 indicating greater semantic similarity. The candidate term with the highest similarity is selected; if the similarity is greater than or equal to a preset threshold... If the similarity is below a preset threshold, the entity will be replaced with a standard term; otherwise, the entity will be sent to manual processing.

[0050] The standardized term combinations are mapped to their corresponding standard codes. A set of standardized terms is input into a pre-trained text classification model. The model calculates the contribution weight of each term to the code classification based on an attention mechanism. The weighted term features are input into the classification layer, and the predicted probability of each candidate standard code is output. The standard code with the highest predicted probability is selected as the mapping result. If the highest probability is greater than or equal to a preset threshold... If the highest probability is below the preset threshold, a mapping will be automatically established; otherwise, it will be pushed to manual processing.

[0051] After the structured encoding and unstructured text are mapped, a standard encoded dataset is generated and stored in the central data server. The central data server is a server device pre-deployed in the hospital's network environment, located in the same local area network as each front-end machine. It is used to receive the local standardized datasets reported by each front-end machine, perform encoding mapping operations, and store the standard encoded datasets.

[0052] Furthermore, the process of parsing data records in a standard encoded dataset into semantic nodes includes:

[0053] Iterate through each data record in the standard encoded dataset and extract key information from the data records; create different types of nodes based on the extracted information and fill the nodes with the information.

[0054] Further explanation is needed regarding the specific transformation operations: When a patient identifier is read from a data record, it is determined whether the corresponding patient node already exists. If not, a patient node is created, and the patient identifier is filled into the node attributes. When a visit identifier is read, a visit node is also determined and created, and the visit identifier and occurrence time are filled into the node attributes. When a business type identifier is read, a corresponding node is created according to the identifier type—a diagnosis node is created and the diagnosis standard code is filled in when the business type identifier is diagnosis; a test node is created and the test item standard code and test value are filled in when the business type identifier is test; a medication node is created and the drug standard code is filled in when the medication is medication; and a document node is created and the standardized terminology is filled in when the document is document.

[0055] Furthermore, the process of establishing relationships between nodes and constructing a semantic network includes:

[0056] Centered on the consultation node, patient nodes, diagnosis nodes, test nodes, medication nodes, and document nodes within the same consultation event group are connected to the consultation node respectively; for examination discovery nodes contained in document nodes, they are connected to their respective document nodes; all nodes and edges constitute an initial semantic network with consultation events as the unit.

[0057] It should be further explained that if one node is a patient node and the patient identifier of another node is the same as that patient node, then an edge is established between the two nodes; if one node is a document node and the other node is a node extracted from that document node, then an edge is established between the two nodes; otherwise, no edge is established.

[0058] Furthermore, the process of detecting codes in subsequent standard coding datasets, incorporating codes that meet the conditions into the semantic network, and updating it includes:

[0059] When a new standard coding dataset is stored in the central data server, the standard codes and standard terms of each data record are extracted. The extracted standard codes and standard terms are compared with the existing nodes in the existing semantic network to determine whether they already exist. If they already exist, the new data record is associated with the existing node. If they do not exist, the code or term is determined to meet the inclusion criteria, a new node is created for the code or term, and the association between the new node and the corresponding medical visit node is established based on the medical visit identifier and business type identifier of the data record to which it belongs. The new node is then connected to the semantic network.

[0060] It should be further explained that the comparison process distinguishes between two scenarios: For standard encoding fields, exact string matching is used; if the new encoding is completely consistent with the encoding attributes of any node in the existing semantic network, it is determined that it already exists. For standard term fields, semantic similarity calculation is used for judgment. For the standard terms to be included, their attribute feature sets are extracted and compared with the attribute feature sets of nodes of the same type in the existing semantic network to calculate the semantic similarity between the two.

[0061] ,

[0062] in, This represents semantic similarity, with values ​​between 0 and 1. For standard codes or standard terms to be included; These are existing standard codes or standard terms of the same type in semantic networks; The set of attribute features for an encoding or term includes encoding type, term category, and business system to which it belongs. If the semantic similarity is greater than or equal to a preset threshold, it is determined that the encoding or term already exists in the semantic network, and the new data record is associated with the existing node. If the semantic similarity is less than the preset threshold, it is determined that the encoding or term meets the inclusion conditions, and the operation of creating a new node and establishing an association is performed. Existing nodes and associations remain unchanged.

[0063] Furthermore, the process of providing a unified data query interface and receiving query conditions includes:

[0064] A unified data query interface is built, providing a unified URL address as the query entry point. Query requests submitted by external systems or users are received through this address. Query conditions are extracted from the received requests and validated. Once validated, the query conditions are passed to the semantic network query engine for subsequent operations.

[0065] It should be further explained that external systems or users submit queries by sending network requests to this URL address and carrying query parameters. The query parameters include one or more of the following: patient identifier, visit identifier, diagnosis code, test item code, drug code, and time range.

[0066] The validation process checks whether the data format of the query parameters is valid, whether the required conditions are missing, and whether the query values ​​conform to the standard encoding rules; if the validation fails, an error message is returned, and if the validation passes, the query conditions are passed to the semantic network query engine.

[0067] Furthermore, the process of locating nodes and associated data in a semantic network includes:

[0068] The semantic network query engine receives the validated query conditions, determines the type of node to be matched based on the type of query conditions, and searches for the target node whose attribute value matches the query value in the corresponding type of node set in the semantic network. After locating the target node, it traverses along the association edges between nodes starting from the target node to obtain all associated nodes connected to the target node and their attribute information.

[0069] Furthermore, the process of returning a result set in a uniform format includes:

[0070] The attribute information of the target node and all its associated nodes obtained through traversal is assembled according to a preset unified format to generate a query result set; the result set is then returned to the query requester.

[0071] It should be further explained that the assembly process is as follows: taking the medical visit event as the basic unit, the attribute information of the diagnosis node, test node, medication node, and document node under the same medical visit node is included in the medical visit event entry; the result set is organized in a hierarchical structure. The first layer is patient information, which includes the patient identifier; the second layer is a list of medical visit events, with each medical visit event including the medical visit identifier and the occurrence time; the third layer is the business information under each medical visit event, which is divided into four categories according to business type: diagnosis information, test information, medication information, and document information. Each category of information is listed in a list format with the corresponding standard codes or standard terms.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications or equivalent substitutions made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A data integration method for online management of medical data, characterized in that, include: S1: Collect raw medical data from various business systems, preprocess it, and generate a local standardized dataset; S2: Read and merge local standardized datasets, construct encoding mapping rules, execute encoding mapping, and generate standard encoded datasets; S3: Parse the data records in the standard coding dataset into semantic nodes, establish the relationships between nodes, and construct a semantic network; detect the codes in the subsequent standard coding dataset, include the codes that meet the conditions into the semantic network, and update the semantic network; S4: Based on the semantic network, it provides a unified data query interface. The interface receives query conditions, locates the corresponding nodes and related data in the semantic network, and returns a result set in a unified format.

2. The data integration method for online management of medical data according to claim 1, characterized in that, The process of collecting raw medical data from various business systems, preprocessing it, and generating a local standardized dataset includes: Multiple front-end servers are deployed in the hospital's network environment. Each front-end server establishes a network connection with the database server of each business system through the hospital's local area network. By using change data capture technology, the front-end server reads the log files of each business system's database in real time. When a new or changed data record appears in the log file, the front-end server immediately captures the data record. The captured data record is then subjected to null value verification, format normalization, and text cleaning. The preprocessed data record is organized into a table format according to a preset data structure and stored in the local hard drive of the front-end server to generate a local standardized dataset.

3. The data integration method for online management of medical data according to claim 2, characterized in that, The process of reading and merging local normalized datasets includes: Read the local standardized datasets stored on the local hard drives of each front-end machine, use patient identifiers and visit identifiers as association keys to associate data records from different business systems to the same visit event, and arrange the merged data records in ascending order of occurrence time to form a dataset to be encoded.

4. The data integration method for online management of medical data according to claim 3, characterized in that, The process of constructing encoding mapping rules and executing encoding mapping includes: For structured coded fields, obtain a pre-defined lookup table of local codes and standard codes for each business system, extract the local code field from the data record, and accurately search and replace it with the standard code in the lookup table; For unstructured text fields, key entities are extracted from the text, the entities are matched with a preset term thesaurus and converted into standard terms, and then the standard terms are mapped to the corresponding standard codes. All data records after mapping are organized into a table format according to a unified data structure to generate a standard coded dataset.

5. The data integration method for online management of medical data according to claim 4, characterized in that, The process of constructing a semantic network includes: Traverse each data record in the standard coded dataset, extract key information from the data records, create different types of nodes based on the extracted information, and fill the information into the node attributes; take the consultation node as the center, connect the patient node, diagnosis node, test node, medication node and document node under the same consultation event to the consultation node respectively; all nodes and their associations constitute a semantic network with consultation events as the unit.

6. The data integration method for online management of medical data according to claim 5, characterized in that, The process of updating the semantic network includes: When a new standard coding dataset is stored, the standard codes and standard terms of each data record are extracted and compared with existing nodes in the existing semantic network. For standard coding fields, exact string matching is used to determine whether they already exist. For standard term fields, semantic similarity of attribute feature sets is calculated to determine whether they already exist. If they already exist, the new data record is associated with the existing node. If they do not exist, the inclusion condition is met, a new node is created and an association is established with the corresponding medical treatment node. The new node is then connected to the semantic network.

7. The data integration method for online management of medical data according to claim 6, characterized in that, The process of providing a unified data query interface and receiving query conditions includes: Build a unified data query interface to receive query requests submitted by external systems or users, extract query conditions from the received requests, and validate the query conditions.

8. The data integration method for online management of medical data according to claim 7, characterized in that, The process of returning the result set includes: The system receives the verified query conditions, determines the type of node to be matched based on the type of query conditions, searches for the target node in the semantic network whose attribute value matches the query value, obtains all related nodes connected to the target node and their attribute information, assembles the obtained information according to a preset unified format, generates a query result set, and returns it.

Images