Data set distribution difference-oriented electronic medical record data representation learning method and system

Abstract

The invention relates to a data set distribution difference-oriented electronic medical record data representation learning method and system, and the method comprises the steps: pre-training a source domain teacher model through a source domain data set of an existing disease, and extracting the health state representation information of a source domain disease patient; training a domain invariant feature extractor as a transition model, modeling general features through an adversarial training strategy, and establishing an independent extraction channel for private features to realize feature alignment among different domains; and migrating parameters of the transition model to a target domain emerging disease prediction model, and performing fine tuning in combination with target domain data to realize accurate prediction of emerging diseases. Through domain invariant feature extraction and model migration optimization, the generalization ability and prediction precision of the model on different data sets are significantly improved, clinical results of patients can be accurately predicted, accurate support is provided for medical decision, and a reliable foundation is laid for the cross-domain prediction problem in medical data analysis.

Description

Technical Field

[0001] The present invention belongs to the medical field, and particularly relates to an electronic medical record data representation learning method and system for dataset distribution differences. Background Art

[0002] With the wide application of electronic medical record (EMR) data in the medical field, improving the accuracy of disease prediction and patient management through data-driven methods has become an important research direction. However, due to the diversity of data sources and the complexity of different disease characteristics, there are significant difficulties in the following aspects for existing disease prediction and patient management methods:

[0003] Cross-dataset feature misalignment problem - due to the differences in data distributions between different hospitals or different diseases, existing models are prone to performance degradation in prediction tasks for new datasets or emerging diseases, and it is difficult to achieve cross-domain generalization ability.

[0004] Scarcity of emerging disease data problem - for emerging diseases, the training data that can be referenced and used is very scarce, which results in general prediction models being difficult to achieve good prediction effects.

[0005] Therefore, how to utilize the existing disease data that is already well-known, combined with a small amount of data reference for emerging diseases, to accurately predict the disease progression of patients (such as death risk, length of hospital stay, etc.) is still an unsolved problem. Summary of the Invention

[0006] Aiming at the defects existing in the prior art, the purpose of the present invention is to provide an electronic medical record data representation learning method and system for dataset distribution differences, a cross-domain prediction model based on representation learning and domain adversarial learning, which can effectively handle the data scarcity and domain feature misalignment problems in the medical data of emerging diseases, and improve the generalization ability of the model on different datasets by extracting domain-invariant feature representations, and can accurately predict the clinical outcomes of patients (such as death risk, length of hospital stay, etc.), providing precise support for medical decision-making. It can effectively solve the cross-domain prediction problem in medical data analysis, improve the efficiency and accuracy of emerging disease diagnosis and treatment, and provide important technical support for the development of intelligent medicine.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] In the first aspect, an electronic medical record data representation learning method for dataset distribution differences, the method includes the following steps:

[0009] S1. Use the source domain dataset of existing diseases to pre-train the source domain teacher model;

[0010] S2. Train a domain-invariant feature extractor as a transition model according to the pre-trained source domain teacher model, and perform domain adversarial learning on the transition model to achieve feature alignment between different domains;

[0011] S3. Transfer the parameters of the transition model to the target domain emerging disease prediction model, and perform fine-tuning in combination with the target domain data to achieve accurate prediction of emerging diseases.

[0012] Furthermore, the source domain teacher model has a multi-channel gated recurrent unit network as the core structure.

[0013] Furthermore, the source domain dataset of existing diseases is the electronic medical records of patients with existing diseases, and the electronic medical records include the basic information of the patients and the physiological index data of each visit.

[0014] Furthermore, step S2 includes the following sub-steps:

[0015] S21. For the common features between diseases, input the source domain dataset and the data of emerging diseases into the multi-channel gated recurrent unit network of the source domain teacher model at the same time, generate the embedding matrix of the common features, and constrain the distribution consistency of the embedding matrix of the common features through a domain classifier;

[0016] S22. For the private features between diseases, establish independent feature extraction channels to generate the corresponding private feature embedding matrices respectively, and train the private feature embedding matrices respectively depending on the supervision signals in the domains where the diseases are located;

[0017] S23. Use the embedding matrix of the common features to obtain the patient health status representation generated by the transition model, and align this representation with the health status representation generated by the source domain teacher model, so as to constrain the transition model to ensure sufficient consistency with the pre-trained source domain teacher model while considering the target domain, and extract the same hidden layer space information.

[0018] Furthermore, in step S21, distillation and domain adversarial training strategies are adopted, and the embedding matrix of the common features is constrained by introducing a domain classifier.

[0019] Furthermore, step S3 includes the following sub-steps:

[0020] S31. For the common features, directly transfer the parameters of the multi-channel gated recurrent unit network corresponding to the features of the transition model to the target domain prediction model;

[0021] S32. For the private features, calculate the DTW distance between the private features of the emerging diseases and the source domain private features through the dynamic time warping algorithm, and find the features with the most similar distribution in time information for parameter transfer.

[0022] Further, the objects predicted by the source domain teacher model and the target domain emerging disease prediction model include the patient's death risk and length of hospital stay.

[0023] In a second aspect, an electronic medical record data representation learning system for dataset distribution differences, the system adopts an electronic medical record data representation learning method for dataset distribution differences described in the first aspect and any optional implementation manner thereof of the present invention, and the system includes:

[0024] A source domain model pre-training module, configured to pre-train a source domain teacher model by using a source domain dataset of existing diseases;

[0025] A transition model training module, configured to train a domain-invariant feature extractor as a transition model according to the pre-trained source domain teacher model, and perform domain adversarial learning on the transition model to achieve feature alignment between different domains;

[0026] A transition model parameter migration module, configured to migrate the parameters of the transition model to the target domain emerging disease prediction model and perform fine-tuning in combination with the target domain data to achieve accurate prediction of emerging diseases.

[0027] Further, the source domain teacher model has a multi-channel gated recurrent unit network as the core structure.

[0028] Further, the transition model training module includes an adversarial training unit and an independent feature extraction channel, configured to model general features and private features respectively.

[0029] The beneficial technical effects of the present invention are as follows: The electronic medical record data representation learning method and system for dataset distribution differences disclosed in the present invention utilize the information mined from the electronic medical record data of hospital patients to help doctors perform rapid clinical diagnosis for emerging diseases.

[0030] Facing the problem of scarce data on emerging diseases, the present invention pre-trains a teacher model on a large-scale source domain dataset and uses a domain-invariant feature extractor to capture domain-invariant features related to downstream tasks, enabling the model to quickly adapt to the target task. In the scenario of emerging disease prediction, in the case of scarce model training data, the method and system disclosed in the present invention still have high prediction accuracy and convergence speed.

[0031] In cross - domain data, due to the significant differences in the distribution of shared features and private features between different data sets, traditional models are prone to performance degradation during the migration process. By designing a domain - invariant feature extractor, this invention combines adversarial learning and dynamic time warping technology, effectively aligning the distributions of shared features in the source domain and the target domain, while fully exploring the information potential of private features in the target domain, achieving the efficient fusion of multi - domain features. It significantly alleviates the negative impact of feature misalignment on model performance, especially demonstrating excellent generalization ability in emerging disease and data - scarce scenarios.

[0032] In addition, the existing disease and emerging disease data used in this invention support different prediction tasks, such as predicting the risk of patient death and the length of hospital stay, which can meet the diverse needs of clinical prediction tasks and provide comprehensive support for the rational allocation of medical resources and the formulation of personalized treatment plans.

[0033] It not only optimizes the doctor's workflow but also promotes the rational allocation of medical resources. It helps the development of precision medicine, introduces intelligent tools to the medical industry, greatly improves the diagnosis and treatment efficiency, and reduces costs. For patients, this technical solution enhances their sense of control over their own conditions, reduces anxiety, and improves the medical experience. It has far - reaching significance in promoting the modernization of clinical medicine and the public health system. Brief Description of the Drawings

[0034] Figure 1 It is a flowchart of an electronic medical record data representation learning method for dataset distribution differences shown in Embodiment 1 of the present invention;

[0035] Figure 2 It is a flowchart of step S1 in an electronic medical record data representation learning method for dataset distribution differences shown in Embodiment 1 of the present invention;

[0036] Figure 3 It is a flowchart of step S2 in an electronic medical record data representation learning method for dataset distribution differences shown in Embodiment 1 of the present invention. Detailed Embodiments

[0037] The present invention will be further described below in conjunction with the drawings and specific embodiments.

[0038] Term Explanation

[0039] Electronic Medical Record

[0040] An Electronic Medical Record (EMR) is also known as a computerized medical record system or a computer-based patient record. It is a digital medical record stored, managed, transmitted, and reproduced using electronic devices (such as computers and health cards), replacing the handwritten paper-based medical record. Its content includes all the information in the paper-based medical record.

[0041] Representation Learning

[0042] Representation learning is a deep learning method whose purpose is to automatically learn effective feature representations from raw data through a model without manual feature design. In many tasks, the high-dimensional complexity and diversity of data make it difficult to directly analyze and model using raw data. Representation learning significantly improves the prediction performance and generalization ability of the model by automatically extracting low-dimensional, compact, and semantically meaningful features. Especially in deep learning, representation learning is widely used in fields such as image classification, natural language processing, and medical data analysis.

[0043] Dynamic Time Warping

[0044] Dynamic Time Warping (DTW) is a classic sequence alignment algorithm used to measure the similarity between two time series, even if they may have non-linear alignment on the time axis. The main advantage of DTW is that it allows sequences to be stretched or compressed locally to find the optimal matching path. This method is widely used in fields such as speech recognition, handwriting recognition, and medical time series analysis. For example, when processing multi-dimensional time series in electronic medical records, DTW can effectively align the disease course data of different patients for unified analysis.

[0045] Domain-Adversarial Learning

[0046] Domain-adversarial learning is a machine learning technique used to solve cross-domain problems, aiming to reduce the difference in feature distributions between different domains (such as the source domain and the target domain) through adversarial training. In this method, by introducing a domain discriminator, the model can learn domain-invariant feature representations, thereby improving the generalization ability in the target domain. Domain-adversarial learning is widely used in tasks such as transfer learning and domain adaptation, especially suitable for dealing with the problem of model performance degradation caused by data distribution shift, such as cross-hospital or cross-disease prediction tasks in medical data analysis.

[0047] Example 1

[0048] Such as Figure 1As shown, an embodiment of the present invention provides an electronic medical record data representation learning method for dataset distribution differences, and the method includes the following steps:

[0049] S1. Use the source domain dataset of existing diseases to pre-train a source domain teacher model.

[0050] First, use the source domain dataset of existing diseases to pre-train a mature source domain teacher model to provide a strong supervision signal for the prediction of emerging diseases.

[0051] Input the source domain dataset of existing diseases into the source domain teacher model with a multi-channel gated recurrent unit network (GRU) as the core structure, and the health status representation information of patients with source domain diseases can be obtained, which provides a medical information learning reference standard that can be imitated for the subsequent learning of the source domain teacher model on emerging diseases.

[0052] The source domain dataset of existing diseases is the electronic medical records of patients with existing diseases, and the electronic medical records include the basic information of patients and the physiological index data of each visit.

[0053] By using the source domain dataset to pre-train the source domain teacher model, the problem of scarce target domain data is solved, providing strong support for subsequent transfer learning. The multi-channel structure of the teacher model can effectively embed different medical features and capture their dynamic relevance, thus significantly improving the generalization ability of the teacher model for complex medical scenarios. In addition, through comprehensive feature extraction and optimization, when the present invention processes cross-domain migration of electronic medical record data, this structure significantly reduces the training error and prediction bias of the teacher model.

[0054] S2. According to the pre-trained source domain teacher model, train a domain-invariant feature extractor as a transition model, and perform domain adversarial learning on the transition model to achieve feature alignment between different domains.

[0055] As Figure 2 shown, for a certain emerging disease, according to the pre-trained source domain teacher model, train a domain-invariant feature extractor alone as a transition model between different domains.

[0056] Due to the particularity of electronic medical record data, there may be many misaligned index features between the data of emerging diseases and the existing disease datasets, which makes it impossible for excellent prediction models trained on the source domain to directly predict on emerging disease data. In the embodiment of the present invention, the transition model is trained. For the common features and private features between diseases, the common features and private features are modeled respectively. By constraining the feature distribution shift, feature alignment between different domains is achieved, thereby capturing domain-invariant features related to downstream tasks and further improving the generalization ability of the source domain teacher model.

[0057] For the common features between diseases, the source domain dataset and the data of emerging diseases are simultaneously input into the multi-channel gated recurrent unit network of the source domain teacher model to generate the embedding matrix of the common features. To ensure the distribution consistency of the generated feature representations across different domains, distillation and domain adversarial training strategies are adopted. By introducing a domain classifier (DomainClassifier) to constrain the embedding matrix of the common features, it becomes difficult for the domain classifier to distinguish them, ensuring that the embedding matrices of the common features generated by the transition model for the emerging diseases in the source domain and the target domain have similar distributions.

[0058] For the private features between diseases, independent feature extraction channels are established respectively. The training of the private feature embedding matrix only depends on the supervision signals in its own domain, avoiding interference with the domain alignment of the common features.

[0059] Finally, using the embedding matrix of the common features, the patient health state representation generated by the transition model is obtained. This representation is aligned with the health state representation generated by the source domain teacher model, constraining the transition model to be consistent enough with the pre-trained source domain teacher model while considering the target domain, and extracting the same hidden layer space information.

[0060] S3. Transfer the parameters of the transition model to the target domain emerging disease prediction model and fine-tune it with the target domain data to achieve accurate prediction of emerging diseases.

[0061] As Figure 3 shown, after the training of the transition model is completed, the feature representation ability learned from the source domain is transferred to the target task, and fine-tuned with the target domain data to achieve the adaptation and optimization of the target domain emerging disease prediction model for the target task.

[0062] As a domain-invariant feature extractor, the transition model can extract the common information between different diseases. When finally applied to emerging disease clinics, the target domain emerging disease prediction model is fine-tuned based on the parameters learned from the transition model, which can avoid starting from scratch based on the scarce emerging disease data and enable rapid clinical application.

[0063] When migrating the transition model parameters to the target domain emerging disease prediction model, it is also necessary to distinguish between general features and private features. For general features, directly migrate the multi-channel gated recurrent unit network parameters of the corresponding features of the transition model to the target domain emerging disease prediction model. For the private features of emerging diseases, since the source domain teacher model cannot directly supervise these features, by introducing the dynamic time warping algorithm (DTW), calculate the DTW distance between the private features of emerging diseases and all the previous source domain features, and find the features with the most similar distribution in terms of time information as a similar feature for parameter migration, ensuring that for private features, the target domain prediction model can still follow the traces.

[0064] Example 2

[0065] An embodiment of the present invention provides an electronic medical record data representation learning system for dataset distribution differences. The system adopts an electronic medical record data representation learning method according to Embodiment 1 of the present invention and any of its optional embodiments. The system includes:

[0066] A source domain model pre-training module for pre-training a source domain teacher model using the source domain dataset of existing diseases;

[0067] A transition model training module for training a domain-invariant feature extractor as a transition model based on the pre-trained source domain teacher model, and performing domain adversarial learning on the transition model to achieve feature alignment between different domains;

[0068] A transition model parameter migration module for migrating the parameters of the transition model to the target domain emerging disease prediction model and fine-tuning in combination with the target domain data to achieve accurate prediction of emerging diseases.

[0069] The source domain teacher model has a multi-channel gated recurrent unit network as the core structure for extracting the health status representation information of source domain disease patients.

[0070] The transition model training module includes an adversarial training unit and an independent feature extraction channel for respectively modeling general features and private features.

[0071] Survival duration prediction experiments for specific results were conducted on multiple datasets.

[0072] Predict the survival time of patients on the nephrology dataset of Peking University Third Hospital. Compared with the baseline method, the results of MSE (mean square error) and MAD (median absolute deviation) are as follows:

[0073] Method Comparison MSE MAD GRU 700.375 21.710 Transformer 666.060 20.290 Concare 646.039 20.418 StageNet 659.011 21.579 TimeNet 690.704 21.232 T-LSTM 640.689 20.419 Distcare 632.932 20.723 Dann 633.051 20.072 Codats 648.445 20.367 Ours 610.231 19.987

[0074] Predicting the outcomes and survival times of patients on the COVID-19 datasets of Tongji Hospital and Spanish hospitals, and comparing with the baseline methods, the results of MSE (Mean Squared Error) and MAD (Median Absolute Deviation) are as follows:

[0075]

[0076]

[0077] Predicting the outcomes and survival times of patients on the COVID-19 datasets of Tongji Hospital and Spanish hospitals, using 1 / 5 of the samples as the training set for training, simulating the scenario of scarce data volume in emerging diseases, and comparing the effects with the baseline methods are as follows:

[0078]

[0079] The results show that the electronic medical record data representation learning method and system for dataset distribution differences disclosed in the present invention are always superior to the baseline methods in performance and have a higher training convergence speed when dealing with a limited amount of data.

[0080] As can be seen from the above embodiments, in order to solve the problems of extremely scarce electronic medical record data and cross-domain feature misalignment in the context of emerging diseases, the electronic medical record data representation learning method and system for dataset distribution differences disclosed in the present invention achieve accurate prediction of new diseases through domain-invariant feature extraction and model transfer optimization. It can solve the problem of feature distribution shift in the electronic medical record dataset, thereby realizing clinical prediction under the conditions of limited data and feature misalignment. By introducing a transition model between the source domain and target domain datasets, capturing domain-invariant features related to downstream tasks, through the combination of adversarial training and dynamic time warping technology, extracting common features and private features between different diseases, coping with the problem of feature alignment, and through the strong supervision of the source domain teacher model, ensuring that the emerging disease prediction model in the target domain has a trace to follow when learning emerging diseases, and then promoting the learning of the same domain-invariant encoder to achieve outstanding prediction effects.

[0081] The methods and systems described in the present invention are not limited to the embodiments described in the specific embodiments. Those skilled in the art can obtain other embodiments according to the technical solutions of the present invention, which also belong to the scope of the technical innovation of the present invention.

Claims

1. A method for learning electronic medical record data representation based on dataset distribution differences, comprising the following steps: S1. Pre-train the source domain teacher model using the existing source domain dataset of the disease; S2. Based on the pre-trained source domain teacher model, a domain-invariant feature extractor is trained as a transition model, and domain adversarial learning is performed on the transition model to achieve feature alignment between different domains. S3. Migrate the parameters of the transition model to the target domain emerging disease prediction model and fine-tune it based on the target domain data to achieve accurate prediction of emerging diseases.

2. The electronic medical record data representation learning method for dataset distribution differences according to claim 1, characterized in that: The source domain teacher model is based on a multi-channel gated recurrent unit network as its core structure.

3. The electronic medical record data representation learning method for dataset distribution differences according to claim 1, characterized in that: The source domain dataset of the existing disease is the electronic medical records of patients with the existing disease, and the electronic medical records include the basic information of the patients and physiological indicator data of each visit.

4. The electronic medical record data representation learning method for data set distribution differences according to claim 1, characterized in that: Step S2 includes the following sub-steps: S21. For common features between diseases, the source domain dataset and the data of emerging diseases are simultaneously input into the multi-channel gated recurrent unit network of the source domain teacher model to generate an embedding matrix of common features. The distribution consistency of the embedding matrix of common features is constrained by the domain classifier. S22. For the private features between diseases, establish independent feature extraction channels to generate corresponding private feature embedding matrices respectively, and train the private feature embedding matrices respectively based on the supervisory signals of the disease domain; S23. Use the embedding matrix of common features to obtain the patient health status representation generated by the transition model, and use this representation to align with the health status representation generated by the source domain teacher model. Constrain the transition model to consider the target domain while ensuring sufficient consistency with the pre-trained source domain teacher model, and extract the same latent space information.

5. The electronic medical record data representation learning method for dataset distribution differences according to claim 1, characterized in that: In step S21, distillation and domain adversarial training strategies are adopted to constrain the embedding matrix of common features by introducing domain classifiers.

6. The electronic medical record data representation learning method for data set distribution differences according to claim 4, characterized in that: Step S3 includes the following sub-steps: S31. For common features, directly transfer the multi-channel gated recurrent unit network parameters corresponding to the transition model to the target domain prediction model; S32. For private features, the DTW distance between the private features of emerging diseases and the private features of the source domain is calculated through the dynamic time warping algorithm, and the features with the most similar distribution in time information are found for parameter migration.

7. The electronic medical record data representation learning method for dataset distribution differences according to claim 1, characterized in that: The objects predicted by the source domain teacher model and the target domain emerging disease prediction model include the patient's mortality risk and hospitalization duration.

8. A system for learning electronic medical record data representation based on data set distribution differences, the system adopting the method for learning electronic medical record data representation based on data set distribution differences according to any one of claims 1 to 7, characterized in that: The system comprises: The source domain model pre-training module is used to pre-train the source domain teacher model using the existing source domain dataset of the disease; The transition model training module is used to train a domain-invariant feature extractor as a transition model based on the pre-trained source domain teacher model, and perform domain adversarial learning on the transition model to achieve feature alignment between different domains; The transition model parameter migration module is used to migrate the parameters of the transition model to the emerging disease prediction model in the target domain, and fine-tune it in combination with the target domain data to achieve accurate prediction of emerging diseases.

9. The electronic medical record data representation learning system for dataset distribution differences according to claim 8, characterized in that: The source domain teacher model is based on a multi-channel gated recurrent unit network as its core structure.

10. The electronic medical record data representation learning system for dataset distribution differences according to claim 8, characterized in that: The transition model training module includes an adversarial training unit and an independent feature extraction channel for modeling common features and private features respectively.

Images