A semi-supervised conditional transfer learning method for medical feature generation models

Abstract

This invention discloses a semi-supervised conditional transfer learning method for medical feature generation models, comprising: S1, preprocessing the input source domain and target domain data; S2, filtering the source domain data according to preset rules to obtain filtered source domain data; S3, dividing the filtered source domain data into a pre-trained training set and a test set based on the rarity of features; S4, constructing a feature generation model based on the characteristics of the source domain data; S5, filtering the target domain data according to preset rules to obtain filtered target domain data; S6, generating a target domain data representation; S7, generating a dataset with the same dimensionality as the source domain data; S8, merging the filtered source domain data and the target domain data set to form a hybrid dataset; S9, retraining based on the hybrid dataset to obtain a feature generation model with the characteristics of the target domain data; S10, repeating steps S5-S9. This invention enhances the characteristics of the original data through various feature enhancement methods.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and more specifically, to a semi-supervised conditional transfer learning method for medical feature generation models. Background Technology

[0002] In 2018, the article "Multicenter Validation of a Sepsis Prediction Algorithm Using Vital Signs Data from Emergency Departments, General Wards, and Intensive Care Units Only" published in the open edition of the *British Medical Journal*, Volume 8, Issue 1, proposed a homogeneous transfer learning method for sepsis prediction. The same features are extracted from two isomorphic datasets, but their data distributions differ. This work uses six vital sign features learned from the source domain to predict sepsis in the target domain. However, isomorphic transfer learning is incompatible with most (if not all) healthcare applications because they are typically associated with different functionalities and data structures.

[0003] In 2022, the paper "Efficient Conditional Pre-training for Transfer Learning," published in the *IEEE / CFF Conference Proceedings on Computer Vision and Pattern Recognition* (pages 4241–4250), proposed an efficient filtering method for selecting relevant subsets from pre-training datasets. Furthermore, it was found that reducing image resolution during the pre-training step offers a significant trade-off between cost and performance. The technique was validated by pre-training on ImageNet in both unsupervised and supervised settings and fine-tuning it on different target datasets and tasks. The proposed method significantly reduces pre-training costs and provides substantial performance improvements. Finally, by adjusting available models on subsets and pre-training on datasets filtered from larger datasets, the current standard for ImageNet pre-training is improved by 1–3%.

[0004] In summary, existing technologies cannot transfer existing models to scenarios with heterogeneous data, and performance degrades after transfer due to varying data distributions. Therefore, it is indeed necessary to develop a semi-supervised conditional transfer learning method for medical feature generation models. Summary of the Invention

[0005] The purpose of this invention is to provide a semi-supervised conditional transfer learning method for medical feature generation models, so as to overcome the shortcomings of existing technologies.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A semi-supervised conditional transfer learning method for medical feature generation models includes the following steps:

[0008] S1. Perform feature preprocessing on the input source domain and target domain data;

[0009] S2. Filter the source domain data according to the preset rules to obtain the filtered source domain data;

[0010] S3. Divide the filtered source domain data into a pre-training training set and a test set according to whether the features are rare;

[0011] S4. Construct a feature generation model based on the characteristics of source domain data;

[0012] S5. Filter the target domain data according to the preset rules to obtain the filtered target domain data;

[0013] S6. Input the filtered target domain data features into the feature generation model based on the characteristics of the source domain data to generate a target domain data representation.

[0014] S7. Combine the target domain data representation generated in step S6 as a pseudo-label with the target domain data filtered in step S5 to generate a dataset with the same dimension as the source domain data.

[0015] S8. Combine the source domain data selected in step S2 with the target domain set generated in step S7 to form a mixed dataset;

[0016] S9. Retrain the hybrid dataset to obtain a feature generation model with the characteristics of the target domain data;

[0017] S10. Repeat steps S5-S9 until the performance of the feature generation model with the characteristics of the target domain data reaches the preset requirements. Then, transfer the feature generation model to the target domain dataset using a semi-supervised conditional transfer learning method.

[0018] Further, step S1 specifically includes:

[0019] S10. Construct and record the actual testing data of all patients throughout the entire observation period;

[0020] S11. Infer missing values ​​from known values;

[0021] S12. Set a sliding window to observe the recent historical data.

[0022] S13. A forward imputation method was used to fill the missing values ​​in the longitudinal data with the latest available measurements.

[0023] S14. Construct clinical scoring features.

[0024] Furthermore, in step S2, the filtering of source domain data according to preset rules specifically involves removing data with missing values ​​greater than 80%.

[0025] Furthermore, step S3 specifically includes:

[0026] S30. The source domain data after filtering is filtered using the data statistics enhancement component to obtain multiple one-dimensional matrices;

[0027] S31. Divide the filtered source domain data into a pre-trained training set and a test set.

[0028] Furthermore, in step S4, a tree-based ensemble learning algorithm is used to construct a feature generation model based on the characteristics of the source domain data.

[0029] Furthermore, in step S5, filtering the target domain data according to preset rules specifically involves removing data with more than 80% missing values.

[0030] Compared with existing technologies, the advantages of this invention are as follows: This invention combines the characteristics of semi-supervised transfer learning and conditional transfer learning, improving the universality and performance reduction of medical models caused by different data characteristics. The machine learning method adopted by this invention ensures that all parameters are intuitive and transparent, have interpretability, and are easy to adjust according to actual application scenarios. At the same time, this invention strengthens the characteristics of the original data through various feature enhancement methods. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of the semi-supervised conditional transfer learning method for medical feature generation models according to the present invention.

[0033] Figure 2 This is a flowchart of steps S1-S4 in this invention. Detailed Implementation

[0034] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0035] See Figure 1As shown, this embodiment discloses a semi-supervised conditional transfer learning method for medical feature generation models, mainly comprising two parts: first, modeling the source domain data to construct a model based on the characteristics of the source domain data; and then, a transfer learning method that transfers the model constructed in the source domain to the target domain. After inputting the source domain and the target domain (which have different dimensions, the source domain has more features than the target domain, and the source domain contains features of the target domain, i.e., all features of the target domain are present in the source domain, but features of the source domain may not be present in the target domain), its basic detection part follows... Figure 1 The basic process shown is executed. This method specifically includes the following steps:

[0036] Step S1: Perform feature preprocessing on the input source domain and target domain data. The data includes many patients, and each patient has several observation points.

[0037] For details, see the pseudocode. Figure 2 The input raw data is preprocessed as follows: (1. Add two features, missingness features and differential features, to record the characteristics of the original dataset; 2. Empirical imputation of missing data; 3. Sliding window data augmentation; 4. Forward imputation; 5. Add some commonly used medical features).

[0038] In this step, a data frame encompassing the entire observation period for all patients is first constructed. Before preprocessing, missing data in the raw data are described from two perspectives: missing data and discrimination. Missing data records the measurement frequency and time interval of the raw data. This distinction reflects the difference between the current and previous observation values ​​of the same feature. Two new features are added based on the actual detection data in the raw data.

[0039] Subsequently, some empirical methods are used to infer missing values ​​from known values ​​(e.g., systolic blood pressure (SBP), diastolic blood pressure (DBP), and mean arterial pressure (MAP), which can be calculated using the formula MAP = DBP + 1 / 3(SBP + DBP)). Lateral imputation of the two-dimensional matrix: Based on some data from the current observation point and using medical common sense, the missing features of the current observation point are imputed.

[0040] Furthermore, electronic medical record data may contain various elements, including demographics, vital signs, and laboratory and test results. For all vital signs and laboratory test results, first- and second-order statistical characteristics—minimum, maximum, mean, median, standard deviation, and variance—are derived using a six-hour sliding window to better reflect changes during the observation period. These statistical characteristics are crucial for indicating processes that are significant in the time domain, thus improving model performance. Specifically, by setting a sliding window, recent historical records are recorded and added as features to each observation point.

[0041] Next, a forward imputation method was used to fill in the missing values ​​of the longitudinal data with the latest available measurements. Typically, these features are imputed with available corresponding values ​​to avoid forward bias in the predictions. Otherwise, the "NAN" is reserved for other features that have no values ​​at all. This is then performed longitudinally on the two-dimensional matrix.

[0042] Finally, eight clinical scoring features were constructed to further enhance model performance. In addition to the SOFA score (Sequential Organ Failure Assessment), six components that contribute to the SOFA score were selected as clinical scoring features. This was done to gain a clearer understanding of the rationale behind the SOFA score. Quick-SOFA (qSOFA) was also selected as one of the eight clinical scoring features. Compared to SOFA, qSOFA is a clinically accessible and rapid indicator commonly used for diagnosis and reflecting a patient's infection status. Based on the preceding data, some scoring features were calculated for each observation point and added as new features to each observation point.

[0043] Step S2: Filter the source domain data according to the preset rules (remove data with more than 80% missing values) to obtain the filtered source domain data.

[0044] Step S3: Divide the filtered source domain data into a pre-training training set and a test set according to whether the features are rare. Rare means that they do not exist in the target domain data.

[0045] Specifically, the steps are as follows: S30, use the data statistics enhancement component to filter the source domain data to obtain multiple one-dimensional matrices; here, the basic features serve as the knowledge source, which is a multi-dimensional data matrix, while the generated features, as pre-trained labels, are the parts that need to be predicted. Here, the generated feature part (multi-dimensional test data matrix) of the data filtered by the data statistics enhancement component is decomposed into multiple one-dimensional matrices to allow for more targeted modeling of features with different distributions and characteristics. Step S31, divide the filtered source domain data into a pre-trained feature set (basic features) and labels (generated features) according to the features, that is, match the pre-trained set with each label subset and the feature set to form multiple datasets for training machine learning models.

[0046] Step S4: Construct a feature generation model based on the characteristics of the source domain data. Tree-based ensemble learning algorithms can be used, or other algorithms can be used to generate pre-trained models. However, traditional machine learning methods are less effective than this approach, while deep neural network methods lack interpretability and are unstable. The hyperopt automatic parameter tuning method uses five-fold cross-validation on the data and then takes the average of the cross-validation results as the final result.

[0047] Step S5: Filter the target domain data according to the preset rules (remove data with more than 80% missing values) to obtain the filtered target domain data.

[0048] Step S6: Input the filtered target domain data features into the feature generation model based on the characteristics of the source domain data to generate a target domain data representation.

[0049] Step S7: Combine the target domain data representation generated in step S6 as a pseudo-label with the target domain data filtered in step S5 to generate a dataset with the same dimensions as the source domain data.

[0050] Step S8: Combine the source domain data selected in step S2 with the target domain set generated in step S7 to form a hybrid dataset.

[0051] S9. A feature generation model with the characteristics of the target domain data is obtained by retraining based on the hybrid dataset.

[0052] Step S10: Repeat steps S5-S9 until the performance of the feature generation model with the characteristics of the target domain data reaches the preset requirements (that is, the performance no longer improves). Then, transfer the feature generation model to the target domain dataset using a semi-supervised conditional transfer learning method.

[0053] This invention combines the features of semi-supervised transfer learning and conditional transfer learning, improving the universality and performance reduction of medical models caused by different data characteristics. The machine learning method used in this invention ensures that all parameters are intuitive and transparent, have interpretability, and can be easily adjusted according to actual application scenarios. At the same time, this invention enhances the characteristics of the original data through various feature enhancement methods.

[0054] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, the patent owner may make various modifications or alterations within the scope of the appended claims, as long as they do not exceed the protection scope described in the claims of the present invention, they shall be within the protection scope of the present invention.

Claims

1. A semi-supervised conditional transfer learning method for medical feature generation models, characterized in that, Includes the following steps: S1. Perform feature preprocessing on the input source domain and target domain data; Step S1 specifically includes: S10, constructing and recording the actual test results for all patients throughout the entire observation period; S11, inferring missing values ​​from known values; S12, setting up a sliding window to observe the recent historical data; S13, using a forward imputation method to fill the missing values ​​in the longitudinal data with the latest available measurements; S14, constructing clinical scoring features. S2. Filter the source domain data according to the preset rules to obtain the filtered source domain data; S3. The filtered source domain data is divided into a pre-trained training set and a test set according to whether the features are rare. Rare means that they do not exist in the target domain data. Step S3 specifically includes: S30. The filtered source domain data is filtered using a data statistics enhancement component to obtain multiple one-dimensional matrices; S31. The filtered source domain data is divided into a pre-trained feature set and a label according to the features. S4. Construct a feature generation model based on the characteristics of source domain data; S5. Filter the target domain data according to the preset rules to obtain the filtered target domain data; S6. Input the filtered target domain data features into the feature generation model based on the characteristics of the source domain data to generate a target domain data representation. S7. Combine the target domain data representation generated in step S6 as a pseudo-label with the target domain data filtered in step S5 to generate a dataset with the same dimension as the source domain data. S8. Combine the source domain data selected in step S2 with the target domain set generated in step S7 to form a mixed dataset; S9. Retrain the hybrid dataset to obtain a feature generation model with the characteristics of the target domain data; S10. Repeat steps S5-S9 until the performance of the feature generation model with the characteristics of the target domain data reaches the preset requirements. Then, transfer the feature generation model to the target domain dataset using a semi-supervised conditional transfer learning method.

2. The semi-supervised conditional transfer learning method for medical feature generation models according to claim 1, characterized in that, In step S2, filtering the source domain data according to preset rules specifically involves removing data with more than 80% missing values.

3. The semi-supervised conditional transfer learning method for medical feature generation models according to claim 1, characterized in that, In step S4, a tree-based ensemble learning algorithm is used to construct a feature generation model based on the characteristics of the source domain data.

4. The semi-supervised conditional transfer learning method for medical feature generation models according to claim 1, characterized in that, In step S5, the target domain data is filtered according to preset rules, specifically by removing data with more than 80% missing values.

Images