A mixed type health data minority class enhancement method based on correlation regularization double generation

Abstract

This invention discloses a minority class augmentation method for mixed-type health data based on correlation regularization dual generation, belonging to the field of health and medical data processing technology. The method includes the following steps: acquiring a health dataset and separating continuous features and binary classification features; using interpolation to oversample the continuous features to generate synthetic continuous features; constructing a conditional generator based on a multilayer perceptron to predict the probability distribution of binary classification features using continuous features as conditions; introducing a correlation regularization loss function to constrain the correlation matrix between binary classification features and the continuous-binary cross-domain correlation matrix of the generated samples to remain consistent with the real samples during training; using the trained conditional generator to generate corresponding binary classification features for the synthetic continuous features, and obtaining binary classification values ​​through Bernoulli sampling; concatenating the synthetic continuous features with the generated binary classification features to obtain complete synthetic minority class samples. This invention effectively solves the problem that traditional interpolation methods cannot handle mixed-type data and destroy the correlation structure between features through a conditional generation mechanism and correlation regularization constraints. The generated synthetic samples outperform traditional methods in terms of statistical similarity and data utility.

Description

Technical Field

[0001] This invention relates to the field of health and medical data processing technology, specifically to a minority class augmentation method for mixed-type health data based on Correlation-Regularized Dual Generation (CRDG), which is applicable to the preprocessing and sample augmentation of imbalanced health data in scenarios such as health status assessment and chronic disease risk prediction. Background Technology

[0002] With the accelerating aging of the population, chronic disease risk assessment and health status prediction based on health data are of great significance for public health decision-making and personalized health management.

[0003] In real-world health datasets, the distribution of samples across different health status categories is often severely imbalanced. Taking the CHARLS dataset as an example, samples with a health status of "very good" (level 5) account for only about 11%, representing a typical minority class. Machine learning models trained on imbalanced datasets tend to favor the majority class, resulting in a severe deficiency in their ability to identify the minority class, thus impacting the clinical application value of health risk assessment models.

[0004] Traditional data augmentation methods mainly include SMOTE (Synthetic Minority Oversampling) and its variants (Borderline-SMOTE, ADASYN, etc.). These methods generate synthetic samples by linear interpolation between minority class samples, effectively alleviating class imbalance. However, health data typically contains both continuous features (such as physical measurements and blood test indicators like blood pressure, blood sugar, and blood lipids) and binary features (such as health behavior and disease state labels like smoking status, alcohol consumption, and hypertension), making it a typical mixed-type data. Traditional SMOTE-like methods only support interpolation of continuous features and cannot directly process binary features. Current practices usually treat binary features as continuous values, interpolate and then round them, or perform independent random sampling after interpolation. These practices destroy the inherent correlation structure between features, leading to a decrease in the quality of the generated synthetic samples and affecting the training effect of subsequent models.

[0005] Therefore, how to effectively maintain the correlation structure between continuous features and binary classification features in mixed-type data and generate high-quality synthetic samples when performing minority class sample augmentation on health data is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The present invention aims to solve the following technical problems existing in the prior art: When performing minority class sample augmentation on mixed-type health data, the traditional SMOTE interpolation method only supports the generation of continuous features and lacks a reasonable generation mechanism for binary features. The existing simple processing method for binary features will destroy the correlation structure between continuous features and binary features, as well as between binary features themselves, resulting in the distortion of feature association patterns in the generated samples, thereby reducing the quality of the synthesized samples and their utility in training health status assessment models.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A minority class augmentation method for hybrid-type health data based on relevance regularization dual generation includes the following steps: Step 1: Obtain a health dataset containing continuous features and binary classification features, and identify minority class samples; Step 2: Separate the continuous feature matrix X_cont and the binary classification feature matrix X_bin from the minority class samples; Step 3: Oversample the continuous features of the minority class samples using an interpolation method to generate a synthetic continuous feature matrix X'_cont; the interpolation method is SMOTE, Borderline-SMOTE, or ADAS. Any of YN; Step 4: Construct a conditional generator G, which is a multilayer perceptron network that takes continuous features as input and outputs the probability distribution of binary classification features; its network structure is as follows: the input layer is the continuous feature dimension n_cont, which passes through the first hidden layer (128 neurons) and the second hidden layer (64 neurons) and the ReLU activation function, and the output layer is the binary classification feature dimension n_bin; Step 5: Train the conditional generator G on real minority class samples. The training loss function consists of the binary cross-entropy loss L_BCE and the correlation regularization loss L_corr: L = L_BCE+ λ·L_corr; The correlation regularization loss consists of the mean square error of the correlation matrix between binary features and the mean square error of the continuous-binary cross-domain correlation matrix; Step 6: Input the continuous feature matrix X'_cont into the trained conditional generator G to obtain the binary feature probability distribution P_bin, and perform Bernoulli sampling on the probability distribution to obtain the binary feature matrix X'_bin; Step 7: Concatenate the synthesized continuous feature matrix X'_cont with the generated binary feature matrix X'_bin to obtain the complete synthesized minority class sample.

[0009] Compared with the prior art, the present invention has the following beneficial effects:

[0010] (1) This invention constructs a condition generator to generate corresponding binary classification features for each synthetic sample based on continuous features, so that the generation of binary classification features is associated with continuous features, thus avoiding the loss of correlation caused by traditional independent random sampling.

[0011] (2) The present invention introduces correlation regularization loss, which effectively maintains the correlation structure between mixed-type features by minimizing the difference between generated samples and real samples in the correlation matrix between binary classification features and the cross-domain correlation matrix, and the generated synthetic samples have higher quality;

[0012] (3) This invention is compatible with a variety of traditional interpolation methods such as SMOTE, Borderline-SMOTE, and ADASYN, and can be embedded as a general module into existing data augmentation processes, with good scalability and applicability;

[0013] (4) Experimental results of the present invention on health and medical data show that, compared with the pure interpolation method without using the CRDG module, the samples generated by the method of the present invention have significantly improved in the dimensions of statistical similarity (KS test, MMD distance) and data utility (classification F1 score), which can effectively improve the ability of the health status assessment model to identify minority classes. Attached Figure Description

[0014] Figure 1 This is an overall flowchart of the method according to an embodiment of the present invention;

[0015] Figure 2 This is a schematic diagram of the architecture of the CRDG module in an embodiment of the present invention;

[0016] Figure 3 This is a visualization comparison of t-SNE dimensionality reduction between samples generated by each method in the embodiments of the present invention and real minority class samples; wherein Figure 3 (a) is the SMOTE+CRDG method. Figure 3 (b) is the Borderline-SMOTE+CRDG method. Figure 3 (c) is the ADASYN+CRDG method. Figure 3 (d) represents the CTGAN method;

[0017] Figure 4 This is a heatmap comparison of the correlation matrix differences between samples generated by different methods and real samples in the embodiments of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are not intended to limit the scope of protection of the present invention.

[0019] Example 1: Data Sources and Preprocessing

[0020] This embodiment uses data from the China Health and Retirement Longitudinal Study (CHARLS) from 2015 to 2018. The CHARLS project, hosted by the National School of Development at Peking University, employs a multi-stage probability sampling method, covering 150 county-level units and 450 village-level units nationwide. Its aim is to collect high-quality micro-health data representative of Chinese people aged 45 and above and their spouses. After data cleaning and missing value removal, a dataset containing 2665 samples and 37 features was finally obtained.

[0021] The target variable was the health status level, divided into 5 levels: very poor (level 1), poor (level 2), fair (level 3), good (level 4), and very good (level 5). Of these, 294 samples (health status level 5) were in the minority category, accounting for 11.0% of the total sample. The feature space included 28 continuous features (including age, BMI, systolic blood pressure, diastolic blood pressure, pulse, respiratory function, grip strength, height, weight, waist circumference, upper arm length, knee height, as well as white blood cell count, hemoglobin, hematocrit, mean corpuscular volume, platelets, triglycerides, creatinine, blood urea nitrogen, high-density lipoprotein cholesterol, low-density lipoprotein cholesterol, total cholesterol, glucose, uric acid, cystatin C, C-reactive protein, and glycated hemoglobin, etc.) and 7 binary features (including gender, smoking status, alcohol consumption, hypertension, dyslipidemia, diabetes, and pulmonary heart disease).

[0022] The dataset was randomly split into training and test sets at a 7:3 ratio based on health status. This stratification strategy ensures that the distribution ratio of each health level in the training and test sets remains consistent with the original dataset. After splitting, the training set contains approximately 1865 records (about 205 of which are from the minority class), and the test set contains approximately 800 records (about 89 of which are from the minority class). It is important to note that minority class augmentation is only performed on the minority class samples in the training set; the minority class samples in the test set remain unchanged to ensure the objectivity and comparability of the evaluation. Continuous features are standardized using Z-scores; that is, after fitting the mean and standard deviation on the training set, the data is standardized separately for both the training and test sets to avoid data leakage.

[0023] Example 2: Experimental Environment and Parameter Settings

[0024] The experimental environment for this embodiment is as follows: operating system: Windows 11; programming language: Python 3.13; deep learning framework: PyTorch 2.x; machine learning libraries: scikit-learn, balanced-learn, and XGBoost. The interpolation method parameters were set as follows: K-nearest neighbor number K = min(5, n_minority-1), random seed: 42. The hyperparameters of the CRDG module were set as follows: number of hidden layer neurons: 128 and 64, learning rate: 0.001, optimizer: Adam, training epochs: 200, batch size: 32, regularization coefficient λ = 0.1. All methods uniformly generated 200 synthetic samples, which is approximately 97.6% of the original minority class samples (approximately 205 samples).

[0025] Example 3: Interpolation method for generating continuous features

[0026] This embodiment employs three interpolation methods—SMOTE, Borderline-SMOTE, and ADASYN—to oversample the 28-dimensional continuous features of minority class samples. SMOTE generates new samples by performing linear interpolation along the lines connecting minority class samples and their K nearest neighbors. The formula for calculating the synthesized sample is: x_new = x_i + δ × (x_j - x_i), where x_i is the selected minority class sample, x_j is its randomly selected K nearest neighbor, and δ is a random number within the interval [0,1]. Borderline-SMOTE improves upon this by selecting only samples located in the class boundary region as the interpolation base point, resulting in synthesized samples concentrated near the classification decision boundary. ADASYN further introduces an adaptive weighting mechanism, automatically calculating the synthesis difficulty weight based on the proportion of majority class samples in the K nearest neighbors of each minority class sample, generating more synthesized samples in difficult regions.

[0027] Each of the three interpolation methods independently generates 200 28-dimensional synthetic continuous feature matrices, which serve as inputs to the subsequent CRDG module.

[0028] Example 4: CRDG Module Architecture and Training

[0029] The CRDG (Correlation-Regularized Dual Generation) module is the core innovation of this invention, and its structure is as follows: Figure 2As shown, the core of the CRDG module is a conditional generator, a three-layer fully connected neural network. The input layer receives a 28-dimensional continuous feature vector, which undergoes non-linear feature transformation through 128-dimensional and 64-dimensional hidden layers (with ReLU activation function). The output layer has 7 neurons, corresponding to the logits values ​​of 7 binary classification features. The output is mapped to probability values ​​through a sigmoid function, and then sampled by Bernoulli to obtain the final 0 / 1 binary classification result. The entire network contains only about 23,000 trainable parameters, making it a lightweight network structure.

[0030] The loss function of the CRDG module consists of two parts: the binary cross-entropy loss L_BCE and the correlation regularization loss L_corr: L = L_BCE + λ·L_corr, where the correlation regularization loss L_corr contains two sub-terms:

[0031] (a) Difference in correlation matrices between binary features, L_binary. The Pearson correlation coefficient matrix R_syn (7×7) between the seven binary features of the generated samples is calculated, and its mean squared error is calculated compared with the binary correlation matrix R_real of the real minority class samples. This constraint ensures that the binary features in the generated samples maintain correlations consistent with the real data, such as the positive correlation between "hypertension" and "dyslipidemia".

[0032] (b) Cross-domain correlation matrix difference L_cross. Continuous features and binary features are concatenated into a 35-dimensional joint feature vector. The cross-domain correlation matrices R_cross_syn and R_cross_real for generated samples and real samples are calculated respectively, and the mean squared error between them is calculated. This constraint ensures that the cross-domain correlation structure between continuous features and binary features is maintained, such as the strong positive correlation between "systolic blood pressure" and "whether or not one has hypertension".

[0033] The CRDG module is trained on real minority class samples, using the 28-dimensional continuous features of the real minority class samples as input and the corresponding 7-dimensional binary classification features as supervision signals. It employs a "train once, infer many times" strategy, meaning the CRDG model is trained only once on real minority class samples to infer corresponding binary classification features for the synthetic continuous features generated by the three interpolation methods, significantly reducing computational overhead.

[0034] Example 5: Generation and splicing

[0035] After training, the CRDG module is used to generate corresponding binary classification features for the synthetic continuous features generated by each interpolation method. Specifically, 200 28-dimensional synthetic continuous features are input into a conditional generator. The generator outputs the logits values ​​of seven binary classification features, which are converted into probability values ​​using the Sigmoid function. Then, Bernoulli sampling is performed on each probability value (with probability p taking a value of 1 and probability 1-p taking a value of 0), resulting in the final 0 / 1 binary classification feature values. The generated seven-dimensional binary classification features are concatenated with the 28-dimensional synthetic continuous features to form 200 complete 35-dimensional synthetic minority class samples.

[0036] Example 6: Experimental Results and Analysis

[0037] This experiment evaluates three methods—SMOTE+CRDG, Borderline-SMOTE+CRDG, and ADASYN+CRDG—from two dimensions: statistical similarity and data utility, with a pure interpolation method (without the CRDG module) serving as a control.

[0038] Statistical similarity assessment

[0039] The following indicators were used for statistical similarity assessment: (1) MMD (Maximum Mean Difference), used to measure the consistency of high-dimensional joint distributions; (2) KS statistic, used to measure the consistency of individual continuous feature distributions; (3) Wasserstein distance, used to measure the minimum transportation cost between distributions; and (4) Frobenius norm, used to measure the difference in correlation matrices. The assessment results are shown in Table 1.

[0040] Table 1 Comparison of Statistical Similarity Evaluation Indicators for Each Method

[0041] SMOTE+CRDG 0.007443 0.0909 0.1180 0.7782 Borderline+CRDG 0.005200 0.0752 0.0805 0.8381 ADASYN+CRDG 0.013412 0.1588 0.2089 1.3886

[0042] As shown in the table above, the Borderline-SMOTE combined with CRDG method performs best in MMD, KS statistic, and Wasserstein distance, with values ​​of 0.005200, 0.0752, and 0.0805, respectively, indicating that its generated synthetic samples are closest to the true minority class in terms of distribution consistency. SMOTE+CRDG follows closely behind, while ADASYN+CRDG, due to its adaptive weighting strategy generating more samples in difficult regions, results in a slight decrease in statistical indicators. Regarding correlation matrix preservation, the Frobenius differences among the three methods remain at a low level (0.78~1.39), validating the effectiveness of the CRDG module in preserving the correlation structure between features.

[0043] Figure 3The diagram presents a visual comparison of t-SNE dimensionality reduction between samples generated by each method and real minority class samples. As can be observed from the figure, the samples (red scatter points) generated by the three interpolation methods combined with the CRDG module are well embedded into the distribution region of the real minority class samples (blue scatter points), with no obvious outliers or distribution shifts. This indicates that the binary classification features generated by the CRDG module and the continuous features generated by the interpolation methods have good consistency.

[0044] Figure 4 A heatmap comparison of the correlation matrix differences between samples generated by different methods and real samples is presented. The color intensity of the heatmaps shows that the correlation matrix difference after combining the traditional interpolation method with the CRDG module is significantly smaller than that of the deep generation method, indicating that the CRDG module has a significant advantage in preserving the correlation structure between features.

[0045] Data utility assessment

[0046] The data utility evaluation adopted a paradigm of "augmented data training and full test set evaluation". 200 synthetic samples generated by each method were concatenated with the original training set to form an augmented training set. This augmented training set was then used to train five classifiers: SVM, RandomForest, XGBoost, MLP, and TabNet, and the classification performance was evaluated on the original test set. The original training set without any augmentation was used as a baseline control. The evaluation results are shown in Table 2.

[0047] Table 2. Comparison of Macro F1 scores for different methods on different classifiers

[0048] SVM 0.8316 0.8435 0.8375 0.8397 RandomForest 0.7673 0.7960 0.7809 0.7916 XGBoost 0.8032 0.7924 0.7995 0.8054 MLP 0.8198 0.8140 0.8133 0.8294 TabNet 0.7584 0.7913 0.7800 0.8151

[0049] As shown in the table above, after enhancement by the CRDG module, the Macro F1 scores of most classifiers improved compared to the baseline. Among them, ADASYN+CRDG performed best on MLP and TabNet, achieving F1 scores of 0.8294 and 0.8151 respectively, representing improvements of 1.2% and 7.5% compared to the baseline. SMOTE+CRDG achieved the best results on SVM and RandomForest, with F1 scores of 0.8435 and 0.7960 respectively. Experimental results demonstrate that the CRDG module proposed in this invention can effectively improve the quality of minority class sample enhancement in mixed-type health data, thereby improving the minority class recognition performance of downstream classification tasks.

[0050] Overall Conclusion

[0051] In summary, the proposed method for minority class enhancement of mixed-type health data based on correlation regularization dual generation has the following outstanding advantages: (1) It realizes conditional probability modeling from continuous features to binary features through a condition generator, avoiding the arbitrariness of binary feature generation in traditional methods; (2) It effectively maintains the correlation structure between mixed-type features through correlation regularization loss, improving the quality of synthetic samples; (3) It is compatible with various traditional interpolation methods, with wide applicability and good scalability. Experimental results verify that the method of the present invention is superior to traditional methods in both statistical similarity and data utility dimensions, which is of great significance for improving the prediction performance of health status assessment models in the minority class.

Claims

1. A method for minority class augmentation of hybrid-type health data based on relevance regularization dual generation, characterized in that, Includes the following steps: Step 1: Obtain a health dataset containing continuous features and binary classification features, and identify minority class samples within it; Step 2: Separate the continuous feature matrix X_cont and the binary feature matrix X_bin from the minority class samples; Step 3: Oversample the continuous feature matrix X_cont using an interpolation method to generate a synthetic continuous feature matrix X'_cont; Step 4: Construct a condition generator G, which is a multilayer perceptron network that takes continuous features as input and outputs the probability distribution of binary classification features. Its network structure is as follows: the input layer dimension is the number of continuous features n_cont, which is transformed by the first hidden layer, the second hidden layer and the ReLU activation function, and the output layer dimension is the number of binary classification features n_bin. Step 5: Train the conditional generator G on real minority class samples. The training loss function is L = L_BCE + λ·L_corr, where L_BCE is the binary cross-entropy loss, L_corr is the correlation regularization loss, and λ is the regularization coefficient. Step 6: Input the synthesized continuous feature matrix X'_cont into the trained conditional generator G to obtain the probability distribution P_bin of the binary classification features, and perform Bernoulli sampling on the probability distribution to obtain the binary classification feature matrix X'_bin; Step 7: Concatenate the synthetic continuous feature matrix X'_cont with the binary classification feature matrix X'_bin to obtain the complete synthetic minority class sample.

2. The method according to claim 1, characterized in that, The interpolation method in step 3 is any one of SMOTE (Synthetic Minority Oversampling), Borderline-SMOTE, or ADASYN (Adaptive Synthetic Sampling).

3. The method according to claim 1, characterized in that, In step 4, the first hidden layer of the condition generator has 128 neurons, the second hidden layer has 64 neurons, and the activation function is ReLU.

4. The method according to claim 1, characterized in that, The method for calculating the correlation regularization loss L_corr in step 5 includes: L_binary = MSE(R_syn, R_real), where R_syn is the Pearson correlation coefficient matrix of the generated sample binary classification feature vector, and R_real is the Pearson correlation coefficient matrix of the real sample binary classification feature vector; L_cross = MSE(R_cross_syn, R_cross_real), where R_cross_syn is the Pearson correlation coefficient matrix calculated by concatenating the continuous features of the generated sample with the binary classification features, and R_cross_real is the Pearson correlation coefficient matrix calculated by concatenating the continuous features of the real sample with the binary classification features. L_corr = L_binary + L_cross.

5. The method according to claim 4, characterized in that, The Pearson correlation coefficient matrix is ​​calculated using a numerically stable covariance method. After centering the feature matrix, the covariance matrix is ​​calculated, divided by the outer product of the diagonal standard deviation vectors, and the result is clipped in the interval [-1,1] to avoid numerical instability caused by zero variance features.

6. The method according to claim 1, characterized in that, In step 5, the regularization coefficient λ ranges from 0.01 to 1.

0.

7. The method according to claim 1, characterized in that, In step 5, the condition generator is trained using the Adam optimizer with a learning rate of 0.001, 200 training epochs, and a batch size of 32.

8. The method according to claim 1, characterized in that, The health dataset is derived from the China Health and Retirement Longitudinal Study (CHARLS) data. The continuous features include anthropometric indicators and blood test indicators, and the binary features include health behavior and disease status labels.

9. The method according to claim 1, characterized in that, The specific method of Bernoulli sampling in step 6 is as follows: for each binary feature with a probability value p∈[0,1], the probability p is set to 1 and the probability 1-p is set to 0.

10. The method according to claim 1, characterized in that, The condition generator adopts a strategy of training once and inferring multiple times. That is, after training once on real minority class samples, it can infer and generate corresponding binary classification features for synthetic continuous features generated by various interpolation methods.

Images