Secondary blood index dataset generation method and system based on feature dynamic screening

Abstract

The application discloses a secondary blood index dataset generation method and system based on feature dynamic screening, and the application collects basic information and basic blood indexes of research objects, and obtains effective samples through preprocessing; and the effective samples are divided into a non-chronic non-infectious disease control group and a target disease group, and population statistical characteristic difference analysis is performed; secondary blood indexes are calculated according to the basic blood indexes, and population statistical characteristics after screening are integrated to generate an enhanced dataset; the enhanced dataset is divided into a training set and a test set according to disease state stratified sampling, continuous type indexes of the training set and the test set are synchronized and standardized, then an adaptive elastic network based on stability selection is used to screen an optimized feature set with non-zero coefficients, and finally, feature synchronization filtering is performed to integrate and form a final dataset. The application can solve the problems of low quality of an existing blood multidimensional index dataset, non-uniform processing flow and feature redundancy leading to difficulty in model selection.

Description

Technical Field

[0001] This invention relates to the field of medical information technology, specifically to a method and system for generating a secondary blood indicator dataset based on dynamic feature screening. Background Technology

[0002] In the interdisciplinary application of machine learning and healthcare, model performance is highly dependent on the quality and standardization of the training data. Research on machine learning models based on multidimensional blood indicators faces many challenges that need to be addressed.

[0003] The sources of raw clinical data are generally complex. Control groups (people without chronic non-communicable diseases) often include patients with common chronic diseases (such as hypertension and diabetes), leading to baseline shifts. Disease population data lack strict time definitions, causing distortions in the correlation between indicators and pathological states. In addition, raw data commonly contain outliers, missing values, and dimensional differences, which can cause abnormal weight convergence when used directly for model training (such as decision trees splitting too deeply due to extreme values, and neural networks experiencing gradient explosion due to dimensional differences).

[0004] Data preprocessing (such as outlier removal and standardization methods) relies on researchers' experience and lacks a unified process, making it difficult to compare the performance of models trained on different datasets, which seriously hinders the reuse of technology and the promotion of results.

[0005] Commonly used multidimensional indicators (such as basic blood indicators, NLR, PLR, etc.) exhibit strong mathematical correlations (e.g., SII can be derived from platelet count and NLR). Directly inputting these indicators into a model can lead to multicollinearity, reducing model stability (e.g., increased hyperplane oscillation in support vector machines within a high-dimensional redundant feature space). This multicollinearity problem results in inaccurate model parameter estimation and decreased predictive performance. This interference from redundant features makes it difficult for researchers to objectively evaluate the true performance of different models.

[0006] Blood multidimensional indicators have become important biomarkers reflecting the body's immune status and pathological processes due to their ease of acquisition and low detection costs. However, the aforementioned drawbacks lead to unstable model performance. Therefore, it is necessary to construct a standardized, low-redundancy, and cross-comparable dataset of blood multidimensional indicators to provide a reliable benchmark for model selection and optimization. Summary of the Invention

[0007] This invention addresses the problems of low quality, inconsistent processing procedures, and difficulty in model selection due to feature redundancy in existing multidimensional blood indicator datasets. It proposes a method and system for generating secondary blood indicator datasets based on dynamic feature selection. By strictly defining the target disease group as patients who have not received drug treatment or surgery within 14 days of diagnosis, the influence of treatment intervention on indicators is avoided from the data source, ensuring a direct correlation between disease group indicators and pathological states, thus solving the problem of distorted correlation between disease population indicators and pathological states. Through standardized data collection, hierarchical processing, feature selection, and unified integration, a high-quality dataset is generated, providing benchmark data support for the training, evaluation, and optimization of various types of machine learning models.

[0008] This invention is achieved through the following technical solution. A method for generating a secondary blood indicator dataset based on dynamic feature selection includes the following steps:

[0009] S1: Collect basic information and basic blood indicators of the research subjects. The basic information includes demographic characteristics, personal identification information and clinical basic information. Match the basic information and basic blood indicators, then delete duplicate data and data with missing basic blood indicators, retain the samples aged 20-80 years, and finally obtain the effective samples.

[0010] S2: Based on the diagnostic information in the basic clinical information, the valid samples are divided into a control group without chronic non-communicable diseases and a target disease group;

[0011] S3: Perform statistical difference analysis on the demographic characteristics of the control group without chronic non-communicable diseases and the target disease group, screen out key characteristics that are significantly related to the target disease, and obtain the screened demographic characteristics.

[0012] S4: Calculate secondary blood indicators for the control group without chronic non-communicable diseases and the target disease group based on basic blood indicators, and integrate the screened demographic features to generate an enhanced dataset;

[0013] S5: Based on disease status stratified sampling, the augmented dataset is divided into training and test sets. Continuous indicators of the training and test sets are simultaneously standardized to obtain standardized training and test sets. An adaptive elastic network based on stability selection is used to filter the optimized feature set with non-zero coefficients. Based on the optimized feature set, the training and test sets are simultaneously filtered to form the final dataset.

[0014] Further optimization involves using an adaptive elastic network based on stability selection to filter out the optimized feature set with non-zero coefficients. The specific steps are as follows:

[0015] Phase 1: Run the ElasticNet network on the training set. First, optimize the regularization strength λ and the mixed parameter α of L1 and L2 penalties of the ElasticNet network simultaneously through 5-fold cross-validation. Then, fit the ElasticNet network on the training set based on the optimal hyperparameters to obtain the coefficients of the features. Calculate the adaptive weight w for each feature j based on the coefficients. j =1 / (|β j |+ε), where β j Let be the coefficient of feature j, and ε be a constant;

[0016] Phase 2: Introduce adaptive weights into the penalty term of the elastic network to construct an adaptive elastic network. Simultaneously optimize two hyperparameters of the adaptive elastic network through K-fold cross-validation: the regularization strength λ and the mixed parameter α of L1 regularization and L2 regularization penalties.

[0017] Phase 3, Stability Selection: The adaptive elastic network is enhanced using a stability selection framework. Sub-training sets are generated through B-time bootstrap sampling. The adaptive elastic network is run on each sub-training set and the selected features are recorded. Finally, the frequency of each feature being selected is calculated and recorded as the stability score.

[0018] Phase 4: Select features with stability scores higher than the stability score threshold to form an optimized feature set.

[0019] Further optimization, step S2 also includes: removing outliers from the baseline blood indicators in the control group without chronic non-communicable diseases, wherein individuals with chronic non-communicable diseases are excluded from the control group without chronic non-communicable diseases; for the target disease group, samples that have not received treatment within 14 days after diagnosis are selected, and the original data of the baseline blood indicators are retained.

[0020] Further optimization involves performing Z-score standardization on the continuous metrics of the augmented dataset when the metrics are continuous metrics. This is done by calculating the mean μ and standard deviation σ based on the training set.

[0021] Further optimization involves matching the basic information of the personal identification information with the basic blood indicators in step S1, and then deleting the personal identification information.

[0022] Further preferably, the secondary blood indicators are derived from basic blood indicators through fixed mathematical formulas and are closely related to chronic diseases, including but not limited to: neutrophil-to-lymphocyte ratio (NLR), lymphocyte-to-monocyte ratio (LMR), platelet-to-lymphocyte ratio (PLR), systemic immune inflammation index (SII), systemic inflammatory response index (SIRI), neutrophil-monocyte-to-lymphocyte ratio (NMLR), neutrophil-to-lymphocyte-to-platelet ratio (NLPR), neutrophil-to-monocyte ratio (NMR), neutrophil-to-platelet ratio (NPR), and red blood cell distribution width-to-platelet ratio (RPR). These secondary blood indicators have all been verified by research and can effectively reflect the body's immune inflammatory state, organ dysfunction, and other pathological processes, and are highly correlated with the risk of chronic diseases such as cancer and cardiovascular diseases.

[0023] This invention also provides a system for generating secondary blood indicator datasets based on dynamic feature filtering, comprising:

[0024] Data preprocessing module: used to collect basic information and basic blood indicators of research subjects. The basic information includes demographic characteristics, personal identification information and basic clinical information; the basic information and basic blood indicators are matched, and then duplicate data and data with missing basic blood indicators are deleted, retaining the samples aged 20-80 years old, and finally obtaining the effective samples;

[0025] Data grouping and feature extraction module: used to divide the effective samples into a control group without chronic non-communicable diseases and a target disease group based on the diagnostic information in the clinical basic information; to perform statistical difference analysis on the demographic characteristics of the control group without chronic non-communicable diseases and the target disease group, to screen out key features that are significantly related to the target disease, and to obtain the screened demographic characteristics.

[0026] Indicator Calculation Module: Used to calculate secondary blood indicators for the control group without chronic non-communicable diseases and the target disease group based on basic blood indicators, and to integrate the screened demographic features to generate an enhanced dataset;

[0027] Dataset optimization module: It is used to perform stratified sampling based on disease status, divide the augmented dataset into training set and test set, and simultaneously standardize the continuous indicators of the training set and test set to obtain standardized training set and standardized test set. It uses an adaptive elastic network based on stability selection to filter the optimized feature set with non-zero coefficients, and performs simultaneous feature filtering on the training set and test set based on the optimized feature set to integrate and form the final dataset.

[0028] Further preferably, the data grouping and feature extraction module includes:

[0029] Grouping Unit: This unit is used to divide the valid samples into a control group without chronic non-communicable diseases and a target disease group based on diagnostic information in the clinical baseline information; this unit also supports intelligent grouping based on multiple diagnostic criteria such as ICD coding;

[0030] Difference Analysis Unit: Statistical difference analysis is performed on the demographic characteristics of the control group without chronic non-communicable diseases and the target disease group to screen out key characteristics that are significantly related to the target disease and obtain the screened demographic characteristics.

[0031] The indicator calculation module includes:

[0032] Outlier handling unit: Used to perform outlier handling operations. Outliers are removed from baseline blood indicators of the control group without chronic non-communicable diseases using box plot method, while the original data of baseline blood indicators of the target disease group are retained.

[0033] Enhanced dataset generation unit: Calculates secondary blood indicators and integrates filtered demographic features to form an enhanced dataset.

[0034] The dataset optimization module includes:

[0035] Division Unit: Used to perform stratified sampling, stratified sampling according to disease status, ensuring that the training set and test set are balanced in terms of disease type distribution, and using random seeds to ensure the reproducibility of sampling results;

[0036] Standardized Units: Perform Z-score standardization on continuous metrics of augmented datasets, automatically store the mean and standard deviation of the training set, and apply them to the test set to avoid data leakage;

[0037] Feature dynamic optimization unit: Optimizes the feature set by filtering using an adaptive elastic network based on stability selection;

[0038] Integration Unit: Synchronously filters features from the training and test sets, integrates data and metadata to generate the final dataset.

[0039] This invention distinguishes the nature of outliers between the control group and the disease group by using strict sample inclusion / exclusion criteria and differential outlier handling, combined with clinical pathology logic: outliers in the control group are removed using box plots to ensure baseline purity, while the original data in the disease group is retained and only extreme detection errors (such as values ​​exceeding the linear range of detection) are removed. This ensures that the baseline purity of the control group is maintained without losing key pathological information in the disease group, thus ensuring the consistency and accuracy of the data baseline.

[0040] A multi-feature optimization mechanism is implemented through statistical difference analysis and an adaptive elastic network based on stability selection: First, basic features related to the disease are screened through statistical tests; then, ElasticNet fitting and adaptive weight penalty are used to jointly shrink (L2 regularization) and sparsify (L1 regularization) the coefficients of relevant features, avoiding random elimination of collinear features by single regularization; finally, stability selection (multiple subsampling validation) ensures the robustness of the screened features, solving the multicollinearity problem and improving the reproducibility of the feature selection results. This correlation screening, adaptive regularization, and stability validation effectively eliminate redundant features while retaining key clinical indicators.

[0041] Through standardization and metadata recording, Z-score standardization is used to eliminate dimensional differences, and metadata ensures process traceability, guaranteeing the comparability and reproducibility of the final dataset. The generated final dataset can directly support performance comparisons of multiple models and the selection of the optimal architecture, solving the technical problems of incomparability of existing datasets and subjective model selection. Attached Figure Description

[0042] Figure 1 This is a flowchart of the method of the present invention;

[0043] Figure 2 Box plot comparing the distribution of individual inflammatory markers;

[0044] Figure 3 A flowchart illustrating the process of filtering an optimized feature set with non-zero coefficients using an adaptive elastic network based on stability selection;

[0045] Figure 4 A heatmap showing the correlation between inflammatory markers;

[0046] Figure 5 A bar chart (AUC, sensitivity, specificity) evaluating the five-fold cross-validation performance of the decision tree model (DT) on this dataset.

[0047] Figure 6 A bar chart (AUC, sensitivity, specificity) evaluating the five-fold cross-validation performance of Random Forest (RF) on this dataset.

[0048] Figure 7 The bar chart shows the performance evaluation of Naive Bayes (NBC) on this dataset using five-fold cross-validation (AUC, sensitivity, specificity).

[0049] Figure 8 A bar chart (AUC, sensitivity, specificity) evaluating the five-fold cross-validation performance of the Support Vector Machine (SVM) on this dataset.

[0050] Figure 9A bar chart of the five-fold cross-validation performance evaluation (AUC, sensitivity, specificity) of K-Nearest Neighbors (KNN) on this dataset.

[0051] Figure 10 A bar chart (AUC, sensitivity, specificity) evaluating the five-fold cross-validation performance of the BP neural network on this dataset.

[0052] Figure 11 A bar chart (AUC, sensitivity, specificity) showing the performance evaluation of XGBoost on this dataset using five-fold cross-validation.

[0053] Figure 12 The ROC curve of the best-fit model trained on the test set based on the final dataset obtained in this invention. Detailed Implementation

[0054] The present invention will be further illustrated below with reference to embodiments, focusing on the construction process of a method and system for generating secondary blood index datasets for predicting the occurrence of chronic diseases, without limiting the scope of protection of the present invention.

[0055] Example 1: Example of Building a Cancer Prediction Model Dataset

[0056] This embodiment illustrates a cancer model development scenario, but the same method can be applied to other disease types.

[0057] S1: Collect basic information of research subjects (including demographic characteristics such as gender and age; personal identification information; and basic clinical information including diagnostic information, tumor type, duration of drug use, and surgical time) and basic blood parameters (including neutrophil count, lymphocyte count, monocyte count, platelet count, and red blood cell distribution width). See the detailed procedure below. Figure 1 Then, preprocessing was performed: basic information and basic blood indicators were accurately matched using personal identification information such as name, ID number, hospital number, and examination sequence number. Then, these personal identification information were deleted using R language; duplicate data was deleted (if multiple blood tests were performed on the same patient on the same date, only the first test result was retained); data with missing basic blood indicators were deleted (if core indicators such as neutrophil count and lymphocyte count were missing, the sample was directly removed); samples aged 20-80 years were retained (to exclude abnormal fluctuations in indicators that may be caused by being too young or too old), and finally, valid samples were obtained.

[0058] S2: Based on the diagnostic information in the clinical baseline information, the valid samples were divided into a control group without chronic non-communicable diseases and a target disease group (cancer group); Control group without chronic non-communicable diseases: To ensure the consistency of baseline characteristics of the control group and to avoid interference from other chronic diseases in the correlation analysis between blood indicators and the target disease, patients with common chronic non-communicable diseases that are known to affect fluctuations in blood indicators were excluded, as were patients with hypertension, type 2 diabetes, cancer (excluding mild skin cancer), chronic lung disease, heart problems (heart attack, coronary heart disease, angina pectoris, congestive heart failure), and stroke. Outliers in the baseline blood indicators were removed using a box plot method (IQR = Q3 - Q1, with outlier determination ranges being below Q1 - 1.5IQR or above Q3 + 1.5IQR, where IQR is the interquartile range, Q1 is the first quartile, and Q3 is the third quartile), ultimately retaining 13,423 samples. For the target disease group (cancer group): samples that had not received treatment within 14 days of diagnosis were selected through electronic medical record time verification, and the original data of the baseline blood indicators were retained (only values ​​exceeding the linear range of the test were reviewed; if confirmed to be outside the range, the extreme value record was removed), ultimately retaining 2,503 samples.

[0059] S3: Statistical differences (t-test, chi-square test) were analyzed in demographic characteristics and baseline blood parameters between the control group without chronic non-communicable diseases and the target disease group (cancer group). For demographic characteristics, key features significantly correlated with the target disease (P < 0.05) were selected. Simultaneous analysis of baseline blood parameters was only used to objectively demonstrate the differences in baseline characteristics between the two groups and was not included in the key feature selection. Clinical baseline information (such as drug use time and surgery time) was not included in this stage of the difference analysis because it is not a universal characteristic possessed by all study subjects or has significant deficiencies. The analysis specifically included two types of information: first, demographic characteristics (including age and sex) used as the key feature selection criteria; and second, baseline blood parameters (including neutrophil count (GRA), lymphocyte count (LYM), monocyte count (MONO), platelet count (PLT), and erythrocyte distribution width (RDW)) used only for baseline characteristic comparison. Statistical analysis showed that the differences in the above features were all statistically significant (P < 0.05), and the results are shown in Table 1.

[0060] Table 1. Comparison of basic information and basic blood indicators between people without chronic non-communicable diseases and cancer patients

[0061]

[0062] Note: GRA = Neutrophil Count; LYM = Lymphocyte Count; MONO = Monocyte Count; PLT = Platelet Count; RDW = Red Blood Cell Distribution Width

[0063] S4: Calculate the secondary blood indicators for the control group without chronic non-communicable diseases and the target disease group based on the basic blood indicators, and integrate the screened basic information to generate an enhanced dataset;

[0064] The secondary blood indicators are calculated according to the preset formula. The specific calculation logic for each secondary blood indicator is as follows:

[0065] Neutrophil-to-lymphocyte ratio (NLR): obtained by dividing the neutrophil count by the lymphocyte count;

[0066] Lymphocyte to monocyte ratio (LMR): obtained by dividing the lymphocyte count by the monocyte count;

[0067] Platelet-to-lymphocyte ratio (PLR): obtained by dividing the platelet count by the lymphocyte count;

[0068] Systemic immune inflammation index (SII): obtained by multiplying the neutrophil count and platelet count, and then dividing by the lymphocyte count;

[0069] Systemic Inflammatory Response Index (SIRI): Calculated by multiplying the neutrophil count and monocyte count, and then dividing by the lymphocyte count.

[0070] Neutrophil-monocyte-lymphocyte ratio (NMLR): obtained by summing the neutrophil count and the monocyte count, and then dividing by the lymphocyte count;

[0071] The neutrophil-lymphocyte-platelet ratio (NLPR) is obtained by dividing the neutrophil count by the product of the lymphocyte count and the platelet count, and then multiplying by 1000.

[0072] Neutrophil to monocyte ratio (NMR): obtained by dividing the neutrophil count by the monocyte count;

[0073] Neutrophil-to-platelet ratio (NPR): Calculated by dividing the neutrophil count by the platelet count and then multiplying by 1000;

[0074] Red blood cell distribution width to platelet ratio (RPR): Calculated by dividing red blood cell distribution width (RDW) by platelet count (PLT).

[0075] After the secondary blood indicators were calculated, the Shapiro-Wilk test was used to verify the normality of each indicator. The results showed that none of the secondary blood indicators conformed to a normal distribution (P<0.05). Therefore, subsequent statistical analysis used non-parametric tests to compare the differences between the two groups. The aforementioned secondary blood indicators specifically include: neutrophil-to-lymphocyte ratio (NLR), lymphocyte-to-monocyte ratio (LMR), platelet-to-lymphocyte ratio (PLR), systemic immune inflammation index (SII), systemic inflammatory response index (SIRI), neutrophil-monocyte-to-lymphocyte ratio (NMLR), neutrophil-to-lymphocyte-to-platelet ratio (NLPR), neutrophil-to-monocyte ratio (NMR), neutrophil-to-platelet ratio (NPR), and erythrocyte distribution width-to-platelet ratio (RPR). These indicators, along with the baseline information after screening (age and sex), formed an enhanced dataset. The distribution differences of these secondary blood indicators in different populations are shown in Table 2.

[0076] Table 2. Distribution of secondary blood indicators in different populations

[0077]

[0078] S5: Stratified sampling based on disease status at an 8:2 ratio divides the augmented dataset into training and testing sets, ensuring a balanced distribution of disease types between the two sets; Z-score parameters (mean μ, standard deviation σ) are calculated based on the training set, and continuous indicators in both the training and testing sets are standardized simultaneously. )). Figure 2 The distribution characteristics of standardized secondary blood indicators in the training set were shown, resulting in a standardized training set and a standardized test set. For the standardized training set and the standardized test set, an adaptive elastic network based on stability selection was used to filter the optimized feature set with non-zero coefficients. Based on the optimized feature set, the features of the training set and the test set were simultaneously filtered to ensure that the feature spaces of the two sets were consistent. The dataset was then integrated to form the final dataset, which was accompanied by metadata (including standardization parameters, filtering paths, differential analysis results, indicator calculation methods, etc.).

[0079] like Figure 3 As shown, the optimal feature set with non-zero coefficients is selected using an adaptive elastic network based on stability selection. The specific steps are as follows:

[0080] Phase 1: Initial Fitting and Adaptive Weight Calculation of ElasticNet: First, run the ElasticNet on the training set. Simultaneously optimize the regularization strength λ and the hybrid parameter α of L1 and L2 penalties using 5-fold cross-validation. In this embodiment, the optimal hyperparameters are determined to be λ=0.0018 and α=0.7. Then, fit the ElasticNet on the training set based on the optimal hyperparameters to obtain the coefficients (estimated values) of the features. Based on these coefficients, calculate the adaptive weight w for each feature j. j =1 / (|β j |+ε), where β j ε is a coefficient, and ε is a very small constant used to prevent the denominator from being zero. This step aims to impose a smaller penalty on important features and a larger penalty on irrelevant features.

[0081] Phase 2, Adaptive Elastic Network Optimization: Adaptive weights are introduced into the penalty term of the elastic network to construct an adaptive elastic network. Two hyperparameters of the adaptive elastic network are simultaneously optimized through K-fold cross-validation: the regularization strength λ and the hybrid parameter α of L1 regularization and L2 regularization penalty, to obtain the optimal hyperparameters that fit the adaptive weights.

[0082] Phase 3, Stability Selection: To improve the stability and reproducibility of feature selection, a stability selection framework is used to enhance the adaptive elastic network. Sub-training sets are generated through B bootstrap iterations (e.g., B=100), with each sub-training set being the same size as the original training set. The adaptive elastic network is run on each sub-training set, and the selected features (features with non-zero coefficients) are recorded. Finally, the frequency of selection for each feature is calculated and recorded as a stability score.

[0083] Phase Four: Determining the Optimized Feature Set: Based on a preset stability score threshold (e.g., 0.6-0.8, this range is based on empirical values ​​from statistical learning and achieves a good balance between controlling the false detection rate (FDR) and retaining valid features; in practical applications, the stability score threshold can be flexibly adjusted according to specific requirements for model sparsity or FDR; in this embodiment, the stability score threshold is set to 0.6), features with stability scores higher than the stability score threshold are selected to form a high-confidence optimized feature set. This process effectively controls the false detection rate during the selection process, ensuring the robustness of the feature set.

[0084] Correlation heatmap among secondary blood indicators before feature screening is shown below Figure 4 As shown in A, Figure 4 Figure B shows a heatmap of correlations among secondary blood indicators retained after screening using an adaptive elastic network based on stability selection. (Comparison) Figure 4 China A and Figure 4As can be seen from B, the strong correlation within the core feature set retained after screening is significantly reduced, and the feature selection results have higher stability and interpretability.

[0085] Based on the final dataset obtained in this embodiment, the seven pre-built models are trained and evaluated:

[0086] Decision Tree (DT): Optimizing the complexity parameter cp, five-fold cross-validation results show an AUC of 0.637-0.695, a sensitivity of 0.301-0.412, and a specificity of 0.940-0.958. Figure 5 );

[0087] Random Forest (RF): Optimized feature sampling ratio mtry, five-fold cross-validation results showed an AUC of 0.820-0.861, sensitivity of 0.658-0.754, and specificity of 0.828-0.784. Figure 6 );

[0088] Naive Bayes (NBC): With Laplace smoothing and five-fold cross-validation enabled, the AUC values ​​were 0.750-0.800, the sensitivity was 0.552-0.640, and the specificity was 0.810-0.893. Figure 7 );

[0089] Support Vector Machine (SVM): Optimized radial basis function kernel parameter γ, five-fold cross-validation results show AUC values ​​of 0.794-0.822, sensitivity of 0.690-0.710, and specificity of 0.852-0.889. Figure 8 );

[0090] K-Nearest Neighbors (KNN): Optimizing the number of nearest neighbors k, five-fold cross-validation results showed an AUC of 0.775-0.811, a sensitivity of 0.627-0.666, and a specificity of 0.814-0.841. Figure 9 );

[0091] Backpropagation (BP) neural network: Optimized number of hidden layer neurons, five-fold cross-validation results showed AUC values ​​of 0.859-0.885, sensitivity of 0.711-0.786, and specificity of 0.813-0.866. Figure 10 );

[0092] XGBoost: Optimizing tree depth and learning rate, five-fold cross-validation results show an AUC of 0.814-0.854, a sensitivity of 0.690-0.739, and a specificity of 0.787-0.832. Figure 11 ).

[0093] Comprehensive evaluation shows that the BP neural network performs optimally on this final dataset (highest AUC with minimal fluctuation and highest sensitivity). Applying the optimal model (BP neural network) trained on this final dataset to the independent test set yields an area under the ROC curve (AUC) of 0.870 and a sensitivity of 0.746. Figure 12 This verifies the supporting role of the dataset in the model's generalization ability.

[0094] This embodiment demonstrates that the final dataset constructed by the present invention, through an adaptive elastic network based on stability selection, not only effectively reduces feature redundancy of indicators, but also improves the robustness and reproducibility of the feature set through stability selection. The selected indicators have low correlation and can stably reflect the pathological characteristics of the disease, which can accurately support model performance comparison and objective optimization, fully demonstrating its technical value as a benchmark for model selection.

[0095] Example 2: This example provides a system for generating secondary blood indicator datasets based on dynamic feature filtering, including:

[0096] Data preprocessing module: This module is used to collect basic information and basic blood indicators of the research subjects. The basic information includes demographic characteristics, personal identification information, and basic clinical information. The basic information and basic blood indicators are matched, and then duplicate data and data with missing basic blood indicators are deleted. Samples aged 20-80 years are retained to obtain the effective samples. This module is also responsible for the preliminary cleaning and format standardization of the data.

[0097] Data grouping and feature extraction module: This module is used to divide the effective samples into a control group without chronic non-communicable diseases and a target disease group based on the diagnostic information in the clinical basic information; to perform statistical difference analysis on the demographic characteristics of the control group without chronic non-communicable diseases and the target disease group, to screen out key features that are significantly related to the target disease, and to obtain the screened demographic characteristics; this module has built-in multiple statistical test algorithms and can automatically output the significance analysis results.

[0098] The indicator calculation module is used to calculate secondary blood indicators for the control group without chronic non-communicable diseases and the target disease group based on basic blood indicators, and integrate the screened demographic features to generate an enhanced dataset. This module includes an extensible indicator calculation library that supports user-defined derivative indicator formulas.

[0099] The dataset optimization module is used for stratified sampling based on disease status. It divides the augmented dataset into training and test sets, and simultaneously standardizes continuous indicators in both sets to obtain standardized training and test sets. An adaptive elastic network based on stability selection filters for optimized feature sets with non-zero coefficients. Based on these optimized feature sets, the training and test sets are simultaneously filtered to form the final dataset. The stratified sampling unit of this module supports multi-dimensional stratification settings (e.g., disease status + age + gender). The standardization unit can automatically identify continuous indicators (e.g., age, blood indicator values) and categorical indicators (e.g., gender, disease type), performing Z-score standardization only on continuous indicators.

[0100] The data grouping and feature extraction module includes:

[0101] Grouping Unit: This unit is used to divide the valid samples into a control group without chronic non-communicable diseases and a target disease group based on diagnostic information in the clinical baseline information; this unit also supports intelligent grouping based on multiple diagnostic criteria such as ICD coding;

[0102] Difference Analysis Unit: Statistical difference analysis is performed on the demographic characteristics of the control group without chronic non-communicable diseases and the target disease group to screen out key characteristics that are significantly related to the target disease and obtain the screened demographic characteristics.

[0103] The indicator calculation module includes:

[0104] Outlier handling unit: Used to perform outlier handling operations. For the baseline blood indicators of the control group without chronic non-communicable diseases, outliers are removed using box plot method. For the baseline blood indicators of the target disease group, the original data are retained (only values ​​that exceed the linear range of the test are reviewed. If it is confirmed that the value exceeds the linear range, the extreme value record is removed).

[0105] Enhanced dataset generation unit: Calculates secondary blood indicators and integrates filtered demographic features to form an enhanced dataset.

[0106] The dataset optimization module includes:

[0107] Division Unit: Used to perform stratified sampling, stratified sampling according to disease status, ensuring that the training set and test set are balanced in terms of disease type distribution, and using random seeds to ensure the reproducibility of sampling results;

[0108] Standardized Units: Perform Z-score standardization on continuous metrics of augmented datasets, automatically store the mean and standard deviation of the training set, and apply them to the test set to avoid data leakage;

[0109] Feature Dynamic Optimization Unit: This unit optimizes the feature set through an adaptive elastic network based on stability selection. It incorporates modules for initial fitting, adaptive weight calculation, cross-validation parameter optimization, and stability selection resampling, automatically outputting a highly stable optimized feature set and its stability score.

[0110] Integration Unit: Simultaneously filters features from the training and test sets, integrates data and metadata (including standardized parameters, screening paths, differential analysis results, indicator calculation methods, etc.) to generate the final dataset.

[0111] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for generating a secondary blood indicator dataset based on feature dynamic screening, characterized in that, Comprising the following steps: S1: Collecting the basic information and basic blood indicators of the research subjects, the basic information including demographic characteristics, personal identification information and clinical basic information; matching the basic information and basic blood indicators, then deleting the repeated data and the data with missing basic blood indicators, retaining the samples with an age of 20-80 years old, and finally obtaining the effective samples; S2: Dividing the effective samples into a non-chronic non-communicable disease control group and a target disease group according to the diagnosis information in the clinical basic information; eliminating outliers in the basic blood indicators in the non-chronic non-communicable disease control group, which needs to exclude individuals with chronic non-communicable diseases; for the target disease group, selecting samples that have not received treatment within 14 days after diagnosis, and retaining the original data of the basic blood indicators; S3: Performing statistical difference analysis on the demographic characteristics of the non-chronic non-communicable disease control group and the target disease group, screening out key features significantly related to the target disease, and obtaining screened demographic characteristics; S4: Calculating the secondary blood indicators of the non-chronic non-communicable disease control group and the target disease group according to the basic blood indicators, and integrating the screened demographic characteristics to generate an enhanced data set; S5: Stratified sampling according to the disease state, dividing the enhanced data set into a training set and a test set, synchronously standardizing the continuous indicators of the training set and the test set to obtain a standardized training set and a standardized test set, selecting an optimized feature set with non-zero coefficients based on stability selection adaptive elastic network, and synchronously filtering the training set and the test set based on the optimized feature set to form a final data set; Wherein, the specific steps of selecting an optimized feature set with non-zero coefficients based on stability selection adaptive elastic network are as follows: Stage one: run elastic network on training set, first optimize elastic network regularization strength λ and the mixed parameter α of L1 and L2 penalty through 5-fold cross-validation, then fit elastic network on training set based on the optimal hyperparameters to obtain the coefficients of features, and calculate adaptive weight w for each feature j according to the coefficient j =1 / (|β j |+ε), where β j is the coefficient of feature j, and ε is a constant Phase two: Introducing adaptive weights into the penalty term of elastic network to build adaptive elastic network, and synchronously optimizing two hyperparameters of adaptive elastic network: regularization strength λ and mixing parameter α of L1 regularization and L2 regularization penalty through K-fold cross-validation; Phase three: Enhancing adaptive elastic network using stability selection framework, generating sub-training sets through B times bootstrap sampling, running adaptive elastic network on each sub-training set and recording the selected features, and finally calculating the frequency of each feature being selected, denoted as stability score; Phase four: Selecting features with a stability score higher than a stability score threshold to form an optimized feature set.

2. The method of claim 1, wherein, When the indicators of the enhanced data set are continuous indicators, calculate the mean μ and standard deviation σ based on the training set, and perform Z-score standardization on the continuous indicators of the training set and the test set.

3. The method of claim 1, wherein, In step S1, the matching of the basic information and the basic blood indicators is realized according to the personal identification information, and then the personal identification information is deleted.

4. The method of claim 1, wherein the method further comprises: The secondary blood indicators are indicators closely related to chronic diseases derived from the basic blood indicators through fixed mathematical formulas, including: neutrophil-to-lymphocyte ratio, lymphocyte-to-monocyte ratio, platelet-to-lymphocyte ratio, systemic immune inflammation index, systemic inflammatory response index, neutrophil-to-monocyte-to-lymphocyte ratio, neutrophil-to-lymphocyte-to-platelet ratio, neutrophil-to-monocyte ratio, neutrophil-to-platelet ratio, and red blood cell distribution width-to-platelet ratio.

5. A system for implementing the method of generating a secondary blood indicator dataset based on feature dynamics dynamic filtering according to any one of claims 1-4, characterized in that, It comprises: a data preprocessing module for collecting basic information and basic blood indicators of the research subjects, the basic information including demographic characteristics, personal identification information and clinical basic information; matching the basic information and the basic blood indicators, then deleting repeated data and data with missing basic blood indicators, and retaining samples with an age of 20-80 years old, to finally obtain effective samples; a data grouping and feature extraction module for dividing the effective samples into a non-chronic non-communicable disease control group and a target disease group according to the diagnosis information in the clinical basic information; performing statistical difference analysis on the demographic characteristics of the non-chronic non-communicable disease control group and the target disease group, screening out key features significantly related to the target disease, and obtaining screened demographic characteristics; an indicator calculation module for calculating secondary blood indicators of the non-chronic non-communicable disease control group and the target disease group according to the basic blood indicators, and integrating the screened demographic characteristics to generate an enhanced data set; a data set optimization module for stratified sampling according to disease states, dividing the enhanced data set into a training set and a test set, synchronously normalizing continuous indicators of the training set and the test set to obtain a standardized training set and a standardized test set, selecting an optimized feature set with non-zero coefficients through adaptive elastic net screening based on stability selection, and performing feature synchronous filtering on the training set and the test set based on the optimized feature set to integrate and form a final data set.

6. The system of claim 5, wherein, The data grouping and feature extraction module comprises: a grouping unit for dividing the effective samples into a non-chronic non-communicable disease control group and a target disease group according to the diagnosis information in the clinical basic information; 7. The system of claim 5, wherein, a difference analysis unit for performing statistical difference analysis on the demographic characteristics of the non-chronic non-communicable disease control group and the target disease group, screening out key features significantly related to the target disease, and obtaining screened demographic characteristics. The indicator calculation module comprises: an outlier processing unit for performing outlier processing operations, using the box plot method to remove outliers for the basic blood indicators of the non-chronic non-communicable disease control group, and retaining original data for the basic blood indicators of the target disease group; 8. The system of claim 5, wherein, an enhanced data set generation unit for calculating secondary blood indicators and integrating screened demographic characteristics to form an enhanced data set. The data set optimization module comprises: a division unit for performing stratified sampling operations and stratified sampling according to disease states; a standardization unit for performing Z-score standardization on continuous indicators of the enhanced data set, automatically storing the mean and standard deviation of the training set, and applying them to the test set; Feature dynamic optimization unit: optimize feature set by adaptive elastic network screening based on stability selection; Integration unit: synchronize the features of training set and test set, integrate data and meta information to generate final dataset.

Images