Medical data analysis and comparison system based on big data

Abstract

The invention discloses a medical data analysis and comparison system based on big data, and relates to the technical field of medical data analysis, medical data are collected from various data sources, preprocessing and feature extraction are carried out, and subtype complexity features and sample data inclination features of rare medical patients are identified, so that the medical data are analyzed and compared. After the sufficiency of the number of rare medical patient data samples is judged, the rare medical patient samples are increased by using a data enhancement technology, undersampling is performed on common medical patient samples, a data set is balanced, a K-fold cross validation technology is used for circularly training and verifying a model, and the stability of the model is evaluated; according to the method, the medical patient identification accuracy is evaluated by comprehensively analyzing the sample number judgment result and the model stability, and finally, the result is visually displayed, so that the rare medical patient identification accuracy and the model robustness are effectively improved, and a reliable analysis result can still be provided under the condition of unbalanced medical data.

Description

Technical Field

[0001] The present invention relates to the technical field of medical data analysis, and in particular to a medical data analysis and comparison system based on big data. Background Art

[0002] Medical data analysis and comparison based on big data refers to the use of existing technologies including machine learning, artificial intelligence, data mining and statistical analysis to extract valuable information from multi-source data such as electronic health records (EHR), genomic data, imaging data and real-time health monitoring data by integrating and processing them. However, when processing and integrating multi-source data, if the number of samples of rare medical patient data is insufficient, the analysis model may be biased towards common medical patients and ignore rare medical patients. As a result, the model may not be able to correctly identify rare medical patients, resulting in misdiagnosis or missed diagnosis, affecting patient health. Summary of the invention

[0003] The purpose of the present invention is to provide a medical data analysis and comparison system based on big data to address the deficiencies in the background technology.

[0004] In order to achieve the above-mentioned object, the present invention provides the following technical solutions: a medical data analysis and comparison system based on big data, comprising a data acquisition module, a data processing module, a sample data addition module, a model stability evaluation module and a display module;

[0005] Data acquisition module: used to collect relevant medical data from various data sources, pre-process the collected data, and extract rare medical patient features from the pre-processed medical data;

[0006] Data processing module: judging whether the number of rare medical patient data samples is sufficient based on the extracted subtype complexity characteristics of rare medical patients and the tilt characteristics of patient sample data;

[0007] Sample data augmentation module: When the number of rare medical patient data is insufficient, the number of rare medical patient samples is increased through data augmentation technology, oversampling rare medical patient data and undersampling common medical patient data to balance the data set;

[0008] Model stability evaluation module: Use K-fold cross-validation technology to divide the data set into K subsets, perform training and validation in a cycle, and evaluate the stability of the analysis model;

[0009] Display module: Comprehensively analyze the judgment results of the number of rare medical patient data samples and the stability of the analysis model, evaluate the accuracy of the medical patient identification results of the analysis model, and display them visually.

[0010] Preferably, in the data processing module, a subtype distribution anomaly index is generated according to the extracted subtype complexity characteristics of rare medical patients, and the method for obtaining the subtype distribution anomaly index is:

[0011] Obtain clinical and genetic data of rare medical patients and annotate the number of subtypes M of N patients, select clinical features related to the subtypes of rare medical patients, calculate the selected features, and calculate the baseline distribution of each subtype under normal circumstances. The expression is: Where n i is the number of patients of the ith subtype, where i∈{1,2,…,M}, qi is the baseline distribution of each subtype i, and the baseline distribution is used to establish a data set Q={q1,q2,…,qi,…,qn}; and the current baseline distribution pi is calculated to establish a data set P={p1,p2,…,pi,…,pn}; the actual distribution frequency of each subtype in the current sample is calculated, that is, the subtype distribution abnormality index is calculated, and the specific calculation expression is: Where DK is the subtype distribution abnormality index.

[0012] Preferably, a data category imbalance index is generated according to the extracted patient sample data tilt feature, and the method for obtaining the data category imbalance index is:

[0013] Collect patient sample data from different sources, label each patient sample, determine the category it belongs to, mark the number of samples in the dataset as W, and divide it into k categories, and count the number of samples W in each category s s ; Calculate the proportion of each category, the expression is: Among them, G s Represents the sample proportion of category s, and calculates the proportion of expected uniform distribution. The expression is: Among them, k represents the total number of categories, and the data category imbalance index is calculated as follows: Among them, YB is the data category imbalance index.

[0014] Preferably, the subtype distribution anomaly index and the data category imbalance index are normalized, and the adequacy assessment coefficient of the number of rare medical patient data samples is calculated using the normalized subtype distribution anomaly index and the data category imbalance index.

[0015] Preferably, the sufficiency assessment coefficient of the acquired number of rare medical patient data samples is compared with a preset reference threshold of the sufficiency assessment coefficient of the number of rare medical patient data samples. If the sufficiency assessment coefficient is greater than or equal to the reference threshold of the sufficiency assessment coefficient, the number of rare medical patient data samples is sufficient, and a data sufficiency signal is generated; if the sufficiency assessment coefficient is less than the reference threshold of the sufficiency assessment coefficient, the number of rare medical patient data samples is insufficient, and a data insufficiency signal is generated.

[0016] Preferably, in the model stability evaluation module, a K-fold cross-validation technique is used to divide the data set into K subsets, and training and validation are performed cyclically to evaluate the stability of the analysis model;

[0017] Collect patient data of rare and common medical patients from various data sources, including clinical data and genetic data, select K value, randomly divide the entire dataset into K subsets, and define metrics for evaluating model performance, including accuracy, precision, recall, and F1 score;

[0018] The i-th subset is used as the validation set, and the remaining K-1 subsets are combined as the training set;

[0019] Train the model using the training set;

[0020] Use the validation set to evaluate model performance and record the evaluation results;

[0021] The evaluation indicators of the K-times validation results are collected and the corresponding data sets are established. The standard deviation of each data set is calculated. The standard deviation of each data set is weighted moving averaged to calculate the model stability analysis coefficient.

[0022] Preferably, the acquired model stability analysis coefficient is compared with the model stability analysis coefficient reference threshold. If the model stability analysis coefficient is greater than or equal to the model stability analysis coefficient reference threshold, the model stability is high, and a model stability signal is generated; if the model stability analysis coefficient is less than the model stability analysis coefficient reference threshold, the model stability is low, and a model instability signal is generated.

[0023] Preferably, in the display module, a comprehensive analysis is performed on the judgment results of the number of rare medical patient data samples and the stability of the analysis model to evaluate the accuracy of the medical patient identification results of the analysis model, specifically:

[0024] The sufficiency evaluation coefficient of the number of rare medical patient data samples and the model stability analysis coefficient are used as the input items of fuzzy logic, and the accuracy of the medical patient identification results of the analysis model is used as the output item of fuzzy logic;

[0025] The fuzzy sets of the adequacy evaluation coefficients and model stability analysis coefficients of the number of rare medical patient data samples are defined respectively;

[0026] Formulate fuzzy rules, convert specific values ​​into fuzzy values, perform reasoning based on the fuzzy rule base, and convert fuzzy values ​​into specific numerical outputs.

[0027] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0028] 1. This invention solves the problem of model bias caused by insufficient sample size by integrating multi-source medical data and extracting and evaluating features of rare disease patient data. The number of rare disease samples is increased through data enhancement technology, and common disease samples are undersampled to effectively balance the data set. At the same time, the K-fold cross-validation technology is used to evaluate and optimize the model stability, and the accuracy of the model is comprehensively analyzed through fuzzy logic to provide a reliable basis for clinical decision-making.

[0029] 2. This invention not only improves the accuracy of rare disease identification and the robustness of the model, but also intuitively displays the number of data samples and the results of model stability analysis. Through visual display, medical practitioners can have a more comprehensive understanding of model performance, make more accurate treatment decisions, and ultimately improve patient health management results. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0031] Figure 1 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] For examples, see Figure 1 As shown, the medical data analysis and comparison system based on big data described in this embodiment includes a data acquisition module, a data processing module, a sample data addition module, a model stability evaluation module and a display module;

[0034] Data acquisition module: used to collect relevant medical data from various data sources, pre-process the collected data, and extract rare medical patient features from the pre-processed medical data;

[0035] Data processing module: judging whether the number of rare medical patient data samples is sufficient based on the extracted subtype complexity characteristics of rare medical patients and the tilt characteristics of patient sample data;

[0036] Sample data augmentation module: When the number of rare medical patient data is insufficient, the number of rare medical patient samples is increased through data augmentation technology, oversampling rare medical patient data and undersampling common medical patient data to balance the data set;

[0037] Model stability evaluation module: Use K-fold cross-validation technology to divide the data set into K subsets, perform training and validation in a cycle, and evaluate the stability of the analysis model;

[0038] Display module: Comprehensively analyze the judgment results of the number of rare medical patient data samples and the stability of the analysis model, evaluate the accuracy of the medical patient identification results of the analysis model, and display them visually.

[0039] In the data acquisition module, relevant medical data is collected from various data sources, and the collected data is preprocessed, and rare medical patient features are extracted from the preprocessed medical data, specifically:

[0040] Identify the sources of medical data that need to be collected, such as hospital electronic medical record systems (EMR), medical equipment data, laboratory test results, gene sequencing data, patient questionnaires, etc. Specifically: Hospital EMR system: collects patient diagnosis, treatment and follow-up records. Laboratory system: collects test results of biological samples such as blood and urine. Gene sequencing data: collects patient genome data. Medical equipment: collects data such as electrocardiograms and blood pressure monitoring. Patient questionnaires: collect information such as patients' medical history and living habits.

[0041] Collect relevant medical data from various data sources using API, database query, file import, etc. Store the collected data in a secure database to ensure data integrity and security.

[0042] Preprocess the medical data in the database, including: Data cleaning: processing missing values, duplicate values ​​and outliers in the data. Data standardization: converting data into a unified format and unit to ensure data consistency. Data integration: integrating data from different data sources to build a comprehensive data set. Data conversion: segmenting and encoding text data, and normalizing numerical data.

[0043] Select features related to rare medical patients from the preprocessed data. Use machine learning and statistical methods to extract key features of rare medical patients. Perform engineering processing on the extracted features to construct feature vectors. Specifically include:

[0044] Feature selection: Based on medical knowledge and literature, select clinical features, test indicators, gene variants, etc. related to rare medical patients. Use feature selection algorithms (such as LASSO regression, recursive feature elimination, etc.) to automatically select the most important features.

[0045] Feature extraction: Use principal component analysis (PCA) to reduce dimensionality and extract the most important principal components in the data. Use clustering algorithms (such as K-means, hierarchical clustering) to identify characteristic patterns of rare medical patients. Use genomic methods to identify genetic variants associated with rare medical patients.

[0046] Feature engineering: Extract features from time series data, such as extracting the heart rate variability feature of an electrocardiogram. Construct feature interaction terms, such as taking the product or ratio of multiple test indicators as a new feature. Perform feature transformation, such as logarithmic transformation, square root transformation, etc.

[0047] Data processing module: Based on the extracted subtype complexity characteristics of rare medical patients and the skew characteristics of patient sample data, determine whether the number of rare medical patient data samples is sufficient.

[0048] The subtype distribution anomaly index is generated according to the extracted subtype complexity characteristics of rare medical patients. The method for obtaining the subtype distribution anomaly index is as follows:

[0049] Obtain clinical and genetic data of rare medical patients and annotate the number of subtypes M of N patients, select clinical features related to the subtypes of rare medical patients, calculate the selected features, and calculate the baseline distribution of each subtype under normal circumstances. The expression is: Where n i is the number of patients of the ith subtype, where i∈{1,2,…,M}, qi is the baseline distribution of each subtype i, and the baseline distribution is used to establish a data set Q={q1,q2,…,qi,…,qn}; and the current baseline distribution pi is calculated to establish a data set P={p1,p2,…,pi,…,pn}; the actual distribution frequency of each subtype in the current sample is calculated, that is, the subtype distribution abnormality index is calculated, and the specific calculation expression is: Where DK is the subtype distribution abnormality index.

[0050] The larger the subtype distribution anomaly index is, the greater the diversity of the number of samples of rare medical patient data is. This means that in the data set, the distribution of each subtype is significantly different from the expected baseline distribution, and there may be more different types of subtypes or some subtypes have abnormally high or low frequencies, reflecting the increased diversity and complexity of the samples.

[0051] The data category imbalance index is generated according to the extracted patient sample data tilt characteristics, and the method for obtaining the data category imbalance index is:

[0052] Collect patient sample data from different sources, label each patient sample, determine the category it belongs to, mark the number of samples in the dataset as W, and divide it into k categories, and count the number of samples W in each category s s ; Calculate the proportion of each category, the expression is: Among them, G s Represents the sample proportion of category s, and calculates the proportion of expected uniform distribution. The expression is: Among them, k represents the total number of categories, and the data category imbalance index is calculated as follows: Among them, YB is the data category imbalance index.

[0053] The larger the data category imbalance index, the smaller the diversity of the number of samples of rare medical patient data. A larger imbalance index means that the number of samples in some categories is significantly larger than that in other categories, indicating that there is a serious category imbalance problem in the dataset, reflecting the low diversity of the number of samples, which may cause the model to favor larger categories during training and ignore the characteristics of smaller categories.

[0054] The subtype distribution anomaly index and the data category imbalance index are normalized, and the adequacy assessment coefficient of the number of rare medical patient data samples is calculated based on the normalized subtype distribution anomaly index and the data category imbalance index.

[0055] For example, the present invention can use the following formula to calculate the adequacy evaluation coefficient of the number of rare medical patient data samples, and the calculation expression is: Where WP is the adequacy assessment coefficient of the number of rare medical patient data samples, DK is the subtype distribution anomaly index, YB is the data category imbalance index, a1 and a2 are the proportional coefficients of the subtype distribution anomaly index and the data category imbalance index, respectively, and a2>a1>0.

[0056] The sufficiency assessment coefficient of the obtained number of rare medical patient data samples is compared with the pre-set reference threshold of the sufficiency assessment coefficient of the number of rare medical patient data samples. If the sufficiency assessment coefficient is greater than or equal to the reference threshold of the sufficiency assessment coefficient, it means that the number of rare medical patient data samples is sufficient, and a data sufficiency signal is generated at this time; if the sufficiency assessment coefficient is less than the reference threshold of the sufficiency assessment coefficient, it means that the number of rare medical patient data samples is insufficient, and a data insufficiency signal is generated at this time.

[0057] Sample data augmentation module: When the number of rare medical patient data is insufficient, data augmentation technology is used to increase the number of rare medical patient samples, oversample rare medical patient data, undersample common medical patient data, and balance the data set.

[0058] Collect patient data of rare and common medical patients from various data sources, including clinical data and genetic data. Clean, standardize and extract features from the data to ensure data quality and consistency.

[0059] Choose appropriate data augmentation methods based on the data type, such as synthetic minority oversampling technique (SMOTE), autoencoder generation, data transformation, etc. Apply data augmentation techniques: Oversample rare medical patient data to increase the number of samples.

[0060] Select rare medical patient sample data, calculate K nearest neighbors, randomly select several samples from the K nearest neighbors, generate new sample data, and the calculation expression is: xnew = xi + δ*(xnearest_neighbor-xi); where xnew is the newly generated sample, xi is the existing minority class sample, xnearest_neighbor is a sample in the K nearest neighbors, and δ is a random number between 0 and 1.

[0061] Undersampling common medical patient data can use methods such as random undersampling and cluster undersampling. Undersampling common medical patient data to reduce the number of samples includes:

[0062] Randomly selected sample: A subset of samples is randomly selected from a common medical patient sample.

[0063] Construct a new dataset: Merge the selected samples with the rare medical patient samples to construct a new balanced dataset.

[0064] Merge data: Merge the augmented rare medical patient data with the undersampled common medical patient data.

[0065] Divide the dataset: Divide the dataset into training set, validation set and test set. Divide the training set: used for model training, accounting for 80% of the total data. Divide the validation set: used for model hyperparameter tuning, accounting for 10% of the total data. Divide the test set: used for final model evaluation, accounting for 10% of the total data.

[0066] Model stability evaluation module: Use K-fold cross-validation technology to divide the data set into K subsets, perform training and validation in a cycle, and evaluate the stability of the analysis model;

[0067] Collect patient data of rare and common medical patients from various data sources, including clinical data and genetic data. Clean, standardize and extract features from the data to ensure data quality and consistency.

[0068] Select the K value, commonly used choices are 5 or 10. Randomly divide the entire dataset into K subsets (folds). Define the metrics used to evaluate the performance of the model, including accuracy, precision, recall, and F1 score.

[0069] For each subset i (i from 1 to K):

[0070] Training set and validation set division: the i-th subset is used as the validation set, and the remaining K-1 subsets are combined as the training set.

[0071] Model training: Use the training set to train the model.

[0072] Model validation: Use the validation set to evaluate model performance and record the evaluation results.

[0073] Record results: Record the evaluation indicators of this training and verification.

[0074] The evaluation indicators of the K-times validation results are collected and the corresponding data sets are established. The standard deviation of each data set is calculated. The standard deviation of each data set is weighted moving averaged to calculate the model stability analysis coefficient.

[0075] The obtained model stability analysis coefficient is compared with the model stability analysis coefficient reference threshold. If the model stability analysis coefficient is greater than or equal to the model stability analysis coefficient reference threshold, it means that the model stability is high, and a model stability signal is generated at this time; if the model stability analysis coefficient is less than the model stability analysis coefficient reference threshold, it means that the model stability is low, and a model instability signal is generated at this time.

[0076] Display module: Comprehensively analyze the judgment results of the number of rare medical patient data samples and the stability of the analysis model, evaluate the accuracy of the medical patient identification results of the analysis model, and display them visually.

[0077] The sufficiency evaluation coefficient WP of the number of rare medical patient data samples and the model stability analysis coefficient HG are used as the input items of fuzzy logic, and the accuracy of the medical patient identification results of the analysis model is used as the output item of fuzzy logic;

[0078] The fuzzy sets of the adequacy evaluation coefficients and model stability analysis coefficients of the number of rare medical patient data samples are defined respectively;

[0079] Fuzzy set definition: Input item WP: divided into low, medium, and high. Input item HG: divided into unstable, medium stable, and stable. Output item accuracy: divided into low, medium, and high.

[0080] Fuzzy rule base design: If WP is low and HG is unstable, the accuracy is low. If WP is low and HG is moderately stable, the accuracy is low. If WP is low and HG is stable, the accuracy is moderate. If WP is moderate and HG is unstable, the accuracy is low. If WP is moderate and HG is moderately stable, the accuracy is moderate. If WP is moderate and HG is stable, the accuracy is high. If WP is high and HG is unstable, the accuracy is moderate. If WP is high and HG is moderately stable, the accuracy is high. If WP is high and HG is stable, the accuracy is high.

[0081] Fuzzification: Convert specific values ​​into fuzzy values.

[0082] Rule reasoning: Reasoning based on the fuzzy rule base.

[0083] Defuzzification: Convert fuzzy values ​​into concrete numerical outputs.

[0084] Format the chart: WP on the horizontal axis. HG on the vertical axis. Use color or size to indicate accuracy.

[0085] Draw scatter points: For each sample, draw a point, the position is determined by WP and HG, and the color or size is determined by accuracy.

[0086] Add a legend: The legend shows how color or size corresponds to accuracy.

[0087] The horizontal axis (X axis) indicates the sample adequacy assessment coefficient (WP). The vertical axis (Y axis) indicates the model stability analysis coefficient (HG). Each data point represents a sample. The color of the data point indicates the accuracy. The darker the color, the higher the accuracy.

[0088] Define a color map (e.g. from light blue to dark blue) to represent different values ​​of accuracy. Assign a corresponding color to each sample based on its accuracy.

[0089] For example, using the following color map:

[0090] Accuracy between 0.6-0.7: light blue

[0091] Accuracy between 0.7-0.8: Blue

[0092] Accuracy between 0.8-0.9: dark blue

[0093] For the given dataset:

[0094] Sample 1: WP = 0.2, HG = 0.85, Accuracy = 0.70 (blue)

[0095] Sample 2: WP = 0.4, HG = 0.90, Accuracy = 0.75 (blue)

[0096] Sample 3: WP = 0.3, HG = 0.80, Accuracy = 0.65 (light blue)

[0097] Sample 4: WP = 0.5, HG = 0.95, Accuracy = 0.85 (dark blue)

[0098] Sample 5: WP = 0.25, HG = 0.87, Accuracy = 0.72 (blue)

[0099] Data point location: The location of the data point on the scatter plot is determined by WP and HG. For example, the location of sample 1 is (0.2, 0.85).

[0100] Data point color: The color of the data point reflects the accuracy. For example, the color of sample 1 is blue, indicating that its accuracy is between 0.7-0.8.

[0101] In this embodiment, relevant medical data is collected from various data sources, and the collected data is preprocessed, and rare medical patient features are extracted from the preprocessed medical data; whether the number of rare medical patient data samples is sufficient is judged based on the extracted subtype complexity features of rare medical patients and the tilt features of patient sample data; when the number of rare medical patient data is insufficient, the number of rare medical patient samples is increased by data enhancement technology, the rare medical patient data is oversampled, and the common medical patient data is undersampled to balance the data set; the K-fold cross-validation technology is used to divide the data set into K subsets, and training and verification are performed cyclically to evaluate the stability of the analysis model; the judgment results of the number of rare medical patient data samples and the stability of the analysis model are comprehensively analyzed, the accuracy of the medical patient identification results of the analysis model is evaluated, and the results are visualized. The accuracy of rare medical patient identification and the robustness of the model are effectively improved, ensuring that reliable analysis results can still be provided in the case of data imbalance.

[0102] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0103] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0104] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0108] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0109] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A medical data analysis and comparison system based on big data, Features: It includes data acquisition module, data processing module, sample data addition module, model stability evaluation module and display module; Data acquisition module: used to collect relevant medical data from various data sources, pre-process the collected data, and extract rare medical patient features from the pre-processed medical data; Data processing module: judging whether the number of rare medical patient data samples is sufficient based on the extracted subtype complexity characteristics of rare medical patients and the tilt characteristics of patient sample data; Sample data augmentation module: When the number of rare medical patient data is insufficient, the number of rare medical patient samples is increased through data augmentation technology, oversampling rare medical patient data and undersampling common medical patient data to balance the data set; Model stability evaluation module: Use K-fold cross-validation technology to divide the data set into K subsets, perform training and validation in a cycle, and evaluate the stability of the analysis model; Display module: Comprehensively analyze the judgment results of the number of rare medical patient data samples and the stability of the analysis model, evaluate the accuracy of the medical patient identification results of the analysis model, and display them visually.

2. A medical data analysis and comparison system based on big data according to claim 1, Features: In the data processing module, the subtype distribution anomaly index is generated according to the extracted subtype complexity characteristics of rare medical patients. The method for obtaining the subtype distribution anomaly index is: Obtain clinical and genetic data of rare medical patients and annotate the number of subtypes M of N patients, select clinical features related to the subtypes of rare medical patients, calculate the selected features, and calculate the baseline distribution of each subtype under normal circumstances. The expression is: Where n i is the number of patients of the ith subtype, where i∈{1,2,…,M}, qi is the baseline distribution of each subtype i, and the baseline distribution is used to establish a data set Q={q1,q2,…,qi,…,qn}; and the current baseline distribution pi is calculated to establish a data set P={p1,p2,…,pi,…,pn}; the actual distribution frequency of each subtype in the current sample is calculated, that is, the subtype distribution abnormality index is calculated, and the specific calculation expression is: Where DK is the subtype distribution abnormality index.

3. A medical data analysis and comparison system based on big data according to claim 2, Features: The data category imbalance index is generated according to the extracted patient sample data tilt characteristics, and the method for obtaining the data category imbalance index is: Collect patient sample data from different sources, label each patient sample, determine the category it belongs to, mark the number of samples in the dataset as W, and divide it into k categories, and count the number of samples W in each category s s ; Calculate the proportion of each category, the expression is: Among them, G s Represents the sample proportion of category s, and calculates the proportion of expected uniform distribution. The expression is: Among them, k represents the total number of categories, and the data category imbalance index is calculated as follows: Among them, YB is the data category imbalance index.

4. A medical data analysis and comparison system based on big data according to claim 3, Features: The subtype distribution anomaly index and the data category imbalance index are normalized, and the adequacy assessment coefficient of the number of rare medical patient data samples is calculated based on the normalized subtype distribution anomaly index and the data category imbalance index.

5. A medical data analysis and comparison system based on big data according to claim 4, Features: The sufficiency assessment coefficient of the obtained number of rare medical patient data samples is compared with the pre-set reference threshold of the sufficiency assessment coefficient of the number of rare medical patient data samples. If the sufficiency assessment coefficient is greater than or equal to the reference threshold of the sufficiency assessment coefficient, the number of rare medical patient data samples is sufficient, and a data sufficiency signal is generated; if the sufficiency assessment coefficient is less than the reference threshold of the sufficiency assessment coefficient, the number of rare medical patient data samples is insufficient, and a data insufficiency signal is generated.

6. A medical data analysis and comparison system based on big data according to claim 1, Features: In the model stability assessment module, the K-fold cross-validation technique is used to divide the data set into K subsets, and training and validation are performed cyclically to evaluate the stability of the analysis model; Collect patient data of rare and common medical patients from various data sources, including clinical data and genetic data, select K value, randomly divide the entire dataset into K subsets, and define metrics for evaluating model performance, including accuracy, precision, recall, and F1 score; The i-th subset is used as the validation set, and the remaining K-1 subsets are combined as the training set; Train the model using the training set; Use the validation set to evaluate model performance and record the evaluation results; The evaluation indicators of the K-times validation results are collected and the corresponding data sets are established. The standard deviation of each data set is calculated. The standard deviation of each data set is weighted moving averaged to calculate the model stability analysis coefficient.

7. A medical data analysis and comparison system based on big data according to claim 6, Features: The obtained model stability analysis coefficient is compared with the model stability analysis coefficient reference threshold. If the model stability analysis coefficient is greater than or equal to the model stability analysis coefficient reference threshold, the model stability is high, and a model stability signal is generated at this time; if the model stability analysis coefficient is less than the model stability analysis coefficient reference threshold, the model stability is low, and a model instability signal is generated at this time.

8. The medical data analysis and comparison system based on big data according to claim 1, Features: In the display module, a comprehensive analysis is conducted on the judgment results of the number of rare medical patient data samples and the stability of the analysis model to evaluate the accuracy of the medical patient identification results of the analysis model. Specifically: The sufficiency evaluation coefficient of the number of rare medical patient data samples and the model stability analysis coefficient are used as the input items of fuzzy logic, and the accuracy of the medical patient identification results of the analysis model is used as the output item of fuzzy logic; The fuzzy sets of the adequacy evaluation coefficients and model stability analysis coefficients of the number of rare medical patient data samples are defined respectively; Formulate fuzzy rules, convert specific values ​​into fuzzy values, perform reasoning based on the fuzzy rule base, and convert fuzzy values ​​into specific numerical outputs.

Images