Intelligent medical diagnosis method and device based on semi-supervised depth non-parallel support vector machine
The SDNSVM method improves glioma grading diagnosis by leveraging self-supervised feature augmentation, pseudo-labels, and class balancing to enhance learning from limited data, addressing data insufficiency and class imbalance, thereby improving accuracy and efficiency.
Patent Information
- Application Number
- CN202410967070.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-18
AI Technical Summary
The grading diagnosis of brain glioma faces challenges due to the high cost of obtaining samples with diagnostic information, leading to insufficient data for model learning, and an imbalance in glioma dataset categories, causing the model to ignore minority classes and produce incorrect judgments.
An intelligent medical diagnosis method based on a semi-supervised deep non-parallel support vector machine (SDNSVM) is proposed, utilizing a self-supervised numerical feature augmentation mechanism, pseudo-labels, class balancing, and a non-parallel intra-class and inter-class loss mechanism to enhance learning from small amounts of labeled and unlabeled data, improving accuracy and efficiency.
The method effectively addresses the issues of insufficient data and class imbalance, enhancing the model's generalization ability, recognition of minority classes, and overall classification accuracy for glioma grading.
Smart Images

Figure CN119069109A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer-aided medical diagnosis, in particular to intelligent medical diagnosis data mining tasks for genetic information and clinical characteristics of different patients. Specifically, the present invention proposes an intelligent medical diagnosis method and device based on a semi-supervised deep nonparallel support vector machine (SDNSVM) for hierarchical diagnosis of brain gliomas. Background Art
[0002] With the development of intelligent medical diagnosis technology, the accuracy and efficiency of medical diagnosis have been significantly improved. Intelligent medical diagnosis is mainly used in the early identification of diseases, prognosis prediction, formulation of treatment plans, reduction of misdiagnosis and missed diagnosis rates, and real-time monitoring and early warning of patients. By combining advanced machine learning technology, intelligent medical diagnosis systems can analyze a large amount of clinical data and imaging information, provide auxiliary decision support, and improve the accuracy and efficiency of diagnosis.
[0003] Among many medical applications, glioma grading diagnosis has particularly important research value and broad application prospects. Glioma refers to tumors originating from brain glial cells and is one of the most common primary intracranial tumors. Although its pathogenesis has not yet been fully clarified, the following two key factors have a significant impact on technological progress: long-term exposure to high-dose ionizing radiation is a known predisposing factor, and high-penetrance genetic mutations associated with certain rare syndromes are also important risk factors. The annual incidence of glioma in my country is 5 to 8 / 100,000, and the 5-year mortality rate is second only to pancreatic cancer and lung cancer among all tumors. According to histological and imaging standards, gliomas are mainly divided into two categories: low-grade glioma (LGG) and glioblastoma multiforme (GBM). Low-grade gliomas grow slowly and have low malignancy; glioblastoma multiforme grows rapidly, has high malignancy and poor prognosis. In addition to traditional histological and imaging features, clinical and genetic mutation factors (such as the patient's age, gender, and specific gene mutation status) also play an important role in accurate grading and treatment strategy formulation.
[0004] However, in the field of machine learning, the grading diagnosis method of brain glioma faces two major challenges: First, it is expensive to obtain samples with diagnostic information, and it is difficult to obtain enough data, resulting in the model being unable to learn enough information from a small amount of data, which in turn affects the classification effect. Second, there is an imbalance in the categories of glioma datasets, that is, low-grade glioma (LGG) data is far more than glioblastoma multiforme (GBM) data, causing the model to ignore minority classes during the learning process and produce incorrect judgments. Therefore, grading diagnosis of brain glioma is an imbalanced learning problem.
[0005] In this context, relevant scholars have proposed several solutions to address the above challenges: [1] Yoon J, Zhang Y, Jordon J, et al. Vime: Extending the success of self-and semi-supervised learning to tabular domain [J]. Advances in Neural Information Processing Systems, 2020, 33: 11033-11043. [2] Pant H, Sharma M, Soman S. Twin neural networks for the classification of large unbalanced datasets [J]. Neurocomputing, 2019, 343: 34-49. [3] Qi Z, Tian Y, Shi Y. Laplacian twin support vector machine for semi-supervised classification [J]. Neural networks, 2012, 35: 46-53. [4] Ren Y, Zhu H, Tian Y, et al. A Laplacian svm based semi-supervised classification using multi-local linear model [J]. IEEE Transactions on Electrical and Computer Engineering. Electronic Engineering, 2021, 16(3): 455-463. Reference [1] pre-trains tabular data through self-supervision, then uses semi-supervision for classification tasks, and uses mask vectors to corrupt data to expand tabular data. Reference [2] extends TWSVM to the neural network framework, and proposes the construction of a dual neural network to address the problem of class imbalance, in which two neural networks are trained to learn linear classifiers in the feature space respectively, similar to the hyperplane model of TWSVM, to handle imbalanced data sets. Reference [3] addresses the semi-supervised problem by introducing a manifold regularization term combined with SVM to solve the problem. Reference [4] improves theoretical interpretability by incorporating potential boundary information into the kernel function through an interpretable model. Although these methods have alleviated the imbalance problem of semi-supervised learning and SVM to a certain extent, the following challenges still exist:
[0006] 1. In the literature [1], pre-training is required before semi-supervised training, which significantly increases the training time.
[0007] 2. Although the literature [2] solved the problem of class imbalance, the multi-classification problem requires solving a large number of parameters, which increases the amount of calculation and time.
[0008] 3. References [3] and [4] did not consider the imbalance between classes, which resulted in the model being biased towards the minority class.
[0009] To this end, the present invention proposes an intelligent medical diagnosis method based on a semi-supervised deep non-parallel support vector machine to more effectively address these challenges and improve the accuracy and efficiency of grading diagnosis of brain gliomas. Summary of the invention
[0010] In order to overcome the difficulty of grading and diagnosing gliomas in smart medical treatment, the present invention proposes an intelligent medical diagnosis method and device based on a semi-supervised deep nonparallel support vector machine (SDNSVM). This method is implemented through a neural network framework, combined with the loss function of a non-parallel support vector machine, making full use of a small amount of labeled data and a large amount of unlabeled data, solving the problems of insufficient data and imbalanced categories, and improving the accuracy and efficiency of grading and diagnosing gliomas.
[0011] Specifically, the technical innovations of the present invention include the following aspects:
[0012] 1) This paper introduces a self-supervised numerical feature augmentation mechanism. A feature generator is constructed by using mask vectors and data features to generate new features to augment the original table data, which is used to enhance the learning of the relationship between features and improve the generalization ability of the model.
[0013] 2) This paper proposes a semi-supervised learning strategy for pseudo-labels. By using the model's prediction results for unlabeled data as pseudo-labels, the training set is expanded and the model performance is improved. In order to ensure the high quality of pseudo-labels, this paper introduces a confidence screening mechanism to enhance the reliability of unsupervised sample classification.
[0014] 3) In order to solve the problem of class imbalance, the present invention randomly samples from the minority class samples and adds new samples to balance the proportion of each class sample. This allows the model to avoid preference for the majority class during the learning process, improves the model's recognition ability for the minority class, and ensures accuracy and fairness of classification.
[0015] 4) The present invention constructs a non-parallel intra-class and inter-class loss mechanism. To handle complex multi-class learning scenarios,
[0016] By combining the local information within and between classes and adopting the parameter sharing mechanism to find the optimal hyperplane,
[0017] It enhances the learning ability of multi-category information and improves the learning efficiency of the model.
[0018] In order to facilitate better understanding and description, the present invention formulates a unified symbol representation rule, in which scalars are represented by lowercase or uppercase letters without bold, such as: x, X. Vectors are represented by lowercase letters with bold, such as: x. Matrices are represented by uppercase letters with bold, such as: X. Table 1 is a description of the symbols involved in the present invention.
[0019]
[0020]
[0021] Table 1 SDNSVM symbol description table
[0022] The present invention is an intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine (SDNSVM). The overall process is as follows: Figure 1 As shown in FIG. 1 , the intelligent medical diagnosis method based on the semi-supervised deep non-parallel support vector machine is described as follows:
[0023] Step 1: Construct and preprocess the medical dataset.
[0024] This step realizes the construction and preprocessing of medical data sets, including data screening and preprocessing, data normalization and data labeling, and imbalanced data balancing. The specific description is as follows:
[0025] (1.1) Data screening and preprocessing. First, the clinical medical data were screened and analyzed to extract the 20 most frequently mutated gene loci and two clinical characteristics: gender and age of diagnosis. To ensure the integrity and quality of the data, the raw data were preprocessed, including filtering instances of missing key information, converting the age of diagnosis from a string format to a continuous value, and excluding irrelevant columns (such as medical record number, initial diagnosis, and project).
[0026] (1.2) Data normalization and data labeling. In order to meet the input requirements of the model, the data is normalized. Normalization ensures that the numerical ranges of different features are consistent, thereby improving the effect of model training. Specifically, after normalization, the data is labeled according to the doctor’s diagnosis results to obtain supervised samples. where x i ∈R d is the sample feature of dimension d, y i is the label corresponding to the sample, n l is the supervised sample size; for unlabeled data, it is the unsupervised sample Where n u is the unsupervised sample size.
[0027] (1.3) Balance the imbalanced data. Check the preprocessed glioma dataset to ensure the accuracy and consistency of the data. To address the class imbalance problem in the dataset, randomly sample the minority class samples and increase the number of minority class samples to balance the data. This prevents the model from having a preference for the majority class during the learning process, thereby improving the recognition ability of the minority class.
[0028] First calculate the number of samples in each category and record it as n k , and then determine the target number of samples. Assume that the category with the most samples is n max , then the goal is to increase the number of samples of all categories to n max Then calculate the number of samples that need to be added. For each minority class k, the number of samples that need to be added is:
[0029]
[0030] Finally, randomly select samples from the minority class k samples and add them to the dataset. Through this process, the number of samples in all classes will tend to be the same, thus balancing the dataset.
[0031] Step 2: Augment supervised and unsupervised samples.
[0032] This step realizes the augmentation function of the dataset samples. By constructing a mask generator and a feature generator and integrating the augmented output, the training effect and generalization ability of the model are improved. Inspired by self-supervised learning, by i ,u i A series of enhancement transformations are performed to generate new training samples, learn the potential relationships between data samples, and thus learn useful feature representations to achieve diversity in data samples.
[0033] The specific description is as follows:
[0034] (2.1) Construct a mask generator. The generator outputs a binary mask of dimension D, denoted as m = [m1, ..., m D ] T ∈{0, 1} D , where m D With probability p m It is randomly sampled from the Bernoulli distribution, that is:
[0035]
[0036] (2.2) Construct a feature generator (Pretext generator). The generator takes the input mask m and calculates the corresponding subtraction operation (1-m), and multiplies (1-m) and the original sample feature element by element to obtain (1-m)⊙s. The original feature sample s (which can be a supervised sample or an unsupervised sample) is randomly shuffled column by column to obtain Multiply element-wise with the mask m, denoted as The final operation can be expressed as:
[0037]
[0038] Among them, g m is the feature generator, is the enhancing features, ⊙ is the element-by-element multiplication, s is the sample, is a randomly shuffled feature matrix. Formula (3) is interpreted as retaining the original samples of the part with a value of 1 in mask m plus the part with a value of 0 in mask m.
[0039] (2.3) Integrate the augmented output. According to formula (3), take the supervised sample x, unsupervised samples u1, u2 and mask m for sample s respectively, and get the corresponding supervised sample augmented features And unsupervised sample augmentation features
[0040] Step 3: Calculate the reconstruction loss and consistency loss.
[0041] This step implements the function of calculating reconstruction and consistency loss, including building a feature representation network model, calculating the logits corresponding to the feature, calculating the reconstructed feature representation, calculating the reconstruction loss of the sample, and calculating the mask reconstruction loss. The reconstruction loss measures the difference between the model's reconstructed features and the original features, while the consistency loss ensures that the output of the neural network remains consistent under different conditions. The network model structure diagram of the present invention is shown in the figure below: Figure 2 The specific description is as follows:
[0042] (3.1) Constructing a feature representation network model F. In order to fully utilize the powerful feature extraction capability of the neural network, the present invention uses a network model to extract the feature representation of the sample in the nonlinear space, replacing the nonlinear transformation implemented by the traditional kernel function. Specifically, the network F consists of an encoder f enc and decoder f dec The structure of the encoder f enc is responsible for mapping the input data into a compact feature space, while the decoder f dec It is used to reconstruct the input data, and the specific form is as follows:
[0043]
[0044] in, is a function composite symbol. F is composed of encoder f enc and decoder f dec The composite structure is used as the overall network. The encoder-decoder structure can not only effectively capture the complex nonlinear characteristics of the sample, but also improve the accuracy and robustness of feature expression.
[0045] (3.2) Calculate the corresponding logits of the features. Splice input encoder f enc , and get the corresponding logits, recorded as
[0046]
[0047] Among them, W e , where W e and b e are the network weights and biases, is the vector concatenation symbol. By concatenating the augmented features, it is input into the encoder and the corresponding logits are output.
[0048] (3.3) Calculate the reconstructed feature representation. Concatenate the input encoder-decoder model to obtain supervised sample reconstruction features Unsupervised reconstruction features and mask reconstruction features
[0049]
[0050] Among them, W d and b d are the network weights and biases. By concatenating the augmented features and inputting them into the encoder-decoder model, the corresponding reconstructed features are output.
[0051] (3.4) Calculate the reconstruction loss of the sample. The reconstruction loss can evaluate the difference between the autoencoder reconstructed sample and the original sample. The calculation process is as follows: Figure 3 shown.
[0052] Calculate the supervised sample mean squared error loss:
[0053]
[0054] Among them, n l is the supervised sample size, the original sample features x and the reconstructed sample features When minimizing this loss, we can reduce x and The difference between.
[0055] Calculate the reconstruction loss for unsupervised samples:
[0056]
[0057] Among them, n u is the unsupervised sample size, the original unsupervised sample feature u1 and the reconstructed unsupervised sample feature In order to minimize this loss, reduce u1 and The difference between them is calculated to ensure the consistency between the reconstructed features and the original features.
[0058] (3.5) Calculate the mask reconstruction loss. For the original mask m and the reconstructed mask Calculate the cross entropy loss:
[0059]
[0060] Where D is the size of the mask m. The original mask m and the reconstructed mask are evaluated by calculating the cross entropy loss. The difference in probability distributions.
[0061] (3.6) Calculate the consistency loss of unsupervised samples. Calculate the unsupervised sample logits, and Consistency loss:
[0062]
[0063] Among them, n u is the unsupervised sample size. The prediction results after enhancement under the same sample are: and The model can still remain consistent, which is used to evaluate the model's ability to remain robust to data changes at the prediction level.
[0064] Step 4: Use confidence to filter high-quality pseudo labels.
[0065] This step achieves effective use of unsupervised samples by generating and screening high-quality pseudo labels. This includes generating pseudo label vectors for unsupervised samples and screening out high-quality pseudo labels to enhance the reliability of unsupervised sample predictions. By screening pseudo labels using confidence thresholds, it can be ensured that the pseudo labels used by the model during training have high accuracy and credibility. The specific description is as follows:
[0066] (4.1) Create a pseudo label vector for the unsupervised sample. According to formula (5), For unsupervised samples, only encoder f enc The obtained logits. have:
[0067]
[0068] in, Output for each category, k is the category. A vector representing the output for each class.
[0069] The formula (11) is obtained Taking the absolute value, we get The activation function softmin is used to map a real number between 0 and 1, which is recorded as And normalization ensures that the sum is 1, that is:
[0070]
[0071] Where σ is the symbolic representation of the activation function softmin. The sum of all probabilities is 1.
[0072] (4.2) Filter high-quality pseudo labels. Introduce a scalar hyperparameter confidence threshold τ∈[0, 1]. If the prediction result is above this threshold, the pseudo label of this category will be retained. On this basis, increasing the confidence threshold can filter pseudo label samples with higher credibility. First, take its q k The maximum probability is compared with the confidence threshold τ, and those greater than the threshold are retained.
[0073]
[0074] in, is the indicator function, max(q k ) is the maximum category probability of the unsupervised sample. When the maximum category probability is greater than the confidence threshold τ, Otherwise it is 0.
[0075] According to formula (13), if it is greater than the confidence threshold τ, the pseudo label is taken out and recorded as
[0076]
[0077] Among them, the pseudo label Q k The category with the highest probability.
[0078] Step 5: Construct the classification task loss function.
[0079] This step implements the loss function design of the classification task, including constructing a non-parallel intra-class loss function and a non-parallel inter-class loss function. By combining the local information within and between classes and adopting a parameter sharing mechanism, the present invention finds an optimal hyperplane f for each class. k (x), thereby improving the classification accuracy and the robustness of the model, in the following form:
[0080]
[0081] Among them, f k (x) is the proximal hyperplane of class k. k and b k are the weights and biases corresponding to the hyperplane, f enc is the encoder, and x is the feature of the sample. The present invention can be used for multi-classification, in which a similar "one-to-rest" approach is used to handle multi-classification learning problems, that is, for K-class classification, K binary classifiers are constructed, the k-th class in the k-th classifier is the positive class, and the remaining classes are trained as negative classes. The process of calculating intra-class and inter-class losses in the present invention is as follows: Figure 4 shown.
[0082] (5.1) Construct a non-parallel intra-class loss function. Construct an intra-class loss function for the kth class of all sample points s. The output of the kth class is required to be the minimum. Take a sample point i as an example, according to the true label y and the pseudo label First filter out the output of the kth category, recorded as:
[0083]
[0084] Then calculate the minimum distance of the kth class, and the loss function is constructed as follows:
[0085]
[0086] Take the 2-norm for the k-th class and find the distance from the k-th class to the origin.
[0087] In addition, the minimum output of classes other than the kth class is required The distance from the hyperplane to the kth class is at least α. Taking a sample point i as an example, the loss function is constructed as follows:
[0088]
[0089] Where α∈R is the margin. Formula (18) obtains the minimum output of the class except the k-th class, and Formula (19) requires that the class except the k-th class and the hyperplane to which the k-th class belongs have a margin distance.
[0090] (5.2) Construct a non-parallel inter-class loss function. For all sample points s, the kth class and the classes other than the kth class Construct an inter-class loss function. Require the minimum output of all classes except the kth class The distance from the kth class is at least α. Taking a sample point i as an example, the loss function is constructed as follows:
[0091]
[0092] Where α∈R is the margin. Formula (20) requires that the classes other than the k-th class have a margin distance from the k-th class.
[0093] Step 6: Construct the overall loss function of the model.
[0094] This step integrates an overall loss function, including promoting the compactness of intra-class samples, the discrimination of inter-class samples, model reconstruction accuracy and prediction consistency. Combining these target loss functions into a unified loss function can guide the model to continuously optimize during the training process, thereby improving the overall generalization ability of the model. The specific description is as follows:
[0095] (6.1) Fusion of intra-class and inter-class loss functions. In this sub-step, the intra-class loss function and the inter-class loss function are fused to optimize both intra-class compactness and inter-class discrimination. Specifically, by combining equations (17), (19) and (20), the fusion is:
[0096]
[0097] Among them, w ik , is a regularization term, λ1∈R, λ2∈R, λ3∈R are trade-off parameters. Analyzing formula (21), the first term of the minimized objective function is the regularization term, in order to minimize the structural risk, that is, to balance the model complexity and model accuracy to avoid overfitting; the second term brings the k-th class sample point closer to the hyperplane of that class; the third term requires that the distance from other class sample points to the k-th class sample point is at least one margin, that is, the slack variable is the measurement error when the constraint condition is not met, aiming to maximize the separation of the two classes of sample points; the fourth term requires that the distance from other class sample points to the hyperplane to which the k-th class sample points belong is at least one margin, that is, other classes are as far away from the hyperplane to which the k-th class sample points belong as possible.
[0098] The supervised loss function is:
[0099]
[0100] Indicates the loss for all supervised samples.
[0101] The unsupervised loss is:
[0102]
[0103] Analyze formula (23), is the indicator function, max(q ik ) is the maximum category probability of the unsupervised sample. When the maximum category probability is greater than the confidence threshold τ, Otherwise, it is 0, which means that high-quality pseudo labels are selected and added to the training.
[0104] (6.2) Construct a unified overall model loss function. In this sub-step, the comprehensive intra-class and inter-class losses are combined with the model reconstruction loss and prediction consistency loss to construct a unified overall model optimization loss function. Based on the above losses (22), (23) and the reconstruction losses (7), (8), (9) and consistency loss (10) of the steps, the overall loss function is integrated.
[0105] Loss = λ x ·(L xr +L x )+λ u ·(L ur +L uc +L u )+λ m ·L m (twenty four)
[0106] Among them, λ x ∈R,λ u ∈R,λ m ∈R is a trade-off parameter.
[0107] Step 7: Medical diagnosis grading and evaluation stage.
[0108] This step aims to achieve hierarchical diagnosis of glioma data. For a new data sample, i.e., glioma data x, the category of the new sample is determined by calculating the distance from x to each non-parallel proximal hyperplane. Each non-proximal hyperplane represents a category (low-grade glioma LGG, glioblastoma multiforme GBM). The category of the closest hyperplane is the category of the new sample. The decision function is as follows:
[0109]
[0110] Here, |·| represents an absolute value, k=1,…, K represents the kth category.
[0111] The present invention aims at the problem of class imbalance in brain glioma grading estimation under a small amount of data, which can be used to identify tumors and predict their malignancy and recurrence risk. The SDNSVM method is proposed. When the data set has characteristics such as a small amount of data and class imbalance learning, the SDNSVM method performs better.
[0112] The advantages of the present invention are:
[0113] 1) This paper introduces a self-supervised numerical feature augmentation mechanism. A feature generator is constructed by using mask vectors and data features to generate new features to augment the original table data, which is used to enhance the learning of the relationship between features and improve the generalization ability of the model.
[0114] 2) This paper proposes a semi-supervised learning strategy for pseudo-labels. By using the model's prediction results for unlabeled data as pseudo-labels, the training set is expanded and the model performance is improved. In order to ensure the high quality of pseudo-labels, this paper introduces a confidence screening mechanism to enhance the reliability of unsupervised sample classification.
[0115] 3) In order to solve the problem of class imbalance, the present invention randomly samples from the minority class samples and adds new samples to balance the proportion of each class sample. This allows the model to avoid preference for the majority class during the learning process, improves the model's recognition ability for the minority class, and ensures accuracy and fairness of classification.
[0116] 4) The present invention constructs a non-parallel intra-class and inter-class loss mechanism. In order to handle complex multi-class learning scenarios, by combining the local information within and between classes and adopting a parameter sharing mechanism to find the optimal hyperplane, the learning ability of multi-class information is enhanced and the learning efficiency of the model is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0117] Figure 1 The figure is a general flow chart of the method of the present invention.
[0118] Figure 2 This is a structural diagram of the network model of the present invention.
[0119] Figure 3 Flowchart for calculating reconstruction and consistency loss for the present invention.
[0120] Figure 4 This is a flow chart of calculating intra-class and inter-class losses in the present invention. DETAILED DESCRIPTION
[0121] The preferred embodiments of the present invention are given below in conjunction with the accompanying drawings to illustrate the technical solution of the present invention in detail. The embodiments described here are only used to explain the present invention and are not intended to limit the present invention.
[0122] Example 1
[0123] This embodiment relates to an intelligent medical diagnosis method based on a semi-supervised deep non-parallel support vector machine, and the specific steps are as follows:
[0124] Step 1: Construct and preprocess the medical dataset.
[0125] This step realizes the construction and preprocessing of medical data sets, including data screening and preprocessing, data normalization and data labeling, and imbalanced data balancing. The specific description is as follows:
[0126] (1.1) Data screening and preprocessing. First, the clinical medical data were screened and analyzed to extract the 20 most frequently mutated gene loci and two clinical characteristics: gender and age of diagnosis. To ensure the integrity and quality of the data, the raw data were preprocessed, including filtering instances of missing key information, converting the age of diagnosis from a string format to a continuous value, and excluding irrelevant columns (such as medical record number, initial diagnosis, and project).
[0127] (1.2) Data normalization and data labeling. In order to meet the input requirements of the model, the data is normalized. Normalization ensures that the numerical ranges of different features are consistent, thereby improving the effect of model training. Specifically, after normalization, the data is labeled according to the doctor’s diagnosis results to obtain supervised samples. where x i ∈R d is the sample feature of dimension d, y i is the label corresponding to the sample, n l is the supervised sample size; for unlabeled data, it is the unsupervised sample Where n u is the unsupervised sample size.
[0128] (1.3) Balance the imbalanced data. Check the preprocessed glioma dataset to ensure the accuracy and consistency of the data. To address the class imbalance problem in the dataset, randomly sample the minority class samples and increase the number of minority class samples to balance the data. This prevents the model from having a preference for the majority class during the learning process, thereby improving the recognition ability of the minority class.
[0129] First calculate the number of samples in each category and record it as n k , and then determine the target number of samples. Assume that the category with the most samples is n max , then the goal is to increase the number of samples of all categories to n max Then calculate the number of samples that need to be added. For each minority class k, the number of samples that need to be added is:
[0130]
[0131] Finally, randomly select samples from the minority class k samples and add them to the dataset. Through this process, the number of samples in all classes will tend to be the same, thus balancing the dataset.
[0132] Step 2: Augment supervised and unsupervised samples.
[0133] This step realizes the augmentation function of the dataset samples. By constructing a mask generator and a feature generator and integrating the augmented output, the training effect and generalization ability of the model are improved. Inspired by self-supervised learning, by i ,u i A series of enhancement transformations are performed to generate new training samples, learn the potential relationships between data samples, and thus learn useful feature representations to achieve diversity in data samples.
[0134] The specific description is as follows:
[0135] (2.1) Construct a mask generator. The generator outputs a binary mask of dimension D, denoted as m = [m1, ..., m D ] T ∈{0, 1} D , where m D With probability p m It is randomly sampled from the Bernoulli distribution, that is:
[0136]
[0137] (2.2) Construct a feature generator (Pretext generator). The generator takes the input mask m and calculates the corresponding subtraction operation (1-m), and multiplies (1-m) and the original sample feature element by element to obtain (1-m)⊙s. The original feature sample s (which can be a supervised sample or an unsupervised sample) is randomly shuffled column by column to obtain Multiply element-wise with the mask m, denoted as The final operation can be expressed as:
[0138]
[0139] Among them, g m is the feature generator, is the enhancing features, ⊙ is the element-by-element multiplication, s is the sample, is a randomly shuffled feature matrix. Formula (3) is interpreted as retaining the original samples of the part with a value of 1 in mask m plus the part with a value of 0 in mask m.
[0140] (2.3) Integrate the augmented output. According to formula (3), take the supervised sample x, unsupervised samples u1, u2 and mask m for sample s respectively, and get the corresponding supervised sample augmented features And unsupervised sample augmentation features
[0141] Step 3: Calculate the reconstruction loss and consistency loss.
[0142] This step implements the function of calculating reconstruction and consistency loss, including building a feature representation network model, calculating the logits corresponding to the feature, calculating the reconstructed feature representation, calculating the reconstruction loss of the sample, and calculating the mask reconstruction loss. The reconstruction loss measures the difference between the model's reconstructed features and the original features, while the consistency loss ensures that the output of the neural network remains consistent under different conditions. The network model structure diagram of the present invention is shown in the figure below: Figure 2 The specific description is as follows:
[0143] (3.1) Constructing a feature representation network model F. In order to fully utilize the powerful feature extraction capability of the neural network, the present invention uses a network model to extract the feature representation of the sample in the nonlinear space, replacing the nonlinear transformation implemented by the traditional kernel function. Specifically, the network F consists of an encoder f enc and decoder f dec The structure of the encoder f enc is responsible for mapping the input data into a compact feature space, while the decoder f dec It is used to reconstruct the input data, and the specific form is as follows:
[0144]
[0145] in, is a function composite symbol. F is composed of encoder f enc and decoder f dec The composite structure is used as the overall network. The encoder-decoder structure can not only effectively capture the complex nonlinear characteristics of the sample, but also improve the accuracy and robustness of feature expression.
[0146] (3.2) Calculate the corresponding logits of the features. Splice input encoder f enc , and get the corresponding logits, recorded as
[0147]
[0148] Among them, W e , where W e and b e are the network weights and biases, is the vector concatenation symbol. By concatenating the augmented features, it is input into the encoder and the corresponding logits are output.
[0149] (3.3) Calculate the reconstructed feature representation. Concatenate the input encoder-decoder model to obtain supervised sample reconstruction features Unsupervised reconstruction features and mask reconstruction features
[0150]
[0151] Among them, W d and b d are the network weights and biases. By concatenating the augmented features and inputting them into the encoder-decoder model, the corresponding reconstructed features are output.
[0152] (3.4) Calculate the reconstruction loss of the sample. The reconstruction loss can evaluate the difference between the autoencoder reconstructed sample and the original sample. The calculation process is as follows: Figure 3 shown.
[0153] Calculate the supervised sample mean squared error loss:
[0154]
[0155] Among them, n l is the supervised sample size, the original sample features x and the reconstructed sample features When minimizing this loss, we can reduce x and The difference between.
[0156] Calculate the reconstruction loss for unsupervised samples:
[0157]
[0158] Among them, n u is the unsupervised sample size, the original unsupervised sample feature u1 and the reconstructed unsupervised sample feature In order to minimize this loss, reduce u1 and The difference between them is calculated to ensure the consistency between the reconstructed features and the original features.
[0159] (3.5) Calculate the mask reconstruction loss. For the original mask m and the reconstructed mask Calculate the cross entropy loss:
[0160]
[0161] Where D is the size of the mask m. The original mask m and the reconstructed mask are evaluated by calculating the cross entropy loss. The difference in probability distributions.
[0162] (3.6) Calculate the consistency loss of unsupervised samples. Calculate the unsupervised sample logits, and Consistency loss:
[0163]
[0164] Among them, n u is the unsupervised sample size. The prediction results after enhancement under the same sample are: and The model can still remain consistent, which is used to evaluate the model's ability to remain robust to data changes at the prediction level.
[0165] Step 4: Use confidence to filter high-quality pseudo labels.
[0166] This step achieves effective use of unsupervised samples by generating and screening high-quality pseudo labels. This includes generating pseudo label vectors for unsupervised samples and screening out high-quality pseudo labels to enhance the reliability of unsupervised sample predictions. By screening pseudo labels using confidence thresholds, it can be ensured that the pseudo labels used by the model during training have high accuracy and credibility. The specific description is as follows:
[0167] (4.1) Create a pseudo label vector for the unsupervised sample. According to formula (5), For unsupervised samples, only encoder f enc The obtained logits. have:
[0168]
[0169] in, Output for each category, k is the category. A vector representing the output for each class.
[0170] The formula (11) is obtained Taking the absolute value, we get The activation function softmin is used to map a real number between 0 and 1, which is recorded as And normalization ensures that the sum is 1, that is:
[0171]
[0172] Where σ is the symbolic representation of the activation function softmin. The sum of all probabilities is 1.
[0173] (4.2) Filter high-quality pseudo labels. Introduce a scalar hyperparameter confidence threshold τ∈[0, 1]. If the prediction result is above this threshold, the pseudo label of this category will be retained. On this basis, increasing the confidence threshold can filter pseudo label samples with higher credibility. First, take its qk The maximum probability is compared with the confidence threshold τ, and those greater than the threshold are retained.
[0174]
[0175] in, is the indicator function, max(q k ) is the maximum category probability of the unsupervised sample. When the maximum category probability is greater than the confidence threshold τ, Otherwise it is 0.
[0176] According to formula (13), if it is greater than the confidence threshold τ, the pseudo label is taken out and recorded as
[0177]
[0178] Among them, the pseudo label Q k The category with the highest probability.
[0179] Step 5: Construct the classification task loss function.
[0180] This step implements the loss function design of the classification task, including constructing a non-parallel intra-class loss function and a non-parallel inter-class loss function. By combining the local information within and between classes and adopting a parameter sharing mechanism, the present invention finds an optimal hyperplane f for each class. k (x), thereby improving the classification accuracy and the robustness of the model, in the following form:
[0181]
[0182] Among them, f k (x) is the proximal hyperplane of class k. k and b k are the weights and biases corresponding to the hyperplane, f enc is the encoder, and x is the feature of the sample. The present invention can be used for multi-classification, in which a similar "one-to-rest" approach is used to handle multi-classification learning problems, that is, for K-class classification, K binary classifiers are constructed, the k-th class in the k-th classifier is the positive class, and the remaining classes are trained as negative classes. The process of calculating intra-class and inter-class losses in the present invention is as follows: Figure 4 shown.
[0183] (5.1) Construct a non-parallel intra-class loss function. Construct an intra-class loss function for the kth class of all sample points s. The output of the kth class is required to be the minimum. Take a sample point i as an example, according to the true label y and the pseudo label First filter out the output of the kth category, recorded as:
[0184]
[0185] Then calculate the minimum distance of the kth class, and the loss function is constructed as follows:
[0186]
[0187] Take the 2-norm for the k-th class and find the distance from the k-th class to the origin.
[0188] In addition, the minimum output of classes other than the kth class is required The distance from the hyperplane to the kth class is at least α. Taking a sample point i as an example, the loss function is constructed as follows:
[0189]
[0190] Where α∈R is the margin. Formula (18) obtains the minimum output of the class except the k-th class, and Formula (19) requires that the class except the k-th class and the hyperplane to which the k-th class belongs have a margin distance.
[0191] (5.2) Construct a non-parallel inter-class loss function. For all sample points s, the kth class and the classes other than the kth class Construct an inter-class loss function. Require the minimum output of all classes except the kth class The distance from the kth class is at least α. Taking a sample point i as an example, the loss function is constructed as follows:
[0192]
[0193] Where α∈R is the margin. Formula (20) requires that the classes other than the k-th class have a margin distance from the k-th class.
[0194] Step 6: Construct the overall loss function of the model.
[0195] This step integrates an overall loss function, including promoting the compactness of intra-class samples, the discrimination of inter-class samples, the model reconstruction accuracy and the prediction consistency. Combining these target loss functions into a unified loss function can guide the model to continuously optimize during the training process, thereby improving the overall generalization ability of the model. The specific description is as follows:
[0196] (6.1) Fusion of intra-class and inter-class loss functions. In this sub-step, the intra-class loss function and the inter-class loss function are fused to optimize both intra-class compactness and inter-class discrimination. Specifically, by combining equations (17), (19) and (20), the fusion is:
[0197]
[0198] Among them, w ik , is a regularization term, λ1∈R, λ2∈R, λ3∈R are trade-off parameters. Analyzing formula (21), the first term of the minimized objective function is the regularization term, in order to minimize the structural risk, that is, to balance the model complexity and model accuracy to avoid overfitting; the second term brings the k-th class sample point closer to the hyperplane of that class; the third term requires that the distance from other class sample points to the k-th class sample point is at least one margin, that is, the slack variable is the measurement error when the constraint condition is not met, aiming to maximize the separation of the two classes of sample points; the fourth term requires that the distance from other class sample points to the hyperplane to which the k-th class sample points belong is at least one margin, that is, other classes are as far away from the hyperplane to which the k-th class sample points belong as possible.
[0199] The supervised loss function is:
[0200]
[0201] Indicates the loss for all supervised samples.
[0202] The unsupervised loss is:
[0203]
[0204] Analyze formula (23), is the indicator function, max(q ik ) is the maximum category probability of the unsupervised sample. When the maximum category probability is greater than the confidence threshold τ, Otherwise, it is 0, which means that high-quality pseudo labels are selected and added to the training.
[0205] (6.2) Construct a unified overall model loss function. In this sub-step, the comprehensive intra-class and inter-class losses are combined with the model reconstruction loss and prediction consistency loss to construct a unified overall model optimization loss function. Based on the above losses (22), (23) and the reconstruction losses (7), (8), (9) and consistency loss (10) of the steps, the overall loss function is integrated.
[0206] Loss = λ x ·(L xr +L x )+λ u ·(L ur +L uc +L u )+λ m ·L m (twenty four)
[0207] Among them, λ x ∈R,λ u ∈R,λ m ∈R is a trade-off parameter.
[0208] Step 7: Medical diagnosis grading and evaluation stage.
[0209] This step aims to achieve hierarchical diagnosis of glioma data. For a new data sample, i.e., glioma data x, the category of the new sample is determined by calculating the distance from x to each non-parallel proximal hyperplane. Each non-proximal hyperplane represents a category (low-grade glioma LGG, glioblastoma multiforme GBM). The category of the closest hyperplane is the category of the new sample. The decision function is as follows:
[0210]
[0211] Here, |.| represents the absolute value, k=1,…, K represents the kth category.
[0212] In order to verify the robustness of the method of the present invention under very little labeled data, the experiment was conducted on the medical data set of glioma with only 5%, 10%, and 20% labeled data in the training set. In order to ensure the fairness of the experimental results, the present invention conducted experiments based on different random number seeds, taking five seeds of 0, 1, 2, 3, and 4 respectively. Deep Neural Networks (DNN), SVM, and Lap-SVM were selected as comparison methods in the experiment, among which DNN and SVM are supervised methods, and Lap-SVM and SDNSVM are semi-supervised methods. Accuracy (ACC), recall, precision, F1 Score, and G_mean are used as method evaluation indicators, which are specifically defined as follows:
[0213] The accuracy (ACC) is the proportion of correct categories predicted among all predictions.
[0214]
[0215] Among them, TP is a true positive example, TN is a true negative example, FP is a false positive example, and FN is a false negative example.
[0216] Recall reflects the model's ability to identify positive samples. It refers to the proportion of all actual k-th class samples that are correctly predicted as the k-th class.
[0217]
[0218] Precision reflects the proportion of samples predicted by the model to be positive that are actually positive. It refers to the proportion of all samples predicted to be in the kth class that are actually in the kth class.
[0219]
[0220] The F1 Score combines precision and recall and provides a harmonic mean of the two.
[0221]
[0222] G_mean reflects the overall performance of the model when dealing with imbalanced data. By comprehensively considering the recognition capabilities of positive and negative classes, G_mean provides a more comprehensive imbalanced classification evaluation indicator.
[0223]
[0224] The medical diagnosis output of the present invention is for two different brain glioma grades: low-grade glioma (LGG) and glioblastoma multiforme (GBM). In order to verify the robustness of the SDNSVM method in the case of a small amount of labeled data, three sets of scenarios with different numbers of labels were designed, specifically dividing 5%, 10%, and 20% of the labeled data in the training set data. Tables 2, 3, and 4 record the hierarchical diagnosis estimation performance comparison of the SDNSVM method of the present invention with DNN, SVM, and Lap-SVM methods on five evaluation indicators when there are only 5%, 10%, and 20% labeled data.
[0225]
[0226] Table 2 Comparison of the performance of glioma grading diagnosis estimation of each method at 5% label ratio
[0227] The results in Table 2 show that when the proportion of labeled data is 5%, the SDNSVM method proposed in this invention has superior performance, is almost unaffected by the small amount of labeled data, and performs well compared with other algorithms. In the case of DNN and SVM supervision, the ACC of SDNSVM is 3.96% higher than that of the supervised method; compared with Lap-SVM, the performance of SDNSVM is improved by 14.07%. In terms of Recall, Precision and F1 Score indicators, the SDNSVM method achieves the best. In addition, in terms of G_mean indicator, SDNSVM far exceeds Lap-SVM, indicating that SDNSVM performs better in dealing with class imbalance problems.
[0228]
[0229] Table 3 Comparison of the performance of glioma grading diagnosis estimation of each method under 10% label ratio
[0230] The results in Table 3 show that when the labeled data accounts for 10%, Lap-SVM shows certain instability and lack of accuracy in classifying samples. In contrast, the performance of SDNSVM is significantly better than Lap-SVM, and it shows better classification results in comparison with supervised methods. SDNSVM achieves the best results in F1 Score and G_mean indicators, indicating that it has good performance in dealing with class imbalance problems.
[0231]
[0232] Table 4 Comparison of the performance of glioma grading diagnosis estimation of each method under 20% label ratio
[0233] The results in Table 4 show that when the proportion of labeled data is 20%, SDNSVM still performs well, and all indicators are significantly better than Lap-SVM; compared with supervised methods such as DNN and SVM, SDNSVM still shows certain advantages under the same proportion of labeled data, verifying the effectiveness of semi-supervised methods.
[0234] From the results of Table 2, Table 3 and Table 4, it can be seen that as the amount of labeled data decreases and the categories are unbalanced, SDNSVM can still maintain a high classification performance. In general, the SDNSVM method proposed in the present invention shows good robustness in the classification diagnosis scenario of brain glioma with unbalanced categories.
[0235] Example 2
[0236] The present embodiment relates to an intelligent medical diagnosis device based on a semi-supervised deep non-parallel support vector machine, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the intelligent medical diagnosis method based on the semi-supervised deep non-parallel support vector machine of Example 1.
[0237] The above are preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereto. Any transformation or replacement that a person skilled in the art can think of within the technical scope disclosed by the present invention without creative work is included in the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope defined by the claims.
Claims
1. An intelligent medical diagnosis method based on a semi-supervised deep non-parallel support vector machine comprises the following steps: Step 1: Construct and preprocess medical datasets; Including data screening and preprocessing, data normalization and data labeling, and imbalanced data balancing; Step 2: Augment supervised and unsupervised samples. By building a mask generator and a feature generator, the augmented output is integrated to improve the training effect and generalization ability of the model. Inspired by self-supervised learning, the original training data x i ,u i Perform a series of enhancement transformations to generate new training samples, learn the potential relationship between data samples, and thus learn useful feature representations to achieve diversity in data samples; Step 3: Calculate the reconstruction loss and consistency loss, including building a feature representation network model, calculating the corresponding logits of the features, calculating the reconstructed feature representation, calculating the reconstruction loss of the sample, and calculating the mask reconstruction loss; the reconstruction loss measures the difference between the model's reconstructed features and the original features, while the consistency loss ensures that the output of the neural network remains consistent under different conditions; Step 4: Use confidence to select high-quality pseudo labels to achieve effective use of unsupervised samples; this includes generating pseudo label vectors for unsupervised samples and selecting high-quality pseudo labels to enhance the reliability of unsupervised sample predictions; By using confidence thresholds to filter pseudo labels, we can ensure that the pseudo labels used by the model during training have high accuracy and credibility. Step 5: Construct the classification task loss function, including constructing a non-parallel intra-class loss function and a non-parallel inter-class loss function; by combining the local information within and between classes and adopting a parameter sharing mechanism, find an optimal hyperplane f for each category k (x), thereby improving classification accuracy and model robustness; Step 6: Construct the overall loss function of the model, including promoting the compactness of intra-class samples, the discrimination of inter-class samples, the model reconstruction accuracy and the prediction consistency; integrating these target loss functions into a unified loss function can guide the model to continuously optimize during the training process, thereby improving the overall generalization ability of the model; Step 7: Medical diagnosis grading evaluation to achieve graded diagnosis of glioma data; for a new data sample, i.e. glioma data x, the category of the new sample is determined by calculating the distance from x to each non-parallel proximal hyperplane; each non-proximal hyperplane represents a category (low-grade glioma LGG, glioblastoma multiforme GBM); the category of the closest hyperplane is the category of the new sample.
2. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: Step 1 specifically includes: (1.1) Data screening and preprocessing: First, the clinical medical data were screened and analyzed to extract the 20 most frequently mutated gene loci and two clinical characteristics: gender and diagnosis age. To ensure the integrity and quality of the data, the raw data were preprocessed, including filtering instances of missing key information, converting the diagnosis age from a string format to a continuous value, and excluding irrelevant columns (such as medical record number, initial diagnosis, and project). (1.2) Data normalization and data labeling: In order to meet the input requirements of the model, the data is normalized. Normalization ensures that the numerical ranges of different features are consistent, thereby improving the effect of model training. Specifically, after normalization, the data is labeled according to the doctor's diagnosis results to obtain supervised samples. where x i ∈R d is the sample feature of dimension d, y i is the label corresponding to the sample, n l is the supervised sample size; for unlabeled data, it is the unsupervised sample Where n u is the unsupervised sample size; (1.3) Balance the imbalanced data: Check the preprocessed glioma dataset to ensure the accuracy and consistency of the data; To address the problem of class imbalance in the dataset, randomly sample the minority class samples to increase the number of minority class samples so that the data reaches a balanced state; This prevents the model from having a preference for the majority class during the learning process, thereby improving the recognition ability of the minority class; First calculate the number of samples in each category and record it as n k , and then determine the target number of samples; assuming that the category with the most samples is n max , then the goal is to increase the number of samples of all categories to n max ; Then calculate the number of samples that need to be increased. For each minority class k, the number of samples that need to be increased is: Finally, randomly select samples from the minority class k samples and add these samples to the dataset; through this process, the number of samples for all classes will tend to be the same, thus balancing the dataset.
3. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: Step 2 specifically includes: (2.1) Construct a mask generator; the generator outputs a binary mask of dimension D, denoted as m = [m1, ..., m D ] T ∈{0, 1} D , where m D With probability p m It is randomly sampled from the Bernoulli distribution, that is: (2.2) Construct a feature generator (Pretext generator); the generator takes the input mask m and calculates the corresponding subtraction operation (1-m), and multiplies (1-m) and the original sample feature element by element to obtain (1-m)⊙s; the original feature sample s (which can be a supervised sample or an unsupervised sample) is randomly shuffled column by column to obtain Multiply element-wise with the mask m, denoted as The final operation can be expressed as: Among them, g m is the feature generator, is the enhancing features, ⊙ is the element-by-element multiplication, s is the sample, is a randomly shuffled feature matrix; Formula (3) is interpreted as retaining the original samples of the part with a value of 1 in mask m plus the part with a value of 0 in mask m; (2.3) Integrate the augmented output; According to formula (3), take the supervised sample x, unsupervised samples u1, u2 and mask m for sample s respectively, and obtain the corresponding supervised sample augmented features And unsupervised sample augmentation features 4. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: Step three specifically includes: (3.1) Construct a feature representation network model F. In order to fully utilize the powerful feature extraction capability of the neural network, the network model F is used to extract the feature representation of the sample in the nonlinear space, replacing the nonlinear transformation implemented by the traditional kernel function. Specifically, the network F consists of an encoder f enc and decoder f dec The structure of encoder f enc is responsible for mapping the input data into a compact feature space, while the decoder f dec It is used to reconstruct the input data, and the specific form is as follows: in, is a function composite symbol; F is composed of encoder f enc and decoder f dec The composite structure is used as an overall network; the encoder-decoder structure can not only effectively capture the complex nonlinear characteristics of the sample, but also improve the accuracy and robustness of feature expression; (3.2) Calculate the corresponding logits of the features; Splice input encoder f enc , and get the corresponding logits, recorded as Among them, W e , where W e and b e are the network weights and biases, is the vector concatenation symbol; by concatenating the augmented features, it is input into the encoder and the corresponding logits are output; (3.3) Calculate the reconstructed feature representation; augment the feature Concatenate the input encoder-decoder model to obtain supervised sample reconstruction features Unsupervised reconstruction features and mask reconstruction features Among them, W d and b d are the network weights and biases; by concatenating the augmented features and inputting them into the encoder-decoder model, the corresponding reconstructed features are output; (3.4) Calculate the reconstruction loss of the sample; the reconstruction loss can evaluate the difference between the autoencoder reconstructed sample and the original sample. The calculation process is shown in Figure 3. Calculate the supervised sample mean squared error loss: Among them, n l is the supervised sample size, the original sample features x and the reconstructed sample features When minimizing this loss, we can reduce x and The difference between Calculate the reconstruction loss for unsupervised samples: Among them, n u is the unsupervised sample size, the original unsupervised sample feature u1 and the reconstructed unsupervised sample feature In order to minimize this loss, reduce u1 and The difference between them is calculated to ensure the consistency between the reconstructed features and the original features; (3.5) Calculate the mask reconstruction loss; for the original mask m and the reconstructed mask Calculate the cross entropy loss: Where D is the size of the mask m; the original mask m and the reconstructed mask are evaluated by calculating the cross entropy loss The difference in probability distribution between (3.6) Calculate the consistency loss of unsupervised samples; Calculate the unsupervised sample logits, and Consistency loss: Among them, n u is the unsupervised sample size; the prediction results after enhancement under the same sample, and The model can still remain consistent, which is used to evaluate the model's ability to remain robust to data changes at the prediction level.
5. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: Step 4 specifically includes: (4.1) Create a pseudo label vector for the unsupervised sample; according to formula (5), For unsupervised samples, only encoder f enc The obtained logits; for have: in, Output for each category, k is the category; A vector representing the output for each category; The formula (11) is obtained Taking the absolute value, we get The activation function softmin is used to map a real number between 0 and 1, which is recorded as And normalization ensures that the sum is 1, that is: Among them, σ is the symbolic representation of the activation function softmin; the sum of all probabilities is 1; (4.2) Filter high-quality pseudo labels; introduce a scalar hyperparameter confidence threshold τ∈[0, 1], and retain the category pseudo label if the prediction result is above this threshold; on this basis, increasing the confidence threshold can filter pseudo label samples with higher credibility; first take its q k The maximum probability is compared with the confidence threshold τ, and those greater than the threshold are retained; in, is the indicator function, max(q k ) is the maximum category probability of the unsupervised sample. When the maximum category probability is greater than the confidence threshold τ, Otherwise it is 0; According to formula (13), if it is greater than the confidence threshold τ, the pseudo label is taken out and recorded as Among them, the pseudo label for q k The category with the highest probability.
6. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: Step 5 specifically includes: The optimal hyperplane f k (x) is in the following form: Among them, f k (x) is the proximal hyperplane of class k. k and b k are the weights and biases corresponding to the hyperplane, f enc is the encoder, x is the feature of the sample; and it can be used for multi-classification, in which a similar "one-to-rest" approach is used to handle multi-classification learning problems, that is, for K-class classification, K binary classifiers are constructed, the k-th classifier is the positive class, and the remaining classes are trained as negative classes; the process of calculating intra-class and inter-class losses is as follows: (5.1) Construct a non-parallel intra-class loss function. Construct an intra-class loss function for the kth class of all sample points s. The output of the kth class is required to be the minimum. Take a sample point i as an example, according to the true label y and the pseudo label First filter out the output of the kth category, recorded as: Then calculate the minimum distance of the kth class, and the loss function is constructed as follows: Take the 2-norm for the k-th class and find the distance from the k-th class to the origin; In addition, the minimum output of classes other than the kth class is required The distance from the hyperplane to the kth class is at least α. Taking a sample point i as an example, the loss function is constructed as follows: Where α∈R is the margin. Formula (18) obtains the minimum output of the class except the k-th class, and Formula (19) requires that the class except the k-th class and the hyperplane to which the k-th class belongs have a margin distance; (5.2) Construct a non-parallel inter-class loss function. For all sample points s, the kth class and the classes other than the kth class Construct an inter-class loss function. Require the minimum output of all classes except the kth class The distance from the kth class is at least α. Taking a sample point i as an example, the loss function is constructed as follows: Among them, α∈R is the margin, and formula (20) requires that the classes other than the k-th class have a margin distance from the k-th class.
7. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: Step 6 specifically includes: (6.1) Fusion of intra-class and inter-class loss functions: In this sub-step, the intra-class loss function and the inter-class loss function are fused to simultaneously optimize the intra-class compactness and inter-class discrimination; specifically, by combining equations (17), (19) and (20), the fusion is performed: Among them, w ik , w ik is a regularization term, λ1∈R, λ2∈R, λ3∈R are trade-off parameters. Analyzing formula (21), the first term of the minimized objective function is the regularization term, which is used to minimize the structural risk, that is, to balance the model complexity and model accuracy to avoid overfitting; the second term brings the k-th sample point closer to the hyperplane of that class; the third term requires that the distance from other class sample points to the k-th class sample point is at least one margin, that is, the slack variable is the measurement error when the constraint condition is not met, which aims to maximize the separation of the two classes of sample points; the fourth term requires that the distance from other class sample points to the hyperplane to which the k-th class sample points belong is at least one margin, that is, other classes are as far away from the hyperplane to which the k-th class sample points belong as possible; The supervised loss function is: Indicates loss for all supervised samples; The unsupervised loss is: Analyze formula (23), is the indicator function, max(q ik ) is the maximum category probability of the unsupervised sample. When the maximum category probability is greater than the confidence threshold τ, Otherwise, it is 0, which means that high-quality pseudo labels are selected and added to the training; (6.2) Construct a unified overall model loss function; In this sub-step, the comprehensive intra-class and inter-class losses are combined with the model reconstruction loss and prediction consistency loss to construct a unified overall model optimization loss function; Based on the above losses (22), (23) and the reconstruction losses (7), (8), (9) and consistency loss (10) of the steps, the overall loss function is integrated; Loss=λ x ·(L xr +L x )+λ u ·(L ur +L uc +L u )+λ m ·L m (24) Among them, λ x ∈R,λ u ∈R,λ m ∈R is a trade-off parameter.
8. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: Step 7 specifically includes: realizing hierarchical diagnosis of brain glioma data; for a new data sample, i.e., brain glioma data x, the category of the new sample is determined by calculating the distance from x to each non-parallel proximal hyperplane; each non-proximal hyperplane represents a category (low-grade glioma LGG, glioblastoma multiforme GBM); the category of the closest hyperplane is the category of the new sample; the decision function is as follows: Here, |.| represents the absolute value, k=1,…, K represents the kth category.
9. The intelligent medical diagnosis method based on semi-supervised deep non-parallel support vector machine according to claim 1, characterized in that: The data described in step 1 are from the lower-grade glioma (LGG) and glioblastoma (GBM) brain glioma sub-projects of the TCGA (The Cancer Genome Atlas) project; The confidence threshold described in step 4 is set to 0.95; The trade-off parameter λ described in step 6 x ,λ u ,λ m The value of is 1; The types of brain gliomas described in step seven include: low-grade glioma (LGG) and glioblastoma multiforme (GBM).
10. An intelligent medical diagnostic device based on a semi-supervised deep non-parallel support vector machine, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the intelligent medical diagnosis method based on a semi-supervised deep non-parallel support vector machine as described in any one of claims 1 to 9.
Citation Information
Patent Citations
Semi-supervised fault classification method based on weighted feature alignment auto-encoder
CN113222045A
Medical image processing method based on robust multi-projection support vector machine
CN113449772A
Comparative multi-view depth fusion clustering method based on dual-data enhancement
CN117934889A
Medical image classification method and system based on semi-supervised multi-task learning, and medium
CN118038171A
Method and device for classifying nodes of graph
CN118339563A
Cited By
Intelligent early screening system for lung cancer
CN119446496A
Medical data anomaly detection method based on machine learning
CN120280173A
A machine learning-based medical data anomaly detection method
CN120280173B
