Multi-model fusion framework integrating ensemble learning and attention mechanism

Abstract

The invention provides a multi-model fusion framework MEDMFF combining ensemble learning and an attention mechanism, which is applied to pathological tissue slice analysis, is used for predicting the curative effect of breast cancer neoadjuvant chemotherapy (NAC), and belongs to the technical field of medical image processing. According to the method, firstly, a pathological section is preprocessed through an image color normalization technology, and dyeing differences are eliminated; then, a Cellprofiler and ResNet50 are used for extracting various features including texture features and local region features from the breast cancer pathological section, and key features are screened out through feature engineering; then, prediction results of a plurality of machine learning base models are integrated, a weighted average attention mechanism training meta-model is introduced, and an integrated learning system is constructed. By fusing the advantages of various machine learning algorithms, the accuracy and stability of curative effect prediction are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a multi-model fusion framework integrating ensemble learning and attention mechanism. More specifically, the present invention proposes a multi-model fusion framework integrating ensemble learning and attention mechanism, which combines the advantages of BYOL contrastive learning and ensemble learning, aiming to improve the accuracy and stability of efficacy prediction.

[0002] Background Area Technology

[0003] Breast cancer is one of the most common malignant tumors in women, with high morbidity and mortality. Neoadjuvant chemotherapy (NAC) is an important treatment strategy for patients with locally advanced breast cancer, aiming to reduce tumor size, lower clinical stage, and improve patient prognosis. However, the heterogeneity of breast cancer leads to significant differences in the efficacy of NAC in patients. Therefore, accurately predicting the efficacy of NAC and formulating individualized treatment plans for each patient is a key clinical challenge. Although pathological sections have great potential in predicting the efficacy of neoadjuvant chemotherapy for breast cancer, their application is relatively limited. Extracting features with predictive value from high-dimensional pathological section data is also a technical challenge.

[0004] As a powerful machine learning method, ensemble learning improves the generalization ability and prediction accuracy of the overall model by integrating the advantages of multiple prediction models. However, the results of ensemble learning are highly dependent on the learning degree of the integrated models, and there may be inconsistencies in the degree of fit between models.

[0005] In order to solve these problems, the present invention introduces contrastive learning and an attention mechanism when training the meta-model, weighting the models with better fitting degree so that the meta-model is more inclined to select the results of excellent models. Summary of the invention

[0006] The present invention aims to utilize the pathological section data of breast cancer tissue before neoadjuvant chemotherapy to propose an improved ensemble learning framework. By combining the advantages of contrastive learning and ensemble learning, a new hybrid architecture is proposed to improve the prediction accuracy of NAC efficacy of breast cancer and enhance the generalization ability.

[0007] The core innovation of this invention is to introduce BYOL contrastive learning on the ResNet50 framework to more effectively train the feature extractor, enhance the importance of the extracted features, and thus improve the prediction accuracy. In addition, adding a weighted average attention module to the ensemble learning and color normalizing the slices further improves the prediction accuracy.

[0008] Specifically, the method of the present invention comprises the following steps:

[0009] S1, using Cycle-GAN based image color normalization;

[0010] S2, using Resnet50 and ensemble learning as the basic framework;

[0011] S3, adding the Byol contrast learning module to the feature extraction module;

[0012] S4, select SVM, RandomForest and Logistics Regression as base models, and SVM as meta-model;

[0013] S5. Add a weighted average attention module to the meta-model;

[0014] S6, feature data preprocessing uses Z_score normalization, and then uses U test, Spearman and lasso methods to design feature screening to train the base model. Combined with the base model output, the meta model is trained to obtain the final model. The beneficial effects of the present invention are as follows:

[0015] (1) This paper introduces Cycle-GAN image color normalization to solve the impact of staining differences in different slices on accuracy.

[0016] (2) The present invention introduces the BYOL module, which can perform training without labels through self-supervised learning capabilities without labels. BYOL can enhance feature representation, has high training stability, and reduces dependence on negative samples.

[0017] (3) The present invention further improves the acquisition of important features inside the slice by using Cellprofiler to extract features, and processing together with resnet50 features is beneficial to improving the accuracy of model prediction.

[0018] (4) The present invention further improves the accuracy of the results by proposing a weighted average attention module.

[0019] (5) The present invention uses a series of data preprocessing methods to remove most of the redundancy in the data, reduce the data dimension, improve the accuracy of model training, shorten the training time, and reduce the probability of model overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the steps of the construction method of the present invention;

[0021] Figure 2 It is a schematic diagram of the overall framework of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0023] Example:

[0024] Cooperate Figure 1 and Figure 2 As shown, the present invention discloses a multi-model fusion framework that integrates ensemble learning and attention mechanism, and its overall architecture includes the following main modules:

[0025] Step S1: Collecting breast cancer pathological tissue sections from different patients.

[0026] Step S2: Normalize the image color of the slices through Cycle-GAN.

[0027] Step S3: Use Resnet50 with BYOL to learn its features.

[0028] Step S4: Use Cellprofiler and Resnet50 for feature extraction respectively.

[0029] Step S5: Use U-test, spearman, and Lasso to screen features and divide the data set.

[0030] Step S6: Train SVM, RandomForest, and Logistics regression to obtain intermediate results.

[0031] Step S7: Obtain a new dataset training meta-model by weighted average attention to obtain a trained model.

[0032] Step S8: Put the data to be predicted into the final prediction result.

[0033] Furthermore, the Resnet50 basic framework in step S3 is used for comparative learning.

[0034] Further, the step S5 is implemented in the following manner: first, the original features are subjected to Z-score normalization, U test, Spearman correlation analysis and Lasso regression screening in sequence. Among them, Lasso regression uses an adaptive algorithm to automatically select the optimal parameters for feature screening. In the feature division process, a stratified sampling method is used to ensure the applicability and stability of the training model.

[0035] Furthermore, the steps of step S6 are as follows: (1) initializing three base models; (2) selecting the best parameters and obtaining the result model according to the ROC and AUC values ​​of the model training results through grid screening.

[0036] Furthermore, the weighted average attention model mentioned in step S7 has the following formula:

[0037]

[0038] where w i is the weight of the i-th model, f i (x) is the output of the model, N is the number of base models, and M is the number of features.

[0039] About w i , the formula is as follows:

[0040]

[0041] Where D i (x) is the square of the deviation of the prediction of the ith model. N is the number of base models.

Claims

1. A multi-model fusion framework that integrates ensemble learning and attention mechanism, It is characterized in that The following steps are involved: S1, using Cycle-GAN based image color normalization; S2, using Resnet50 and ensemble learning as the basic framework; S3, adding the Byol contrast learning module to the feature extraction module; S4, select SVM, RandomForest and Logistics Regression as base models, and SVM as meta-model; S5. Add a weighted average attention module to the meta-model; S6. Feature data preprocessing uses Z_score normalization, and then U test, Spearman and lasso methods are used to design feature screening to train the base model. Combined with the base model output, the meta-model is trained to obtain the final model.

2. The algorithm according to claim 1, It is characterized in that In step S1, the image color is normalized by using Cycle-GAN.

3. The algorithm according to claim 1, It is characterized in that The Byol contrastive learning module in step S3 achieves effective feature learning through self-supervised learning.

4. The algorithm according to claim 1, It is characterized in that The weighted average attention module in step S5 improves the model accuracy by evaluating the output of the base model, using the evaluation results as weight parameters, and then weighting the outputs. The formula is defined as: where w i is the weight of the i-th model, f i (x) is the output of the model, N is the number of base models, and M is the number of features. About w i , the formula is as follows: Where D i (x) is the square of the deviation of the prediction of the ith model. N is the number of base models.

5. The algorithm according to claim 1, It is characterized in that The four methods mentioned in step S6 will process the features in turn to obtain features after dimensionality reduction.

Images