Breast cancer multi-classification method, device, medium and product based on dual-mode image

Abstract

This application discloses a method, device, medium, and product for multi-classification of breast cancer based on bimodal images, relating to the field of machine learning technology. The method includes: acquiring data to be classified; obtaining image depth feature groups based on the data using a preset depth feature extraction model; wherein the data to be classified includes PET and CT images corresponding to breast cancer; acquiring clinical feature groups from the data to be classified; performing feature filtering on the features in the image depth feature groups and clinical feature groups to obtain target feature groups; and obtaining classification results by applying the target feature groups to a preset breast cancer classification model, wherein the classification results include one of multiple breast cancer subtypes. This application can accurately identify multiple breast cancer subtypes.

Description

Technical Field

[0001] This application relates to the field of machine learning, and in particular to a method, device, medium, and product for multi-classification of breast cancer based on bimodal images. Background Technology

[0002] Breast cancer is a highly heterogeneous disease, classified into multiple subtypes based on molecular characteristics. Currently, several classification systems exist in clinical practice, and molecular subtyping of breast cancer is of great significance for the precise diagnosis and treatment of patients.

[0003] Currently, the only way to obtain the results is through puncture biopsy or postoperative pathological specimens. This technique is invasive, has poor timeliness, is one-sided, difficult to obtain, and is easily affected by tumor heterogeneity.

[0004] In existing technologies, the biological information in PET / CT images can be deeply mined, and computer models can be built using deep learning methods to effectively solve this problem. Current related research has established binary classification models, and the similarity between different breast cancer subtypes is high. If more classification results are to be identified, the accuracy is difficult to guarantee. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, medium, and product for multi-classification of breast cancer based on bimodal images, which can accurately identify multiple breast cancer subtypes.

[0006] To achieve the above objectives, this application provides the following solution:

[0007] Firstly, this application provides a method for multi-classification of breast cancer based on bimodal images, including:

[0008] Obtain the data to be classified, and obtain image depth feature groups based on the data to be classified using a preset depth feature extraction model; wherein, the data to be classified includes PET images and CT images corresponding to breast cancer;

[0009] Obtain the clinical feature group of the data to be classified, and perform feature filtering on the features in the image depth feature group and the clinical feature group to obtain the target feature group;

[0010] The target feature group is classified using a preset breast cancer classification model, and the classification result includes one of multiple breast cancer subtypes.

[0011] Optionally, before acquiring the data to be classified, the breast cancer multi-classification method based on bimodal images further includes: acquiring the preset deep feature extraction model; specifically including:

[0012] Obtain PET images, PET mask images, CT images, and CT mask images corresponding to breast lesions;

[0013] The PET images, PET mask images, CT images, CT mask images, and corresponding classification labels are used as a bimodal dataset; the classification label is one of several breast cancer subtypes.

[0014] The preset deep feature extraction model is obtained by performing transfer training on the pre-trained preset feature extraction model based on the bimodal dataset.

[0015] Optionally, before obtaining the classification result of the target feature group through a preset breast cancer classification model, the breast cancer multi-classification method based on bimodal images further includes:

[0016] Obtaining the preset breast cancer classification model specifically includes:

[0017] A training dataset is obtained based on the features and classification labels in the target feature group. A preset classification model is then trained based on the training dataset to obtain a preset breast cancer classification model. The classification label is one of several breast cancer subtypes.

[0018] Optionally, the preset deep feature extraction model is a trained SEResNext50 network; the preset breast cancer classification model is a trained LightGBM classification model.

[0019] Optionally, the step of performing feature filtering on the features in the image depth feature group and the clinical feature group to obtain the target feature group includes:

[0020] Preprocess the features in the image depth feature group and the clinical feature group to obtain the first feature group;

[0021] The target feature group is obtained by sequentially performing variance filtering, correlation filtering, and feature importance selection on the first feature group based on a threshold.

[0022] Optionally, the first feature group is subjected to variance filtering, correlation filtering, and feature importance selection based on a threshold to obtain the target feature group;

[0023] Obtain the variance of the features in the first feature group, and select the features whose variance is greater than a first threshold to form a second feature group;

[0024] The feature correlation of the features in the second feature group is obtained according to the Pearson method or the Spearman method, and the features whose feature correlation is less than the second threshold are selected to form the third feature group.

[0025] The feature importance of the third feature group is obtained according to the feature importance module in the preset breast cancer classification model, and the features whose feature importance is greater than the third threshold are selected to form the target feature group.

[0026] Secondly, this application provides a breast cancer multi-classification device based on dual-modal images, comprising:

[0027] The feature extraction module is used to acquire data to be classified and to obtain image depth feature groups based on the data to be classified using a preset depth feature extraction model; the data to be classified includes PET images and CT images corresponding to breast cancer.

[0028] The filtering module is used to obtain the clinical feature group of the data to be classified, and to perform feature filtering on the features in the image depth feature group and the clinical feature group to obtain the target feature group.

[0029] The identification module is used to obtain classification results for the target feature group through a preset breast cancer classification model, wherein the classification results include one of multiple breast cancer subtypes.

[0030] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the breast cancer multi-classification method based on bimodal images as described above.

[0031] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the breast cancer multi-classification method based on bimodal images described above.

[0032] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the breast cancer multi-classification method based on bimodal images described above.

[0033] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0034] This application provides a method, device, medium, and product for multi-classification of breast cancer based on bimodal images. It obtains image depth feature groups through a preset depth feature extraction model, and performs feature filtering on the features in the image depth feature groups and the clinical feature groups to obtain target feature groups. The obtained features are highly correlated with the classification results. Then, the target feature groups are input into the classification results obtained by the preset breast cancer classification model, which can improve the accuracy of classification. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A flowchart illustrating a breast cancer multi-classification method based on bimodal images provided in this application embodiment;

[0037] Figure 2 This is a schematic diagram of the method for obtaining a preset depth feature extraction model provided in an embodiment of this application;

[0038] Figure 3 for Figure 1 A detailed flowchart of step 1020;

[0039] Figure 4 A schematic diagram of the ROC curve of the preset breast cancer classification model provided in the embodiments of this application;

[0040] Figure 5 A schematic diagram of the precision-recall curve of the preset breast cancer classification model provided in the embodiments of this application;

[0041] Figure 6 A schematic diagram of the confusion matrix of the preset breast cancer classification model provided in this application;

[0042] Figure 7 A schematic diagram illustrating the category prediction error of the preset breast cancer classification model provided in the embodiments of this application;

[0043] Figure 8 A schematic diagram of the functional modules of a breast cancer multi-classification device based on bimodal images provided in an embodiment of this application;

[0044] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Specifically, breast cancer is a highly heterogeneous disease, classified into multiple subtypes based on molecular characteristics. Currently, clinical classification based on hormone receptors and HER2 status is the most commonly used method, using three biomarkers: estrogen receptor (ER) and progesterone receptor (PR) status based on hormone receptor (HR) status, and HER2 (ERBB2) status. This results in four main categories: Luminal A, Luminal B, HER2-positive, and triple-negative. Luminal B can be further divided into Luminal B (her2-) and Luminal B (her2+). Accurate classification is crucial for individualized treatment decisions. For example, Luminal patients may benefit from endocrine therapy, HER2-positive patients require anti-HER2 targeted therapy, and triple-negative breast cancer patients may respond to immunotherapy. Currently, molecular classification is gradually moving from research to clinical practice, guiding precision diagnosis and treatment of breast cancer. The purpose of this application is to provide a multi-classification method for breast cancer based on bimodal imaging, capable of accurately obtaining breast cancer subtypes.

[0048] In one exemplary embodiment, such as Figure 1 As shown, a method for multi-classification of breast cancer based on bimodal images is provided, including the following steps 1010 to 1030. Wherein:

[0049] Step 1010: Obtain the data to be classified. Based on the data to be classified, obtain the image depth feature groups through a preset depth feature extraction model. The data to be classified includes PET images and CT images corresponding to breast cancer.

[0050] As one implementation method, the original 3D PET model and the original 3D CT model of the case are obtained. The original 3D PET model and the original 3D CT model are sliced ​​at the same location to obtain PET images and CT images respectively. The PET images and CT images are preprocessed (e.g., foreground cropping and size adjustment are performed sequentially). The preprocessed images are input into a preset depth feature extraction model to obtain the output image depth feature group.

[0051] Step 1020: Obtain the clinical feature group of the data to be classified, and perform feature filtering on the features in the image depth feature group and the clinical feature group to obtain the target feature group;

[0052] Specifically, target feature groups are formed by selecting features that are strongly correlated with the typing results through feature filtering.

[0053] Step 1030: Obtain the classification result of the target feature group through the preset breast cancer classification model. The classification result includes one of the breast cancer subtypes.

[0054] Specifically, the classification results are one of the following: Luminal A, Luminal B (her2-), Luminal B (her2+), Her2 positive, or triple negative.

[0055] Alternatively, for the original 3D PET model and the original 3D CT model of the same case, slices can be taken at different locations to obtain different subtyping data. For different subtyping data, steps 1010 to 1030 can be repeated to obtain the corresponding breast cancer subtyping results for diagnosis and analysis of a single case.

[0056] Steps 1010 to 1030 above obtain image deep feature groups through a preset deep feature extraction model. Feature filtering is performed on the features in the image deep feature groups and clinical feature groups to obtain target feature groups. The obtained features are highly correlated with the classification results. Then, the target feature groups are input into the classification results obtained by the preset breast cancer classification model, which can improve the accuracy of classification.

[0057] In another exemplary embodiment of this application, in order to accurately obtain image depth feature groups, before obtaining the data to be classified, the breast cancer multi-classification method based on bimodal images provided in this application further includes: obtaining a preset depth feature extraction model; such as Figure 2 As shown, this specifically includes steps 2010 to 2020:

[0058] Step 2010: Obtain the PET image, PET mask image, CT image, and CT mask image corresponding to the breast lesion;

[0059] As one embodiment, step 2010 includes:

[0060] Step 2011: Obtain the original 3D PET data and corresponding original 3D CT data of multiple cases. Perform masking on the original 3D PET data to obtain the 3D PET lesion ROI (as a mask map for subsequent PET images). Perform masking on the original 3D CT data to obtain the 3D CT lesion ROI (as a mask map for subsequent CT images).

[0061] Specifically, raw 3D CT data and raw 3D PET data are both medical imaging technologies that generate three-dimensional volumetric data. They are neither two-dimensional images in the traditional sense nor artificially constructed models, but they can be further processed by displaying them in slice form or three-dimensional visualization.

[0062] Specifically, one or more medical professionals manually delineate and examine the lesion mask, and any disagreements regarding the lesion delineation are resolved through negotiation.

[0063] As one example, the delineation work can be done in 3D slicer software (version 5.5.0, http: / / www.slicer.org). All PET and CT images are imported into 3D slicer software, and the lesion areas in the PET and CT images are delineated separately and saved as mask image files. All mask image files are saved in NRRD format.

[0064] Step 2012: Slice the original 3D CT data to obtain multiple 2D CT images; slice the 3D CT lesion mask to obtain multiple 2D CT mask images; slice the original 3D PET data to obtain multiple 2D PET images; slice the 3D PET lesion mask to obtain multiple 2D PET mask images.

[0065] As one example, slicing can be implemented using in-house software.

[0066] Step 2013: Retain the 2D PET image containing the lesion, the 2D PET mask, the 2D CT image, and the 2D CT mask;

[0067] Step 2020: The 2D slices and masks of the PET images corresponding to the breast lesions, the 2D slices and masks of the CT images corresponding to the breast lesions, and the corresponding classification labels are used as a bimodal dataset; the classification label is one of the various breast cancer subtypes.

[0068] Specifically, a bimodal dataset is constructed, where each dataset includes a PET image, a PET mask, a CT image, a CT mask, and corresponding classification labels for a uniform slice location. The PET mask indicates the ROI of the PET image, and the CT mask indicates the ROI of the CT image.

[0069] Optionally, considering GPU memory capabilities, a bimodal dataset is constructed in which PET images, PET masks, CT images, and CT masks are all foreground-cropped and then resized to (128, 128). For 2D PET and PET mask images, pixel values ​​are normalized to [0, 255].

[0070] Specifically, to address the issue of class imbalance in the training data, the bimodal dataset also includes images that have been data-augmented using methods such as random flipping, random 90° rotation, random contrast adjustment, and random Gaussian sharpening.

[0071] As one example, the classification label is one of Luminal A, Luminal B (her2-), Luminal B (her2+), Her2 positive, or triple negative.

[0072] Step 2030: Perform transfer training on the pre-trained preset feature extraction model based on the bimodal dataset to obtain the preset deep feature extraction model.

[0073] Optionally, the preset feature extraction model can be pre-trained to obtain a pre-trained preset feature extraction model.

[0074] Specifically, a pre-trained model refers to a model that has been trained on a large-scale dataset (such as ImageNet) beforehand. The model parameters are initialized using information from this large-scale dataset, and then fine-tuned or through transfer learning to adapt the model to a specific task. Alternatively, a pre-trained feature extraction model can be used directly. The 2D deep neural network pre-trained model used in this application's embodiments comes from https: / / github.com / Cadene / pretrained-models.pytorch.

[0075] Optionally, the pre-trained preset feature extraction model can be a pre-trained model based on 2D EfficientNet-B0 or ​​2D SEResNext50.

[0076] Preferably, a pre-trained 2D SEResNext50 network is used as the pre-trained preset feature extraction model;

[0077]

[0078] Specifically, in the 2D SEResNext50 network used in this embodiment, the Adam (adaptive moment estimation) optimizer algorithm is selected for model optimization. The cross-entropy loss function after label smoothing is chosen. Label smoothing is a regularization method to prevent overfitting, which can reduce the model's overfitting to noisy data and improve the model's robustness.

[0079] Furthermore, the 2D SEResNext50 network architecture used in the embodiments of this application is shown in Table 1, comprising a Conv1 layer, a MaxPool layer, a Stage1 layer, a Stage2 layer, a Stage3 layer, a Stage4 layer, an AvgPool layer, and an FC layer connected in sequence. Stages 1 through 4 are all SE-ResNeXt Bottleneck modules, and each SE-ResNeXt Bottleneck includes a 1×1 conv, a 3×3 group conv, a 1×1 conv, an SE module, and a residual block connected in sequence.

[0080] According to the steps of this application, the obtained image depth features are 2048 features output from the Stage4 layer.

[0081] As an optional implementation, to mitigate overfitting due to insufficient training data, the dropout rate is set to 0.5. In the Adam optimizer, the learning rate is set to 0.0001, and the weight decay is set to 0.0001. The weight decay is an L2 regularization used to avoid overfitting. The deep learning training epochs are set to 500, and the batch size is set to 30.

[0082] In another exemplary embodiment of this application, in order to obtain a target feature group that is strongly correlated with the classification result, such as Figure 3 Step 1020 above is replaced by steps 301 to 305:

[0083] Step 301: Obtain the clinical feature groups of the data to be classified;

[0084] As one example, the clinical characteristic group includes age, clinical stage, and SUV characteristics;

[0085] Specifically, the clinical staging includes four stages: stage 1, stage 2, stage 3, and stage 4.

[0086] Specifically, the SUV value is obtained by calculating the SUV value from the above PET images. V The feature matrix, i.e., SUV features.

[0087] Step 302: Preprocess the features in the image depth feature group and clinical feature group to obtain the first feature group;

[0088] Specifically, the features in the image depth feature group are the features output by the feature output layer of the pre-trained preset feature extraction model; the features in the clinical feature group are clinical features that are strongly correlated with the classification results.

[0089] Specifically, preprocessing includes outlier handling, Z-score standardization, and Yeo-Johnson power transformation, with the aim of making the random variable closer to a normal distribution.

[0090] Step 303: Obtain the variance of the features in the first feature group, and select features with variance greater than the first threshold to form the second feature group;

[0091] As an optional embodiment, the first threshold is 0.3.

[0092] Step 304: Obtain the feature correlation of the features in the second feature group according to the Pearson method or the Spearman method, and select features with feature correlation less than the second threshold to form the third feature group;

[0093] As an optional embodiment, the second threshold is 0.9.

[0094] Step 305: Obtain the feature importance of the third feature group according to the feature importance module in the preset breast cancer classification model, and select features with feature importance greater than the third threshold to form the target feature group.

[0095] As an optional embodiment, the third threshold is 50%.

[0096] The preset breast cancer classification model is a model obtained by training the LightGBM classification model, and the feature importance module is the feature importance module in the LightGBM classification model, which can be used to calculate the importance of features and match the preset breast cancer classification model in the next step.

[0097] In another exemplary embodiment of this application, in order to obtain a preset breast cancer classification model, as shown in the figure, the method of this embodiment further includes the following steps:

[0098] The training dataset is obtained based on the features and classification labels in the target feature group; the classification label is one of the various breast cancer subtypes; the preset classification model is trained based on the training dataset to obtain the preset breast cancer classification model.

[0099] Specifically, the classification labels are: Luminal A, Luminal B (her2-), Luminal B (her2+), Her2 positive, or triple negative.

[0100] Specifically, the preset classification model is the LightGBM classification model, and its training process is similar to that of the above models, so it will not be described again here.

[0101] Specifically, during the training of both the pre-set deep feature extraction model and the pre-set breast cancer classification model, a stratified 5-fold cross-validation method was employed. Four folds served as the training set for model training, while the remaining fold served as the validation set for evaluating model performance. Finally, the performance of the classification model was comprehensively evaluated based on the results of all five cross-training iterations.

[0102] In one specific embodiment, the training in this application is implemented based on the Medical Open Network for Artificial Intelligence (MONAI) framework (version 1.3.0, https: / / monai.io). The training environment is a Windows 11 operating system and an Nvidia RTX A2000 GPU with 8GB of RAM.

[0103] Example 1

[0104] In this embodiment, according to the dual-modal image-based breast cancer multi-classification method provided in this application, the masking and labeling of the original 3D PET and original 3D CT models are performed by at least one physician with more than 5 years of nuclear medicine experience, and a physician with more than 10 years of experience examines all ROI delineation images. Disagreements regarding lesion delineation are ultimately resolved through negotiation.

[0105] The classification labels all use molecular typing (label=0, 1, 2, 3, 4, corresponding to five types: Luminal A, Luminal B (her2-), Luminal B (her2+), Her2 positive, and triple negative, respectively).

[0106] To obtain the necessary dataset, 3D PET and 3D CT images from 506 eligible patients were included. First, the images were sliced, resulting in 6529 2D slices. The number of slices for each of the five categories (0-4) were 1088, 1312, 1385, 1371, and 1373, respectively. All 2D slices were stratified and randomly divided into a training set and a test set at a 9:1 ratio. The training set consisted of 5876 cases (979:1181:1246:1234:1236 for the five categories), while the test set consisted of 653 cases (109:131:139:137:137 for the five categories). The same slicing and dataset division were performed on the 506 3D PET and 3D CT ROI images.

[0107] Among them, 2050 clinical feature groups of image depth features were obtained, and 3 clinical features were obtained; according to the screening method provided in the embodiments of this application, 105 features were obtained to form a target feature group.

[0108] Comparative Example 1

[0109] Similar to the dataset in Example 1, the EfficientNet-B0 model is trained using the training method of the preset deep feature extraction model based on the above bimodal dataset to obtain the first classification model. The data to be classified is then input into the first classification model to obtain the classification result.

[0110] Comparative Example 2

[0111] Similar to the dataset in Example 1, the SEResNext50 model is trained using the above-mentioned bimodal dataset according to the training method of the preset deep feature extraction model to obtain the second classification model. The data to be classified is then input into the second classification model to obtain the classification result.

[0112] Comparative Example 3

[0113] Obtain the predicted probabilities of the five categories in the classification results of Comparative Example 1 and the predicted probabilities of the five categories in the classification results of Comparative Example 2; calculate the average of the two predicted probabilities to obtain the classification result.

[0114] To evaluate the performance of the pre-defined breast cancer classification model (LightGBM) obtained in the method of this application (Example 1) and the methods of Comparative Examples 1-3, macro metrics were used to evaluate the performance of different classification models. Specifically, the following five metrics were used: Macro Area Under the Receiver Operating Characteristic Curve (Macro AUC), Macro Accuracy (MacroACC), Macro Precision (Macro Precision), Macro Recall (Macro Recall), and Macro F1 Score (Macro F1-score). Macro Accuracy (Macro ACC) is defined as the ratio of the number of correctly classified samples to the total number of samples. The other four macro metrics are defined as the arithmetic mean of the metrics for each category.

[0115]

[0116] in, This represents the total number of categories.

[0117] Specifically, the performance comparison of the methods is shown in Table 2. As can be seen from Table 2, the LightGBM model provided by the method of this application has a much better performance than the other three comparative models.

[0118] Figure 4 The ROC of the preset breast cancer classification model provided in this application embodiment for each classification is shown. The model's prediction performance for each classification is not significantly different, that is, the model's prediction performance is stable and reliable. Figure 5 The precision-recall curve shows that the bias (false positives and false negatives) of the preset breast cancer classification model is controlled within a reasonable range. Figure 6 To establish the confusion matrix of the breast cancer classification model for each category of data samples, the number of predicted samples for each category by the model is described in detail. Figure 7 The image displays the number of samples correctly and incorrectly classified by the pre-defined breast cancer classification model. The image shows that the highest number of correctly predicted samples were within the same category. In summary, the pre-defined breast cancer classification model proposed in this application demonstrates reliable stability and accuracy.

[0119]

[0120] This application provides a breast cancer multi-classification method based on bimodal images, which has the following technical advantages:

[0121] A pre-defined feature extraction model was trained using a dual-modal dataset consisting of PET images + PET mask images and CT images + CT mask images. The trained model was then used to acquire image depth features. The dual-modal training provided complementary information from both PET and CT images. Simultaneously, the introduction of Regions of Interest (ROIs) enabled the model to focus on key areas highly relevant to the task, effectively suppressing background noise and improving the targeting of feature extraction. The image depth features obtained using this training method more accurately reflect the essential attributes of breast cancer subtyping images.

[0122] Based on the method described in this application, target features are obtained by sequentially filtering acquired image depth features and clinical features through variance, correlation, and feature importance. By calculating the correlation coefficient between each feature and the target variable, the contribution strength of each feature to the prediction task can be quantified, and highly correlated features can be selected accordingly. The feature importance evaluation module built into the LightGBM model can effectively identify and select the key features most influential on the model's predictions. Simultaneously, this method retains the most representative subset of features in the data, ensuring the interpretability of the model and facilitating the analysis of the intrinsic relationship between each key feature and the prediction target. This feature importance-based selection strategy enables the model to maintain high performance while possessing advantages in computational efficiency and interpretability, providing a reliable feature foundation for subsequent modeling and analysis.

[0123] LightGBM, based on a feature importance calculation mechanism (such as split count and information gain) of decision trees, can objectively quantify the contribution of each feature to the final prediction result. By selecting features with high importance ranking, the model's running efficiency and generalization ability can be significantly improved. On the one hand, feature selection reduces the interference of redundant or noisy features, allowing the model to focus more on the truly discriminative feature dimensions, thereby improving prediction accuracy. On the other hand, it reduces the dimensionality of the feature space, effectively alleviating the curse of dimensionality problem, making model training faster and more stable.

[0124] The built-in feature importance evaluation module is used for screening, and it is highly integrated with the LightGBM model itself to achieve closed-loop optimization of feature selection and model training, which further improves the accuracy of classification.

[0125] The method in this application is more flexible. The target features obtained vary depending on different data points, making it more adaptable and flexible for different classification tasks.

[0126] In summary, the method proposed in this application can accurately classify complex breast cancer subtypes, and it has certain advantages in terms of accuracy, processing speed, flexibility and robustness.

[0127] This application also provides an application scenario in which the above-described multi-classification method for breast cancer based on bimodal images is applied. Specifically, the multi-classification method for breast cancer based on bimodal images provided in this embodiment can be applied to the clinical diagnosis of breast cancer, helping doctors to make diagnoses and predicting disease progression and patient prognosis.

[0128] Based on the same inventive concept, this application also provides a dual-modal image-based breast cancer multi-classification device for implementing the aforementioned dual-modal image-based breast cancer multi-classification method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more dual-modal image-based breast cancer multi-classification device embodiments provided below can be found in the above-described limitations of the dual-modal image-based breast cancer multi-classification method, and will not be repeated here.

[0129] In one exemplary embodiment, such as Figure 8 As shown, a breast cancer multi-classification device based on bimodal images is provided, including:

[0130] The feature extraction module is used to acquire the data to be classified and to obtain image depth feature groups based on the data to be classified using a preset depth feature extraction model; the data to be classified includes PET images and CT images corresponding to breast cancer;

[0131] The filtering module is used to obtain the clinical feature group of the data to be classified, and to filter the features in the image depth feature group and the clinical feature group to obtain the target feature group.

[0132] The identification module is used to obtain classification results for target feature groups through a preset breast cancer classification model. The classification results include one of multiple breast cancer subtypes.

[0133] As an optional implementation, a model module is also included, which is used for:

[0134] Before acquiring the data to be classified, a preset deep feature extraction model is obtained;

[0135] As an optional implementation, the model module is used for:

[0136] Obtain PET images, PET mask images, CT images, and CT mask images corresponding to breast lesions;

[0137] PET images, PET mask images, CT images, CT mask images, and corresponding classification labels are used as a bimodal dataset; the classification label is one of several breast cancer subtypes.

[0138] Based on the bimodal dataset, a pre-trained preset feature extraction model is transferred to obtain a preset deep feature extraction model.

[0139] As an optional implementation, the model module is used for:

[0140] Before obtaining the classification results of the target feature group through the preset breast cancer classification model, obtain the preset breast cancer classification model.

[0141] As an optional implementation, the model module is also specifically used for:

[0142] The training dataset is obtained based on the features and classification labels in the target feature group; the classification label is one of the various breast cancer subtypes; the preset classification model is trained based on the training dataset to obtain the preset breast cancer classification model.

[0143] As an optional implementation, the default deep feature extraction model is the trained SEResNext50 network; the default breast cancer classification model is the trained LightGBM classification model.

[0144] As an optional implementation, the filtering module is used to: preprocess the features in the image depth feature group and the clinical feature group to obtain the first feature group;

[0145] The target feature group is obtained by sequentially performing variance screening, correlation screening, and feature importance selection on the first feature group based on a threshold.

[0146] As an optional implementation, the filtering module is also used for:

[0147] The target feature group is obtained by sequentially performing variance filtering, correlation filtering, and feature importance selection on the first feature group based on thresholds;

[0148] Obtain the variance of the features in the first feature group, and select features with variance greater than the first threshold to form the second feature group;

[0149] The feature correlation of the features in the second feature group is obtained according to the Pearson method or the Spearman method, and features with feature correlation less than the second threshold are selected to form the third feature group.

[0150] The feature importance of the third feature group is obtained from the feature importance module in the preset breast cancer classification model, and features with feature importance greater than the third threshold are selected to form the target feature group.

[0151] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores processed data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a multi-classification method for breast cancer based on bimodal images.

[0152] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0154] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0155] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0158] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. In summary, the content of this specification should not be construed as a limitation of this application.

Claims

1. A breast cancer multi-classification device based on bimodal images, characterized in that, The breast cancer multi-classification device based on bimodal images includes: The feature extraction module is used to acquire data to be classified and to obtain image depth feature groups based on the data to be classified using a preset deep feature extraction model. The data to be classified includes PET and CT images corresponding to breast cancer. The preset deep feature extraction model is a trained SEResNext50 network. The SEResNext50 network includes a Conv1 layer, a MaxPool layer, a Stage1 layer, a Stage2 layer, a Stage3 layer, a Stage4 layer, an AvgPool layer, and an FC layer connected sequentially. The filtering module is used to obtain the clinical feature group of the data to be classified, and to perform feature filtering on the features in the image depth feature group and the clinical feature group to obtain the target feature group; wherein, the clinical feature group includes age, clinical stage and SUV feature, and the SUV feature is an SUV feature matrix obtained by calculating SUV value based on the PET image; The identification module is used to obtain classification results based on the target feature group through a preset breast cancer classification model. The classification results are used to indicate one of Luminal A, Luminal B (her2-), Luminal B (her2+), Her2 positive, and triple negative. The preset breast cancer classification model is a trained LightGBM classification model. The model module is used for: Obtain PET images, PET mask images, CT images, and CT mask images corresponding to breast lesions; The PET images, PET mask images, CT images, CT mask images, and corresponding classification labels are used as a bimodal dataset; the classification labels are used to indicate one of Luminal A type, Luminal B (her2-) type, Luminal B (her2+) type, Her2 positive type, and triple negative type. The preset deep feature extraction model is obtained by performing transfer training on the pre-trained preset feature extraction model based on the bimodal dataset. The filtering module is also used to: preprocess the features in the image depth feature group and the clinical feature group to obtain a first feature group; Obtain the variance of the features in the first feature group, and select the features whose variance is greater than a first threshold to form a second feature group; The feature correlation of the features in the second feature group is obtained according to the Pearson method or the Spearman method, and the features whose feature correlation is less than the second threshold are selected to form the third feature group. The feature importance of the third feature group is obtained according to the feature importance module in the preset breast cancer classification model, and the features whose feature importance is greater than the third threshold are selected to form the target feature group.

Images