A breast cancer image multi-classification method and device and a storage medium

Abstract

The application discloses a breast cancer image multi-classification method and device and a storage medium. The method comprises the following steps: acquiring first characteristic target data based on a full tumor apparent diffusion coefficient imaging sequence histogram according to first image data of a plurality of target objects to be classified; combining feature sequences of the first image data of the plurality of target objects according to a preset target combination sequence, fusing the obtained combination sequence results according to the target objects, and obtaining second characteristic target data based on imaging features; combining the first characteristic target data and the second characteristic target data to obtain third characteristic target data, taking the third characteristic target data as an input of a trained first classification network, and outputting a first classification result. The classification effect of the breast cancer image can be improved.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, and in particular to a method, apparatus and storage medium for multi-classification of breast cancer images. Background Technology

[0002] Breast cancer is the most common malignant tumor among women worldwide, and its incidence increases with age. Breast cancer is a highly heterogeneous tumor, and its different biological behaviors in clinical practice are mainly determined by its different intrinsic gene phenotypes. Based on the expression of estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor-2 (HER-2) detected by immunohistochemical examination and fluorescence in situ hybridization, breast cancer is classified into four subtypes: Luminal A, Luminal B, HER-2 enriched, and triple-negative breast cancer. These molecular subtypes have different clinical manifestations, therapeutic targets, therapeutic responses, and long-term survival, leading to systemic treatment strategies based on different molecular subtypes.

[0003] Currently, immunohistochemical examination of biopsies is the primary clinical method for assessing the molecular subtype of breast cancer before treatment. However, pathological examination, as an invasive procedure, not only causes pain for patients but also suffers from sampling bias, making it difficult to capture overall histological and genetic phenotype information of the tumor, and the information provided is often delayed. Therefore, it is necessary to develop a new alternative approach to extract features from the entire tumor as characterizing tumor phenotype and prognostic indicators, such as the expression status and molecular subtype of molecular receptors, and to dynamically monitor changes in tumor biology in real time during treatment.

[0004] Currently, multiparametric magnetic resonance imaging (mMRI) shows promising applications in the diagnosis, molecular subtype differentiation, treatment efficacy evaluation, and prognostic prediction of breast cancer patients. Diffusion-weighted imaging (DWI) and its derived apparent diffusion coefficient (ADC) map can quantitatively reflect the degree of water molecule diffusion within the tumor and reflect changes in the spatial composition of tissue structure under pathophysiological conditions at the molecular level. Previous studies have utilized ADC histogram analysis from breast cancer mMR imaging to differentiate between different molecular subtypes. However, most studies only plotted regions of interest (ROIs) at a single level of the tumor or across the entire tumor to obtain the mean, minimum, and maximum ADC values ​​within the ROI as cutoff values ​​to distinguish different molecular subtypes. However, these indicators do not fully reflect tumor heterogeneity. Furthermore, absolute ADC values ​​are easily affected by multiple b-values ​​set in MR scanning protocols at different medical centers and by different MR scanners, resulting in poor universality. Previous studies have successfully assessed the prognosis of metastatic renal cell carcinoma patients receiving neoadjuvant sunitinib by analyzing the proportion of tumors below the 25th percentile in the average ADC value of each voxel in the overall tumor ADC histogram. These studies suggest that this tumor region represents the area with the most significant restriction of water molecule diffusion within metastatic renal cell carcinoma and may also be the region with the highest tumor cell density. This invention tentatively names this region tumor proliferative burden (TPB) and extends its definition to the proportion of tumor volume at the lowest overall ADC value relative to the total tumor volume. However, no studies have yet applied TPB to the differentiation of triple-negative breast cancer from the other three breast cancer subtypes (non-triple-negative breast cancer). In recent years, radiomics has been widely used as a quantitative, non-invasive preoperative tumor assessment method in the auxiliary diagnosis of breast cancer. However, current techniques only perform radiomics analysis on single or two sequences to identify triple-negative breast cancer. In reality, different sequences can be viewed as multimodal images describing the tumor from different perspectives. Fusion of imaging information from multiple sequences can better characterize tumor features and, theoretically, improve the classification performance of radiomics models. Currently, no studies have attempted to fully leverage the potential of integrating radiomics features obtained from multiple MRI sequences routinely scanned in clinical practice to differentiate between triple-negative and non-triple-negative breast cancer. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by proposing a multi-classification method, device, and storage medium for breast cancer images. This method can combine tumor proliferation burden parameters to improve the differentiation performance between triple-negative and non-triple-negative breast cancer, thereby achieving better classification results for breast cancer images.

[0006] In a first aspect, the present invention provides a multi-classification method for breast cancer images, comprising:

[0007] Based on the first image data of multiple target objects to be classified, the corresponding first characteristic target data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient is obtained; wherein, the first image data contains multiple feature sequences, and the apparent diffusion coefficient imaging sequence histogram includes tumor proliferation burden;

[0008] According to the preset target combination sequence, the feature sequences of the first image data of multiple target objects are combined, and the resulting combination sequence is fused according to the target objects to obtain the second characteristic target data based on image omics features.

[0009] The first feature target data and the second feature target data are combined to obtain the third feature target data. The third feature target data is used as the input of the trained first classification network to output the first classification result. The first classification network contains one or more classifiers.

[0010] This invention uses the histogram of the entire tumor's apparent diffusion coefficient imaging sequence as the first characteristic target data based on the histogram of the apparent diffusion coefficient imaging sequence. This allows for the investigation of the impact of tumor proliferation burden indicators on the classification performance of triple-negative and non-triple-negative breast cancer. This is because the form of the histogram based on the entire tumor's apparent diffusion coefficient imaging sequence can fully reflect the heterogeneity information of the tumor. Secondly, by combining all different sequences, it is beneficial to complement multi-sequence information. The multiple second characteristic target data obtained based on radiomics features are fused with the first characteristic target data based on the histogram of the entire tumor's apparent diffusion coefficient imaging sequence and used as input to the first classification network. This allows for the construction of a high-performance and robust classification system, achieving better classification results for breast cancer images. This invention not only fully utilizes the overall apparent diffusion coefficient imaging sequence information of the tumor but also fuses imaging information from multiple sequences for classification. The fused imaging information can better describe the characteristics of the tumor, thereby improving classification accuracy, reducing the workload of manual classification, and increasing efficiency.

[0011] Furthermore, the obtained combined sequences are fused according to the target object to obtain the second characteristic target data based on radiomics features, including:

[0012] Multiple feature sequences in the combined sequence are fused according to the same target object to obtain second characteristic target data based on radiomics features; wherein, the combined sequence contains at least two feature sequences.

[0013] This invention employs the fusion of all different sequences, which facilitates information complementarity among multiple sequences and avoids the significant bias that a single sequence might have on the overall classification effect after processing a sequence individually, thereby improving the classification effect of breast cancer images.

[0014] Furthermore, the fusion of multiple feature sequences in the combined sequence according to the same target object to obtain the second characteristic target data based on radiomics features specifically involves:

[0015] Based on different feature sequences, the fourth characteristic target data corresponding to the multiple target objects based on radiomics are obtained sequentially, and the transformation matrix of the fourth characteristic target data corresponding to the multiple target objects is calculated based on the combined sequence.

[0016] Based on the transformation matrix, the fourth characteristic target data corresponding to the feature sequences belonging to the same target object are fused to obtain the second characteristic target data based on radiomics features.

[0017] This invention fuses characteristic sample data for each combination sequence according to the target, which can reduce the dimensionality of the combination sequence and obtain the second characteristic target data corresponding to the target object, so that the third classification network can use different classifiers for classification.

[0018] Furthermore, the multi-classification method for breast cancer images further includes: using the first feature target data as input to a trained second classification network to output a second classification result; wherein the second classification network includes one or more classifiers.

[0019] Furthermore, the multi-classification method for breast cancer images further includes: using the second feature target data as input to a trained third classification network to output a third classification result; wherein the third classification network contains one or more classifiers.

[0020] This invention uses first characteristic target data and second characteristic target data as inputs to the second classification network and the third classification network, respectively, to obtain corresponding classification results. It can obtain multi-classification of breast cancer images based on different extracted characteristic target data and can explore the influence of different breast cancer characteristic target data on breast cancer image classification.

[0021] Furthermore, the trained first classification network includes:

[0022] Based on the second image data of multiple sample objects to be classified, the corresponding first characteristic sample data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient is obtained; wherein, the second image data contains multiple feature sequences, and the apparent diffusion coefficient imaging sequence histogram includes tumor proliferation burden;

[0023] Multiple combinations of second image data are obtained to form multiple combination sequences. Each combination sequence is then fused according to the sample object to obtain second characteristic sample data based on image omics features.

[0024] The first feature sample data and the second feature sample data are combined, and the initial first classification network is trained based on the obtained third feature sample data to obtain the trained first classification network; wherein, the first classification network contains one or more classifiers.

[0025] Furthermore, the step of combining the feature sequences of the first image data of multiple target objects according to a preset target combination sequence includes:

[0026] Based on multiple trained sub-classification networks, calculate the AUC index of each sub-classification network under the corresponding combined sequence in turn;

[0027] The optimal combination sequence with the best AUC index is selected as the target combination sequence. Based on the target combination sequence, the feature sequences of the first image data of multiple target objects are combined.

[0028] Each subclassification network contains one or more classifiers, and the subclassification network with the best AUC index is selected as the trained third classification network.

[0029] Preferably, the feature sequence of the first image data includes an apparent diffusion coefficient imaging sequence, a diffusion-weighted imaging sequence, a T2-weighted imaging sequence, and an early sequence of dynamic enhancement scanning.

[0030] Secondly, the present invention provides a multi-classification device for breast cancer images, comprising:

[0031] The first characteristic target data unit is used to obtain corresponding first characteristic target data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient according to the first image data of multiple target objects to be classified; wherein, the first image data includes multiple feature sequences, and the imaging sequence histogram of apparent diffusion coefficient includes tumor proliferation burden.

[0032] The second characteristic target data unit is used to combine the feature sequences of the first image data of multiple target objects according to the preset target combination sequence, and fuse the obtained combination sequence results according to the target objects to obtain the second characteristic target data based on image omics features.

[0033] The third characteristic target data unit is used to combine the first characteristic target data and the second characteristic target data to obtain the third characteristic target data;

[0034] The first classification network unit is used to take the third feature target data as input to the trained first classification network so as to output a first classification result; wherein the first classification network includes one or more classifiers.

[0035] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to perform the multi-classification method for breast cancer images as described in the first aspect. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the multi-classification method for breast cancer images provided in an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of the ADC histogram and corresponding histogram indices provided in an embodiment of the present invention;

[0038] Figure 3 This is a flowchart illustrating the testing process of the multi-classification method for breast cancer images provided in an embodiment of the present invention.

[0039] Figure 4 This is a flowchart illustrating the testing process of the initial first classification network in the multi-classification method for breast cancer images provided in this embodiment of the invention.

[0040] Figure 5 This is a schematic diagram of the process for obtaining second characteristic sample data in the multi-classification method for breast cancer images provided in this embodiment of the invention;

[0041] Figure 6 This is a flowchart illustrating the training process of the multi-classification method for breast cancer images provided in this embodiment of the invention.

[0042] Figure 7 This is a schematic diagram of feature selection in the first classification network provided in an embodiment of the present invention;

[0043] Figure 8 This is a schematic diagram of the structure of the multi-classification device for breast cancer images provided in an embodiment of the present invention;

[0044] Figure 9This is a schematic diagram of the structure of the computer-readable storage medium provided in the embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The purpose of this invention is to provide a method, device, and storage medium for classifying breast cancer images. This is achieved by studying the histogram of the apparent diffusion coefficient (ADC) imaging sequence for different molecular subtypes of breast cancer, especially the tumor proliferative burden (TPB). This is because TPB can provide comprehensive information on tumor heterogeneity. By fusing imaging information from multi-parameter image data, the tumor can be better characterized and the accuracy of classification can be improved.

[0047] See Figure 1 This is a flowchart illustrating the multi-classification method for breast cancer images provided in this embodiment of the invention, including steps S11 to S13, specifically:

[0048] Step S11: Based on the first image data of multiple target objects to be classified, obtain the corresponding first characteristic target data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient; wherein, the first image data contains multiple feature sequences, and the imaging sequence histogram of the apparent diffusion coefficient includes the tumor proliferation burden.

[0049] It is worth noting that the first image data consists of MRI images corresponding to multiple target objects. Based on various magnetic resonance imaging (MRI) scans, MRI images containing multiple feature sequences are obtained. These MRI images can be viewed as multimodal images describing the tumor from different perspectives. Mask image data can be obtained from the MRI images, including: delineating the target area for each layer of each multimodal MRI image according to the user-input operation parameters, obtaining the two-dimensional region of interest (ROI) for each layer, and then saving the two-dimensional ROIs of all layers as three-dimensional mask image data; the users include at least two experienced radiology experts.

[0050] Feature extraction based on the whole-tumor epigenetic diffusion coefficient imaging sequence histogram was performed on the mask image data of the ADC sequence in MRI images to obtain the first characteristic target data of multiple target objects corresponding to the whole-tumor epigenetic diffusion coefficient imaging sequence histogram. Specifically, 3D ROIs were delineated on the ADC images to obtain the corresponding ADC histograms, and ADC histogram indicators, including tumor proliferation burden (TPB), were extracted for different molecular subtypes of breast cancer. Indicator extraction was performed on multiple target objects in turn to obtain multiple first characteristic sample data samples based on the whole-tumor epigenetic diffusion coefficient imaging sequence histogram.

[0051] Step S12: According to the preset target combination sequence, the feature sequences of the first image data of multiple target objects are combined, and the resulting combination sequence is fused according to the target objects to obtain the corresponding second characteristic target data based on image omics features.

[0052] Preferably, the feature sequence of the first image data includes an apparent diffusion coefficient imaging sequence, a diffusion-weighted imaging sequence, a T2-weighted imaging sequence, and an early sequence of dynamic enhancement scanning.

[0053] It is worth noting that the early sequence of dynamic contrast-enhanced scanning is the early enhancement (DCE2) sequence of dynamic contrast-enhanced (DCE) scanning. The first image data includes: MRI images and corresponding mask images. Therefore, both the MRI images and the mask images include ADC sequences, diffusion-weighted imaging (DWI) sequences, T2-weighted imaging (T2WI) sequences, and DCE2 sequences.

[0054] This invention employs the fusion of all different sequences, which facilitates information complementarity among multiple sequences and avoids the significant bias that a single sequence might have on the overall classification effect after processing a sequence individually, thereby improving the classification effect of breast cancer images.

[0055] The process involves fusing the obtained combined sequences according to the target object to obtain second characteristic target data based on radiomics features. This includes fusing multiple sequences in the combined sequence according to the same target object to obtain second characteristic target data based on radiomics features. The combined sequence contains at least two feature sequences.

[0056] Specifically, fusing multiple sequences in the combined sequence according to the same target object to obtain second characteristic target data based on radiomics features includes: sequentially obtaining fourth characteristic target data based on radiomics corresponding to the multiple target objects according to different feature sequences; calculating the transformation matrix of the fourth characteristic target data corresponding to the multiple target objects according to the combined sequence; and fusing the fourth characteristic target data corresponding to the feature sequences belonging to the same target object according to the transformation matrix to obtain second characteristic target data based on radiomics features.

[0057] It is worth noting that, based on different feature sequences, the fourth characteristic target data corresponding to the multiple target objects is obtained in sequence. Specifically, the fourth characteristic target data based on radiomics is obtained by sequentially extracting radiomics features from the mask image data corresponding to different feature sequences, and then obtaining the corresponding fourth characteristic target data for all target objects in sequence.

[0058] This invention employs the method of fusing each MRI combination sequence according to the target object, which can reduce the dimensionality of the MRI combination sequence and obtain the second characteristic target data corresponding to the target object, so that the third classification network can use different classifiers for classification.

[0059] The process of combining the feature sequences of the first image data of multiple target objects according to a preset target combination sequence includes: calculating the AUC index of each sub-classification network under the corresponding combination sequence according to multiple trained sub-classification networks; selecting the combination sequence with the best AUC index as the target combination sequence; and combining the feature sequences of the first image data of multiple target objects according to the target combination sequence. Each sub-classification network contains one or more classifiers, and the sub-classification network with the best AUC index is selected as the trained second classification network.

[0060] Furthermore, it includes: using the first feature target data as input to a trained second classification network to output a second classification result; wherein the second classification network contains one or more classifiers. And, using the second feature target data as input to a trained third classification network to output a third classification result; wherein the third classification network contains multiple third sub-classification networks, each of which contains one or more classifiers.

[0061] It is worth noting that the number of classifiers can include one or more. When the number of classifiers includes multiple classifiers, the classification network is specifically a multi-classification model system; that is, a multi-classification method for breast cancer images is a multi-classification model system. Considering that the classification ability of a classification model is related to the original classifier it uses, and that different classifiers may produce inconsistent results even when applied to the same task, in this embodiment, it is preferable to input training data into multiple different classifiers for training to construct a multi-classification model system, which includes multiple classification models. According to the theorem of "no free lunch," any classifier has its advantages and disadvantages. Comparing the performance of multiple different classifiers compared to a single classifier can provide a more robust classification system with higher classification accuracy.

[0062] Preferably, the first classification network, the second classification network, and the third classification network are all multi-classification models; wherein, the number of multi-classification models is determined according to the number of feature selection algorithms and the number of classifiers.

[0063] For example, if n feature selection algorithms and m classifiers are set, m×n classification models can be obtained. It can be understood that the first, second, and third classification networks each include corresponding m×n classification models. In the third classification network, the fourth feature sample data based on radiomics for each combined sequence, along with the corresponding label data, are input into each of the aforementioned m×n classification models. The specified feature selection algorithm in each classification model first obtains a feature representative set, then inputs this feature representative set into the classifier, calls the `predict` function for prediction, saves and outputs the corresponding third classification result.

[0064] This invention uses first characteristic target data and second characteristic target data as inputs to the second classification network and the third classification network, respectively, to obtain corresponding classification results. It can obtain multi-classification of breast cancer images based on different extracted characteristic target data and can explore the influence of different breast cancer characteristic target data on breast cancer image classification.

[0065] Step S13: Combine the first feature target data and the second feature target data to obtain the third feature target data, and use the third feature target data as the input of the trained first classification network to output the first classification result; wherein, the first classification network contains one or more classifiers.

[0066] For example, 129 patients with pathologically confirmed triple-negative breast cancer and non-triple-negative breast cancer were selected as the target subjects for the test; among these 129 patients, there were 11 patients with triple-negative breast cancer and 118 patients with non-triple-negative breast cancer. Four types of sequence data were collected, including images obtained from preoperative MRI scans of the 11 triple-negative breast cancer patients based on four different MRI sequences included in the target combination sequence: ADC sequence, DWI sequence, T2W sequence I, and DCE2 sequence. These images served as the first images for the multimodal test, and the constructed first, second, and third classification networks were tested.

[0067] Tumor type labeling was performed on the target objects of each test to obtain label data for each target object. For each MRI sequence, 129 corresponding multimodal test images were first obtained from MRI scans of 129 target objects. Then, ADC histogram index extraction was performed on the ADC images in the multimodal test images to obtain 129 first characteristic target data based on ADC histograms. Radiomics features were then extracted from each multimodal image data to obtain 129 first characteristic target data based on radiomics. The label data of the first characteristic target data corresponds to the label data of its respective target object. The specific steps for histogram index extraction of ADC test images and radiomics feature extraction of multimodal test images include the histogram index extraction and radiomics feature extraction methods in steps S21-S23.

[0068] Based on the transformation matrix of the ADC+DWI+T2WI+DCE2 combined sequence, and according to four different target combination sequences including ADC sequence, DWI, T2WI and DCE2, the feature sequences of 129 radiomics-based first characteristic target data are combined to obtain 129 radiomics-based second characteristic target data. Combining the 129 ADC histogram-based first characteristic target data with the 129 radiomics-based second characteristic target data, 129 third characteristic target data are obtained.

[0069] 129 first-characteristic target data based on ADC histograms, 129 second-characteristic target data based on radiomics, and 129 third-characteristic target data were input into the second, third, and first classification networks, respectively. This allowed the 150 classification models of the classification networks to classify each characteristic target data one by one and output a probability score for each characteristic target data belonging to triple-negative breast cancer and non-triple-negative breast cancer. Based on the probability scores output by the 150 classification models, the tumor type classification result of each test target object was determined.

[0070] Based on the tumor type classification results of 129 pathologically confirmed triple-negative breast cancer and non-triple-negative breast cancer subjects, the classification performance of the classification models with the largest AUC for the second, third, and first classification networks was calculated, including AUC, classification accuracy, sensitivity, and specificity, as shown in Table 1 below. Here, f represents the performance of each classification network compared to the first classification network using the Wilcoxon signed-rank test; a p-value < 0.05 is considered statistically significant and is indicated in bold.

[0071] Table 1. Comparison of classification performance of the classification models with the highest AUC values ​​among the three classification networks.

[0072]

[0073] The test results show that the breast cancer image classification method disclosed in this invention can construct a multi-classification model system that fully utilizes the heterogeneity of tumors and integrates image information from multiple MRI sequences. On one hand, it generates the first characteristic target data to be tested based on the whole tumor ADC histogram. On the other hand, it uses the proposed combination algorithm to combine the radiomics-based first characteristic target data from different MRI sequences to generate new second characteristic target data. Furthermore, it combines the first and second characteristic target data to generate new third characteristic target data. These target data are processed by different classifiers and feature selection algorithms, allowing for the construction of multiple classification models composed of different types of classifiers and feature selection algorithms for performance comparison. This results in more reliable classification results and improves the robustness of the classification system.

[0074] It is worth noting that in the entire classification test process, the target object's first characteristic target data based on the ADC histogram is first obtained. Then, the transformation matrix of the combination sequence based on the feature sequence of the first characteristic target data is obtained to obtain the second characteristic target data based on the influencing omics features. Finally, the first characteristic target data based on the ADC histogram and the second characteristic target data based on radiomics are merged into a third characteristic target data.

[0075] The target subjects are patients undergoing breast tumor type prediction. For the first characteristic target data, breast cancer images to be classified are first acquired based on ADC scan sequences. Then, ADC histogram indicators are extracted from each of the target subjects' breast cancer images to obtain the first characteristic target data based on the ADC histogram. For the second characteristic target data, based on the K types of MRI sequences included in the preset target combination sequence, K breast cancer images to be classified are first acquired based on the K types of MRI sequences. Then, radiomics features are extracted from each of the K types of breast cancer images to obtain K radiomics-based first characteristic target data. Finally, based on the final transformation matrix obtained by concatenating the transformation matrices of each radiomics feature in the preset target combination sequence, the K radiomics-based first characteristic target data are fused into a new feature space to obtain a radiomics-based second characteristic target data. For example, after determining the above ADC+DWI+T2WI+DCE2 fusion sequence as the preset target combination sequence, K equals 4, and images of the target object obtained by scanning based on four different MRI sequences, ADC, DWI, T2WI and DCE2, can be used as breast cancer images to be classified.

[0076] The target data for the first characteristic based on ADC histograms, the target data for the second characteristic based on radiomics, and the target data for the third characteristic are input into the trained second classification network, the trained third classification network, and the trained first classification network, respectively. The target label data for the target object is determined based on the output results of the classification networks. Preferably, each trained classification network contains 150 classification models that process the target data for the first, second, and third characteristics, respectively, and output a probability score indicating that the target data belongs to a certain type of breast cancer. Finally, based on the probability scores output by the 150 classification models, the target label data for the target object is determined. The target label data can be 0 or 1, i.e., referring to non-triple-negative breast cancer or triple-negative breast cancer. See also... Figure 3 This is a flowchart illustrating the testing process for multi-classification of breast cancer provided in an embodiment of the present invention. The flowchart includes the testing processes of a second classification network, a first classification network, and a third classification network. The first characteristic target data containing histogram features of ADC, the third characteristic target data combining the first characteristic target data with the second characteristic target data containing combined sequence radiomics features, and the second characteristic target data containing combined sequence radiomics features are used as inputs to the first classification network, the second classification network, and the third classification network, respectively, and the second classification results for triple-negative breast cancer and non-triple-negative breast cancer are output.

[0077] See Figure 4The flowchart of the initial first classification network testing process of the multi-classification method for breast cancer images provided in this embodiment of the invention includes steps S21 to S23, specifically:

[0078] Step S21: Based on the second image data of multiple sample objects to be classified, obtain the corresponding first characteristic sample data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient; wherein, the second image data contains multiple feature sequences, and the imaging sequence histogram of the apparent diffusion coefficient includes the tumor proliferation burden.

[0079] The process involves delineating the target area of ​​an MRI image with an ADC sequence based on user-input parameters, obtaining a two-dimensional region of interest (ROI), and then saving the corresponding ROI as three-dimensional mask image data to obtain the mask image data corresponding to the ADC sequence. Feature extraction based on the histogram of the imaging sequence of the whole tumor apparent diffusion coefficient is then performed on the mask image data corresponding to the ADC sequence. The users include at least two experienced radiology experts.

[0080] It is worth noting that, based on the acquisition of second image data corresponding to multiple sample objects for training, and the tumor type labeling of each sample object, label data for each sample object is obtained; wherein, the label data is used to characterize the tumor type to which the sample object belongs, and the tumor type includes at least two; and by drawing 3DROIs on the ADC image, the corresponding ADC histogram is obtained, and ADC histogram indicators, including tumor proliferation burden (TPB), for different molecular subtypes of breast cancer are extracted to obtain multiple first characteristic sample data based on ADC histograms. It can be understood that the multiple first characteristic sample data based on ADC histograms under the ADC sequence correspond to the label data of their respective sample objects.

[0081] Preferably, the tumor type includes triple-negative breast cancer (TNBC) and non-triple-negative breast cancer (non-TNBC).

[0082] For example, if the tumor types include only triple-negative breast cancer and non-triple-negative breast cancer, then the label data can include two categories: one for triple-negative breast cancer and the other for non-triple-negative breast cancer. For instance, the label data could be set to 0 or 1, with 0 representing non-triple-negative breast cancer and 1 representing triple-negative breast cancer; and vice versa. If the tumor types include more than two categories, in addition to continuing to use numbers, binary numbers or one-hot encoding can also be used for labeling, without limitation here.

[0083] Table 2 Clinical information of TNBC and non-TNBC subjects

[0084]

[0085] For example, suppose there are 337 subjects in the training sample, including 337 patients with pathologically confirmed different molecular subtypes of breast cancer. Images were collected from each of the 337 subjects before surgery using four different MRI sequences: ADC, DWI, T2WI, and DCE2, as multimodal sample images. Among these 337 patients with pathologically confirmed different molecular subtypes of breast cancer, there are 43 patients with triple-negative breast cancer and 294 patients with non-triple-negative breast cancer. See Table 2 for clinical information of TNBC and non-TNBC subjects. The data in the table represent the number of tumors, with percentages in parentheses. * indicates that the data is an average, and the range is indicated in parentheses. † indicates other invasive cancers including 1 case of TNBC malignant phyllodes tumor, 1 case of neuroendocrine carcinoma, and 2 cases of non-TNBC lobular carcinoma in situ. a, b, c, d, and e represent Student's t-test, Mann-Whitney U test, Pearson chi-square test, continuity correction of chi-square test, and Fisher exact test, respectively. A p-value < 0.05 is considered statistically significant and is indicated in bold.

[0086] Based on the target samples during training, first characteristic sample data based on the ADC histogram is obtained, which can first yield ADC histograms for different molecular subtypes throughout the entire tumor region. For example, 59 ADC histogram indicators, including tumor proliferation burden (TPB), are included. Referring to Figure 2, this is a schematic diagram of the ADC histogram and corresponding histogram indicators provided in this embodiment of the invention. Figure 2(a) is a schematic diagram of the ADC histogram, and Figure 2(b) is a schematic diagram of the histogram indicators corresponding to the ADC histogram. Figure 2 contains 59 ADC histogram indicators; specific ADC histogram indicators are shown in Table 3.

[0087] Table 3. 59 ADC Histogram Indicators

[0088]

[0089] It is worth noting that the first characteristic target data or first characteristic sample data based on radiomics for extracting the target objects of the test or the target samples of the training can be obtained by using the open-source Python package Pyradiomics to extract radiomics features from each region of interest. This yields radiomics features of various multimodal sample images, which serve as the first characteristic target data or first characteristic sample data samples. In other words, the radiomics features of various feature sequences are used as the first characteristic target data or first characteristic sample data samples. Due to severe data imbalance in the sample objects, the label data corresponding to the first characteristic sample data can be standardized, and then the Synthetic Minority Oversampling (SMOTE) algorithm can be used to balance the data by introducing synthetic feature samples to oversample the minority class of triple-negative breast cancer samples before further processing, thus overcoming the negative impact of data imbalance.

[0090] Preferably, the radiomics features include 109 features.

[0091] For example, multiple radiomics features can be divided into three categories: shape features, first-order statistical features (histogram analysis), and second-order statistical features (image grayscale distribution, commonly known as texture features), see Table 4, 109 radiomics features.

[0092] Table 4 109 radiomics features

[0093]

[0094] It is worth noting that during the testing and training processes, the specific steps for extracting histogram indicators from ADC test images and extracting radiomics features from multimodal test images can all use the histogram indicator extraction and radiomics feature extraction methods used in steps S21 to S23.

[0095] Step S22: Combine various second image data in different ways to obtain multiple combination sequences, and fuse each combination sequence according to the sample object to obtain second characteristic sample data based on image omics features.

[0096] For example, by combining the ADC sequence, DWI sequence, T2WI sequence and DCE2 sequence in the second image data, 11 combined sequences can be obtained, including: ADC+DWI, ADC+T2WI, ADC+DCE2, DWI+T2WI, DWI+DCE2, T2WI+DCE2, ADC+DWI+T2WI, ADC+DWI+DCE2, ADC+T2WI+DCE2, DWI+T2WI+DCE2, ADC+DWI+T2WI+DCE2, where each combined sequence contains at least two different feature sequences.

[0097] It is worth noting that each combined sequence is fused according to the sample object to obtain the second characteristic sample data based on radiomics features. This includes fusing multiple feature sequences in the combined sequence according to the same sample object to obtain the second characteristic target data based on radiomics features; wherein the combined sequence contains at least two feature sequences. Specifically, the fourth characteristic target data based on radiomics corresponding to the multiple sample objects is obtained sequentially according to different feature sequences, and the transformation matrix of the fourth characteristic target data corresponding to the multiple sample objects is calculated according to the combined sequence; according to the transformation matrix, the fourth characteristic target data corresponding to the feature sequences belonging to the same sample object are fused to obtain the second characteristic target data used for radiomics features. It is worth noting that the second characteristic target data obtained at this time is used for training.

[0098] It should be noted that for each combined sequence including at least two MRI sequences, the first characteristic sample data of each sample object under each MRI sequence includes the aforementioned 109 radiomics features. The MRI sequences include: ADC sequence, DWI sequence, T2WI sequence, and DCE2 sequence. For each radiomics feature, the fourth characteristic target data corresponding to the multiple target objects can be obtained based on the first characteristic sample data of all sample objects included in the combined sequence. Based on the combined sequence corresponding to the fourth characteristic target number, the transformation matrix of the fourth characteristic target data corresponding to the multiple target objects is calculated. Specifically, a multi-sequence feature matrix is ​​constructed for each radiomics feature. This multi-sequence feature matrix includes the feature matrices of various MRI sequences to be fused based on the combined sequence. Then, based on the proposed feature fusion method, the transformation matrix of each radiomics feature is calculated from the multi-sequence feature matrix. Then, based on the transformation matrix, the multi-sequence feature matrices of each radiomics feature are fused to obtain the fourth characteristic sample data of each radiomics feature, that is, the transformation matrix of the fourth characteristic target data corresponding to the multiple target objects is obtained.

[0099] The process of obtaining the second characteristic sample data according to step S22 includes sub-steps S221 to S238, see below. Figure 5 This is a schematic diagram illustrating the process of obtaining second characteristic sample data in the multi-classification method for breast cancer images provided in this embodiment of the invention. For example, for each combined sequence, the following calculation process is performed once for each of the 109 radiomics features, specifically as follows:

[0100] Sub-step S221: Calculate the first feature vector of each type of sample object based on the feature sequence of the radiomics features and the obtained multi-sequence feature matrix.

[0101] Specifically, based on the feature sequences of radiomics features and the obtained multi-sequence feature matrix, the first feature vector of each type of sample object and the mean of the first feature vectors of all sample objects are calculated.

[0102] Preferably, the first feature vector of the sample object and the mean of the first feature vectors of all sample objects can be expressed as follows:

[0103] ,

[0104] ,

[0105] in, Represents the number of tumor categories, Representing the Number of samples in the class =1,…,c; Representing the The class of One sample, .

[0106] Sub-step S222: Calculate the inter-class scattering matrix and covariance matrix of the multi-sequence feature matrix based on the first feature vector.

[0107] Specifically, based on the first feature vector and the mean of the first feature vectors of all sample objects, the inter-class scattering matrix and covariance matrix of the multi-sequence feature matrix are calculated.

[0108] Preferably, the inter-class scattering matrix and covariance matrix of the multi-sequence feature matrix can be expressed as follows:

[0109] ,

[0110] ,

[0111] Where T is the transpose symbol.

[0112] Sub-step S223: Based on the inter-class scattering matrix and the covariance matrix, diagonalize the transpose of the obtained inter-class scattering matrix to obtain the diagonalized second eigenvector matrix.

[0113] Preferably, the transpose of the inter-class scattering matrix can be expressed as:

[0114] .

[0115] Preferably, the second eigenvector matrix can be represented as:

[0116] ,

[0117] in, This represents the eigenvalue matrix.

[0118] Sub-step S224: Based on the second eigenvector matrix, obtain the third eigenvectors corresponding to the first r largest first eigenvalues, and form a third eigenvector matrix by combining the third eigenvectors corresponding to the first r largest first eigenvalues; where r is a positive integer.

[0119] Preferably, the third eigenvector matrix can be represented as:

[0120] ,

[0121] in, Representing the fused dimension, preferably, .

[0122] Sub-step S225: Based on the inter-class scattering matrix, the covariance matrix, and the third eigenvector matrix, obtain the first r second eigenvalues ​​of the inter-class scattering matrix and the corresponding fourth eigenvector.

[0123] Preferably, the second feature value and its corresponding fourth eigenvector The relationship can be expressed as:

[0124] ,

[0125] Sub-step S226: Based on the first r second eigenvalues ​​and the corresponding fourth eigenvectors, obtain the transformation matrix of each radiomics feature.

[0126] It is understandable that, based on the currently calculated combination sequence, the fourth characteristic target data corresponding to the multiple target objects is obtained sequentially according to different feature sequences, and the transformation matrix of the fourth characteristic target data corresponding to the multiple target objects is calculated.

[0127] Preferably, the transformation matrix can be expressed as:

[0128] ,

[0129] Sub-step S227: Based on the multi-sequence feature matrix of each radiomics feature and the corresponding transformation matrix, fuse the multi-sequence feature matrix of each radiomics feature to obtain the first fused vector.

[0130] Preferably, the first fusion vector can be represented as:

[0131] ,

[0132] in, This represents the number of MRI sequences that need to be fused for each combined sequence. Represents the number of samples. Preferably, .

[0133] Sub-step S228: Iterate through each combination sequence in turn to obtain the corresponding first fusion vector, and concatenate all the first fusion vectors. Based on the obtained second fusion vector, divide the data according to different sample objects to obtain the second characteristic sample data based on radiomics that correspond one-to-one with multiple sample objects.

[0134] It is worth noting that for each combined sequence, repeating sub-steps S221-S227 until traversing 109 radiomics features yields a first fusion vector for each radiomics feature. Then, the first fusion vectors of all radiomics features are concatenated to obtain a second fusion vector. This second fusion vector corresponds to all sample objects. Therefore, the second fusion vector can be partitioned according to different sample objects to obtain multiple radiomics-based second characteristic sample data corresponding one-to-one with multiple sample objects. It is understandable that these multiple second characteristic sample data also correspond to the label data of their respective sample objects.

[0135] For example, by dividing the second fusion vector according to different sample objects, N second feature sample data based on radiomics can be obtained, each corresponding to one of the N sample objects; where N is the number of sample objects. It can be understood that the N second feature sample data also correspond to the label data of their respective sample objects.

[0136] Alternatively, sub-steps S236~S228 can be executed after sub-step S225, specifically as follows:

[0137] Sub-step S236: Based on the first r second eigenvalues ​​and the corresponding fourth eigenvectors, obtain the transformation matrix of each image omics feature. Based on each combination sequence, concatenate the transformation matrix to obtain the fusion transformation matrix.

[0138] Sub-step S237: Based on the multi-sequence feature matrix of each radiomics feature and the fusion transformation matrix, fuse the multi-sequence feature matrix of each radiomics feature to obtain the fused second fusion vector.

[0139] Sub-step S238: Iterate through each combination sequence in turn to obtain the corresponding second fusion vector. Based on the multiple second fusion vectors obtained, divide them according to different sample objects to obtain second characteristic sample data based on radiomics that correspond one-to-one with multiple sample objects.

[0140] For example, after traversing all the combined sequences, L final transformation matrices and L second fusion vectors can be obtained. Each second fusion vector includes N radiomics-based second feature sample data. These N radiomics-based second feature data samples are obtained by fusing all the MRI sequences to be fused for the combined sequence to which they belong. Here, L is the number of combined sequences for combining MRI sequences, and N is the number of sample objects.

[0141] Step S23: Combine the first feature sample data and the second feature sample data. Based on the obtained third feature sample data, combine the first feature sample data and the second feature sample data. Based on the obtained third feature sample data, train the initial first classification network respectively to obtain the trained first classification network. The first classification network includes one or more classifiers.

[0142] It is worth noting that the third feature sample data and the label data corresponding to each feature sample data are input into the first classification network for training to obtain the trained first classification network.

[0143] Specifically, during training, a first classification network is established, and the third characteristic sample data are subjected to feature selection by n feature selection algorithms of the corresponding first classification network to obtain n corresponding feature representative sets. Among them, the feature representative set corresponds one-to-one with the feature selection algorithm, and each feature representative set includes n important and meaningful feature representative samples. The label data corresponding to the feature representative set is the same as the label data of the third characteristic sample data to which it belongs. Then, the multiple feature representative sets and the label data corresponding to each feature representative set are sequentially input into m different classifiers for training.

[0144] If n feature selection algorithms and m classifiers are set, the third characteristic sample data based on radiomics of each combined sequence and the corresponding label data are input into each of the above m×n classification models. N-fold cross-validation is used. The specified feature selection algorithm in each classification model will first obtain the feature representative set, then input this feature representative set into the classifier and call the fit function for training, save and output the first classification result.

[0145] Preferably, n can be 15 and m can be 10.

[0146] For example, 15 feature selection algorithms and 10 classifiers are listed in Table 5 below, showing the available feature selection algorithms and classifiers.

[0147] Table 5 Available Feature Selection Algorithms and Classifiers

[0148]

[0149] The above 15 feature selection algorithms and 10 different classifiers can be combined to form a multi-classification model system with 150 classification models. Specifically, the scikit-learn machine learning package in the Python programming language environment can be used to train different classifiers and combine them with the above 15 feature selection algorithms to obtain 150 classification models.

[0150] It is worth noting that the second feature target data also needs to be used as input to the trained third classification network to output the third classification result; wherein, the third classification network contains one or more classifiers.

[0151] Specifically, obtaining the trained third classification network involves: establishing a corresponding third classification network for each combined sequence; sequentially selecting features from the third characteristic sample data based on radiomics for each combined sequence using n feature selection algorithms of the corresponding third classification network to obtain n feature representative sets. It can be understood that multiple initial sub-classification networks are constructed during training, each corresponding to a single combined sequence; the sub-classification network with the best performance after training is used as the trained third classification network. The feature representative sets correspond one-to-one with the feature selection algorithms, and each feature representative set includes n important and meaningful feature representative samples. The label data corresponding to each feature representative set is the same as the label data of the second characteristic sample data to which it belongs. Then, the multiple feature representative sets and their corresponding label data are sequentially input into m different classifiers for training to construct L sub-classification networks; where L is the number of combined sequences.

[0152] In other words, each subclassification network includes multiple classification models. The number of multiclassification models is determined by the number of feature selection algorithms n and the number of classifiers m. Thus, m×n multiclassification models can be calculated, where n and m are both positive integers. Under the framework of the multiclassification model system, it can be understood that each initial subclassification network corresponds to processing a combination sequence, and each initial classification network can contain m×n classification models.

[0153] Furthermore, based on multiple trained sub-classification networks, the AUC index of each sub-classification network under the corresponding combination sequence is calculated sequentially; the combination sequence with the optimal AUC index is selected as the target combination sequence, and the feature sequences of the first image data of multiple target objects are combined according to the target combination sequence; wherein, each sub-classification network contains one or more classifiers, and the sub-classification network with the optimal AUC index is selected as the trained third classification network. Thus, based on the obtained preset target combination sequence, the feature sequences of the first image data of multiple target objects are combined.

[0154] Specifically, during training, the second-feature sample data and the labels corresponding to the second-feature sample data of N sample objects are used as inputs to multiple initial sub-classification networks. Each sub-classification network contains multiple classification models, employing N-fold cross-validation. The specified feature selection algorithm in each classification model first obtains a set of feature representatives, then inputs this set of features into the classifier and calls the `fit` function for training, resulting in the corresponding trained third-classification network. During testing or direct use, the trained third-classification network is directly used, with the second-feature target data as input, outputting the third-classification result.

[0155] For example, based on the classification results of 150 classification models in the initial first classification network, the AUC (Area Under Curve) of each classification model is calculated. The average AUC of these classification models is used as the performance index of the entire multi-class classification model system to evaluate the classification performance of the multi-class classification model system after fusing different combined sequences. The AUC value ranges between 0.5 and 1. The closer the AUC is to 1.0, the higher the realism, indicating better classification performance. The classification performance of single sequences and combined sequences is shown in Table 6 below, which provides an overview of the performance (AUC) of single sequences and combined sequences.

[0156] Table 6. Performance (AUC) Overview of Single Sequences and Combinations

[0157]

[0158] As shown in Table 6 above, when multiple MRI sequences are combined, the AUC obtained by the combined sequences is generally higher than that of the single sequences. The best performance is achieved by the ADC+DWI+T2WI+DCE2 combined sequence, with an AUC value of 0.828, which is higher than the value of the best single ADC sequence (AUC=0.808). Therefore, the subclassification network corresponding to the combined sequence ADC+DWI+T2WI+DCE2 is used as the trained third classification network, and the combined sequence ADC+DWI+T2WI+DCE2 is used as the preset target combined sequence for testing or direct use.

[0159] It is worth noting that in this embodiment, it is preferable to select the classification model corresponding to the sub-classification network with the best performance based on the AUC performance index of each combination sequence as the trained third classification network. That is, in the process of testing or direct use, it is only necessary to process the second characteristic target data according to the third classification network and output the corresponding third classification result.

[0160] It is worth noting that during testing or direct use, the first feature target data must be used as input to the trained second classification network for classification. Specifically, the first feature target data is used as input to the trained second classification network to output a second classification result; wherein the second classification network contains one or more classifiers.

[0161] The trained second classification network is obtained by taking the first feature sample data based on the ADC histogram and the label corresponding to the first feature sample data as the input of the initial second classification network, which contains one or more classifiers.

[0162] Preferably, the initial second classification network comprises one classification network, which in turn contains multiple classifiers. Similar to the trained first classification network, based on n feature selection algorithms and m classifiers, the initial second classification network has m×n classification models. During training, N first-feature sample data based on ADC histograms corresponding to N sample objects and their corresponding labels are used as inputs to the initial second classification network. N-fold cross-validation is employed. The specified feature selection algorithm in each classification model first obtains a set of feature representatives, then inputs this set of features into the classifier and calls the `fit` function for training, resulting in the corresponding trained second classification network. During testing or direct use, the trained first classification network is directly used, with the first-feature target data used as input to the trained second classification network, outputting the second classification result.

[0163] This invention provides the training process for the initial first classification network, the initial second classification network, and the initial third classification network. See [link to documentation]. Figure 6 This is a flowchart illustrating the training process of a multi-classification method for breast cancer images provided in this embodiment of the invention. First characteristic sample data containing histogram features of the ADC (Antibody-Diagnostic Capacitor) is obtained by extracting feature sequences from the sample objects; wherein, the first characteristic sample data includes tumor proliferation burden parameters. Second image data of the sample objects is also subjected to feature extraction to obtain radiomics features containing combined sequences of multiple feature sequences; wherein, the combined sequences include 11 types. The first and second characteristic sample data are used as inputs to the initial second and third classification networks, respectively, and the initial second and third classification networks are trained separately. Furthermore, the first and second characteristic sample data are combined to obtain third characteristic sample data containing histogram features of the ADC and combined sequences of multiple feature sequences, which is then used as input to the first classification network for training. The initial first, second, and third classification networks are all multi-classification models, each containing one classification network, L sub-classification networks, and one classification network, respectively; L is the number of combined sequences.

[0164] The classification performance of the second, third, and first classification networks can be compared based on the best classification model among the 150 classification models included in the multi-classification model system. Performance metrics used to measure the model results include the area under the receiver operating characteristic curve (AUC), accuracy (ACC), sensitivity (SEN), and specificity (SPE). The AUC ranges from 0.5 to 1, while the other three metrics range from 0 to 1. The closer the AUC, accuracy, sensitivity, and specificity values ​​are to 1.0, the higher the realism and the better the classification performance. Table 7 shows the classification performance of the classification models with the highest AUCs among the first, second, and third classification networks. The value 'f' represents the performance of each classification network compared to the first classification network using the Wilcoxon signed-rank test. A p-value < 0.05 is considered statistically significant and is indicated in bold.

[0165] Table 7. Comparison of classification performance of the classification model with the largest AUC among the three classification networks.

[0166]

[0167] As shown in Table 7 above, using only the histogram index of the ADC sequence (second classification network) already yielded good differentiation performance between triple-negative and non-triple-negative breast cancer (AUC=0.808), confirming the effectiveness of using the whole tumor ADC histogram. When using only the radiomics features of the best-performing combination sequence (ADC+DWI+T2WI+DCE2), the classification performance was slightly improved compared to using only the ADC histogram index (AUC=0.818). However, when combining the ADC histogram index and the radiomics features of ADC+DWI+T2WI+DCE2, the classification performance was significantly improved (AUC=0.839), and the improvement was statistically significant.

[0168] Furthermore, this invention sorts the 150 models of the first classification network based on the total number of feature screenings and analyzes the top ten features by importance. See [link to relevant documentation]. Figure 7 This is a schematic diagram of feature selection in the first classification network provided in an embodiment of the present invention. From the above... Figure 7 As can be seen from the data, the first 10 features are all related to tumor proliferation burden (TPB), proving that TPB plays an important role in differentiating triple-negative breast cancer from non-triple-negative breast cancer.

[0169] Preferably, samples from triple-negative breast cancer were set as positive cases, and samples from non-triple-negative breast cancer were set as negative cases, thus differentiating the various classification networks. It can be represented as:

[0170] ,

[0171] in, The representative added up the sample serial numbers that were confirmed to be triple-negative breast cancer. The sequence number represents the i-th sample confirmed as triple-negative breast cancer, arranged in ascending order based on the probability score output by the classification model; To confirm the number of non-triple-negative breast cancers, This refers to the number of triple-negative breast cancers.

[0172] Preferably, the accuracy of each classification network can be expressed as:

[0173] .

[0174] Preferably, the sensitivity of each classification network can be expressed as:

[0175] .

[0176] Preferably, the specificity of each classification network can be expressed as:

[0177] .

[0178] Wherein, TN is the total number of case samples where both the true and predicted categories are negative, TP is the total number of case samples where both the true and predicted categories are positive, FN is the total number of case samples where the true category is positive but the predicted category is negative, and FP is the total number of case samples where the true category is negative but the predicted category is positive.

[0179] This invention uses the histogram of the entire tumor's apparent diffusion coefficient imaging sequence as the first characteristic sample data based on the histogram of the apparent diffusion coefficient imaging sequence. This allows for the investigation of the impact of tumor proliferation burden indicators on the classification performance of triple-negative and non-triple-negative breast cancer. This is because the form of the histogram based on the entire tumor's apparent diffusion coefficient imaging sequence can fully reflect the heterogeneity information of the tumor. Secondly, by combining all different MRI sequences, it is beneficial to complement multi-sequence information. The multiple second characteristic target data obtained based on radiomics features are fused with the first characteristic target data based on the histogram of the entire tumor's apparent diffusion coefficient imaging sequence and used as input to the first classification network. This allows for the construction of a high-performance and robust classification system, achieving better classification results for breast cancer images. This invention not only fully utilizes the overall apparent diffusion coefficient imaging sequence information of the tumor but also fuses imaging information from multiple MRI sequences for classification. The fused imaging information can better describe the characteristics of the tumor, thereby improving classification accuracy, reducing the workload of manual classification, and increasing efficiency.

[0180] See Figure 8 This is a schematic diagram of the structure of a multi-classification device for breast cancer images provided in an embodiment of the present invention, including: a first feature target data unit, a second feature target data unit, a third feature target data unit, and a first classification network unit.

[0181] The first characteristic target data unit is used to obtain the corresponding first characteristic target data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient according to the first image data of multiple target objects to be classified; wherein, the first image data includes multiple feature sequences, and the imaging sequence histogram of apparent diffusion coefficient includes tumor proliferation burden.

[0182] The second characteristic target data unit is used to combine the feature sequences of the first image data of multiple target objects according to the preset target combination sequence, and fuse the obtained combination sequence results according to the target objects to obtain the second characteristic target data based on radiomics features.

[0183] The process involves fusing the obtained combined sequences according to the target object to obtain second characteristic target data based on radiomics features. This includes fusing multiple feature sequences in the combined sequence according to the same target object to obtain second characteristic target data based on radiomics features. The combined sequence contains at least two feature sequences.

[0184] Specifically, the process of fusing multiple sequences in the combined sequence according to the same target object to obtain second characteristic target data based on radiomics features includes: obtaining fourth characteristic target data based on radiomics corresponding to the multiple target objects according to different feature sequences, and calculating the transformation matrix of the fourth characteristic target data corresponding to the multiple target objects according to the combined sequence; and fusing the fourth characteristic target data corresponding to the feature sequences belonging to the same target object according to the transformation matrix to obtain second characteristic target data based on radiomics features.

[0185] The process of combining the feature sequences of the first image data of multiple target objects according to a preset target combination sequence includes: calculating the AUC index of each sub-classification network under the corresponding combination sequence according to multiple trained sub-classification networks; selecting the combination sequence with the best AUC index as the target combination sequence; and combining the feature sequences of the first image data of multiple target objects according to the target combination sequence. Each sub-classification network contains one or more classifiers, and the sub-classification network with the best AUC index is selected as the trained third classification network.

[0186] Furthermore, the method includes: using the first feature target data as input to a trained second classification network to output a second classification result; wherein the second classification network comprises one or more classifiers. Also, using the second feature target data as input to a trained third classification network to output a third classification result; wherein the third classification network comprises one or more classifiers.

[0187] The third characteristic target data unit is used to combine the first characteristic target data and the second characteristic target data to obtain the third characteristic target data.

[0188] The first classification network unit is used to take the third feature target data as input to the trained first classification network so as to output a first classification result; wherein the first classification network includes one or more classifiers.

[0189] The multi-classification device for breast cancer images further includes: a second classification network unit and a third classification network unit; wherein, the second classification network is used to classify breast cancer images based on the first characteristic sample data transmitted by the first characteristic sample data unit, and obtains a second classification result; the third classification network is used to classify breast cancer images based on the second characteristic sample data transmitted by the second characteristic sample data unit, and obtains a third classification result.

[0190] It is worth noting that the first characteristic target data unit is used to obtain the first characteristic target data of the whole tumor epigenetic diffusion coefficient imaging sequence histogram based on the first image data to be classified, and transmits the obtained first characteristic target data to the second characteristic target data unit, the third characteristic target data unit, and the second classification network unit. According to a preset target combination sequence, after receiving the first characteristic target data, the second characteristic target data unit calculates the second characteristic target data based on radiomics features, and transmits the obtained second characteristic target data to the third characteristic target data unit and the third classification network unit. After receiving the first characteristic target data and the second characteristic target data, the third characteristic target data unit combines them to obtain the third characteristic target data, and transmits the third characteristic target data to the first classification network unit. After obtaining the third characteristic target data, the first characteristic target data, and the second characteristic target data, the first classification network unit, and the third classification network unit respectively perform breast cancer image classification, and output the first classification result, the second classification result, and the third classification result respectively.

[0191] Furthermore, the multi-classification device for breast cancer images further includes: a first training unit, a second training unit, a third training unit, a first feature sample data unit, a second feature sample data unit, and a third feature sample data unit; wherein, the first training unit, the second training unit, and the third training unit are respectively used to train the initial first classification network, the initial second classification network, and the initial multiple sub-classification networks to obtain the trained first classification network, the trained second classification network, and the trained third classification network, and transmit the obtained trained first classification network, trained second classification network, and trained third classification network to the first classification network unit, the second classification network unit, and the third classification network unit, respectively, so as to perform breast cancer classification and output the corresponding classification results respectively.

[0192] The system comprises three training units: a first characteristic sample data unit, a second characteristic sample data unit, and a third characteristic sample data unit. The first characteristic sample data unit obtains first characteristic sample data of the whole tumor epigenetic diffusion coefficient imaging sequence histogram based on the trained second image data, and transmits this first characteristic sample data to the second characteristic sample data unit, the third characteristic sample data unit, and the second training unit. The second characteristic sample data unit, upon receiving the first characteristic sample data, calculates second characteristic sample data based on radiomics features, and transmits this second characteristic sample data to the third characteristic sample data unit and the third training unit. The third characteristic sample data unit, upon receiving the first and second characteristic sample data, combines them to obtain the third characteristic sample data, which is then transmitted to the first training unit. The first, second, and third training units, upon obtaining the third characteristic sample data, the first characteristic sample data, and the second characteristic sample data, respectively, train on breast cancer images. The corresponding trained first, second, and third classification networks are then transmitted to the first, second, and third classification network units, respectively, for breast cancer classification, resulting in separate classification outputs.

[0193] Obtaining the trained first classification network includes steps S21~S23, specifically:

[0194] Step S21: Based on the second image data of multiple sample objects to be classified, obtain the corresponding first characteristic sample data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient; wherein, the second image data contains multiple feature sequences, and the imaging sequence histogram of the apparent diffusion coefficient includes the tumor proliferation burden.

[0195] Step S22: Combine various second image data in different ways to obtain multiple combination sequences, and fuse each combination sequence according to the sample object to obtain second characteristic sample data based on image omics features.

[0196] Step S23: Combine the first feature sample data and the second feature sample data, and train the initial first classification network according to the obtained third feature sample data to obtain the trained first classification network; wherein, the first classification network includes one or more classifiers.

[0197] Preferably, the feature sequence of the first image data includes an apparent diffusion coefficient imaging sequence, a diffusion-weighted imaging sequence, a T2-weighted imaging sequence, and an early sequence of dynamic enhancement scanning.

[0198] See Figure 9This is a schematic diagram of the structure of a computer-readable storage medium provided in this embodiment of the invention, including a memory 401 and a processor 402, wherein the computer-readable storage medium stores a computer program; wherein the computer program causes the computer to execute the multi-classification method for breast cancer images as described above.

[0199] This invention uses the histogram of the entire tumor's apparent diffusion coefficient imaging sequence as the first characteristic target data based on the histogram of the apparent diffusion coefficient imaging sequence. This allows for the investigation of the impact of tumor proliferation burden indicators on the classification performance of triple-negative and non-triple-negative breast cancer. This is because the form of the histogram based on the entire tumor's apparent diffusion coefficient imaging sequence can fully reflect the heterogeneity information of the tumor. Secondly, by combining all different sequences, it is beneficial to complement multi-sequence information. The multiple second characteristic target data obtained based on radiomics features are fused with the first characteristic target data based on the histogram of the entire tumor's apparent diffusion coefficient imaging sequence and used as input to the first classification network. This allows for the construction of a high-performance and robust classification system, achieving better classification results for breast cancer images. This invention not only fully utilizes the overall apparent diffusion coefficient imaging sequence information of the tumor but also fuses imaging information from multiple sequences for classification. The fused imaging information can better describe the characteristics of the tumor, thereby improving classification accuracy, reducing the workload of manual classification, and increasing efficiency.

[0200] Those skilled in the art will understand that embodiments of this application may also include computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0201] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0202] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0203] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0204] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multi-classification method for breast cancer images, characterized in that, include: Based on first image data of multiple target objects to be classified, corresponding first characteristic target data based on the histogram of the imaging sequence of the entire tumor apparent diffusion coefficient is obtained; wherein, the first image data contains multiple feature sequences, and the histogram of the imaging sequence of the apparent diffusion coefficient includes tumor proliferation burden; the tumor proliferation burden is the proportion of tumor volume with an overall tumor ADC value within a preset range to the total tumor volume; the preset range is greater than or equal to 1% and less than or equal to 25%. According to the preset target combination sequence, the feature sequences of the first image data of multiple target objects are combined, and the resulting combination sequence is fused according to the target objects to obtain the second characteristic target data based on image omics features. The step of combining the feature sequences of the first image data of multiple target objects according to the preset target combination sequence includes: calculating the AUC index of each sub-classification network under the corresponding combination sequence according to multiple trained sub-classification networks; selecting the combination sequence with the best AUC index as the target combination sequence; and combining the feature sequences of the first image data of multiple target objects according to the target combination sequence. Each subclassification network contains one or more classifiers, and the subclassification network with the best AUC index is selected as the trained third classification network. The step of fusing the obtained combined sequences according to the target object to obtain second characteristic target data based on radiomics features includes: fusing multiple feature sequences in the combined sequence according to the same target object to obtain second characteristic target data based on radiomics features; wherein the combined sequence contains at least two feature sequences; The feature sequence of the first image data includes an apparent diffusion coefficient imaging sequence, a diffusion-weighted imaging sequence, a T2-weighted imaging sequence, and an early sequence of dynamic enhancement scanning. The first feature target data and the second feature target data are combined to obtain the third feature target data. The third feature target data is used as the input of the trained first classification network to output the first classification result. The first classification network contains one or more classifiers.

2. The multi-classification method for breast cancer images as described in claim 1, characterized in that, The process of fusing multiple feature sequences in the combined sequence according to the same target object to obtain second characteristic target data based on radiomics features is as follows: Based on different feature sequences, the fourth characteristic target data corresponding to the multiple target objects based on radiomics are obtained sequentially, and the transformation matrix of the fourth characteristic target data corresponding to the multiple target objects is calculated based on the combined sequence. Based on the transformation matrix, the fourth characteristic target data corresponding to the feature sequences belonging to the same target object are fused to obtain the second characteristic target data based on radiomics features.

3. The multi-classification method for breast cancer images as described in claim 1, characterized in that, Also includes: The first feature target data is used as input to a trained second classification network to output a second classification result; wherein the second classification network contains one or more classifiers.

4. The multi-classification method for breast cancer images as described in claim 1, characterized in that, Also includes: The second feature target data is used as input to the trained third classification network to output the third classification result; wherein the third classification network contains one or more classifiers.

5. The multi-classification method for breast cancer images as described in claim 1, characterized in that, The trained first classification network includes: Based on the second image data of multiple sample objects to be classified, the corresponding first characteristic sample data based on the imaging sequence histogram of the whole tumor apparent diffusion coefficient is obtained; wherein, the second image data contains multiple feature sequences, and the apparent diffusion coefficient imaging sequence histogram includes tumor proliferation burden; Multiple combinations of second image data are obtained to form multiple combination sequences. Each combination sequence is then fused according to the sample object to obtain second characteristic sample data based on image omics features. The first feature sample data and the second feature sample data are combined, and the initial first classification network is trained based on the obtained third feature sample data to obtain the trained first classification network; wherein, the first classification network contains one or more classifiers.

6. A multi-classification device for breast cancer images, characterized in that, include: The first characteristic target data unit is used to obtain corresponding first characteristic target data based on the imaging sequence histogram of the entire tumor apparent diffusion coefficient, according to the first image data of multiple target objects to be classified; wherein, the first image data includes multiple feature sequences, and the apparent diffusion coefficient imaging sequence histogram includes tumor proliferation burden; the tumor proliferation burden is the proportion of tumor volume with an overall tumor ADC value within a preset range to the total tumor volume; the preset range is greater than or equal to 1% and less than or equal to 25%. The second characteristic target data unit is used to combine the feature sequences of the first image data of multiple target objects according to the preset target combination sequence, and fuse the obtained combination sequence results according to the target objects to obtain the second characteristic target data based on image omics features. The step of combining the feature sequences of the first image data of multiple target objects according to the preset target combination sequence includes: calculating the AUC index of each sub-classification network under the corresponding combination sequence according to multiple trained sub-classification networks; selecting the combination sequence with the best AUC index as the target combination sequence; and combining the feature sequences of the first image data of multiple target objects according to the target combination sequence. Each subclassification network contains one or more classifiers, and the subclassification network with the best AUC index is selected as the trained third classification network. The step of fusing the obtained combined sequences according to the target object to obtain second characteristic target data based on radiomics features includes: fusing multiple feature sequences in the combined sequence according to the same target object to obtain second characteristic target data based on radiomics features; wherein the combined sequence contains at least two feature sequences; The feature sequence of the first image data includes an apparent diffusion coefficient imaging sequence, a diffusion-weighted imaging sequence, a T2-weighted imaging sequence, and an early sequence of dynamic enhancement scanning. The third characteristic target data unit is used to combine the first characteristic target data and the second characteristic target data to obtain the third characteristic target data; The first classification network unit is used to take the third feature target data as input to the trained first classification network so as to output a first classification result; wherein the first classification network includes one or more classifiers.

7. A computer-readable storage medium storing a computer program, wherein, The computer program causes the computer to perform a multi-classification method for breast cancer images as described in any one of claims 1-5.

Images