A breast cancer molecular subtyping prediction method, system, medium, device and terminal

Abstract

The present application belongs to the technical field of medical image processing and computer-aided diagnosis, and discloses a breast cancer molecular typing prediction method, system, medium, equipment and terminal. The breast tumor and peritumoral region are extracted and used as samples by using a manual segmentation and boundary expansion method on multi-phase DCE-MRI images of breast cancer. The peritumoral and tumor images are respectively subjected to unsupervised matrix decomposition by using a mixture convex analysis-compartment modeling CAM-CM method, and a heterogenic region image related to tumor heterogeneity is obtained. The heterogenic region image is used as input data to train a built ViT (Vision Transformer) network with automatic search function. The trained network model is used for prediction on a test set, and a breast cancer molecular typing prediction task is completed. The present application effectively extracts pharmacokinetic heterogeneity in breast tumors and presents it in the form of images, retains more breast tumor heterogeneity characteristics, and has higher accuracy compared with traditional imageomics methods.

Description

Technical Field

[0001] This invention belongs to the fields of medical image processing and computer-aided diagnosis technology, and in particular relates to a method, system, medium, device and terminal for predicting molecular subtyping of breast cancer. Background Technology

[0002] Currently, breast cancer is the most common cancer among women worldwide, affecting one in eight to one in ten women at some point in their lives. Early screening and comprehensive treatment plans can effectively reduce breast cancer mortality. Breast cancer exhibits high heterogeneity, which negatively impacts treatment response, prognosis, and is associated with metastasis. Therefore, heterogeneity is one of the most important and clinically relevant areas of breast cancer research.

[0003] With the continuous advancement of clinical medicine, molecular subtyping systems for breast cancer have been gradually refined to assess tumor heterogeneity. Currently, the most mainstream classification method divides breast cancer into four types: Luminal A, Luminal B, HER2 overexpression, and triple-negative. Clinically, immunohistochemistry is used to obtain indicators such as estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2), and proliferating cell nuclear antigen (Ki-67) to determine the molecular subtype of breast cancer. In clinical diagnosis, obtaining the patient's immunohistochemical characteristics (ER, PR, HER2, Ki-67, etc.) to determine the molecular subtype of breast cancer usually requires surgical intervention, which can have adverse effects on the patient. Therefore, a non-invasive diagnostic method is urgently needed to determine the patient's molecular subtype.

[0004] With the development of medical imaging technology, non-invasive diagnostic techniques such as mammography, ultrasound imaging, and magnetic resonance imaging (MRI) are increasingly widely used in determining the molecular subtype of breast cancer. Compared to mammography and ultrasound imaging, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is radiation-free and has high resolution, clearly presenting the histological and morphological features of breast lesions. Furthermore, DCE-MRI, by injecting contrast agents and capturing multiple images, reveals the dynamic characteristics of breast lesions, and the extracted signal intensity-time curves are highly correlated with the heterogeneity of breast cancer. Most current research is based on breast MRI data, using computer diagnostic systems or matrix calculations to directly extract quantitative features related to breast cancer heterogeneity. However, most of these methods only focus on the morphology of breast tumors or features that are intuitively reflected in images, without exploring deeper heterogeneity features such as pixel-time curves that can only be obtained by combining multiple phases of DCE-MRI images. Moreover, there are few dynamic features that can be directly extracted from MRI images, which makes it difficult and challenging to build subsequent prediction models.

[0005] In recent years, research on predicting breast cancer molecular subtyping based on breast tumor heterogeneity using DCE-MRI has gradually emerged. However, the methods for extracting heterogeneous features and the redundancy between these features, along with traditional machine learning methods, limit the accuracy of computer-aided diagnosis. DCE-MRI imaging is complex, making it difficult to develop an effective method to extract heterogeneous features related to breast cancer molecular subtyping, and eliminating redundancy between these features is also challenging. Furthermore, traditional machine learning methods for predicting breast cancer molecular subtyping face difficulties due to the complexity of feature selection and the diversity of classifiers; choosing effective features and a suitable classifier remains a challenge. Therefore, there is an urgent need to design a new method for breast cancer molecular subtyping.

[0006] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0007] (1) In clinical diagnosis, to obtain the patient's immunohistochemical characteristics and thus determine the molecular subtype of breast cancer, surgical means must be used, but invasive methods can have adverse effects on patients.

[0008] (2) Traditional methods for determining the molecular subtype of breast cancer based on breast MRI data mostly focus on the morphology or features that are intuitively reflected in images of breast tumors. They fail to discover the deep-seated heterogeneity features such as pixel-time curves that can only be obtained by combining multiple phases of DCE-MRI images. Furthermore, there are few dynamic features that can be directly extracted from MRI images, which makes it difficult and challenging to establish subsequent prediction models.

[0009] (3) Existing methods for predicting the molecular subtype of breast cancer based on the heterogeneity of breast tumors using DCE-MRI cannot effectively obtain the heterogeneity features related to the molecular subtype of breast cancer. The extraction methods of heterogeneous features and the redundancy between heterogeneous features limit the accuracy of computer-aided diagnosis.

[0010] (4) Traditional machine learning methods for predicting molecular subtypes of breast cancer face challenges in selecting effective features and suitable classifiers due to the complexity of feature selection and the diversity of classifiers. Summary of the Invention

[0011] To address the problems existing in the prior art, this invention provides a method, system, medium, device, and terminal for predicting molecular subtyping of breast cancer, and particularly relates to a method, system, medium, device, and terminal for predicting molecular subtyping of breast cancer based on heteroproton region analysis.

[0012] This invention is implemented as follows: a method for predicting the molecular subtype of breast cancer. The method includes: extracting the breast tumor and peritumoral region from multi-phase DCE-MRI images of breast cancer using manual segmentation and boundary dilation methods, and using these as samples; performing unsupervised matrix factorization on the peritumoral and tumor images using Convex Analysis of Mixtures–Compartment Modeling (CAM-CM) to extract and transform heterogeneity information in breast cancer, obtaining sub-region images with different dynamic heterogeneity; using these heterogeneous sub-region images as input to train a ViT (Vision Transformer) network with automatic search function to extract deep features from the heterogeneous sub-region images; and performing predictions on a test set, automatically searching for different combinations of heterogeneous sub-regions, evaluating the heterogeneity of different heterogeneous sub-regions, and completing the task of predicting the molecular subtype of breast cancer.

[0013] Furthermore, the molecular subtyping method for breast cancer includes the following steps:

[0014] Step 1: Acquisition of breast tumor and peritumoral images: On the original breast DCE-MRI image, the outline of the breast tumor is manually drawn to generate a mask matrix within the tumor. The mask matrix is ​​then multiplied by the pixel matrix of the original image and processed to obtain the breast tumor image. Starting from the boundary of the breast tumor, the size is expanded outward by R pixels and the breast tumor region is removed to obtain the peritumoral image of the breast tumor. On each DCE-MRI image, K slices before and after the largest slice of the tumor are taken along the z-axis to obtain a total of 2K+1 slices, increasing the training data for different tumor images.

[0015] Step two involves performing pixel-time information processing on the obtained peritumoral and intratumoral images to obtain a pixel-time matrix, which is then fed into the CAM-CM algorithm. The CAM-CM algorithm consists of two modules: a nearest neighbor propagation clustering algorithm and a mixture convex analysis algorithm. The nearest neighbor propagation algorithm clusters the pixels in the given pixel-time matrix into O categories without initialization. The mixture convex analysis algorithm analyzes and separates the heterogeneity information contained in each pixel based on the cluster centers obtained from the clustering. After processing by the CAM-CM algorithm, J pixel matrices are obtained. Visualizing these J pixel matrices yields J heterogeneous region images.

[0016] Step 3: Input images of heteroprotic regions within and around the tumor into the constructed ViT network with automatic search function. The network automatically traverses all possible combinations of heteroprotic regions, evaluates the prediction effect of all combinations, and selects the best combination. Calculate the cross-entropy loss between the input sample labels and the prediction results obtained by the network as the loss function. Use the backpropagation algorithm to backpropagate the error layer by layer to the input layer, and update the network parameters using gradient descent until the network converges. Input the validation dataset into the training model to evaluate the model performance.

[0017] Step four: The test dataset is fed into the trained model, and different combinations of heteroprotic region images extracted based on the CAM-CM algorithm are used to predict the molecular subtype of breast cancer. The test results are voted on by multiple slices of a patient to obtain the final prediction result of the molecular subtype, and the network performance is evaluated.

[0018] Furthermore, the segmentation and processing of the tumor region and peritumoral region on the obtained raw DCE-MRI image of the breast in step one includes:

[0019] (1) In the obtained breast DCE-MRI images, the tumor region in the original image is manually segmented;

[0020] (2) Convert the original image and segmented tumor region mask image data generated by the software into a pixel matrix representation;

[0021] (3) In the intratumoral region mask pixel matrix, the dilation method is used to obtain the peritumoral region mask pixel matrix; the pixel matrix of the original image is multiplied with the mask pixel matrices of the intratumoral and peritumoral regions and visualized to obtain the images of the breast tumor and the intratumoral region;

[0022] (4) Use the method of traversing the slices to determine the largest slice of the breast tumor, select K slices above and below the largest slice to obtain a total of 2K+1 slices, and take the corresponding 2K+1 slices from the peritumoral image;

[0023] (5) Use multiple slice images of the processed tumor area and peritumoral area as input data for heteroproton region extraction.

[0024] Furthermore, the extraction of heteroproton regions in the tumor region and peritumoral region in step two includes:

[0025] (1) Obtain the tumor and peritumoral region of the patient’s multi-phase breast DCE-MRI images. Take 2K+1 slices for each phase and expand and merge the pixel matrix of the corresponding slice images in the multi-phase images into a pixel time matrix.

[0026] (2) Iterate through the columns of the pixel-time matrix to remove the background area in the image;

[0027] (3) The processed matrix is ​​fed into the CAM-CM algorithm for processing. The nearest neighbor propagation clustering algorithm is used to cluster the pixels in the input matrix without initialization. Then, the expectation-maximization algorithm is used to optimize the cluster centers obtained after clustering. The cluster centers are processed by the convex optimization algorithm based on the minimum boundary error criterion. Finally, the pixel matrix of the heterogeneous region is solved by least squares fitting.

[0028] (4) Insert the background part back into the heteroproton region matrix and restore it to the size of the original pixel time matrix;

[0029] (5) Extract each row of the restored matrix and perform visualization operations to obtain three images of the heteroproton region;

[0030] (6) Process the 2K+1 slices of all patients in sequence to obtain the heteroprotic region images corresponding to the 2K+1 slices of the tumor area and the peritumoral area of ​​each patient and form a dataset.

[0031] Furthermore, step three, feeding the training dataset into the constructed ViT network with automatic search capabilities to train the network model, includes:

[0032] (1) Two J heterogeneous region images containing the tumor region and the peritumoral region were used as training data and fed into the constructed ViT network with automatic search function;

[0033] (2) The 2J feature maps obtained by L transformer encoding modules are weighted and averaged with the encoding generated by the auto-search encoder to obtain the fused feature map;

[0034] (3) After passing the fused feature map through G transformer coding modules and multilayer perceptron layers, the molecular typing prediction result of the output network is obtained.

[0035] (4) Calculate the classification cross-entropy loss function between the network's prediction results and the true labels of the training data. Then, given the sample label y, the loss function of the network is calculated. i The loss function for i∈(1,…,N) is:

[0036]

[0037] Where N is the number of samples, p i This represents the probability that the network will predict a sample as a positive sample.

[0038] The error is propagated back to the input layer by layer using the backpropagation algorithm, and the network parameters are updated using the gradient descent method until the network converges to a local optimum.

[0039] (5) Feed the validation dataset into the training model and evaluate the model performance.

[0040] Furthermore, step four, testing the trained model on the test set, includes:

[0041] (1) Input the test dataset into the trained model to determine which molecular subtype of the breast tumor corresponding to the heteroprotic region image extracted by CAM-CM belongs to;

[0042] (2) Conduct patient-level voting on the test results, obtain molecular subtyping labels, and evaluate network performance.

[0043] Another object of the present invention is to provide a breast cancer molecular typing prediction system applying the aforementioned breast cancer molecular typing prediction method, the breast cancer molecular typing prediction system comprising:

[0044] The region extraction module is used to extract breast tumors and peritumoral regions from multi-phase DCE-MRI images of breast cancer using manual segmentation and boundary dilation methods and use them as samples.

[0045] The image decomposition module is used to perform unsupervised matrix decomposition on peritumoral and tumor images using the mixture convex analysis-compartment modeling method to obtain heterogeneous sub-region images related to tumor heterogeneity.

[0046] The ViT network construction module is used to construct a ViT network with automatic search function by taking images of heteroprotic regions with different pharmacokinetic heterogeneity characteristics as input and breast cancer molecular subtyping as output.

[0047] The breast cancer subtyping prediction module is used to feed the training dataset into the ViT network to calculate the loss function and train the network; the trained network model is then used to make predictions on the test set to predict the molecular subtyping of breast cancer.

[0048] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the breast cancer molecular subtyping prediction method.

[0049] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the breast cancer molecular subtyping prediction method.

[0050] Another objective of this invention is to provide an information data processing terminal for implementing the aforementioned breast cancer molecular subtyping prediction system.

[0051] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0052] First, addressing the technical problems existing in the prior art and the difficulty of solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:

[0053] The present invention provides a method for predicting the molecular subtype of breast cancer based on heteroprotic region analysis. In multi-phase DCE-MRI images of breast cancer, manual segmentation and boundary dilation are used to extract the breast tumor and peritumoral regions as samples. The Convex Analysis of Mixtures-Compartment Modeling (CAM-CM) algorithm is used to perform unsupervised matrix factorization on the peritumoral and tumor images, respectively, to obtain heteroprotic region images related to tumor heterogeneity. These heteroprotic region images are then used as input data to train a ViT (Vision Transformer) network with automatic search capabilities. The trained network model is then used to make predictions on a test set to complete the prediction of the molecular subtype of breast cancer.

[0054] This invention utilizes the CAM-CM algorithm to extract heterogeneous regions, which effectively reflect the heterogeneity characteristics among different molecular subtypes of breast tumors. Furthermore, the image-based approach includes deeper-level features. The invention employs an automatic search mechanism to effectively remove redundant features between heterogeneous region images, improving model efficiency and accuracy. Additionally, this invention leverages an effective deep learning network model for the classification of breast cancer molecular subtypes, providing an auxiliary diagnostic and treatment approach for breast cancer, avoiding the harm of surgery, and effectively reducing errors caused by manual diagnosis.

[0055] This invention fully utilizes the pharmacokinetic heterogeneity characteristics related to the molecular subtyping of breast cancer contained in breast DCE-MRI data, comprehensively considering information about the breast tumor and its surrounding areas. By using ViT, a method with automatic search capabilities, instead of traditional machine learning methods, redundant information in heterogeneous region images is removed, and deep-level features are extracted, improving the model's predictive accuracy. This invention can be used for the classification of molecular subtyping of breast cancer, providing auxiliary diagnostic and treatment tools for clinicians and serving as a reference for physicians.

[0056] Second, considering the technical solution as a whole or from a product perspective, the technical effects and advantages of the technical solution to be protected by this invention are specifically described as follows:

[0057] This invention effectively extracts the pharmacokinetic heterogeneity contained in breast tumors, and unlike the quantitative features of traditional methods, it presents it in an image-based form, preserving more of the heterogeneous characteristics of breast tumors. The constructed network can automatically search for different combinations of heterogeneous regions, achieving prediction of molecular subtyping of breast cancer with higher accuracy compared to traditional radiomics methods. This invention, while achieving prediction of molecular subtyping of breast cancer, provides a research approach for the pharmacokinetic heterogeneity of breast cancer, which has important reference value for subsequent research on applying the pharmacokinetic heterogeneity of breast cancer to breast cancer diagnosis.

[0058] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0059] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0060] Breast cancer is a common malignant tumor. The application of the molecular subtyping prediction method for breast cancer proposed in this invention is expected to provide an effective means for the early diagnosis, precision treatment, and personalized medicine of breast cancer. Expected benefits include, but are not limited to, the following: improving the accuracy and effectiveness of breast cancer diagnosis, reducing misdiagnosis and missed diagnosis rates, and improving treatment outcomes and survival rates; enabling personalized medicine by tailoring treatment plans to each patient based on their subtyping results, avoiding unnecessary treatments and drug side effects; increasing patient confidence and satisfaction with breast cancer treatment, and enhancing the hospital's brand value and reputation.

[0061] The commercial value of this invention mainly includes: a broad market prospect. Statistics show that over 2 million people worldwide are diagnosed with breast cancer each year, and breast cancer has the highest incidence and mortality rate among female malignant tumors. Therefore, molecular subtyping prediction methods for breast cancer have broad market prospects and potential commercial value; leading technology. Based on research into the heterogeneity of breast cancer and starting from molecular subtyping classification criteria, this invention has high accuracy and reliability, exhibiting significant technological advantages and a leading position compared to other breast cancer molecular subtyping methods; and cooperation opportunities. This invention can provide pharmaceutical companies and medical institutions with opportunities to collaborate, jointly develop and promote related products and services, and achieve a mutually beneficial business model.

[0062] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0063] This invention uniquely focuses on the heterogeneity of breast cancer, using molecular subtyping of breast cancer as a predictive task to assess the dynamic heterogeneity of breast cancer to varying degrees, a method not previously explored in research. Attached Figure Description

[0064] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a flowchart of the molecular subtyping prediction method for breast cancer provided in an embodiment of the present invention;

[0066] Figure 2 This is a ViT network structure diagram with automatic search function provided in the embodiments of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0068] To address the problems existing in the prior art, the present invention provides a method, system, medium, device and terminal for predicting molecular subtyping of breast cancer. The present invention will be described in detail below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, the breast cancer molecular subtyping prediction method provided in this embodiment of the invention includes the following steps:

[0070] S101, on multi-phase DCE-MRI (Dynamic Contrast-Enhanced Magnetic Resonance Imaging) images of breast cancer, the breast tumor and peritumoral region were extracted and used as samples using manual segmentation and boundary dilation methods;

[0071] S102, using the mixture convex analysis-compartment modeling method, unsupervised matrix decomposition was performed on the peritumoral and tumor images to obtain heterogeneous sub-region images related to tumor heterogeneity;

[0072] S103 uses images of heterogeneous regions with different pharmacokinetic heterogeneity as input and breast cancer molecular subtyping as output to construct a ViT (Vision Transformer) network with automatic search function.

[0073] S104: Feed the training dataset into the ViT network, calculate the loss function and train the network; then use the trained network model to make predictions on the test set to complete the task of predicting molecular subtypes of breast cancer.

[0074] As a preferred embodiment, the breast cancer molecular subtyping prediction method based on heteroproton region analysis provided by this invention specifically includes the following steps:

[0075] (1) Acquisition of breast tumor and peritumoral images: On the original breast DCE-MRI image, the outline of the breast tumor is manually drawn to generate a mask matrix inside the breast tumor. After multiplying the matrix with the pixel matrix of the original image, the image is processed to obtain the breast tumor image. Starting from the boundary of the breast tumor, expand outward by R pixels and remove the breast tumor area to obtain the peritumoral image of the breast tumor. On each phase of the DCE-MRI image, along the z-axis, take K slices before and after the slice with the largest tumor to obtain a total of 2K+1 slices to increase the training data of different tumor images.

[0076] The segmentation and processing of the tumor region and peritumoral region on the obtained raw DCE-MRI images of the breast are performed according to the following steps:

[0077] (1.1) In the obtained breast DCE-MRI images, the tumor region in the original image is manually segmented;

[0078] (1.2) Convert the original image and segmented tumor region mask image data generated by the software into a pixel matrix representation;

[0079] (1.3) In the intratumoral region mask pixel matrix, the peritumoral region mask pixel matrix is ​​obtained by dilation. The pixel matrix of the original image is multiplied with the mask pixel matrices of the intratumoral and peritumoral regions and visualized to obtain the images of the breast tumor and the intratumoral region.

[0080] (1.4) Use the method of traversing the slices to find the slice with the largest breast tumor, select K slices above and below the largest slice, for a total of 2K+1 slices, and take the corresponding 2K+1 slices from the peritumoral image.

[0081] (1.5) Multiple slice images of the processed tumor region and peritumoral region are used as input data for heteroproton region extraction.

[0082] (2) After performing pixel-time information processing on the obtained peritumoral and intratumoral images, a pixel-time matrix is ​​obtained and fed into the CAM-CM (Convex Analysis of Mixtures-Compartment Modeling) algorithm. This algorithm mainly consists of two modules: nearest neighbor propagation clustering algorithm and mixture convex analysis algorithm. The nearest neighbor propagation algorithm clusters the pixels in the given pixel-time matrix into O categories without initialization. The mixture convex analysis algorithm can analyze the heterogeneity information contained in each pixel based on the cluster center results obtained by clustering and separate them. After processing by the CAM-CM algorithm, J pixel matrices are obtained, and after visualizing them, J heterogeneous region images are obtained.

[0083] The extraction of heteroprotic regions from the tumor area and the peritumoral region is performed according to the following steps:

[0084] (2.1) Following the same operation in the first step, the tumor and peritumoral region on the breast DCE-MRI images of a patient in multiple phases were obtained (2K+1 slices were still taken in each phase). The pixel matrix of the corresponding slice images in the multiple phase images was expanded and merged into a pixel time matrix.

[0085] (2.2) Iterate through the columns of the pixel-time matrix to remove the background region in the image;

[0086] (2.3) The processed matrix is ​​fed into the CAM-CM algorithm for processing. The algorithm first uses the nearest neighbor propagation clustering algorithm to cluster the pixels in the input matrix without initialization. Then, the expectation-maximization algorithm is used to optimize the cluster centers obtained after clustering. Subsequently, the convex optimization algorithm based on the minimum boundary error criterion is used to further process the cluster centers. Finally, the least squares fitting is used to solve the pixel matrix of the heteroproton region.

[0087] (2.4) Insert the background portion back into the heteroproton region matrix to restore it to the size of the original pixel time matrix;

[0088] (2.5) Extract each row of the restored matrix and perform visualization operations to obtain three images of the heteroproton region;

[0089] (2.6) Perform the above processing on 2K+1 slices of all patients to obtain a dataset consisting of heteroprotic region images on 2K+1 slices of the tumor area and peritumoral area of ​​each patient.

[0090] (3) Input images of heteroprotic regions within and around the tumor into a ViT network with automatic search function, automatically traverse all possible combinations of heteroprotic regions, evaluate the prediction effect of all combinations, and select the best combination; calculate the cross-entropy loss between the label of the input sample and the prediction result obtained by the network as the loss function; use the backpropagation algorithm to backpropagate the error layer by layer to the input layer, and update the network parameters using the gradient descent method until the network converges; input the validation dataset into the training model and evaluate the model performance.

[0091] The ViT network structure with automatic search function provided in this embodiment of the invention is as follows: Figure 2 As shown.

[0092] The training dataset is fed into the constructed ViT network with automatic search capabilities to train the network model, and the following steps are performed:

[0093] (3.1) A total of 2J heteroprotic region images containing the tumor region and the peritumoral region were fed into the constructed ViT network with automatic search function as training data;

[0094] (3.2) The 2J feature maps obtained by passing through L transformer encoding modules are weighted and averaged with the encoding generated by the auto-search encoder to obtain the fused feature map;

[0095] (3.3) The fused feature map is passed through G transformer coding modules and multilayer perceptron layers to output the molecular subtyping prediction result of the network;

[0096] (3.4) Calculate the classification cross-entropy loss function between the network's prediction results and the true labels of the training data as the network's loss function. Use the backpropagation algorithm to propagate the error back to the input layer layer by layer. Update the network parameters using the gradient descent method until the network converges to a local optimum.

[0097] (3.5) Feed the validation dataset into the training model and evaluate the model performance.

[0098] (4) The test dataset is fed into the trained model, and different combinations of heteroprotic region images extracted based on the CAM-CM algorithm are used to predict the molecular subtype of breast cancer. The test results are voted on by multiple slices of a patient to obtain the final prediction result of the molecular subtype and evaluate the network performance.

[0099] The trained model is then tested on the test set, following these steps:

[0100] (4.1) Input the test dataset into the trained model and determine which molecular subtype of the breast tumor corresponding to the heteroprotic region image extracted by CAM-CM belongs to;

[0101] (4.2) Vote on the patient level of the test results, obtain molecular subtyping labels, and evaluate network performance.

[0102] The breast cancer molecular typing system provided in this embodiment of the invention includes:

[0103] The region extraction module is used to extract breast tumors and peritumoral regions from multi-phase DCE-MRI images of breast cancer using manual segmentation and boundary dilation methods and use them as samples.

[0104] The image decomposition module is used to perform unsupervised matrix decomposition on peritumoral and tumor images using the mixture convex analysis-compartment modeling method to obtain heterogeneous sub-region images related to tumor heterogeneity.

[0105] The ViT network construction module is used to construct a ViT network with automatic search function by taking images of heteroprotic regions with different pharmacokinetic heterogeneity characteristics as input and breast cancer molecular subtyping as output.

[0106] The breast cancer subtyping prediction module is used to feed the training dataset into the ViT network to calculate the loss function and train the network; the trained network model is then used to make predictions on the test set to predict the molecular subtyping of breast cancer.

[0107] To demonstrate the inventiveness and technical value of the present invention, this section provides specific product or related technology application examples of the claimed technical solutions. The breast cancer molecular subtyping prediction method provided by this invention can be used for the auxiliary diagnosis of breast cancer molecular subtyping in patients, helping doctors to develop more personalized treatment plans.

[0108] Example 1: A patient discovered a suspicious lump in her breast during breast cancer screening. Further examination and diagnosis confirmed breast cancer. The doctor input the patient's breast DCE-MRI images into the system, which then provides a molecular subtype prediction of the breast cancer. Based on this result, the doctor can develop an appropriate treatment plan.

[0109] Example 2: Two radiologists disagreed on the molecular subtyping of breast cancer based on a patient's breast DCE-MRI images, and a surgical biopsy of the patient was not possible in a short time. By feeding the DCE-MRI images into this invention, the predicted molecular subtyping of the patient's breast cancer can provide a reference for resolving the disagreement between the doctors.

[0110] As a preferred embodiment, the breast cancer molecular subtyping prediction method based on heteroproton region analysis provided by this invention specifically includes the following steps:

[0111] (1) On multi-phase DCE-MRI images of breast cancer, the breast tumor and peritumoral region were extracted as samples using manual segmentation and boundary dilation methods. The specific process is as follows:

[0112] (1a) In this embodiment of the invention, five-stage DCE-MRI images of 185 patients were selected, of which the number of patients with the two molecular subtypes were 132 and 53, respectively. The training set and the test set were divided in a ratio of 8:2. The test set was also used for the validation process, but did not participate in the selection of the model.

[0113] (1b) On the third-stage breast DCE-MRI images of each patient, the breast tumor portion was manually segmented using MITK software and a mask image matrix was generated, where the size of the original image and the mask matrix were both 512×512×106.

[0114] (1c) Based on the tumor mask matrix, expand outward from the tumor boundary. The size of the expansion kernel is 3×3. Subtract the matrix before expansion from the expanded matrix to obtain the mask matrix around the tumor, which is 512×512×106.

[0115] (1d) Multiply the five-stage DCE-MRI images with the tumor mask matrix and the peritumoral mask matrix respectively to obtain the pixel matrix of the tumor region and peritumoral region of each patient in the five stages, with a size of 512×512×106.

[0116] (1e) Traverse the third dimension of the pixel matrix of the tumor, find the slice with the largest tumor, take two slices before and after the slice, for a total of five slices, and at the same time take the corresponding slice in the pixel matrix around the tumor. Each slice is 512×512 in size. Then resize both the image around the tumor and the tumor image to a size of 150×150.

[0117] (2) The CAM-CM algorithm was used to perform unsupervised matrix decomposition on the peritumoral and tumor images to obtain a total of six heterogeneous sub-region images related to tumor heterogeneity. The specific process is as follows:

[0118] (2a) Expand the pixel matrix of the corresponding slice image in the five phase images of each patient from a two-dimensional 150×150 matrix to a one-dimensional 1×22500 matrix, and stack them in rows according to the phase number to form a pixel-time matrix of size 5×22500.

[0119] (2b) Iterate through the columns of the pixel-time matrix, remove those columns with all 0 elements, i.e. the background area in the image, and save the numbers of these columns with all 0 elements. The size of the matrix after removing 0 is 5×P.

[0120] (2c) The processed matrix is ​​fed into the CAM-CM algorithm for further processing: The CAM-CM algorithm first normalizes the pixel value range of all pixels in the input pixel time matrix to the range [0, 1]; secondly, it uses the nearest neighbor propagation clustering algorithm to cluster the P pixels in the 5×P pixel time matrix into L classes, where the specific value of L is automatically determined by the nearest neighbor propagation clustering algorithm; the expectation-maximization algorithm is used to process the cluster center index of the L classes, optimize the number of clusters, and obtain the optimized new cluster centers; the new cluster centers and the number of artificially set heterogeneous sub-regions (3) are used as input, and a convex optimization algorithm based on the minimum boundary error criterion is used to further obtain the composition coefficient C of the three heterogeneous sub-regions. f C s and C p Finally, the non-negative least squares optimization algorithm is used to solve for the pixel values ​​in the three heteroprotic region images based on the heteroprotic region coefficients and the pixel values ​​in the input pixel time matrix.

[0121] (2d) Insert the zero elements back into the heteroproton region matrix according to the number of the column of zeros, and restore the matrix size to 3×22500;

[0122] (2e) Take out each row of the restored matrix, resize it into a 150×150 matrix, and perform visualization operations to obtain three images of the heteroproton region.

[0123] (3) Using six heterogeneous region images corresponding to the peritumoral and intratumoral images as input data, a ViT network with automatic search function was trained. The structure of the network is as follows: Figure 2 As shown, the specific process is as follows:

[0124] (3a) To meet the network's requirements for input image size, the input images are first resized to 256×256 before entering the network. The network needs to input six images simultaneously, namely three heterogeneous region images each from the peritumoral and intratumoral regions. After the images enter the network, they are first divided into 16×16 image blocks. After the image blocks are unfolded into one dimension, each image block is mapped to a high-dimensional space (the dimension is set to 1024 in this example) using block embedding and position embedding, and then concatenated together. A learnable classification head is added to each image block, and finally the embedding vectors corresponding to the six input images are obtained.

[0125] (3b) Next, the embedding vector corresponding to each input image is fed into the constructed encoder module 1. To reduce the network depth and speed up the operation, encoder module 1 contains only one transformer encoder module, which consists of two layers of normalization operations, a multi-head self-attention module, a multilayer perceptron module, and two residual connection structures. After processing by transformer block 1, a total of six feature maps F1-F6 corresponding to different input images are obtained;

[0126] (3c) NAS-Encoder (Network Architecture Search-Encoder), also known as an automatic search encoder, is an encoder-like structure that automatically generates a six-bit code, where each bit consists of either 0 or 1. It uses a method that can randomly generate a six-bit non-repeating sequence of 0s and 1s as the basic architecture of the automatic search encoder, eliminating the case where the six-bit code generated by the automatic search encoder is all 0s. After the first two steps of processing, six feature maps F1-F6 are obtained from the six input images. These six feature maps are then multiplied by the six-bit code (C1-C6) generated by the automatic search encoder and then averaged and fused.

[0127] F=(F1*C1+…+F6*C6) / SUM(C1,…,C6)

[0128] Where SUM(C1,…,C6) represents the number of 1s in the six-bit code. The feature maps F1-F6, which are the product of the six-bit codes C1-C6, correspond to the following input images in sequence: the intratumoral plasma image, the peritumoral plasma image, the intratumoral slow-flow dynamics image, the peritumoral slow-flow dynamics image, the intratumoral fast-flow dynamics image, and the peritumoral fast-flow dynamics image. An automatic search encoder can automatically obtain the performance of different combinations of feature maps in the breast cancer molecular subtyping task, thereby evaluating the importance of different heteroproton region images corresponding to different feature maps for breast cancer molecular subtyping prediction.

[0129] (3d) After the average fusion of feature maps and the search process of the automatic search encoder, the fused feature map F is obtained. Since the output vector of the transformer encoder module is equal to the input vector, multiple transformer encoders can be stacked in practice. To improve the prediction performance of the network, deeper features are extracted from the fused feature map while taking into account the network's computational speed. The constructed encoder module 2 contains two identical transformer encoder modules. The dimension of the input feature map F remains unchanged after passing through encoder module 2. Finally, the feature map obtained after processing by encoder module 2 is fed into the multilayer perceptron classification head, which consists of a layer normalization module and a linear layer, to obtain the final prediction output.

[0130] (3e) Calculate the cross-entropy loss between the input sample label and the network output prediction result as the loss function, given the sample label as y. i For i∈(1,…,N), the loss function is:

[0131]

[0132] Where N is the number of samples, p i This represents the probability that the network predicts the sample as a positive sample.

[0133] (3f) Use the backpropagation algorithm to propagate the error back to the input layer layer by layer, and use the gradient descent method to update the network parameters until the network converges to a local optimum.

[0134] (3g) Feed the validation dataset into the training model and evaluate the model performance.

[0135] (4) The trained network model is used to make predictions on the test set to complete the task of predicting the molecular subtype of breast cancer. The specific process is as follows:

[0136] (4a) Input the test dataset into the trained model and determine which molecular subtype of the breast tumor corresponding to the heterogeneous region image extracted by CAM-CM belongs to;

[0137] (4b) Vote on the patient level of the test results, obtain molecular subtyping labels, and evaluate network performance.

[0138] The evaluation criteria for the breast cancer molecular subtyping method provided in this invention include receiver operating characteristic (ROC) curves and area under the curve (AUC), accuracy (ACC), sensitivity (SEN), and specificity (SPE). These indicators are used to evaluate the predictive effect of breast cancer molecular subtyping. The values ​​of these indicators are all between [0, 1], with values ​​closer to 1 indicating better model predictive performance. The test results on the test data are as follows: in multiple test experiments, AUC was consistently between [0.65, 0.68], ACC between [0.66, 0.76], SEN between [0.66, 0.78], and SPE between [0.70, 0.78]. The experimental results show that this invention achieves good results, and it focuses on the heterogeneity of breast cancer. Compared with other diagnostic methods for breast cancer molecular subtyping, the heterogeneity features extracted by this invention are more intuitive and effective, providing insights for other computer-aided diagnoses of breast cancer.

[0139] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0140] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for predicting the molecular subtype of breast cancer, characterized in that, The method for predicting molecular subtyping of breast cancer includes the following steps: Step 1: Acquisition of breast tumor and peritumoral images: On the original breast DCE-MRI image, the outline of the breast tumor is manually drawn to generate a mask matrix within the tumor. The mask matrix is ​​then multiplied by the pixel matrix of the original image and processed to obtain the breast tumor image. Starting from the boundary of the breast tumor, the size is expanded outward by R pixels and the breast tumor region is removed to obtain the peritumoral image of the breast tumor. On each DCE-MRI image, K slices before and after the largest slice of the tumor are taken along the z-axis to obtain a total of 2K+1 slices, increasing the training data for different tumor images. Step two involves processing the obtained peritumoral and intratumoral images with pixel-time information to obtain a pixel-time matrix, which is then fed into the Mixture Convex Analysis-Compartment Modeling (CAM-CM) algorithm. The CAM-CM algorithm consists of two modules: a nearest neighbor propagation clustering algorithm and a mixture convex analysis algorithm. The nearest neighbor propagation algorithm clusters the pixels in the given pixel-time matrix into O categories without initialization. The mixture convex analysis algorithm analyzes and separates the heterogeneity information contained in each pixel based on the cluster centers obtained from the clustering. After processing by the CAM-CM algorithm, J pixel matrices are obtained. Visualizing these J pixel matrices yields J heterogeneous region images. Step 3: Images of heterogeneous regions within and around the tumor are fed into a ViT network with an automatic search function. The automatic search function of the ViT network is implemented through an automatic search encoder (NAS-Encoder), which automatically traverses all possible combinations of heterogeneous regions, evaluates the prediction effect of all combinations, and selects the best combination. The cross-entropy loss between the input sample labels and the prediction results obtained by the network is calculated as the loss function. The error is backpropagated layer by layer to the input layer using the backpropagation algorithm, and the network parameters are updated using the gradient descent method until the network converges. The validation dataset is fed into the training model to evaluate the model performance. Step four: The test dataset is fed into the trained model, and different combinations of heteroprotic region images extracted based on the CAM-CM algorithm are used to predict the molecular subtype of breast cancer. The test results are voted on by multiple slices of a patient to obtain the final prediction result of the molecular subtype, and the network performance is evaluated.

2. The method for predicting molecular subtyping of breast cancer as described in claim 1, characterized in that, The specific steps in Step One include: (1) In the obtained breast DCE-MRI images, the tumor region in the original image is manually segmented; (2) Convert the original image and segmented tumor region mask image data generated by the software into a pixel matrix representation; (3) In the intratumoral region mask pixel matrix, the dilation method is used to obtain the peritumoral region mask pixel matrix; the pixel matrix of the original image is multiplied with the mask pixel matrix of the intratumoral and peritumoral regions and visualized to obtain the image of the breast tumor and the intratumoral region; (4) Use the method of traversing the slices to determine the largest slice of the breast tumor, select K slices above and below the largest slice to obtain a total of 2K+1 slices, and take the corresponding 2K+1 slices from the peritumoral image; (5) Use multiple slice images of the processed tumor area and peritumoral area as input data for heteroproton region extraction.

3. The method for predicting molecular subtyping of breast cancer as described in claim 1, characterized in that, Step two includes the following steps: (1) Obtain the tumor and peritumoral region of the patient’s multi-phase breast DCE-MRI images. Take 2K+1 slices for each phase and merge the corresponding slice image pixel matrix in the multi-phase images into a pixel time matrix. (2) Iterate through the columns of the pixel-time matrix to remove the background region in the image; (3) The processed matrix is ​​fed into the CAM-CM algorithm for processing. The nearest neighbor propagation clustering algorithm is used to cluster the pixels in the input matrix without initialization. Then, the expectation-maximization algorithm is used to optimize the cluster centers obtained after clustering. The cluster centers are processed using the convex optimization algorithm based on the minimum boundary error criterion. Finally, the pixel matrix of the heterogeneous region is solved by least squares fitting. (4) Insert the background part back into the heteroproton region matrix to restore the original pixel time matrix size; (5) Extract each row of the restored matrix and perform visualization operations to obtain three images of the heteroproton region; (6) Process the 2K+1 slices of all patients in sequence to obtain the heteroproton region images corresponding to the 2K+1 slices of the tumor area and the peritumoral area of ​​each patient and form a dataset.

4. The method for predicting molecular subtyping of breast cancer as described in claim 1, characterized in that, Step three includes the following steps: (1) Two J heterogeneous region images containing the tumor region and the peritumoral region were used as training data and fed into the constructed ViT network with automatic search function; (2) The 2J feature maps obtained by passing through L transformer encoding modules are weighted and averaged with the encoding generated by the auto-search encoder to obtain the fused feature map; (3) After passing the fused feature map through G transformer encoding modules and multilayer perceptron layers, the molecular typing prediction result of the output network is obtained; (4) Calculate the classification cross-entropy loss function between the network's prediction results and the true labels of the training data as the network loss function. Then, given the sample labels, the loss function is... The loss function under this condition is: ； in, N For the number of samples, p i This represents the probability that the network will predict a sample as a positive sample. The error is propagated back to the input layer by layer using the backpropagation algorithm, and the network parameters are updated using the gradient descent method until the network converges to a local optimum. (5) Feed the validation dataset into the training model and evaluate the model performance.

5. The method for predicting molecular subtyping of breast cancer as described in claim 1, characterized in that, Step four includes the following specific steps: (1) Input the test dataset into the trained model to determine the molecular subtype of the breast tumor corresponding to the heteroprotic region image extracted by CAM-CM; (2) Vote on the patient level of the test results, obtain molecular subtyping labels, and evaluate network performance.

6. A breast cancer molecular typing prediction system applying the breast cancer molecular typing prediction method as described in any one of claims 1 to 5, characterized in that, Breast cancer molecular subtyping prediction systems include: The region extraction module is used to extract breast tumors and peritumoral regions from multi-phase DCE-MRI images of breast cancer using manual segmentation and boundary dilation methods and use them as samples. The image decomposition module is used to perform unsupervised matrix decomposition on peritumoral and tumor images using the mixture convex analysis-compartment modeling method to obtain heterogeneous sub-region images related to tumor heterogeneity. The ViT network construction module is used to construct a ViT network with automatic search function by taking images of heteroprotic regions with different pharmacokinetic heterogeneity characteristics as input and breast cancer molecular subtyping as output. The breast cancer subtyping prediction module is used to feed the training dataset into the ViT network to calculate the loss function and train the network; the trained network model is then used to make predictions on the test set to predict the molecular subtyping of breast cancer.

7. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the breast cancer molecular subtyping prediction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the breast cancer molecular subtyping prediction method as described in any one of claims 1 to 5.

9. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the breast cancer molecular subtyping prediction system as described in claim 6.

Images