A deep learning-based breast cancer risk prediction system and method
By combining FFDM and DBT images and clinical information, a deep learning-based breast cancer risk prediction system was developed, achieving deep fusion of multimodal features and risk stratification. This solves the problem of insufficient utilization of image features in existing technologies, improves the accuracy and stability of breast cancer risk prediction, and supports personalized screening.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANFANG HOSPITAL OF SOUTHERN MEDICAL UNIV
- Filing Date
- 2025-06-17
- Publication Date
- 2026-05-15
AI Technical Summary
Existing breast cancer risk prediction models do not make sufficient use of imaging features, lack multimodal feature fusion, make it difficult to achieve accurate prediction and personalized screening, and lack a scientific risk level classification mechanism.
A breast cancer risk prediction system based on deep learning was adopted. Through automatic segmentation of the pectoralis major muscle, deep fusion of multimodal information and risk grading mechanism, and using FFDM and DBT images and clinical information, a breast cancer risk prediction model was constructed, including an acquisition module, a segmentation module, a feature extraction module, a feature fusion module and a risk prediction module. The CLIP model and X-tile software were used for feature extraction and grading.
It improves the accuracy and stability of breast cancer risk prediction, achieves deep fusion of multimodal imaging features and personalized risk level classification, and supports scientific risk stratification management and screening pathways.
Smart Images
Figure CN120708896B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image processing and artificial intelligence-assisted technology, specifically relating to a breast cancer risk prediction system and method based on deep learning. Background Technology
[0002] The development of breast cancer risk prediction models is an important research direction in the field of medical imaging, especially in the context of the national "Two Cancers" screening program. Establishing a precise prediction system suitable for the characteristics of Chinese women's breasts has significant clinical value. Current prediction methods mainly face three technical bottlenecks:
[0003] First, existing models have significant limitations in utilizing imaging features. Traditional prediction methods rely excessively on the subjects' basic clinical information and fail to fully explore the pathological features contained in breast images, resulting in the models' specificity and sensitivity being insufficient to meet clinical needs.
[0004] Secondly, there are technical limitations in the application of image modalities. Currently, mainstream prediction models are mainly constructed based on two-dimensional full-view digital mammography (FFDM) images, while insufficient utilization of feature information from three-dimensional digital breast tomography (DBT). This single-modal analysis method limits further improvement in model performance and also affects the stability of prediction results.
[0005] Furthermore, current technologies have not yet achieved breakthroughs in multimodal feature fusion. Traditional methods such as Logistic regression struggle to effectively integrate two-dimensional and three-dimensional imaging features with clinical information, lacking a deep feature fusion mechanism. This technological deficiency prevents models from fully reflecting the complex characteristics of breast lesions, thus limiting the accuracy of personalized screening programs.
[0006] Furthermore, current research and practice regarding breast cancer risk classification remain weak. Most risk prediction models only output a continuous risk score, lacking a systematic mechanism to scientifically stratify the prediction results and correlate them with clinical intervention plans. This makes it difficult for the models to directly serve clinical decision-making and hinders the implementation of stratified management and personalized screening pathways.
[0007] Therefore, developing a breast cancer risk prediction model based on deep learning, integrating multimodal imaging and clinical information, and further combining it with a scientific and reasonable risk level classification mechanism, is expected to significantly improve the accuracy and stability of the prediction model. Summary of the Invention
[0008] To address the problems existing in the prior art, this invention provides a breast cancer risk prediction system and method based on deep learning. It aims to effectively improve the accuracy and stability of breast cancer risk prediction by automatically segmenting the pectoralis major muscle, deeply fusing multimodal information, and implementing a risk grading mechanism, thereby breaking through the aforementioned technical bottlenecks in the current field of breast cancer risk prediction.
[0009] To achieve the above objectives, the present invention provides the following solution:
[0010] A breast cancer risk prediction system based on deep learning, the system comprising: a data acquisition module, a segmentation module, a feature extraction module, a feature fusion module, a boundary point acquisition module, and a risk prediction module;
[0011] The acquisition module is used to acquire FFDM images and DBT images of the subject's breast and the subject's clinical information, and to perform preprocessing.
[0012] The segmentation module is used to segment the pectoralis major muscle from the preprocessed FFDM image and DBT image to obtain the segmented FFDM image and the segmented DBT image.
[0013] The feature extraction module is used to extract structured features from the FFDM image after segmentation of the pectoralis major muscle, the DBT image after segmentation of the pectoralis major muscle, and the clinical information using a deep learning-based network model, to obtain FFDM image features, DBT image features, and clinical information features.
[0014] The feature fusion module is used to perform nonlinear transformation and fusion of the FFDM image features, the DBT image features and the clinical information features using MLP to obtain fused features, and to obtain the breast cancer risk score of the subject based on the fused features;
[0015] The boundary point acquisition module is used to segment the breast cancer risk score based on X-tile software to obtain the breast cancer risk boundary point.
[0016] The risk prediction module is used to classify the subject's breast cancer risk level based on the breast cancer risk cutoff point, thereby predicting the subject's breast cancer risk.
[0017] Preferably, the clinical information in the acquisition module includes: the subject's previous number of breast biopsies, reproductive history, age of menarche, age of first delivery, breastfeeding history, whether or not menopause has occurred, the subject's family history of breast cancer in first- and second-degree relatives, and the subject's clinical physician's palpation information.
[0018] Preferably, the segmentation module includes: a boundary line unit, a segmentation matrix unit, a first segmented image unit, an expansion and scaling unit, and a second segmented image unit;
[0019] The boundary line unit is used to obtain the boundary line of the pectoralis major muscle and breast contour in the preprocessed FFDM image based on the LIBRA algorithm.
[0020] The segmentation matrix unit is used to obtain the segmentation matrix of the pectoralis major muscle based on the boundary line and using the boundary threshold method.
[0021] The first segmentation image unit is used to multiply the preprocessed FFDM image with the segmentation matrix to obtain the segmented FFDM image of the pectoralis major muscle.
[0022] The expansion scaling unit is used to expand the segmentation matrix to the size of the preprocessed DBT image, and to interpolate and scale the expanded segmentation matrix using cubic interpolation.
[0023] The second segmentation image unit is used to convolve the preprocessed DBT slices with the interpolated and scaled segmentation matrix to obtain a DBT image after segmenting the pectoralis major muscle.
[0024] Preferably, the feature extraction module includes: a model building unit, a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit;
[0025] The model building unit is used to build a CLIP model based on deep learning;
[0026] The first feature extraction unit is used to input the FFDM image after segmenting the pectoralis major muscle into the image encoder of the CLIP model, extract the first feature vector through the model.get_image_features method and perform normalization processing to obtain the FFDM image features;
[0027] The second feature extraction unit is used to input the DBT image after segmenting the pectoralis major muscle into the image encoder, extract the second feature vector, and aggregate the second feature vector using the torch.mean method to obtain the DBT image features;
[0028] The third feature extraction unit is used to input the clinical information into the text encoder of the CLIP model, extract the third feature vector through the model.get_text_features method and perform normalization processing to obtain the clinical information features.
[0029] This invention also provides a deep learning-based method for predicting breast cancer risk, the method comprising:
[0030] S1. Collect FFDM images and DBT images of the subject's breast and the subject's clinical information, and perform preprocessing.
[0031] S2. Perform pectoralis major muscle segmentation on the preprocessed FFDM and DBT images to obtain FFDM and DBT images after pectoralis major muscle segmentation.
[0032] S3. Using a deep learning-based network model, extract the structured features from the FFDM image after segmenting the pectoralis major muscle, the DBT image after segmenting the pectoralis major muscle, and the clinical information, respectively, to obtain FFDM image features, DBT image features, and clinical information features.
[0033] S4. Using MLP, the FFDM image features, the DBT image features, and the clinical information features are nonlinearly transformed and fused to obtain fused features. Based on the fused features, the breast cancer risk score of the subject is obtained.
[0034] S5. The breast cancer risk score is segmented based on X-tile software to obtain the breast cancer risk cutoff point;
[0035] S6. Based on the breast cancer risk cutoff point, classify the subject's breast cancer risk level to achieve breast cancer risk prediction for the subject.
[0036] Preferably, the clinical information in S1 includes: the subject's previous number of breast biopsies, reproductive history, age of menarche, age of first childbirth, breastfeeding history, whether or not menopause has occurred, the subject's family history of breast cancer in first- and second-degree relatives, and the subject's clinician's palpation information.
[0037] Preferably, S2 includes:
[0038] S21. Obtain the boundary line between the pectoralis major muscle and the breast contour in the preprocessed FFDM image based on the LIBRA algorithm;
[0039] S22. Based on the boundary line, the segmentation matrix of the pectoralis major muscle is obtained by using the boundary threshold method;
[0040] S23. Multiply the preprocessed FFDM image with the segmentation matrix to obtain the FFDM image after segmenting the pectoralis major muscle;
[0041] S24. The segmentation matrix is extended to the size of the preprocessed DBT image, and the extended segmentation matrix is interpolated and scaled using cubic interpolation.
[0042] S25. Convolve the preprocessed DBT slices with the interpolated and scaled segmentation matrix to obtain the DBT image after segmenting the pectoralis major muscle.
[0043] Preferably, S3 includes:
[0044] S31. Construct a CLIP model based on deep learning;
[0045] S32. Input the FFDM image after segmenting the pectoralis major muscle into the image encoder of the CLIP model, extract the first feature vector using the model.get_image_features method and perform normalization processing to obtain the FFDM image features;
[0046] S33. Input the DBT image after segmenting the pectoralis major muscle into the image encoder, extract the second feature vector, and aggregate the second feature vector using the torch.mean method to obtain the DBT image features;
[0047] S34. Input the clinical information into the text encoder of the CLIP model, extract the third feature vector using the model.get_text_features method and perform normalization processing to obtain the clinical information features.
[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0049] This invention provides a breast cancer risk prediction system and method based on deep learning, which effectively improves the accuracy and stability of breast cancer risk prediction through automatic pectoralis major muscle segmentation, deep fusion of multimodal information and risk grading mechanism. Attached Figure Description
[0050] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 This is a schematic diagram of a breast cancer risk prediction system module based on deep learning, as described in an embodiment of the present invention.
[0052] Figure 2 This is a block diagram illustrating an embodiment of the deep learning-based breast cancer risk prediction system of the present invention.
[0053] Figure 3 A schematic diagram illustrating the normalization process for different breast thicknesses in an embodiment of the present invention;
[0054] Figure 4This is a schematic diagram of the pectoralis major muscle segmentation process for three-dimensional breast imaging according to an embodiment of the present invention;
[0055] Figure 5 This is a schematic diagram of the breast cancer risk prediction method based on deep learning, as described in an embodiment of the present invention. Detailed Implementation
[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0057] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0058] Example 1
[0059] like Figure 1 As shown, the present invention provides a breast cancer risk prediction system based on deep learning. The system includes: a data acquisition module, a segmentation module, a feature extraction module, a feature fusion module, a boundary point acquisition module, and a risk prediction module.
[0060] The acquisition module is used to acquire FFDM images and DBT images of the subject's breast and the subject's clinical information, and to perform preprocessing.
[0061] The segmentation module is used to segment the pectoralis major muscle from the preprocessed FFDM and DBT images to obtain the segmented FFDM and DBT images of the pectoralis major muscle.
[0062] The feature extraction module is used to extract structured features from the FFDM image after segmentation of the pectoralis major muscle, the DBT image after segmentation of the pectoralis major muscle, and clinical information using a deep learning-based network model, thereby obtaining FFDM image features, DBT image features, and clinical information features.
[0063] The feature fusion module is used to perform nonlinear transformation and fusion of FFDM image features, DBT image features and clinical information features using MLP to obtain fused features. Based on the fused features, the subject's breast cancer risk score is obtained.
[0064] The boundary point acquisition module is used to segment breast cancer risk scores based on X-tile software to obtain breast cancer risk boundary points.
[0065] The risk prediction module is used to classify the breast cancer risk level of subjects based on breast cancer risk cutoff points, thereby enabling the prediction of breast cancer risk for subjects.
[0066] Furthermore, such as Figure 2 The specific implementation process of the present invention is as follows:
[0067] The acquisition module is used to acquire FFDM images and DBT images of the subject's breast, as well as the subject's clinical information, and to perform preprocessing, including:
[0068] FFDM & DBT imaging: A digital mammography machine was used. First, the breast tissue was fully exposed. Compression was applied to the breast using a compression plate, and CC and MLO views were taken. Image reading and evaluation were performed on a standard diagnostic workstation—a Barco 5M monitor. The diagnosing physician described the lesions according to the 5th edition of the 2013 BI-RADS classification, focusing on the size, shape, and accompanying signs of the lesions.
[0069] Clinical information includes: the number of previous breast biopsies, reproductive history, age of menarche, age of first childbirth, breastfeeding history, whether or not the subject has gone through menopause, family history of breast cancer in first-degree relatives such as mothers and sisters, family history of breast cancer in second-degree relatives such as maternal grandmothers, and palpation information from the subject's clinicians, such as whether lesions were palpable, whether the maximum diameter of the lesion was greater than 2 cm, and whether the lesion was fixed.
[0070] The acquired breast imaging data underwent uniform preprocessing operations, including image format conversion, resolution standardization, image enhancement, and grayscale normalization. Since the number of slices corresponding to different thicknesses varies in 3D tomographic images, an image resampling strategy was used to standardize the image slices to meet the requirements of deep learning models for fixed input dimensions.
[0071] Preprocessing of DBT images includes converting all slices in the 3D DBT images into a set of 8-bit PNG images. Given the varying breast compression thickness among each subject, the total number of slices in the captured DBT images varies between 42 and 130. To maintain a consistent input shape, this invention selects an equal number of slices for each input sample, set to 130. Normalization for different breast thicknesses, such as... Figure 3 As shown.
[0072] The segmentation module is used to segment the pectoralis major muscle from the preprocessed FFDM and DBT images to obtain the segmented FFDM and DBT images of the pectoralis major muscle. It includes: boundary line unit, segmentation matrix unit, first segmentation image unit, expansion and scaling unit, and second segmentation image unit.
[0073] Boundary line unit, used to obtain the boundary line of the pectoralis major muscle and breast contour in the preprocessed FFDM image based on the LIBRA algorithm;
[0074] The segmentation matrix unit is used to obtain the segmentation matrix of the pectoralis major muscle based on the boundary line and using the boundary threshold method;
[0075] The first segmentation image unit is used to multiply the preprocessed FFDM image with the segmentation matrix to obtain the segmented FFDM image of the pectoralis major muscle.
[0076] The extended scaling unit is used to extend the segmentation matrix to the size of the preprocessed DBT image and to interpolate and scale the extended segmentation matrix using cubic interpolation.
[0077] The second segmentation image unit is used to convolve the preprocessed DBT slices with the interpolated and scaled segmentation matrix to obtain the DBT image after segmenting the pectoralis major muscle.
[0078] For pectoralis major muscle segmentation in 2D MLO (oblique) FFDM images, the LIBRA (Laboratory for Individualized Breast Radiodensity Assessment) algorithm was used. Inputting the 2D FFDM image in MLO mode, LIBRA obtained the boundary line between the pectoralis major muscle and the breast contour. A boundary thresholding method was applied, setting the pixel values of the pectoralis major muscle region to 0 and representing the remaining region with 1, thus obtaining the segmentation matrix for the pectoralis major muscle. Multiplying the original image with the segmentation matrix of the pectoralis major muscle yielded the segmented FFDM image of the pectoralis major muscle.
[0079] Specifically as follows:
[0080] LIBRA is an algorithm library specifically designed for automatic analysis of breast images, used to automatically identify the boundary between breast tissue and the pectoralis major muscle region. In this invention, it is primarily used for pectoralis major muscle region segmentation on FFDM images in MLO positions. The output includes: breast contour line C_breast and pectoralis major muscle boundary line C_pectoral.
[0081] Based on the boundary line, the boundary threshold method is used to obtain the segmentation matrix of the pectoralis major muscle. That is, based on the pectoralis major muscle boundary line C_pectoral output by LIBRA, a binary segmentation matrix Mp(x,y) is generated:
[0082]
[0083] Here, (x, y) represents the image pixel coordinates. If the pixels within the region defined by the pectoralis major muscle boundary line C_pectoral belong to the pectoralis major muscle region, the corresponding position in Mp is set to 0; other regions are set to 1 to preserve the breast tissue region.
[0084] ③ Multiplication of partitioned matrices
[0085] The preprocessed FFDM image FFDM Element-wise multiplication of (x,y) with the pectoralis major muscle segmentation matrix Mp(x,y) yields the FFDM image of the segmented pectoralis major muscle:
[0086] I seg (x, y) = I FFDM (x, y)·Mp(x, y)
[0087] Among them, I seg (x, y) represents the FFDM image after segmenting the pectoralis major muscle; · represents element-wise multiplication (Hadamard product); after segmentation, the pixel values of the pectoralis major muscle region are set to zero, while the other regions remain unchanged.
[0088] Two-dimensional segmentation matrix expansion and interpolation scaling:
[0089] Since the 3D DBT image is composed of multiple slices, the 2D segmentation matrix Mp(x,y) needs to be expanded to match the size of each preprocessed DBT slice;
[0090] Assuming each slice of the preprocessed DBT is H'×W', Mp(x,y) is scaled by the same scale using the cubic interpolation function imresize():
[0091] Mp _resized =imresize(Mp,H',W','bicubic')
[0092] Here, bicubic means "bicubic interpolation".
[0093] Pectoralis major muscle segmentation using convolution operations:
[0094] The following operations were performed on each preprocessed DBT slice to obtain DBT images after segmentation of the pectoralis major muscle:
[0095]
[0096] in, To segment the i-th layer of the DBT behind the pectoralis major muscle. The image is the i-th layer of the unsegmented pectoralis major muscle DBT.
[0097] like Figure 4As shown, for pectoralis major muscle segmentation in 3D MLO-based DBT images, a segmentation matrix for the pectoralis major muscle is extracted from 2D breast images and then extended to slices of the 3D image. However, the reconstructed DBT slices and FFDM images are not consistent in size, therefore the aforementioned 2D segmentation matrix is not suitable for pectoralis major muscle segmentation in DBT slices, even though each DBT slice has the same aspect ratio as the FFDM image. To solve this problem, the 2D segmentation matrix is scaled proportionally using cubic interpolation to match each DBT slice. Each slice is convolved with the scaled 2D segmentation matrix to obtain the segmented DBT image of the pectoralis major muscle.
[0098] The feature extraction module includes: a model building unit, a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit;
[0099] Model building unit, used to build CLIP models based on deep learning;
[0100] The first feature extraction unit is used to input the FFDM image after segmenting the pectoralis major muscle into the image encoder of the CLIP model, extract the first feature vector through the model.get_image_features method and perform normalization processing to obtain FFDM image features;
[0101] The second feature extraction unit is used to input the DBT image after segmenting the pectoralis major muscle into the image encoder, extract the second feature vector, and aggregate the second feature vector using the torch.mean method to obtain the DBT image features.
[0102] The third feature extraction unit is used to input clinical information into the text encoder of the CLIP model, extract the third feature vector through the model.get_text_features method and perform normalization processing to obtain clinical information features.
[0103] Specifically:
[0104] A deep learning network architecture supporting multimodal input is introduced to simultaneously process breast imaging and clinical text information. This network architecture incorporates both image and language encoding modules, enabling the extraction of representative feature representations from different modalities. Through unified modeling of multimodal data, the model can not only identify suspicious lesions in images but also comprehensively consider individual patient characteristics and medical history, thereby improving the relevance and reliability of risk prediction.
[0105] During model building, a pre-trained CLIP (Contrastive Language-Image Pretraining) model (openai / clip-vit-base-patch16) was selected, and the corresponding processor and model were loaded. All experiments were conducted on a single NVIDIA 4090 GPU using the PyTorch training framework. Image and text features were extracted using `model.get_image_features` and `model.get_text_features`, respectively. AdamW was used as the optimizer, with the learning rate initialized to 1×10⁻⁶. -5 The batch size was set to 128, and the size of all breast images was set to (224, 224), conforming to the CLIP model input specifications to ensure data consistency and standardization. For each batch of data, image features, labels, and clinical information of the subjects were extracted and fed into the CLIP model for processing.
[0106] This invention makes the following improvements to the CLIP model:
[0107] ①Averaging strategy for fragment features of 3D DBT images:
[0108] Encoding each slice of DBT independently and then aggregating them into the overall features of the 3D image using torch.mean is a custom strategy for 3D medical image structure and is not a native function of the CLIP model.
[0109] Through this averaging strategy, the present invention achieves three-dimensional medical image encoding that is not supported by the CLIP native model; solves the problem of inconsistent feature dimensions caused by inconsistent slice numbers; and after aggregation processing, it can more stably reflect the spatial distribution information of the overall breast lesions.
[0110] ② Fusion of three modal features from FFDM, DBT, and clinical text information:
[0111] CLIP was originally designed for image-text matching tasks. This invention achieves cross-modal fusion of image dual-modality and text information through MLP networks, which is an improvement that expands its application structure.
[0112] By fusing trimodal features of FFDM, DBT, and clinical text information, this invention extends the CLIP model from "image-text matching" to "image-text structured classification," improving the generalization ability of the breast cancer risk prediction model in small samples, compensating for the shortcomings of a single modality, and effectively enhancing the reliability of the results.
[0113] For the extraction of features from 2D FFDM images, the preprocessed image is input into the pre-trained CLIP model image encoder, and the first feature vector is extracted by the model.get_image_features method and then normalized.
[0114] For 3D DBT image feature extraction, each slice needs to be feature-encoded separately, and the feature vectors of each slice are averaged to generate a comprehensive 3D image feature vector. Specifically, each slice is input into the same image encoder, the resulting second feature vectors are stacked, and then aggregated using the torch.mean method.
[0115] For text feature extraction, the preprocessed text information is input into the text encoder of the CLIP model, and the third feature vector is extracted by the model.get_text_features method and normalized to ensure that the feature vectors have consistent scale.
[0116] In these steps, the features of 2D FFDM images, 3D DBT images, and clinical information are processed by their respective encoders to obtain normalized feature vectors, which provide a basis for subsequent feature fusion and classification.
[0117] The feature fusion module uses MLP to perform nonlinear transformation and fusion of FFDM image features, DBT image features, and clinical information features to obtain fused features. Based on the fused features, the subject's breast cancer risk score is obtained.
[0118] This paper fuses 2D FFDM image features, 3D DBT image features, and text features by concatenating these feature vectors into a single composite feature vector. To achieve feature fusion, a multilayer perceptron (MLP) is used to perform nonlinear transformations and fusion of features from different modalities. Through multilayer neural networks, more complex feature relationships can be captured. This concatenation method combines features from multiple modalities, enabling the model to consider both image and text information simultaneously, thus improving predictive performance. Subsequently, this fused feature vector is input into a fully connected layer and outputs a breast cancer risk score.
[0119] This invention makes the following improvements when using MLP to perform nonlinear transformation and fusion of features from different modes:
[0120] In the multimodal feature fusion stage, instead of directly concatenating the original feature vectors, a learnable gating module is first used to assign a weight coefficient to each modality before weighted fusion is performed.
[0121] Define three modal features, fFFDM For two-dimensional FFDM image features, f DBT For 3D DBT image features, f text For text features. Define g. j It is a gated vector [g1, g2, g3], representing the importance weights of each mode, satisfying:
[0122]
[0123] Among them, w j It is the j-th learnable parameter, which is normalized by Softmax and used as a weighting coefficient.
[0124] The final fusion feature is f fusion :
[0125] f fusion =g1·f FFDM +g2·f DBT +g3·f text
[0126] The fused features are then fed into the MLP backbone network for further nonlinear transformation and classification prediction.
[0127] The cutoff point acquisition module is used to segment breast cancer risk scores based on X-tile software to obtain breast cancer risk cutoff points:
[0128] To transform continuous breast cancer risk scores into meaningful hierarchical classifications, this invention employs X-tile software to segment the risk scores output by the model in the training set, determining the optimal breast cancer risk cutoff point. X-tile software is a statistical tool used for survival analysis and risk stratification, capable of automatically finding the optimal cutoff threshold in large sample data to maximize the statistical significance of differences between groups.
[0129] The risk prediction module is used to classify the subject's breast cancer risk level based on breast cancer risk cutoff points, thereby predicting the subject's breast cancer risk.
[0130] Based on the breast cancer risk cutoff point, the subjects' breast cancer risk level is classified. Based on the subject's risk level and combined with existing breast cancer screening guidelines, personalized screening recommendations are automatically generated: routine screening is recommended for low-risk subjects, regular imaging follow-up is recommended for intermediate-risk subjects, and combined use of multimodal examinations such as MRI or referral to a specialist for evaluation is recommended for high-risk subjects.
[0131] In summary, this invention provides a breast cancer risk prediction system based on deep learning, which effectively improves the accuracy and stability of breast cancer risk prediction through automatic pectoralis major muscle segmentation, deep fusion of multimodal information, and risk grading mechanism.
[0132] Example 2
[0133] like Figure 5 As shown, this invention provides a deep learning-based method for predicting breast cancer risk, comprising:
[0134] S1. Collect FFDM and DBT images of the subject's breast and the subject's clinical information, and perform preprocessing.
[0135] S2. Perform pectoralis major muscle segmentation on the preprocessed FFDM and DBT images to obtain FFDM and DBT images after pectoralis major muscle segmentation.
[0136] S3. Using a deep learning-based network model, structured features are extracted from the FFDM image after segmentation of the pectoralis major muscle, the DBT image after segmentation of the pectoralis major muscle, and clinical information, respectively, to obtain FFDM image features, DBT image features, and clinical information features.
[0137] S4. Using MLP, nonlinear transformation and fusion of FFDM image features, DBT image features and clinical information features are performed to obtain fused features. Based on the fused features, the breast cancer risk score of the subject is obtained.
[0138] S5. Based on X-tile software, the breast cancer risk score is segmented to obtain the breast cancer risk cutoff point;
[0139] S6. Based on the breast cancer risk cutoff point, the subjects' breast cancer risk level is classified to predict the subjects' breast cancer risk.
[0140] Furthermore, the clinical information in S1 includes: the subject's previous number of breast biopsies, reproductive history, age of menarche, age of first delivery, breastfeeding history, whether or not they are menopausal, family history of breast cancer in the subject's first- and second-degree relatives, and palpation information from the subject's clinician.
[0141] S2. Perform pectoralis major muscle segmentation on the preprocessed FFDM and DBT images to obtain segmented FFDM and DBT images, including:
[0142] S21. Obtain the boundary line between the pectoralis major muscle and the breast contour in the preprocessed FFDM image based on the LIBRA algorithm;
[0143] S22. Based on the boundary line, the segmentation matrix of the pectoralis major muscle is obtained by using the boundary threshold method;
[0144] S23. Multiply the preprocessed FFDM image with the segmentation matrix to obtain the FFDM image after segmenting the pectoralis major muscle;
[0145] S24. Extend the segmentation matrix to match the size of the preprocessed DBT image, and then interpolate and scale the extended segmentation matrix using cubic interpolation.
[0146] S25. Convolve the preprocessed DBT slices with the interpolated and scaled segmentation matrix to obtain the DBT image after segmenting the pectoralis major muscle.
[0147] S3. Using a deep learning-based network model, structured features are extracted from the FFDM image, DBT image, and clinical information after pectoralis major muscle segmentation, respectively, to obtain FFDM image features, DBT image features, and clinical information features, including:
[0148] S31. Construct a CLIP model based on deep learning;
[0149] S32. Input the FFDM image after segmenting the pectoralis major muscle into the image encoder of the CLIP model, extract the first feature vector through the model.get_image_features method and perform normalization processing to obtain the FFDM image features;
[0150] S33. Input the DBT image after segmenting the pectoralis major muscle into the image encoder, extract the second feature vector, and aggregate the second feature vector using the torch.mean method to obtain the DBT image features;
[0151] S34. Input the clinical information into the text encoder of the CLIP model, extract the third feature vector using the model.get_text_features method and perform normalization processing to obtain the clinical information features.
[0152] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A breast cancer risk prediction system based on deep learning, characterized in that, The system includes: an acquisition module, a segmentation module, a feature extraction module, a feature fusion module, a boundary point acquisition module, and a risk prediction module; The acquisition module is used to acquire FFDM images and DBT images of the subject's breast and the subject's clinical information, and to perform preprocessing. The segmentation module is used to segment the pectoralis major muscle from the preprocessed FFDM image and DBT image to obtain the segmented FFDM image and the segmented DBT image. The feature extraction module is used to extract structured features from the FFDM image after segmentation of the pectoralis major muscle, the DBT image after segmentation of the pectoralis major muscle, and the clinical information using a deep learning-based network model, to obtain FFDM image features, DBT image features, and clinical information features. The feature fusion module is used to perform nonlinear transformation and fusion of the FFDM image features, the DBT image features and the clinical information features using MLP to obtain fused features, and to obtain the breast cancer risk score of the subject based on the fused features; The boundary point acquisition module is used to segment the breast cancer risk score based on X-tile software to obtain the breast cancer risk boundary point. The risk prediction module is used to classify the subject's breast cancer risk level based on the breast cancer risk cutoff point, thereby predicting the subject's breast cancer risk. The segmentation module includes: a boundary line unit, a segmentation matrix unit, a first segmented image unit, an expansion and scaling unit, and a second segmented image unit; The boundary line unit is used to obtain the boundary line of the pectoralis major muscle and breast contour in the preprocessed FFDM image based on the LIBRA algorithm. The segmentation matrix unit is used to obtain the segmentation matrix of the pectoralis major muscle based on the boundary line and using the boundary threshold method. The first segmentation image unit is used to multiply the preprocessed FFDM image with the segmentation matrix to obtain the segmented FFDM image of the pectoralis major muscle. The expansion scaling unit is used to expand the segmentation matrix to the size of the preprocessed DBT image, and to interpolate and scale the expanded segmentation matrix using cubic interpolation. The second segmentation image unit is used to convolve the preprocessed DBT slices with the interpolated and scaled segmentation matrix to obtain a DBT image after segmenting the pectoralis major muscle.
2. The breast cancer risk prediction system based on deep learning according to claim 1, characterized in that, The clinical information in the acquisition module includes: the subject's previous number of breast biopsies, reproductive history, age of menarche, age of first delivery, breastfeeding history, whether or not they have gone through menopause, the subject's family history of breast cancer in first- and second-degree relatives, and the subject's clinical physician's palpation information.
3. The breast cancer risk prediction system based on deep learning according to claim 1, characterized in that, The feature extraction module includes: a model building unit, a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit; The model building unit is used to build a CLIP model based on deep learning; The first feature extraction unit is used to input the FFDM image after segmenting the pectoralis major muscle into the image encoder of the CLIP model, extract the first feature vector through the model.get_image_features method and perform normalization processing to obtain the FFDM image features; The second feature extraction unit is used to input the DBT image after segmenting the pectoralis major muscle into the image encoder, extract the second feature vector, and aggregate the second feature vector using the torch.mean method to obtain the DBT image features; The third feature extraction unit is used to input the clinical information into the text encoder of the CLIP model, extract the third feature vector through the model.get_text_features method and perform normalization processing to obtain the clinical information features.
4. A breast cancer risk prediction method based on deep learning, characterized in that, The method includes: S1. Collect FFDM images and DBT images of the subject's breast and the subject's clinical information, and perform preprocessing. S2. Perform pectoralis major muscle segmentation on the preprocessed FFDM and DBT images to obtain FFDM and DBT images after pectoralis major muscle segmentation. S3. Using a deep learning-based network model, extract the structured features from the FFDM image after segmenting the pectoralis major muscle, the DBT image after segmenting the pectoralis major muscle, and the clinical information, respectively, to obtain FFDM image features, DBT image features, and clinical information features. S4. Using MLP, the FFDM image features, the DBT image features, and the clinical information features are nonlinearly transformed and fused to obtain fused features. Based on the fused features, the breast cancer risk score of the subject is obtained. S5. The breast cancer risk score is segmented based on X-tile software to obtain the breast cancer risk cutoff point; S6. Based on the breast cancer risk cutoff point, classify the subject's breast cancer risk level to achieve breast cancer risk prediction for the subject; S2 includes: S21. Obtain the boundary line between the pectoralis major muscle and the breast contour in the preprocessed FFDM image based on the LIBRA algorithm; S22. Based on the boundary line, the segmentation matrix of the pectoralis major muscle is obtained by using the boundary threshold method; S23. Multiply the preprocessed FFDM image with the segmentation matrix to obtain the FFDM image after segmenting the pectoralis major muscle; S24. The segmentation matrix is extended to the size of the preprocessed DBT image, and the extended segmentation matrix is interpolated and scaled using cubic interpolation. S25. Convolve the preprocessed DBT slices with the interpolated and scaled segmentation matrix to obtain the DBT image after segmenting the pectoralis major muscle.
5. The breast cancer risk prediction method based on deep learning according to claim 4, characterized in that, The clinical information in S1 includes: the subject's previous number of breast biopsies, reproductive history, age of menarche, age of first delivery, breastfeeding history, whether or not they have gone through menopause, the subject's family history of breast cancer in first- and second-degree relatives, and the subject's clinical physician's palpation information.
6. The breast cancer risk prediction method based on deep learning according to claim 4, characterized in that, S3 includes: S31. Construct a CLIP model based on deep learning; S32. Input the FFDM image after segmenting the pectoralis major muscle into the image encoder of the CLIP model, extract the first feature vector through the model.get_image_features method and perform normalization processing to obtain the FFDM image features; S33. Input the DBT image after segmenting the pectoralis major muscle into the image encoder, extract the second feature vector, and aggregate the second feature vector using the torch.mean method to obtain the DBT image features; S34. Input the clinical information into the text encoder of the CLIP model, extract the third feature vector using the model.get_text_features method and perform normalization processing to obtain the clinical information features.