Molybdenum target CT-based triple negative breast cancer immunophenotype prediction method and system
By learning the intrinsic features and multi-view features of unlabeled data sets, combined with self-supervised comparative learning and deep learning technology, the difficulty of labeling and data representative problems of TNBC data sets are solved, and the diagnostic accuracy and efficiency of breast cancer are improved.
Patent Information
- Application Number
- CN202411533770.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The privacy of the prior art in triple-negative breast cancer (TNBC) datasets, limitations of acquisition channels and difficulties in manual annotation have led to challenges in the application of deep learning in breast cancer-assisted diagnosis, and comparative learning methods are difficult to ensure training data. The representativeness of the judgment results in low accuracy.
By learning the inherent characteristics of the unlabeled data set, using multi-view features to divide positive and negative samples, combining self-supervised comparison learning and deep learning technology, twin neural networks and multi-head self-attention mechanisms are used to extract and fusion features to improve the accuracy of TNBC's judgment Rate.
It effectively solved the problem of insufficient labeling data, improved the diagnostic efficiency and accuracy of breast cancer, and promoted the accurate identification and judgment accuracy of TNBC characteristics.
Smart Images

Figure CN119027427A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of molybdenum target CT image processing, and in particular to a method and system for predicting the immune phenotype of triple-negative breast cancer based on molybdenum target CT. Background Art
[0002] Triple negative breast cancer (TNBC) refers to breast cancer that is negative for estrogen receptor (ER), progesterone receptor (PR) and proto-oncogene Her-2 in the immunohistochemical examination of cancer tissue. Due to the lack of expression of ER, PR and HER-2 genes, treatment and prognosis are difficult, making early diagnosis particularly critical. Breast mammography CT uses low-dose X-ray imaging to take multiple views from different angles to display the morphology and pathological conditions of the breast, which plays a vital role in the early detection of breast cancer.
[0003] The prior art has the following problems: First, given the high privacy of TNBC datasets, the limitations of access channels, and the difficulty and inefficiency of manual labeling in this field, the application of deep learning in the field of TNBC breast cancer auxiliary diagnosis faces severe challenges.
[0004] Secondly, contrastive learning in existing technologies mostly uses random transformations of data (such as cropping, rotation, color adjustment, etc.) to generate positive and negative samples, which makes it difficult to ensure the representativeness of the training data, resulting in low judgment accuracy of the trained model.
[0005] Therefore, it is urgent to develop a solution to solve the above problems. Summary of the invention
[0006] The purpose of the present invention is to provide a method and system for predicting the immune phenotype of triple-negative breast cancer based on mammography. By learning the intrinsic characteristics of unlabeled data sets, the problem of insufficient labeled data is effectively improved, and the characteristics of multiple views are fully utilized to divide positive and negative samples, which promotes the accurate identification of TNBC features and improves the accuracy of TNBC judgment.
[0007] The present invention provides a method and system for predicting the immunophenotype of triple-negative breast cancer based on mammography, which adopts the following technical solutions: In the first aspect, a method for predicting the immune phenotype of triple-negative breast cancer based on mammography CT specifically includes: Extract and preprocess the mammography CT data in the existing breast cancer database to obtain a public data set, and preprocess the mammography CT images to be predicted to obtain multi-view data; An initial model is constructed based on a twin neural network, and molybdenum target CT images in a public data set are randomly regarded as key-value views. Based on the key-value views, queries are performed to obtain corresponding query views, and the same patient views of the query views are grouped to obtain positive samples, and other molybdenum target CT images are classified as negative samples. The initial model is trained based on the comparison of positive and negative samples to obtain a pre-trained encoder; Based on the pre-trained encoder, multi-scale feature extraction is performed on the multi-view data to obtain a multi-scale feature map, based on the multi-head self-attention mechanism, feature fusion is performed on the multi-scale feature map to obtain a comprehensive view feature map, based on the classifier, the comprehensive view feature map is classified, and the three-negative confidence is obtained and output.
[0008] The beneficial effect of the method and system for predicting the immunophenotype of triple-negative breast cancer based on molybdenum target CT provided by the present invention is to improve the diagnostic efficiency and accuracy of breast cancer by combining self-supervised comparative learning with deep learning technology. First, a generalized feature extractor is obtained by self-supervised comparative pre-training of a public data set with a large sample size; then, the multi-view molybdenum target CT images in the fine-tuning data set are pre-processed and feature extracted, and a weighted bidirectional feature pyramid network (Bidirectional Feature Pyramid Network, BiFPN) is used for multi-scale feature extraction and feature fusion through a multi-head self-attention mechanism, and finally, a multi-layer perceptron (Multilayer Perceptron, MLP) is used for classification to obtain the immunophenotype of TNBC; by learning the intrinsic features of the unlabeled data set, the problem of insufficient labeled data is effectively solved, and the features of multiple views are fully utilized to promote the accurate recognition of TNBC features and improve the accuracy of TNBC judgment.
[0009] Optionally, the initial model is trained based on positive and negative samples to obtain a pre-trained encoder, including: The encoder based on the convolutional neural network extracts features from the key-value view to obtain the baseline features, and the momentum encoder extracts features from the positive and negative samples of the query view to obtain the positive and negative feature vector sets. The InfoNCE loss function is constructed based on the similarity between the positive and negative feature vector sets and the benchmark features. The InfoNCE loss function is minimized based on the gradient descent method to maximize the similarity of the positive feature vectors and minimize the similarity of the negative feature vectors. When the InfoNCE loss function value tends to be stable, the training is terminated to obtain the pre-trained encoder.
[0010] Optionally, an InfoNCE loss function is constructed based on the similarity between the positive and negative feature vector sets and the reference features. The InfoNCE loss function is as follows: ; In the formula, is the number of sample features, is a key-value view representation, is the positive sample representation of the query view, is the negative representation of the query view, The function is the similarity function, is the total number of negative samples of the query view, is the temperature coefficient.
[0011] Optionally, performing multi-scale feature extraction on the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map includes: Extracting features from multi-view data to obtain multi-view features; Build a bidirectional feature pyramid network based on weighted feature fusion mechanism; Image enhancement is performed on the multi-view features based on a bidirectional feature pyramid network to obtain a multi-scale feature map.
[0012] Optionally, feature fusion is performed on multi-scale feature maps based on a multi-head self-attention mechanism to obtain a comprehensive view feature map, and the feature fusion process includes: Based on the self-attention layer and feed-forward neural network, feature extraction is performed on the multi-scale feature map to obtain a feature vector set; Repeat twice to concatenate and downsample adjacent feature vectors in the feature vector set to obtain a comprehensive view feature map.
[0013] Optionally, in the process of extracting features from a multi-scale feature map based on a self-attention layer and a feedforward neural network to obtain a feature vector set, the self-attention layer includes 2 layers of Transformer, the number of heads in each layer is 4, and two downsampling operations are performed between layers. Before the two downsampling operations, adjacent feature vectors are spliced respectively.
[0014] Optionally, the comprehensive view feature map is classified based on a classifier to obtain and output a three-negative confidence, the classifier comprising: an input layer, a hidden layer and an output layer; The input layer is used to receive and transmit the comprehensive view feature map to the hidden layer; The hidden layer consists of 256 neurons, which are used to classify the comprehensive view features and predict the logical value; The output layer normalizes the logical value based on the sigmoid function and limits the output value to 0-1 to obtain the confidence result.
[0015] The second aspect is a triple-negative breast cancer immune phenotype prediction system based on mammography CT, which specifically includes: A data processing module is used to extract and preprocess the mammography CT data in the existing breast cancer database to obtain a public data set, and preprocess the mammography CT images to be predicted to obtain multi-view data; A data training module, which obtains an initial model based on a twin neural network, randomly regards mammography CT images in a public data set as key-value views, performs queries based on the key-value views to obtain corresponding query views, groups views of the same patient in the query views to obtain positive samples, and classifies other mammography CT images as negative samples, and trains the initial model based on a comparison of positive and negative samples to obtain a pre-trained encoder; The triple-negative prediction module extracts multi-scale features from the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map, fuses the multi-scale feature map based on the multi-head self-attention mechanism to obtain a comprehensive view feature map, and classifies the comprehensive view feature map based on the classifier to obtain and output the triple-negative confidence.
[0016] Optionally, the initial model architecture in the data training module includes: an encoder, a decoder, a momentum encoder, a queue, and an InfoNCE loss function; An encoder, composed of a convolutional neural network and a bidirectional feature pyramid network, is used for convolution, downsampling, feature concatenation, and global pooling of the positive and negative samples to obtain feature vectors; The decoder, which consists of a multi-layer perceptron with two hidden layers, is used to map the feature vector to the latent space to obtain the logical value; Momentum encoder, which has the same structure as the above encoder, but does not perform backpropagation during training and uses momentum update method; A queue for storing feature vectors generated by the momentum encoder; InfoNCE loss function, used to optimize the parameters of the initial model, indicates the closeness of positive and negative sample pairs.
[0017] The beneficial effects of the second aspect can refer to the description of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a data flow chart of a method for predicting triple-negative breast cancer immune phenotype based on mammography CT provided by the present invention; Figure 2 It is a schematic diagram of the initial model structure of a triple-negative breast cancer immune phenotype prediction system based on mammography CT provided by the present invention; Figure 3 It is a schematic diagram of the encoder structure in a mammography-based CT-based triple-negative breast cancer immunophenotype prediction system provided by the present invention; Figure 4It is a schematic diagram of a bidirectional feature pyramid network constructed in a triple-negative breast cancer immune phenotype prediction system based on mammography CT provided by the present invention; Figure 5 It is a schematic diagram of a classifier of a triple-negative breast cancer immune phenotype prediction system based on mammography CT provided by the present invention. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0020] The embodiment of the present invention provides a method for predicting the immune phenotype of triple-negative breast cancer based on mammography, and the specific steps include: S1, extracting and preprocessing the mammography CT data in the existing breast cancer database to obtain a public data set, and preprocessing the mammography CT images to be predicted to obtain multi-view data; S2. An initial model is constructed based on a twin neural network, and molybdenum target CT images in a public data set are randomly regarded as key-value views. Based on the key-value views, queries are performed to obtain corresponding query views, and the same patient views of the query views are grouped to obtain positive samples, and other molybdenum target CT images are classified as negative samples. The initial model is trained based on the comparison of positive and negative samples to obtain a pre-trained encoder; S3. Perform multi-scale feature extraction on the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map, perform feature fusion on the multi-scale feature map based on the multi-head self-attention mechanism to obtain a comprehensive view feature map, classify the comprehensive view feature map based on the classifier, and obtain and output the three-negative confidence.
[0021] In some embodiments, when executing step S1, the specific steps include: S1-1, extract and preprocess the mammography CT data in the existing breast cancer database to obtain a public dataset; S1-2, preprocessing the molybdenum target CT image to be predicted to obtain multi-view data; Specifically, when executing step S1-1, data processing is performed on the molybdenum target CT image, including cropping and histogram equalization, to obtain a processed public data set; Furthermore, breast cancer databases are resource collections specifically used to store, analyze, and study breast cancer-related data. These databases usually contain clinical information, gene expression data, pathological images, treatment records, and other types of data from breast cancer patients. Breast cancer databases include: Digital Database for Screening Mammography (DDSM), Molecular Taxonomy Database of the International Breast Cancer Consortium (METABRIC), and TCGA Database (The Cancer Genome Atlas); Specifically, when executing step S1-2, the molybdenum target CT image to be predicted is preprocessed to obtain multi-view data, and the processing process includes image cropping and image conversion.
[0022] In some embodiments, when executing step S2, it specifically includes: S2-1, constructing an initial model based on the twin neural network; S2-2, randomly treating the mammography CT images in the public data set as key-value views, querying based on the key-value views to obtain the corresponding query views, grouping the same patient views of the query views to obtain positive samples, and classifying other mammography CT images as negative samples; S2-3. The initial model is trained based on positive and negative samples to obtain a pre-trained encoder.
[0023] Specifically, when executing step S2-1, the initial model architecture includes: an encoder, a decoder, a momentum encoder, a queue, and an InfoNCE loss function; An encoder, composed of a convolutional neural network and a bidirectional feature pyramid network, is used for convolution, downsampling, feature concatenation, and global pooling of the positive and negative samples to obtain feature vectors; The decoder, which consists of a multi-layer perceptron with two hidden layers, is used to map the feature vector to the latent space to obtain the logical value; Momentum encoder, which has the same structure as the above encoder, but does not perform backpropagation during training and uses momentum update method; A queue for storing feature vectors generated by the momentum encoder; InfoNCE loss function, used to optimize the parameters of the initial model, indicates the closeness of positive and negative sample pairs.
[0024] Specifically, when executing step S2-2, a mammography CT image is regarded as a key view, and different views of the same patient view corresponding to it are regarded as positive samples of the query view, and other mammography CT images are regarded as negative samples of the query view; Furthermore, images of the same patient from different perspectives are used as positive samples, thereby expanding the representation of positive samples from traditional similar samples to similar samples. This is not only conducive to the model learning more representative and rich feature representations, but also significantly enhances the model's insight into the characteristics of mammography CT images.
[0025] Furthermore, in medical diagnosis, especially in image-based diagnostic technologies such as breast X-ray, CT, MRI, etc., view invariance means that under different views or imaging conditions, key features in the image, such as lesions, masses, etc., should remain relatively consistent, and the position and shape of the lesions should be consistent in multiple views.
[0026] For further information, see Figure 1 , input samples Left MLO (left breast oblique view), Left CC (left breast cranio-caudal view), Right MLO (right breast oblique view) and Right CC (right breast cranio-caudal view) to the pre-trained encoder to obtain feature maps of information at different scales, and then input them to the attention fusion module for position encoding, multi-head attention mechanism to extract features and normalize them to obtain the comprehensive view feature map, and output the three-yin confidence through the fully connected layer.
[0027] Specifically, when executing step S2-3, the initial model is trained based on positive sample and negative sample comparison to obtain a pre-trained encoder, including: The encoder based on the convolutional neural network extracts features from the key-value view to obtain the baseline features, and the momentum encoder extracts features from the positive and negative samples of the query view to obtain the positive and negative feature vector sets. The InfoNCE loss function is constructed based on the similarity between the positive and negative feature vector sets and the benchmark features. The InfoNCE loss function is minimized based on the gradient descent method to maximize the similarity of the positive feature vectors and minimize the similarity of the negative feature vectors. When the InfoNCE loss function value tends to be stable, the training is terminated to obtain the pre-trained encoder.
[0028] Furthermore, the InfoNCE loss function is as follows: ; In the formula, is the number of sample features, is a key-value view representation, is the positive sample representation of the query view, is the negative representation of the query view, The function is the similarity function, is the total number of negative samples of the query view, is the temperature coefficient.
[0029] In some embodiments, when executing step S3, it specifically includes: S3-1, performing multi-scale feature extraction on the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map; S3-2, based on the multi-head self-attention mechanism, feature fusion is performed on multi-scale feature maps to obtain a comprehensive view feature map; S3-3. Classify the comprehensive view feature map based on a classifier to obtain and output three-negative confidence.
[0030] Specifically, when executing step S3-1, performing multi-scale feature extraction on the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map includes: Extracting features from multi-view data to obtain multi-view features; Build a bidirectional feature pyramid network based on weighted feature fusion mechanism; Image enhancement is performed on the multi-view features based on a bidirectional feature pyramid network to obtain a multi-scale feature map.
[0031] Specifically, when executing step S3-2, the multi-scale feature map is subjected to feature fusion based on the multi-head self-attention mechanism to obtain a comprehensive view feature map, and the feature fusion process includes: Based on the self-attention layer and feed-forward neural network, feature extraction is performed on the multi-scale feature map to obtain a feature vector set; Repeat twice to concatenate and downsample adjacent feature vectors in the feature vector set to obtain a comprehensive view feature map.
[0032] In fact, in the process of extracting features from multi-scale feature maps based on the self-attention layer and the feedforward neural network to obtain a feature vector set, the self-attention layer contains 2 layers of Transformer, the number of heads in each layer is 4, and two downsampling operations are performed between layers. Before the two downsampling operations, adjacent feature vectors are spliced respectively.
[0033] Specifically, when executing step S3-3, the comprehensive view feature map is predicted based on a classifier to obtain and output the three-negative confidence, and the classifier includes: an input layer, a hidden layer and an output layer; The input layer is used to receive and transmit the comprehensive view feature map to the hidden layer; The hidden layer consists of 256 neurons, which are used to classify the comprehensive view features and predict the logical value; The output layer normalizes the logical value based on the sigmoid function and limits the output value to 0-1 to obtain the confidence result.
[0034] The embodiment of the present invention also provides a triple-negative breast cancer immune phenotype prediction system based on mammography, which specifically includes: A data processing module is used to extract and preprocess the mammography CT data in the existing breast cancer database to obtain a public data set, and preprocess the mammography CT images to be predicted to obtain multi-view data; A data training module, which obtains an initial model based on a twin neural network, randomly regards mammography CT images in a public data set as key-value views, performs queries based on the key-value views to obtain corresponding query views, groups views of the same patient in the query views to obtain positive samples, and classifies other mammography CT images as negative samples, and trains the initial model based on a comparison of positive and negative samples to obtain a pre-trained encoder; The triple-negative prediction module extracts multi-scale features from the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map, fuses the multi-scale feature map based on the multi-head self-attention mechanism to obtain a comprehensive view feature map, and classifies the comprehensive view feature map based on the classifier to obtain and output the triple-negative confidence.
[0035] In some embodiments, see Figure 2 ,The initial model architecture in the data training module includes: encoder, decoder, momentum encoder, queue and InfoNCE loss function; An encoder, composed of a convolutional neural network and a bidirectional feature pyramid network, is used for convolution, downsampling, feature concatenation, and global pooling of the positive and negative samples to obtain feature vectors; The decoder, which consists of a multi-layer perceptron with two hidden layers, is used to map the feature vector to the latent space to obtain the logical value; Momentum encoder, which has the same structure as the above encoder, but does not perform backpropagation during training and uses momentum update method; A queue for storing feature vectors generated by the momentum encoder; InfoNCE loss function, used to optimize the parameters of the initial model, indicates the closeness of positive and negative sample pairs.
[0036] In fact, see Figure 3 , the encoder is applied to the actual scene, and the EfficientNet-B0 model is used as the encoder for feature extraction. The image shape of the input encoder is (3, 224, 224). In order to fully extract the image features, the sample undergoes a series of convolutions, downsampling and feature concatenation to obtain a feature map of size 7*7*1024. Finally, it is globally averaged pooled and then activated by the Relu function to turn the image into a one-dimensional feature vector of length 1024.
[0037] For further information, see Figure 4 The bidirectional feature pyramid network combines the top-down and bottom-up bidirectional path structures to achieve effective aggregation of features at different scales and strengthen feature fusion. By simplifying the network design and introducing a weighted feature fusion mechanism based on importance, compared with other multi-scale extraction modes, this structure optimizes the information flow between feature layers, thereby significantly improving model performance and accelerating model convergence.
[0038] In fact, see Figure 5 The classifier uses a multi-layer perceptron structure, which is responsible for outputting the final triple-negative breast cancer confidence. It has one hidden layer, uses 256 neurons, uses ReLU as the activation function, uses sigmoid for confidence output in the output layer, uses nonlinear enhancement of expression ability, and limits the output to between 0 and 1.
[0039] In fact, the momentum encoder uses the same network architecture as the encoder, but during the training process, back propagation is not performed. Instead, the momentum is updated through the updated value of the encoder. The momentum update formula is as follows: ; in, are the parameters of the momentum encoder, is the parameter of the encoder and m is a constant.
[0040] Furthermore, the difference between the momentum encoder and the encoder is that when the gradient is updated, the encoder parameters are back-propagated, and the momentum encoder calculates the parameter update value by combining its own parameters and the encoder parameters.
[0041] Furthermore, the momentum update performs a filtering operation on the weight changes caused by the drastic back-propagation, thereby enhancing the stability of the training and making it easier to converge.
[0042] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. A method for predicting the immune phenotype of triple-negative breast cancer based on mammography CT. It is characterized in that include: Extract and preprocess the mammography CT data in the existing breast cancer database to obtain a public data set, and preprocess the mammography CT images to be predicted to obtain multi-view data; An initial model is constructed based on a twin neural network, and molybdenum target CT images in a public data set are randomly regarded as key-value views. Based on the key-value views, queries are performed to obtain corresponding query views, and the same patient views of the query views are grouped to obtain positive samples, and other molybdenum target CT images are classified as negative samples. The initial model is trained based on the comparison of positive and negative samples to obtain a pre-trained encoder; Based on the pre-trained encoder, multi-scale feature extraction is performed on the multi-view data to obtain a multi-scale feature map, based on the multi-head self-attention mechanism, feature fusion is performed on the multi-scale feature map to obtain a comprehensive view feature map, based on the classifier, the comprehensive view feature map is classified, and the three-negative confidence is obtained and output.
2. The method for predicting triple-negative breast cancer immune phenotype based on mammography CT according to claim 1, It is characterized in that The initial model is trained based on positive and negative samples to obtain a pre-trained encoder, including: The encoder based on the convolutional neural network extracts features from the key-value view to obtain the baseline features, and the momentum encoder extracts features from the positive and negative samples of the query view to obtain the positive and negative feature vector sets. The InfoNCE loss function is constructed based on the similarity between the positive and negative feature vector sets and the benchmark features. The InfoNCE loss function is minimized based on the gradient descent method to maximize the similarity of the positive feature vectors and minimize the similarity of the negative feature vectors. When the InfoNCE loss function value tends to be stable, the training is terminated to obtain the pre-trained encoder.
3. The method for predicting triple-negative breast cancer immune phenotype based on mammography CT according to claim 2, It is characterized in that The InfoNCE loss function is constructed based on the similarity between the positive and negative feature vector sets and the benchmark features. The InfoNCE loss function is as follows: ; In the formula, is the number of sample features, is a key-value view representation, is the positive sample representation of the query view, is the negative representation of the query view, The function is the similarity function, is the total number of negative samples of the query view, is the temperature coefficient.
4. The method for predicting triple-negative breast cancer immune phenotype based on mammography CT according to claim 1, It is characterized in that Performing multi-scale feature extraction on the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map includes: Extracting features from multi-view data to obtain multi-view features; Build a bidirectional feature pyramid network based on weighted feature fusion mechanism; Image enhancement is performed on the multi-view features based on a bidirectional feature pyramid network to obtain a multi-scale feature map.
5. The method for predicting triple-negative breast cancer immune phenotype based on mammography CT according to claim 1, It is characterized in that Based on the multi-head self-attention mechanism, the multi-scale feature maps are fused to obtain a comprehensive view feature map. The feature fusion process includes: Based on the self-attention layer and feed-forward neural network, feature extraction is performed on the multi-scale feature map to obtain a feature vector set; Repeat twice to concatenate and downsample adjacent feature vectors in the feature vector set to obtain a comprehensive view feature map.
6. The method for predicting triple-negative breast cancer immune phenotype based on mammography CT according to claim 5, It is characterized in that In the process of extracting features from multi-scale feature maps based on the self-attention layer and the feedforward neural network to obtain a feature vector set, the self-attention layer includes 2 layers of Transformer, the number of heads in each layer is 4, and two downsampling operations are performed between the layers. Before the two downsampling operations, adjacent feature vectors are spliced respectively.
7. The method for predicting triple-negative breast cancer immune phenotype based on mammography CT according to claim 6, It is characterized in that Classifying the comprehensive view feature map based on a classifier to obtain and output a three-negative confidence, the classifier comprising: an input layer, a hidden layer and an output layer; The input layer is used to receive and transmit the comprehensive view feature map to the hidden layer; The hidden layer consists of 256 neurons, which are used to classify the comprehensive view features and predict the logical value; The output layer normalizes the logical value based on the sigmoid function and limits the output value to between 0 and 1 to obtain the confidence result.
8. A triple-negative breast cancer immune phenotype prediction system based on mammography CT. It is characterized in that include: A data processing module is used to extract and preprocess the mammography CT data in the existing breast cancer database to obtain a public data set, and preprocess the mammography CT images to be predicted to obtain multi-view data; A data training module, which obtains an initial model based on a twin neural network, randomly regards mammography CT images in a public data set as key-value views, performs queries based on the key-value views to obtain corresponding query views, groups views of the same patient in the query views to obtain positive samples, and classifies other mammography CT images as negative samples, and trains the initial model based on a comparison of positive and negative samples to obtain a pre-trained encoder; The triple-negative prediction module extracts multi-scale features from the multi-view data based on the pre-trained encoder to obtain a multi-scale feature map, fuses the multi-scale feature map based on the multi-head self-attention mechanism to obtain a comprehensive view feature map, and classifies the comprehensive view feature map based on the classifier to obtain and output the triple-negative confidence.
9. The triple-negative breast cancer immune phenotype prediction system based on mammography CT according to claim 8, It is characterized in that The initial model architecture in the data training module includes: encoder, decoder, momentum encoder, queue, and InfoNCE loss function; An encoder, composed of a convolutional neural network and a bidirectional feature pyramid network, is used for convolution, downsampling, feature concatenation, and global pooling of the positive and negative samples to obtain feature vectors; The decoder, which consists of a multi-layer perceptron with two hidden layers, is used to map the feature vector to the latent space to obtain the logical value; Momentum encoder, which has the same structure as the above encoder, but does not perform backpropagation during training and uses momentum update method; A queue for storing feature vectors generated by the momentum encoder; InfoNCE loss function, used to optimize the parameters of the initial model, indicates the closeness of positive and negative sample pairs.
Citation Information
Patent Citations
Digital breast tomographic image focus positioning device
CN114092463A
Mammary gland molybdenum target image segmentation method and system based on multi-view self-supervised deep learning
CN115170505A
Breast cancer full-slice image classification method combining self-supervision and weak supervision learning
CN117237733A
