A self-supervised method for classifying breast cancer

Abstract

This invention relates to the field of medical technology and discloses a self-supervised breast cancer classification method, comprising the following steps: S1: Image acquisition and dataset creation: acquiring contrast-enhanced spectral images of different modalities and multimodal contrast-enhanced spectral mammography images; S2: Data preprocessing: preprocessing the multimodal contrast-enhanced spectral mammography images, cropping the images, and performing pixel mean squarening; S3: Image feature extraction: extracting image features using a Vision Transformer model and providing image category predictions; S4: Loss calculation: calculating the distortion entropy of the predicted classification result and the information entropy loss between the category confidences generated by the classification branches of the two spectral images. This invention improves speed while ensuring accuracy, balancing the relationship between accuracy and efficiency, and proposes a breast cancer image classification model with high accuracy, fast speed, strong robustness, and good adaptability.

Description

Technical Field

[0001] This invention relates to the field of medical technology, specifically to a self-supervised method for classifying breast cancer. Background Technology

[0002] Breast cancer is the second leading cause of death among women worldwide, and among other types of cancer, it is considered the leading cause of death for women in most countries. Breast cancer screening methods mainly include mammography, breast ultrasound, breast MRI, and contrast-enhanced mammography. Contrast-enhanced mammography (also known as breast contrast-enhanced imaging) is an emerging breast cancer detection technology that uses a dual-energy contrast-enhanced mammography device to acquire images. The prototype of this device is a mammography system, modified to achieve dual-energy exposure. This method can simultaneously obtain multiple modalities of breast images, thus capturing the characteristics of different tumor morphologies. It also overcomes the tissue overlap phenomenon in mammography, providing high-resolution tumor information, making it a promising imaging technology.

[0003] Contrast-enhanced mammography is highly effective in detecting breast cancer. In clinical practice, however, the examination of breast images by clinicians is typically done manually, which is time-consuming, labor-intensive, and costly. Furthermore, radiologists may miss primary lesions due to overlapping dense tissues or variations in the shape, size, and boundaries of the mass. Machine learning-based image analysis methods can improve the accuracy of breast cancer detection and can serve as a "second opinion" to enhance the radiologist's work. Currently, this field has accumulated a certain research foundation, largely utilizing techniques from machine learning and deep learning. Machine learning-based recognition methods typically involve preprocessing and feature selection, failing to achieve an end-to-end detection process, and their accuracy is easily affected by various environmental disturbances. While deep learning-based recognition methods have significantly improved accuracy and can achieve an end-to-end detection process, the convolutional neural network model's reliance on regions of interest can lead to the neglect of overall image features. Furthermore, most existing AI-based methods directly apply methods for processing natural images to medical images without considering the specific characteristics of medical images. Target breast cancer image recognition is often accompanied by interference such as tumor occlusion, overlapping breast cancer images, and changes in lighting conditions, which drastically increases the recognition difficulty and reduces the recognition effectiveness of various automated classification technologies.

[0004] Therefore, there is an urgent need for an effective classifier to accurately help doctors predict this chronic disease. Using artificial intelligence to classify diseases from breast spectral images is an inevitable trend in future medicine, and breast spectral image classification, as a crucial component, has significant implications for the real-world application of cancer identification and classification. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a self-supervised breast cancer disease classification method, which solves the problems mentioned in the background section.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention provides the following technical solution: a self-supervised method for classifying breast cancer, comprising the following steps:

[0009] S1: Image Acquisition and Dataset Creation

[0010] Acquire contrast-enhanced spectroscopy images of different modalities and multimodal contrast-enhanced spectroscopy mammography images;

[0011] S2: Data Preprocessing

[0012] Data preprocessing was performed on multimodal contrast-enhanced spectral mammography images, and the images were cropped and pixel averaged.

[0013] S3: Image Feature Extraction

[0014] The Vision Transformer model is used to extract image features and provide image category predictions;

[0015] S4: Loss Calculation

[0016] Calculate the distortion entropy of the predicted classification result and the information entropy loss between the class confidences generated by the classification branches of the two energy spectrum images;

[0017] S5: Model Training

[0018] Based on the calculation results of the distortion entropy, the optimal model is obtained by fitting the training data through gradient backpropagation and iterative training of the model.

[0019] S6: Generate prediction results

[0020] Multimodal contrast-enhanced spectral mammography images are input into a classifier, which outputs the image classification results.

[0021] Preferably, the multimodal contrast-enhanced spectral mammography images in step S1 include low-energy spectral images and contrast-enhanced spectral images.

[0022] Preferably, the specific process of step S1 includes: S1.1: Given an unlabeled low-energy spectral mammography image training set S1 and an unlabeled contrast-enhanced mammography image training set S2, we randomly select a batch of spectral images of size B. And randomly select a batch of images of the same size B.

[0023] Preferably, the specific process of step S2 includes: S2.1: cropping the acquired breast spectral image: first, convert the spectral image into a grayscale image, then use coordinate matching technology to segment the region of interest of the target image into sub-images, and then scan the sub-images row by row. By traversing the image, determine whether a column is empty, that is, calculate whether the pixel value in this column is a completely black background. In the breast spectral image, as long as there are non-completely black pixels, the image area is considered to be an effective area that needs to be recognized and learned. In the grayscale image, the grayscale value of a pure black pixel is 0, and the representative value of a white pixel is 255. Therefore, in actual operation, a threshold close to 0 is set: if the grayscale value of the current pixel is lower than the threshold, the pixel can be determined to be black. If all the pixels in a row are black, the column can be deleted.

[0024] S2.2: Image enhancement and mean normalization of breast spectral images: The number of images is increased by translating, rotating, flipping, and adding noise to prevent overfitting of the network and enhance the robustness of the classification network. Global contrast normalization is then performed on the enhanced breast spectral images. Specifically, for an M×N pixel image, μ represents the mean of all pixels, σ is the standard deviation of all sample pixels, and the average intensity of each breast spectral image is calculated.

[0025] 4. Preferably, the specific process of step S3 includes:

[0026] S3.1: Image feature extraction: The processed energy spectrum images are output to the VisionTransformer in batches, including low-energy energy spectrum images and energy spectrum contrast enhancement images. Each batch of images is processed at a time. However, when performing feature extraction, each energy spectrum image is cut into non-overlapping image blocks and then input into the VisionTransformer classification model. The features contained in the deep feature map are gradually enriched through multi-head attention layers.

[0027] S3.2: Using Vision Transformer as the backbone architecture, image features are extracted; the feature maps output from VIT are fused using a lateral connection method to enrich the semantic representation capacity of the underlying feature maps, and finally, a classification judgment is made on the features; each energy spectrum image is cut into non-overlapping image blocks, and an additional classification block of the same size as the image block is set to learn the information of the entire image. The initial parameters are randomly initialized, and then position encoding and image information encoding are performed on each image block. The position encoding and image encoding information are then fused to obtain a vector matrix. This vector matrix is ​​then normalized, and the output vector features are input into the multi-head self-attention module. In the Vision Transformer self-attention module, let the input of each image block be X. i The input to the entire image is the input X of each image patch. i The matrix X is linearly transformed to obtain matrices Q, K, and V. Then, a self-attention operation is performed as follows to obtain the information representation of each image patch:

[0028] The feature outputs of each image patch are feature maps. These features are fused in a lateral connection manner and then integrated into the classification block. This enriches the semantic representation capacity of the underlying feature maps. The classification block learns the feature representations of all image patches for the image, giving it the same semantic capacity as the entire image. Finally, the classification block is input into a multilayer perceptron, and the output is then fed into a sigmoid activation function to obtain the class distribution.

[0029] 5. Preferably, the specific process of step S4 includes:

[0030] S4.1:

[0031] For the self-supervised learning network of the Siam structure, its model parameters are randomly initialized, and contrast-enhanced spectral mammography images of different modalities are input into the Vision Transformer classification model in step S3;

[0032] S4.2: Input the two breast spectral images into the Vision Transformer classification encoder with a Siamese structure. The output of the Transformer classification encoder is a P-value indicating whether the breast disease is benign or malignant (C-class). k =f θ (X k probability distribution on ) P i k P represents k The probability distribution of the i-th row, i.e., the i-th sample, is obtained using two modal energy spectrum images P. 1 and P 2Given the probability distribution, we define the objective function as follows:

[0033]

[0034] The category distribution is input into the distortion loss function to calculate the distortion entropy L(P). 1 P 2 Distortion entropy is a method for calculating the similarity between two samples based on information entropy, with the goal of making L(P) the most similar sample. 1 P 2 As small as possible, specifically D KL (P i 1 ||P i 2 The following formula provides an explanation.

[0035]

[0036] The first term D in formula (2) KL (||) represents the Kuhlbek-Leibele divergence between two probability distributions, also known as relative entropy. Relative entropy is a method used to describe the difference between two probability distributions; it is an asymmetric method. For example, P i 1 Relative to P i 2 The entropy is not equal to P i 2 Relative to P i 1 The entropy of P is selected in this invention. i 1 Relative to P i 2 Entropy, minimize To ensure consistency in the prediction of energy spectra of different breast modalities, energy spectrum images of different modalities of the same breast need to be identified as the same class. In formula (2), the second term H() represents the entropy of a probability distribution, where the negative logarithm of the probability represents the amount of information carried when a possible event occurs. The higher the probability, the smaller the effective information carried. Multiplying the information of each possible situation by its probability of occurrence and then summing them represents an expected value of all information in the entire system. The formula is as follows:

[0037]

[0038] for Minimizing the entropy of the class distribution for each sample regularizes the output distribution to sharpness, resulting in a deterministic class assignment for each image. Furthermore, the features of samples assigned to the same class become more compact. The third term in formula (2) maximizes the entropy of different samples. The entropy of the average distribution between them makes the predictions for different samples dispersed across class C, avoiding the network from assigning all images to the same class. Since there is a derivation relationship between cross entropy and relative entropy, formula (2) is derived to obtain the similarity function of the two probability distributions as shown in formula (4).

[0039]

[0040] The loss function described above is similar to the cross-entropy loss in supervised learning, where CE() represents cross-entropy. In the absence of ground truth labels for breast spectral images, it minimizes the cross-entropy loss between the class distributions of different modalities of breast spectra from the same patient. It can maximize the diversity of predictions by maximizing the diversity of predicted benign and malignant categories.

[0041] (III) Beneficial Effects

[0042] Compared with existing technologies, this invention provides a self-supervised method for classifying breast cancer, which has the following beneficial effects:

[0043] 1. This invention first preprocesses the acquired contrast-enhanced spectral mammography images, and then inputs the preprocessed contrast-enhanced spectral mammography images of the two modalities into a self-supervised classification network to extract image features before classification. The entire self-supervised structure adopts a Siamese network, in which the feature extraction and classification model is implemented using VisionTransformer. An information entropy-based loss function is used to make the images of the same patient in two different modalities more similar, and the feature differences of the breast images of two different patients in different modalities become larger, thereby providing more useful information for accurate classification and improving the accuracy of mammography image classification.

[0044] 2. This invention can comprehensively utilize the characteristics of contrast-enhanced mammography images to improve the accuracy of contrast-enhanced spectral mammography image classification and reduce the cost of manual annotation.

[0045] 3. This invention improves speed while ensuring accuracy, balancing the relationship between accuracy and efficiency, and proposes a breast cancer image classification model that is highly accurate, fast, robust, and adaptable. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the flowchart structure of the self-supervised contrast-enhanced spectral mammography image classification method in the self-supervised breast cancer disease classification method of the present invention;

[0047] Figure 2 This is a schematic diagram of the structure of breast cancer images under different interference scenarios collected in a self-supervised breast cancer disease classification method of the present invention.

[0048] Figure 3 This is a schematic diagram of the model dataset partitioning graph structure in a self-supervised breast cancer disease classification method of the present invention;

[0049] Figure 4 This is a schematic diagram of the model structure in a self-supervised breast cancer disease classification method of the present invention;

[0050] Figure 5 This is a schematic diagram of the Vision Transformer module structure of the present invention;

[0051] Figure 6 This is a model embodiment diagram of a self-supervised breast cancer disease classification method of the present invention. Detailed Implementation

[0052] like Figure 1-6 As shown, the present invention provides a technical solution: a self-supervised breast cancer disease classification method, comprising the following steps:

[0053] S1: Image Acquisition and Dataset Creation

[0054] Multimodal contrast-enhanced spectral mammography images are acquired, including low-energy spectral images and contrast-enhanced spectral images. The specific process includes: given an unlabeled low-energy spectral mammography image training set S1 and an unlabeled contrast-enhanced spectral mammography image training set S2, we randomly select a batch of spectral images of size B. And randomly select a batch of images of the same size B.

[0055] S2: Data Preprocessing

[0056] Data preprocessing of multimodal contrast-enhanced spectral mammography images, including image cropping and pixel averaging, is performed. The specific steps include:

[0057] S2.1:

[0058] The acquired breast spectral images are cropped as follows: First, the spectral images are converted into grayscale images. Then, coordinate matching technology is used to segment the region of interest of the target image into sub-images. The sub-images are then scanned row by row. By traversing the image, it is determined whether a column is empty, i.e., whether the pixel value in this column is a completely black background. In the breast spectral image, as long as there are non-completely black pixels, the image area is considered to be a valid area that needs to be identified and learned. In the grayscale image, the grayscale value of a pure black pixel is 0, and the representative value of a white pixel is 255. Therefore, in actual operation, a threshold close to 0 is set: if the grayscale value of the current pixel is lower than the threshold, the pixel can be determined to be black. If all the pixels in a row are black, the column can be deleted.

[0059] S2.2:

[0060] Image enhancement and mean normalization are performed on breast spectral images: The number of images is increased by translation, rotation, flipping, and noise addition to prevent overfitting in the network and enhance the robustness of the classification network. Global contrast normalization is then performed on the enhanced breast spectral images. Specifically, for an M×N pixel image, μ represents the mean of all pixels, σ is the standard deviation of all sample pixels, and the average intensity of each breast spectral image is calculated.

[0061] S3: Image Feature Extraction

[0062] The Vision Transformer model is used to extract image features and provide image category predictions. The specific process of this step includes:

[0063] S3.1:

[0064] Image feature extraction: The processed energy spectrum images are output to the VisionTransformer in batches, including low-energy energy spectrum images and energy spectrum contrast enhancement images. Each batch of images is processed at a time. However, when performing feature extraction, each energy spectrum image is cut into non-overlapping image blocks and then input into the VisionTransformer classification model. The features contained in the deep feature map are gradually enriched through a multi-head attention layer.

[0065] S3.2:

[0066] Using Vision Transformer as the backbone architecture, image features are extracted. The feature maps output from VIT are fused using lateral connections to enrich the semantic representation capacity of the lower-level feature maps, and finally, a classification judgment is made based on the features. Each energy spectrum image is sliced ​​into non-overlapping image blocks, and an additional classification block of the same size as the image block is set up to learn the information of the entire image. Initial parameters are randomly initialized. Then, position encoding and image information encoding are performed on each image block. The position encoding and image encoding information are then fused to obtain a vector matrix. This vector matrix is ​​then normalized, and the output vector features are input into a multi-head self-attention module. In the Vision Transformer self-attention module, let X be the input for each image block. i The input to the entire image is the input X of each image patch. i The matrix X is linearly transformed to obtain matrices Q, K, and V. Then, a self-attention operation is performed as follows to obtain the information representation of each image patch:

[0067]

[0068] The feature outputs of each image patch are feature maps. Feature fusion is performed in a lateral connection manner and all features are fused into the classification block to enrich the semantic representation capacity of the low-level feature maps. The classification block learns the feature representations of all image patches for this image, so that the classification block has the same level of semantic capacity as the whole image. Finally, the classification block is input into a multilayer perceptron, and the output is input into a sigmoid activation function to obtain the class distribution.

[0069] S4: Loss Calculation

[0070] The specific steps for calculating the distortion entropy of the predicted classification result and the information entropy loss between the class confidence scores of the two energy spectrum images include:

[0071] S4.1:

[0072] For the self-supervised learning network of the Siam structure, its model parameters are randomly initialized, and contrast-enhanced spectral mammography images of different modalities are input into the Vision Transformer classification model in step S3;

[0073] S4.2:

[0074] Multimodal breast spectral images are input into a Vision Transformer classification encoder with a Siamese structure. The output of the Transformer classification encoder is a P-value indicating benign or malignant breast disease (C-class). k =f θ (X k probability distribution on ) P i k P represents k The probability distribution of the i-th row, i.e., the i-th sample, is obtained using two modal energy spectrum images P. 1 and P 2 Given the probability distribution, we define the objective function as follows:

[0075]

[0076] The category distribution is input into the distortion loss function to calculate the distortion entropy L(P). 1 P 2 Distortion entropy is a method for calculating the similarity between two samples based on information entropy, with the goal of making L(P) the most similar sample. 1 P 2 As small as possible, specifically D KL (P i 1 ||P i 2 The following formula provides an explanation.

[0077]

[0078] The first term D in formula (2) KL () represents the Kuhlbek-Leibele divergence between two probability distributions, also known as relative entropy. Relative entropy is a method used to describe the difference between two probability distributions; it is an asymmetric method. For example, P i 1 Relative to P i 2 The entropy is not equal to P i 2 Relative to P i 1 The entropy of P is selected in this invention. i 1 Relative to P i 2 Entropy, minimize To ensure consistency in the prediction of different modalities of breast energy spectrum, the multimodal energy spectrum images of the same patient's breast need to be identified as the same class. In formula (2), the second term H() represents the entropy of a probability distribution, where the negative logarithm of the probability represents the amount of information carried when a possible event occurs. The higher the probability, the smaller the effective information carried. Multiplying the information of each possible situation by its probability of occurrence and then summing them represents an expected value of all the information in the entire system. The formula is as follows:

[0079]

[0080] for Minimizing the entropy of the class distribution for each sample regularizes the output distribution to sharpness, resulting in a deterministic class assignment for each image. Furthermore, the features of samples assigned to the same class become more compact. The third term in formula (2) maximizes the entropy of different samples. The entropy of the average distribution between them makes the predictions for different samples dispersed across class C, avoiding the network from assigning all images to the same class. Since there is a derivation relationship between cross entropy and relative entropy, formula (2) is derived to obtain the similarity function of the two probability distributions as shown in formula (4).

[0081]

[0082] The loss function described above is similar to the cross-entropy loss in supervised learning, where CE() represents cross-entropy. In the absence of ground truth labels for breast spectral images, it minimizes the cross-entropy loss between the class distributions of different modalities of breast spectra from the same patient. It can maximize the diversity of predictions by maximizing the diversity of predicted benign and malignant categories;

[0083] S5: Model Training

[0084] Based on the calculation results of the distortion entropy, the optimal model is obtained by fitting the training data through gradient backpropagation and iterative training of the model.

[0085] S6: Generate prediction results

[0086] Multimodal contrast-enhanced spectral mammography images are input into a classifier, and the image classification results are output.

[0087] according to Figure 6 The system that is aware of the present invention includes:

[0088] The image input module takes the acquired contrast-enhanced mammogram images and inputs them into the system, awaiting further processing.

[0089] The preprocessing module performs preprocessing on the contrast-enhanced breast spectral imaging image, cropping the black-edge background from the breast spectral image;

[0090] The image enhancement module performs image enhancement processing and global contrast homogenization processing on the cropped breast spectral imaging image.

[0091] The feature extraction module extracts features from the scale-adjusted breast spectral image based on a preset deep neural network classification model, and fuses the features from two views and two modalities of the same breast.

[0092] The classification result output module classifies the energy spectrum image into benign or malignant types.

[0093] In summary, this invention can comprehensively utilize the characteristics of contrast-enhanced mammography images to improve the accuracy of contrast-enhanced spectral mammography image classification, reduce the cost of manual annotation, and improve speed while ensuring accuracy. It balances the relationship between accuracy and efficiency, and proposes a breast cancer image classification model that is highly accurate, fast, robust, and adaptable.

Claims

1. A self-supervised breast cancer disease classification method, characterized in that, Includes the following steps: S1: Image Acquisition and Dataset Creation Multimodal contrast-enhanced spectral mammography images were acquired, including low-energy spectral images and contrast-enhanced spectral images. S2: Data Preprocessing Data preprocessing was performed on multimodal contrast-enhanced spectral mammography images, and the images were cropped and pixel averaged. S3: Image Feature Extraction The Vision Transformer model is used to extract image features and provide image category predictions; S4: Loss Calculation Calculate the distortion entropy of the predicted classification result and the information entropy loss between the class confidences generated by the classification branches of the two energy spectrum images; S4.1： For the self-supervised learning network of the Siam structure, its model parameters are randomly initialized, and contrast-enhanced spectral mammography images of different modalities are input into the Vision Transformer classification model in step S3; S4.2： Two breast spectral images were input into a Vision Transformer classification encoder with a Siamese structure. The output of the Transformer classification encoder was a classification of breast disease into benign and malignant categories (C-level). probability distribution on ), express The Line, i.e., the first The probability distribution of each sample is obtained using two modal energy spectrum images. and Given the probability distribution, we define the objective function as follows: （1） The category distribution is input into the distortion loss function to calculate the distortion entropy. Distorted entropy is a method for calculating the similarity between two samples based on information entropy, with the goal of making the similarity between two samples equal to the information entropy between them. As small as possible, specific The following formula is explained. （2） The first term D in formula (2) KL ( ) represents the Kuhlbek-Leibele divergence between two probability distributions, also known as relative entropy. Relative entropy is a method used to describe the difference between two probability distributions; it is an asymmetric method of description. Compared to The entropy is not equal to Compared to The entropy, selected relatively Entropy, minimize To ensure consistency in the prediction of energy spectra of different breast modalities, energy spectrum images of different modalities of the same breast need to be identified as the same class. In formula (2), the second term H() represents the entropy of a probability distribution, where the negative logarithm of the probability represents the amount of information carried when a possible event occurs. The higher the probability, the smaller the effective information carried. Multiplying the information of each possible situation by its probability of occurrence and then summing them represents an expected value of all information in the entire system. The formula is as follows: （3） for Minimizing the entropy of the class distribution for each sample regularizes the output distribution to sharpness, which gives each image a deterministic class assignment. Furthermore, the features of samples assigned to the same class will be more compact. The third term in formula (2) maximizes the entropy of different samples. The entropy of the average distribution between them makes the predictions for different samples dispersed across class C, avoiding the network from assigning all images to the same class. Since there is a derivation relationship between cross entropy and relative entropy, formula (2) is derived to obtain the similarity function of the two probability distributions as shown in formula (4). （4） The loss function described above is similar to the cross-entropy loss in supervised learning, where CE() represents cross-entropy. In the absence of ground truth labels for breast spectral images, it minimizes the cross-entropy loss between the class distributions of different modalities of breast spectra from the same patient. It can maximize the diversity of predictions by maximizing the diversity of predicted benign and malignant categories; S5: Model Training Based on the calculation results of the distortion entropy, the optimal model is obtained by fitting the training data through gradient backpropagation and iterative training of the model. S6: Generate prediction results Multimodal contrast-enhanced spectral mammography images are input into a classifier, which outputs the image classification results.

2. The self-supervised breast cancer disease classification method according to claim 1, characterized in that, The specific process of step S1 includes: S1.1： Given an unlabeled low-energy spectral mammography image training set S1 and an unlabeled contrast-enhanced mammography image training set S2, we randomly select a batch of spectral images X1⊂S1 of size B, and randomly select a batch of images X2⊂S2 of size B.

3. The self-supervised breast cancer disease classification method according to claim 1, characterized in that, The specific process of step S2 includes: S2.1： The acquired breast spectral images are cropped as follows: First, the spectral images are converted into grayscale images. Then, coordinate matching technology is used to segment the region of interest of the target image into sub-images. The sub-images are then scanned row by row. By traversing the image, it is determined whether a column is empty, i.e., whether the pixel value in this column is a completely black background. In the breast spectral image, as long as there are non-completely black pixels, the image area is considered to be a valid area that needs to be identified and learned. In the grayscale image, the grayscale value of a pure black pixel is 0, and the representative value of a white pixel is 255. Therefore, in actual operation, a threshold close to 0 is set: if the grayscale value of the current pixel is lower than the threshold, the pixel can be determined to be black. If all the pixels in a row are black, the column can be deleted. S2.2： Image enhancement and averaging are performed on breast spectral images: The number of images is increased through translation, rotation, flipping, and noise addition to prevent overfitting in the network and enhance the robustness of the classification network. Global contrast normalization is then applied to the enhanced breast spectral images. Specifically, for… Image of pixel size, using This represents the mean value of all pixels in the image. Calculate the average intensity of each breast spectral image based on the standard deviation of all sample pixels. .

4. The self-supervised breast cancer disease classification method according to claim 1, characterized in that, The specific process of step S3 includes: S3.1： Image feature extraction: The processed energy spectrum images are output to the Vision Transformer in batches, including low-energy energy spectrum images and energy spectrum contrast enhancement images. Each batch of images is processed at a time. However, when performing feature extraction, each energy spectrum image is cut into non-overlapping image blocks and then input into the Vision Transformer classification model. The features contained in the deep feature map are gradually enriched through a multi-head attention layer. S3.2： Using Vision Transformer as the backbone architecture, image features are extracted. The feature maps output from VIT are fused using lateral connections to enrich the semantic representation capacity of the lower-level feature maps, and finally, a classification judgment is made based on the features. Each spectral image is sliced ​​into non-overlapping image blocks, and an additional classification block of the same size as the image block is set up to learn the information of the entire image. Initial parameters are randomly initialized. Then, position encoding and image information encoding are performed on each image block. The position encoding and image encoding information are then fused to obtain a vector matrix. This vector matrix is ​​then normalized, and the output vector features are input into a multi-head self-attention module. In the Vision Transformer self-attention module, the input for each image block is set as follows: The input for the entire image is the input for each image patch. The matrix formed After a linear transformation, the Q, K, and V matrices are obtained. Then, a self-attention operation is performed as follows to obtain the information representation of each image patch: The feature outputs of each image patch are feature maps. These features are fused in a lateral connection manner and then integrated into the classification block. This enriches the semantic representation capacity of the underlying feature maps. The classification block learns the feature representations of all image patches for the image, giving it the same semantic capacity as the entire image. Finally, the classification block is input into a multilayer perceptron, and the output is then fed into a sigmoid activation function to obtain the class distribution.

Images