Breast cancer lesion identification method and device

Abstract

The application discloses a breast cancer lesion identification method and device, applied to the technical field of medical imaging, and comprises the following steps: acquiring a first sample set and a second sample set, the first sample set comprising a plurality of two-dimensional images corresponding to a plurality of normal sample DBT images respectively, the second sample set comprising a plurality of two-dimensional images corresponding to a plurality of normal sample DBT images respectively and a plurality of two-dimensional images corresponding to a plurality of abnormal sample DBT images respectively, each abnormal sample DBT image having corresponding lesion annotation information; pre-training a diffusion model by using the first sample set to obtain a pre-trained diffusion model; taking the pre-trained diffusion model as a teacher network and taking an identification model as a student network, and performing knowledge distillation-based training processing on the identification model by using the second sample set to obtain a trained identification model; and performing identification processing on a to-be-tested DBT image by using the trained identification model to obtain a breast cancer lesion identification result. The application can effectively improve the identification precision of breast cancer lesions.

Description

TECHNICAL FIELD

[0001] The present application relates to the medical imaging field, in particular to a breast cancer lesion identification method and device. BACKGROUND

[0002] Digital breast tomosynthesis (DBT) technology is an advanced breast screening technology combining three-dimensional tomography and two-dimensional synthesis. In the related art, a neural network model is trained using an existing image sample set to obtain a trained identification model, which is used to identify the lesion area of breast cancer. However, the feature information in the lesion area of breast cancer is often complex and diverse, and the current identification model is usually a convolutional neural network-based identification model, which has limited feature extraction capability, resulting in low identification accuracy of breast cancer lesions. SUMMARY

[0003] The embodiments of the present application provide a breast cancer lesion identification method and device to improve the identification accuracy of breast cancer lesions.

[0004] In one aspect, the embodiments of the present application provide a breast cancer lesion identification method, comprising the following steps:

[0005] Obtain a first sample set and a second sample set, the first sample set comprising a plurality of two-dimensional images corresponding to a plurality of sample DBT images without breast cancer lesions, and the second sample set comprising a plurality of two-dimensional images corresponding to a plurality of sample DBT images without breast cancer lesions and a plurality of two-dimensional images corresponding to a plurality of sample DBT images containing breast cancer lesions, each sample DBT image containing breast cancer lesions having corresponding lesion annotation information;

[0006] Pre-train a diffusion model using the first sample set to obtain a pre-trained diffusion model;

[0007] Use the pre-trained diffusion model as a teacher network and an identification model as a student network, and perform knowledge distillation-based training processing on the identification model using the second sample set to obtain a trained identification model;

[0008] Use the trained identification model to identify the to-be-tested DBT image to obtain a breast cancer lesion identification result.

[0009] In another aspect, the embodiments of the present application provide a breast cancer lesion identification device, comprising:

[0010] The acquisition module is configured to acquire a first sample set and a second sample set, the first sample set including a plurality of two-dimensional images respectively corresponding to a plurality of breast cancer lesion-free sample DBT images, and the second sample set including a plurality of two-dimensional images respectively corresponding to a plurality of breast cancer lesion-free sample DBT images and a plurality of two-dimensional images respectively corresponding to a plurality of breast cancer lesion-containing sample DBT images, each breast cancer lesion-containing sample DBT image having corresponding lesion annotation information;

[0011] The pre-training module is configured to pre-train the diffusion model using the first sample set to obtain a pre-trained diffusion model.

[0012] The training module is configured to use the pre-trained diffusion model as a teacher network, use an identification model as a student network, and perform knowledge distillation-based training processing on the identification model using the second sample set to obtain a trained identification model.

[0013] The identification module is configured to perform identification processing on a to-be-tested DBT image using the trained identification model to obtain a breast cancer lesion identification result.

[0014] The breast cancer lesion identification method and device provided by the present application have the following beneficial effects: first, a first sample set and a second sample set are acquired, the first sample set including a plurality of two-dimensional images respectively corresponding to a plurality of normal sample DBT images, and the second sample set including a plurality of two-dimensional images respectively corresponding to a plurality of normal sample DBT images and a plurality of two-dimensional images respectively corresponding to a plurality of abnormal sample DBT images, each abnormal sample DBT image having corresponding lesion annotation information; then, a diffusion model is pre-trained using the first sample set to obtain a pre-trained diffusion model; subsequently, the pre-trained diffusion model is used as a teacher network, an identification model is used as a student network, and the identification model is subjected to knowledge distillation-based training processing using the second sample set to obtain a trained identification model; finally, the trained identification model is used to perform identification processing on a to-be-tested DBT image to obtain a breast cancer lesion identification result. As can be seen, the diffusion model is first pre-trained in a self-supervised manner using a large number of normal sample DBT images, and in this process, the diffusion model can learn the distribution rule of breast gland features, then the pre-trained diffusion model is used as a teacher network, an identification model is used as a student network, and the identification model is subjected to knowledge distillation-based training processing using a large number of normal sample DBT images and a large number of abnormal sample DBT images, so as to transfer the feature extraction capability of the diffusion model to the downstream identification model, and finally the trained identification model is used to implement breast cancer lesion identification processing, which can effectively improve the feature extraction capability of the identification model, enable the identification model to effectively capture complex and variable lesion representations in the lesion region of breast cancer, and thus improve the identification accuracy of breast cancer lesions. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a flow chart of a breast cancer lesion recognition method provided by the present application;

[0016] Figure 2A -B is a schematic diagram of a DBT image preprocessing operation provided by the present application;

[0017] Figure 3 is a structural diagram of a diffusion model provided by the present application;

[0018] Figure 4 is a structural diagram of a denoising generation module provided by the present application;

[0019] Figure 5 is a structural diagram of a recognition model provided by the present application;

[0020] Figure 6 is a schematic diagram of distillation learning provided by the present application;

[0021] Figure 7 is an example diagram of three-dimensional aggregation provided by the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0023] The present application will be further described below in combination with the drawings and specific embodiments of the present application. The described embodiments should not be considered as limiting the present application, and all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0024] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0026] Digital Breast Tomosynthesis (DBT) is an advanced breast screening technology combining three-dimensional tomographic imaging and two-dimensional synthesis. In the related art, a neural network model is usually trained using an existing image sample set to obtain a trained recognition model for identifying the lesion area of breast cancer. However, the related art has the following defects:

[0027] 1. The available labeled DBT images are scarce, which leads to insufficient DBT images for training the recognition model. Specifically, first, since the DBT image contains the patient's privacy information, it is usually necessary to go through a complex and strict application process to obtain the right to use the DBT image, which makes it difficult to obtain the DBT image; second, the cost of labeling the DBT image is high, and it often takes multiple experienced radiologists to cooperate to complete the labeling of the lesion area in the DBT image, which makes it difficult to obtain the labeling information of the DBT image; third, since DBT technology is a newly emerging technology, it has developed relatively late, so the labeled DBT image samples are usually few. These factors lead to insufficient DBT images for training the recognition model, so that the recognition model cannot be fully trained.

[0028] 2. The feature information in the lesion area of breast cancer is often complex and diverse, which increases the difficulty of training and feature extraction of the recognition model. Specifically, the current breast cancer lesions can be described as intramammary mass, calcification, structural distortion, asymmetry and other abnormal manifestations. The representation of these lesions is complex and diverse, and some lesions are difficult to find in breast images. Among them, the positioning of intramammary mass often involves three aspects of mass edge, shape and density, and the edge sign of the mass is the most important for judging the nature of the mass. Calcification has complex and diverse types and distribution. Calcification can be divided into typical benign calcification and suspicious calcification in terms of type, and these two types also have multiple morphological characteristics. Calcification can be scattered, clustered, regional, linear and segmental in distribution. Structural distortion refers to the distortion of normal structures without clear visible mass, including radial shadows from a point and local contraction, or edge distortion of the substance. Structural distortion is more difficult to detect than mass and calcification. Asymmetry refers to the asymmetric structure visible only in one projection position compared to the contralateral breast, which is 80% likely to be caused by the overlap of normal tissues. These complex and diverse lesion representations not only increase the difficulty of training the recognition model, but also enhance the ability requirements of the recognition model in feature extraction. However, the current recognition model is usually a recognition model based on convolutional neural network, which is difficult to capture the complex and diverse lesion representation in the lesion area of breast cancer, and its feature extraction capability is low.

[0029] 3. The high-resolution DBT image increases the computational cost of the recognition model. Specifically, in breast X-ray imaging, in order to ensure that sufficient detailed information can be captured, the original resolution of the DBT image is usually large, for example, the pixel size of a DBT image constructed by a certain instrument can reach 3000x4000x65 on average. For the recognition model, the higher the resolution of the input image, the more pixel points the model needs to process, and the more computational resources the model needs. Therefore, the high-resolution DBT image increases the computational cost of the recognition model. Although some related technologies attempt to reduce the size of the DBT image to reduce the computational cost of the recognition model, the way of reducing the DBT image will affect the presentation of key features such as tumor edge, microcalcification, and atypical structure distortion, thereby seriously affecting the performance of the recognition model in identifying breast cancer lesion regions.

[0030] To this end, the embodiments of the present application provide a breast cancer lesion identification method and device, aiming to solve the problems of insufficient DBT image for training the recognition model, the existing recognition model being difficult to capture the complex and variable lesion representation in the lesion region of breast cancer, and the high-resolution DBT image increasing the computational cost of the recognition model, and effectively improve the recognition accuracy of the breast cancer lesion.

[0031] It should be noted that in each specific embodiment of the present disclosure, when it is necessary to process related data such as sample DBT images, to-be-detected DBT images, etc., the permission or consent of the subject will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present disclosure need to obtain related data such as sample DBT images, to-be-detected DBT images, etc., the separate permission or separate consent of the subject will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the separate permission or separate consent of the subject, the necessary related data such as sample DBT images, to-be-detected DBT images, etc. for enabling the embodiments of the present disclosure to function normally will be obtained.

[0032] It should be emphasized that the sample DBT images and to-be-detected DBT images involved in the embodiments of the present application are image data pre-stored in a certain database, rather than image data obtained from a patient in real time. That is, the embodiments of the present application do not take a living human body as the object. Moreover, the final output of the embodiments of the present application is the lesion region in the DBT image, which is provided to the doctor for diagnosis reference, rather than the final diagnosis result, and the final diagnosis of breast cancer still needs to be judged and decided by the doctor. That is, the embodiments of the present application do not directly aim to obtain the diagnosis result of breast cancer. Therefore, the embodiments of the present application do not belong to the diagnosis and treatment method of the disease.

[0033] The implementation steps of the breast cancer lesion identification method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0034] The breast cancer lesion identification method provided by the embodiments of the present application can be applied in a terminal, can be applied in a server, and can also be software running in a terminal or a server, etc. The terminal can be a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The server can be a standalone physical server, can be a server cluster or a distributed system composed of multiple physical servers, can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. In addition, the server can also be a node server in a blockchain network, but is not limited thereto. The blockchain is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc.

[0035] Reference Figure 1 The breast cancer lesion identification method provided by the embodiments of the present application can include the following steps S101-S104.

[0036] S101, acquiring a first sample set and a second sample set.

[0037] It should be noted that the first sample set can include a plurality of two-dimensional images respectively corresponding to a plurality of sample DBT images not containing breast cancer lesions. The second sample set can include a plurality of two-dimensional images respectively corresponding to a plurality of sample DBT images not containing breast cancer lesions, and a plurality of two-dimensional images respectively corresponding to a plurality of sample DBT images containing breast cancer lesions, each of the sample DBT images containing breast cancer lesions having corresponding lesion annotation information. The lesion annotation information includes the category of the breast cancer lesion in the DBT image and the position information of the breast cancer lesion in the sample DBT image, and the category of the breast cancer lesion refers to the breast adenosis lesion category that can be detected in the DBT image, including mass, calcification, structural distortion, asymmetry, etc.

[0038] In this step, first, a plurality of sample DBT images are obtained, which can be divided into two categories containing breast cancer lesions and not containing breast cancer lesions. Among them, the sample DBT image containing the breast cancer lesion has lesion annotation information, which is pre-annotated information. Then, since the DBT image is a three-dimensional image constructed according to different angle projections, and the requirements of the diffusion model and the recognition model for the input data are both two-dimensional images, it is necessary to perform preprocessing such as two-dimensional segmentation on each sample DBT image to obtain a plurality of two-dimensional images corresponding to each sample DBT image, and then construct a first sample set and a second sample set.

[0039] Optionally, the DBT data set disclosed by Duke University is selected, which contains 22032 DBT images of 5060 patients, a total of 4807 normal cases and 201 labeled abnormal cases, and the rest are suspicious cases to be screened. 4600 DBT images (normal cases) not containing breast cancer lesions are selected as the first sample set, and 107 DBT images (normal cases) not containing breast cancer lesions and 101 DBT images (abnormal cases) containing breast cancer lesions are selected as the second sample set. In the first sample set, 4000 cases are the training set, 300 cases are the verification set, and 300 cases are the test set. In the verification and test stages of the recognition model, 50 DBT images (normal cases) containing breast cancer lesions are used as negative samples, and 50 DBT images (abnormal cases) containing breast cancer lesions are used as positive samples, and all parameters of the recognition model are frozen.

[0040] S102, pre-training the diffusion model using the first sample set to obtain a pre-trained diffusion model.

[0041] In this step, before training the recognition model, the diffusion model is first pre-trained in a self-supervised manner based on a plurality of two-dimensional images respectively corresponding to a large number of sample DBT images not containing breast cancer lesions. In the self-supervised pre-training, the diffusion model has different noise levels in the diffusion process, which enables the diffusion model to obtain features of different scales, thereby prompting the diffusion model to learn the gland distribution features of the DBT image in the case of a lack of labeled samples and capture the regularity of the breast gland distribution.

[0042] S103, using the pre-trained diffusion model as a teacher network and the recognition model as a student network, performing knowledge distillation-based training processing on the recognition model using the second sample set to obtain a trained recognition model.

[0043] In this step, the pre-trained diffusion model can learn rich and high-dimensional gland distribution features. The pre-trained diffusion model is used as a teacher network, and the recognition model is used as a student network. The recognition model is trained based on knowledge distillation based on a plurality of two-dimensional images corresponding to a small number of sample DBT images without breast cancer lesions and a plurality of two-dimensional images corresponding to a small number of sample DBT images containing breast cancer lesions. On the one hand, the pre-trained diffusion model as the teacher network can guide the recognition model as the student network to effectively extract the gland distribution features of the DBT image, so that the recognition model can capture the law of breast gland distribution from the DBT image, thereby migrating the feature extraction capability of the upstream diffusion model to the downstream recognition model, and effectively improving the feature extraction capability of the recognition model for the DBT image. On the other hand, the network structure of the pre-trained diffusion model is more complex than that of the recognition model, and the task objectives of the two are different. The task objective of the former is diffusion and generation, and the task objective of the latter is classification and positioning, which easily leads to the general features learned by the pre-trained diffusion model cannot be directly migrated to the task objective of the recognition model. Therefore, the distillation learning is introduced to train the recognition model in this embodiment of the present application. The distillation learning can effectively migrate the feature extraction capability of the DBT image from the upstream diffusion model to the downstream recognition model, reduce the difference between the pre-trained diffusion model and the recognition model in network structure and task objective, realize the knowledge migration from the complex model to the lightweight model, and thus ensure that the feature extraction capability of the recognition model can be improved.

[0044] In S104, the trained recognition model is used to recognize the to-be-tested DBT image to obtain a breast cancer lesion recognition result.

[0045] It should be noted that the breast cancer lesion recognition result is used to indicate the breast cancer lesion region of the to-be-tested DBT image, that is, the position information of the breast cancer lesion region in the to-be-tested DBT image.

[0046] In this step, first, the to-be-tested DBT image is obtained. Then, since the trained recognition model requires two-dimensional images as input data, the to-be-tested DBT image needs to be preprocessed, such as two-dimensional segmentation, to obtain a plurality of two-dimensional images corresponding to the to-be-tested DBT image. The preprocessing operation of the to-be-tested DBT image is the same as the preprocessing operation of the sample DBT image. Then, the plurality of two-dimensional images corresponding to the to-be-tested DBT image are input into the trained recognition model, and the breast cancer lesion region of the to-be-tested DBT image can be recognized by the pre-trained recognition model, thereby realizing the recognition of the breast cancer lesion.

[0047] The steps of the embodiments of the present application will be further described below with reference to the accompanying drawings.

[0048] (I) Preprocessing operation of the DBT image:

[0049] In some embodiments, referring to Figure 2A -B, the obtaining of the first sample set and the second sample set in the step S101 can include:

[0050] obtaining a plurality of sample DBT images without breast cancer lesions and a plurality of sample DBT images containing breast cancer lesions, and performing the following operations for each sample DBT image:

[0051] cropping the sample DBT image to obtain a cropped image of the sample DBT image;

[0052] performing two-dimensional segmentation processing on the cropped image of the sample DBT image to obtain a plurality of two-dimensional images of the sample DBT image.

[0053] In this embodiment, a plurality of sample DBT images without breast cancer lesions and a plurality of sample DBT images containing breast cancer lesions are obtained. In order to avoid the influence of the difference of the input images on the effect of transfer learning, the same preprocessing operation is performed on the pre-training input and the distillation learning input in this embodiment. Specifically, for each sample DBT image, the following operations are performed:

[0054] Since the DBT image is a three-dimensional image constructed according to different angle projections, it has high resolution and contains a large number of black regions without gland information, and the continuous sections of these black regions are similar, so as shown in Figure 2A According to the characteristics of the DBT image, the black region without gland information in the sample DBT image is cropped to obtain a cropped image of the sample DBT image, which helps to reduce the computational complexity of the diffusion model and the recognition model.

[0055] Then, since the requirements of the diffusion model and the recognition model for the input data are both two-dimensional images, and the cropped image of the sample DBT image is a three-dimensional image, the cropped image of the sample DBT image needs to be processed by two-dimensional segmentation in this embodiment. Specifically, first, the sample DBT image is segmented into a plurality of continuous initial two-dimensional images, and the plurality of initial two-dimensional images are combined into a plurality of two-dimensional sections. Each two-dimensional section is composed of a preset number of initial two-dimensional images, and the preset number of images can be flexibly set according to actual conditions, for example, the preset number of images can be 10, but is not limited thereto. As shown in Figure 2B In the space, the sample DBT image is segmented along the The axis is divided into a plurality of two-dimensional sections. Then, the plurality of two-dimensional sections are respectively subjected to a reduction processing to obtain a plurality of reduced two-dimensional sections. The reduction processing here refers to a proper reduction of the width-height ratio of the two-dimensional sections. After that, the plurality of reduced two-dimensional sections are respectively subjected to a padding processing so that each of the reduced two-dimensional sections is padded to a size of 1024x1024 to obtain a plurality of padded two-dimensional sections. Finally, in order to improve the effect of pre-training and the effect of distillation learning, the plurality of padded two-dimensional sections are respectively subjected to a normalization and standardization operation, thereby obtaining a plurality of two-dimensional images of the sample DBT image, the two-dimensional images corresponding one-to-one to the two-dimensional sections. The normalization and standardization operation satisfy the following formulas (1)-(2):

[0056] (1);

[0057] In formula (1), represents the pixel value of the padded two-dimensional section; represents the minimum pixel value of the padded two-dimensional section; represents the maximum pixel value of the padded two-dimensional section; represents the normalized two-dimensional section.

[0058] (2);

[0059] In formula (2), represents the average mean of all normalized two-dimensional sections; represents the average standard deviation of all normalized two-dimensional sections; represents the standardized two-dimensional section, i.e. the two-dimensional image of the sample DBT image.

[0060] (B) Diffusion model and self-supervised pre-training thereof:

[0061] In some embodiments, with reference to Figure 3 the diffusion model can include:

[0062] a diffusion encoding module configured to map the two-dimensional image to a latent feature;

[0063] a quantization module configured to convert the latent feature into an embedding vector closest to a preset codebook;

[0064] a noisy diffusion module configured to add Gaussian noise to the embedding vector based on a preset time step to obtain a noisy feature;

[0065] a denoising generation module configured to perform denoising prediction processing according to the time step and the noisy feature to obtain a predicted noise;

[0066] The inverse decoding module is configured to obtain a generated image of the two-dimensional image according to the added noise feature and the predicted noise.

[0067] In the embodiment, the diffusion model based on the latent space includes a vector quantized generative adversarial network (VQ-GAN), an added noise diffusion module, and a denoising generation module. The diffusion encoding module and the inverse decoding module in the VQ-GAN realize the mapping of the two-dimensional image between the high-dimensional pixel space and the low-dimensional latent space, and the quantization module realizes the conversion between the continuous feature representation and the discrete feature representation. The added noise diffusion module can add noise to the latent space of the input two-dimensional image according to the time step. The denoising generation module can predict the added noise according to the time step, and then predict the low-dimensional latent representation of the two-dimensional image, and restore the generated image through the inverse decoding module in the VQ-GAN.

[0068] The overall diffusion process can be divided into a forward diffusion process and a reverse denoising process, as shown in FIG. 1. Figure 3 The left side of the dashed line is the forward diffusion process, and the right side of the dashed line is the reverse denoising process. In the forward diffusion process, the two-dimensional image is mapped to the low-dimensional continuous latent space by the diffusion encoding module, is converted into a discrete feature representation by the quantization module, and then is subjected to noise addition processing by the added noise diffusion module to obtain the latent representation with added noise . In the reverse denoising process, the latent representation with added noise is input into the embedding layer (Emb) and the positional encoding layer (PE), and the outputs of the two layers are spliced (Concat) to obtain the input of the reverse denoising process. In the reverse denoising process, the denoising prediction processing is performed by the denoising generation module to obtain the predicted noise , and the predicted noise and the latent representation are converted into the latent space by the inverse decoding module, and the latent space is mapped to the pixel space to obtain the generated image of the two-dimensional image . ​​​In this process, the diffusion module does not need to rely on additional lesion annotation information, and it can obtain more rich and high-dimensional breast gland features through self-supervised pre-training and diffusion process, thereby prompting the diffusion model to learn the gland distribution features of the DBT image in the case of lack of labeled samples and capture the rules of breast gland distribution.

[0069] The functions and structures of the components of the diffusion model will be further described below in conjunction with the accompanying drawings.

[0070] 1) a diffusion encoding module, which is denoted as "VQ-GAN encoder" in Figure 3 , the module is the encoder of the VG-GAN, which is composed of multiple convolutional layers and multiple residual blocks. It is assumed that each time an input two-dimensional image is input, the size is , is the number of channels of the two-dimensional image, is the height of the two-dimensional image, is the width of the two-dimensional image, as shown on the left side of the dashed line in Figure 3 , the input two-dimensional image is mapped to a low-dimensional continuous latent space , i.e. latent features, the size becomes , the number of channels of the low-dimensional continuous latent features is the same as the dimension of the embedding vector in the codebook of the quantization module, is the height of the low-dimensional continuous latent features, , is the width of the low-dimensional continuous latent features, , is the down-sampling factor. Wherein, the down-sampling factor can be flexibly set according to the actual situation, for example, the down-sampling factor can be 16, but not limited to this.

[0071] 2) a quantization module, which is denoted as "quantization" in Figure 3 , the module is configured with a codebook with a size of , which comes from a public pre-trained model. Wherein, is the number of embedding vectors in the codebook, is the dimension of the embedding vector. Optionally, , . The latent features are input to the quantization module, and the quantization module converts the latent features into the embedding vector closest to the codebook based on the nearest neighbor search algorithm, and the size of the embedding vector is , wherein Specifically, using a brute-force search algorithm to find the nearest embedding vector to the latent feature would significantly increase the computational cost of the diffusion model. Therefore, this implementation uses a nearest neighbor search algorithm based on a KD-tree (K-Dimensional Tree) to find the nearest embedding vector to the latent feature from the codebook. In the KD-tree-based nearest neighbor search algorithm, the dimension with the largest variance in the embedding vectors is used as the split dimension. Then, the median is selected as the split point on the split dimension. The embedding vectors in the codebook are recursively divided into two parts according to the split point, ultimately constructing a KD-tree. During the traversal of the KD-tree, the latent feature and the embedding vector of the current node are compared in the specified dimension. If the current node is smaller, the left subtree is entered; otherwise, the right subtree is entered. Starting from the root node, the distance between the latent feature and the embedding vector of the current leaf node is calculated and recorded as the minimum distance until the leaf node is reached. Since the comparison is only made in one specified dimension during the traversal, unvisited subtrees may contain closer points. To ensure that the nearest neighbor is found, we need to backtrack to the parent node and check whether the unvisited subtrees may contain a closer point. If so, we update the minimum distance; otherwise, we convert the latent features into the embedding vectors corresponding to the nodes.

[0072] 3) Noise diffusion module, which in Figure 3 The term "noising" is used here. This module adds Gaussian noise to the embedding vector based on a preset time step, resulting in noisy features. This allows the embedding vector to be dediscretized for use in the diffusion process. Specifically, the maximum time step for noise addition needs to be preset before adding noise. And within this range, randomly select the time step for each noise addition. Assume the input data is (i.e., embedding vector), the noise diffusion module follows the specified time steps. Gradually towards data Add Gaussian noise This makes the data It is degraded to pure noise, as shown in the following formula (3):

[0073] (3);

[0074] In equation (3), Indicates the first The data after adding noise satisfies the following formula (4):

[0075] (4);

[0076] In equation (4), Indicates the first The data after adding noise; Represents the predefined first The diffusion rate of the step; represents the Gaussian noise of the t-th step.

[0077] Since the added noise satisfies the Gaussian distribution, it can be calculated by the Bayes formula, the data and the data satisfy the following formula (5):

[0078] (5).

[0079] In formula (5), represents the normal distribution with a mean of 0.

[0080] Therefore, under the given data , the time step , the Gaussian noise of the t-th step , the diffusion rate of the t-th step , the corresponding noisy data after the time step can be directly calculated, and then the latent representation after adding noise , that is, the noisy feature, can be obtained, which can effectively avoid the calculation amount caused by step-by-step noise addition. The Gaussian noise of the t-th step is the same size as the latent feature, both of which are , is the number of two-dimensional images, is the number of latent features, is the dimension of the latent feature.

[0081] 4) A denoising generation module, which is represented as “encoder” and “decoder” in Figure 3 , is a Transformer-based end-to-end neural network model used for the reverse denoising process, and the structure is shown on the right side of the dashed line in Figure 3 . The head of the denoising generation module includes an embedding layer and a position encoding layer, the middle includes a denoising encoding module and a denoising decoding module, and the denoising encoding module and the denoising decoding module are connected by a skip connection for feature fusion, and the skip connection is represented as “Concat and Conv1” in Figure 3 . The tail includes a fully connected layer (Linear) and a reshape layer (Reshape).

[0082] Specifically, the input of the denoising generation module is the time step and the noisy feature representation (i.e., the noisy feature). First, the time step ​​​is encoded into an embedding vector, which has the same dimension as the noisy feature representation , and the final size is . The noisy feature representation is then subjected to fixed sin-cos positional encoding, and concatenated with the embedding vector of the time step to form a vector. Then, they are sequentially input into the denoising encoding module and the denoising decoding module, the denoising encoding module is denoted as “encoder” in Figure 3 , and the denoising decoding module is denoted as “decoder” in Figure 3 . The denoising encoding module is used to encode the input of the denoising encoding module based on the multi-head self-attention mechanism, and the denoising decoding module is used to decode the input of the denoising decoding module based on the multi-head self-attention mechanism. The number of attention heads of the denoising encoding module and the denoising decoding module can be set according to actual conditions, for example, the number of attention heads of the denoising encoding module and the denoising decoding module can be 4, but it is not limited to this. Assuming that the input of each attention head in the denoising encoding module and the denoising decoding module is , multiplying it by the weight , , can obtain the corresponding , , , and the output of a single attention head can be obtained by attention calculation according to the following formula (6) . The output of the denoising encoding module and the output of the denoising decoding module can be obtained by the operation of multiple self-attention heads:

[0083] (6);

[0084] In formula (6), denotes a scaling factor.

[0085] Finally, the output of the denoising decoding module is reshaped after passing through a fully connected layer, and the noise added at the time step , that is, the predicted noise.

[0086] More specifically, the denoising encoding module includes a plurality of sequentially connected denoising encoding layers, and the denoising decoding module includes a plurality of sequentially connected denoising decoding layers, and the plurality of denoising encoding layers and the plurality of denoising decoding layers are connected in a skip manner. The number of denoising encoding layers and the number of denoising decoding layers can be flexibly set according to actual conditions, for example, the number of denoising encoding layers is five, and the number of denoising decoding layers is four. It should be noted that the difference between the number of denoising encoding layers and the number of denoising decoding layers is one.

[0087] The following is described by taking five denoising encoding layers and four denoising decoding layers as an example. As shown in Figure 4 encoder 1 to encoder 5 are the first denoising encoding layer to the fifth denoising encoding layer respectively, and decoder 1 to decoder 4 are the first denoising decoding layer to the fourth denoising decoding layer respectively, the first denoising encoding layer corresponds to the fourth denoising decoding layer, the second denoising encoding layer corresponds to the third denoising decoding layer, the third denoising encoding layer corresponds to the second denoising decoding layer, and the fourth denoising encoding layer and the fifth denoising encoding layer correspond to the first denoising decoding layer. Among them, the denoising encoding layer and the denoising decoding layer are both Transformer blocks, as shown in Figure 3 The Transformer block includes a layer normalization layer (Layer Normalization, LN), a multi-head attention layer (Multi-Head-Attention, MHA), and a feedforward neural network layer (Feedforward Neural Network, FNN), which has the same structure as the existing Transformer block, and will not be described again.

[0088] First, in each denoising encoding layer, the input of the denoising encoding layer is encoded based on the multi-head self-attention mechanism to obtain the output of the denoising encoding layer. Among them, the input of the first denoising encoding layer is a vector obtained by splicing the noised feature after position encoding and the embedding vector of the time step.

[0089] Then, the output of the fifth denoising encoding layer and the output of the fourth denoising encoding layer are spliced in the channel dimension to obtain the first spliced feature, and the first spliced feature is convoluted by the first convolution layer to obtain the first convolution feature. The first convolution feature will be input to the first denoising decoding layer for decoding processing to obtain the output of the first denoising decoding layer.

[0090] The output of the first denoising decoding layer and the output of the third denoising encoding layer are spliced in the channel dimension to obtain the second spliced feature, and the second spliced feature is convoluted by the second convolution layer to obtain the second convolution feature. The second convolution feature is input to the second denoising decoding layer for decoding processing to obtain the output of the second denoising decoding layer.

[0091] The output of the second denoising decoding layer and the output of the second denoising encoding layer are spliced in the channel dimension to obtain the third spliced feature, and the third spliced feature is convoluted by the third convolution layer to obtain the third convolution feature. The third convolution feature is input to the third denoising decoding layer for decoding processing to obtain the output of the third denoising decoding layer.

[0092] The output of the third denoising decoding layer is spliced with the output of the first denoising encoding layer in the channel dimension to obtain a fourth spliced feature, and the fourth spliced feature is subjected to convolution processing through a fourth convolution layer to obtain a fourth convolution feature. The fourth convolution feature is input into the fourth denoising decoding layer for decoding processing to obtain the output of the fourth denoising decoding layer, that is, the output of the denoising generation module.

[0093] The first convolution layer to the fourth convolution layer are used to restore the number of channels, and the convolution kernel and the step length of the first convolution layer to the fourth convolution layer can be flexibly set according to actual conditions. For example, the first convolution layer to the fourth convolution layer are all convolution layers with a convolution kernel size of 3*3 and a step length of 1, but are not limited thereto. It can be understood that the output of a single denoising encoding layer and the output of a single denoising decoding layer are the same in feature size, are spliced in the channel dimension first, at which time the number of channels becomes twice the original number. In order to ensure that the denoising encoding layer and the denoising decoding layer are consistent in the output feature size, the number of channels is restored to the original number of channels through a convolution layer before being input into the denoising decoding layer.

[0094] 5) a reverse decoding module, which can include a reverse pushing module and a diffusion decoding module, the diffusion decoding module is represented as a "VQ-GAN" decoder in Figure 3 , and the diffusion decoding module is a decoder of the VQ-GAN. The decoder can include multiple convolution layers and multiple residual blocks. In the reverse denoising process of the traditional diffusion model, in addition to removing the noise that depends on the model prediction, an additional random noise term is introduced at each step, which makes it necessary to follow a Markov chain to gradually reverse the original data from the final predicted noise. In the reverse denoising process of the present embodiment, the random noise term is removed, and the latent representation of the two-dimensional feature, that is, the latent space , and , is directly reverse pushed through the reverse pushing module according to the above formula (5) from the preset , and then the latent space is mapped to the pixel space through the diffusion decoding module, so as to obtain the generated image of the two-dimensional image . The up-sampling factor of the diffusion decoding module can be flexibly set according to actual conditions. For example, the up-sampling factor of the diffusion decoding module can be 16, but is not limited thereto.

[0095] In some embodiments, referring to Figure 3 , the pre-training of the diffusion model using the first sample set in the above step S102 can include:

[0096] The sample set of the current pre-training batch is input into the diffusion model to obtain the predicted noise set and the generated image set of the current pre-training batch; wherein, the sample set of the current pre-training batch is obtained by batch partitioning the first sample set;

[0097] The target loss for the current pre-training batch is obtained based on the predicted noise set and the generated image set of the current pre-training batch.

[0098] The quantization layer in the diffusion model is updated using the exponential moving average method, and the other parts of the diffusion model, excluding the quantization layer, are updated using the target loss of the current pre-trained batch.

[0099] If the current pre-training batch is not the last pre-training batch, then jump to the next pre-training batch; otherwise, determine the pre-trained diffusion model.

[0100] In this embodiment, the training set of the first sample set is divided into multiple pre-training batch sample sets, and the following operations are performed for each pre-training batch: First, the sample set of the current pre-training batch is input into the diffusion model, and the predicted noise set and generated image set of the current pre-training batch are obtained through forward propagation of the diffusion model. Specifically, for each two-dimensional image... Given a time step The diffusion model predicts the time step by learning the feature distribution of the input data. Noise added And generate a two-dimensional image. Generated image Therefore, the set of predicted noise for the current pre-training batch includes the predicted noise corresponding to multiple two-dimensional images belonging to the current pre-training batch, and the set of generated images for the current pre-training batch includes the generated images corresponding to multiple two-dimensional images belonging to the current pre-training batch.

[0101] Then, based on the predicted noise set and generated image set of the current pre-training batch, the target loss for the current pre-training batch is obtained. Specifically, for each two-dimensional image of the current pre-training batch... Calculate two-dimensional images Corresponding prediction noise and real noise The Euclidean distance between them serves as the diffusion loss. Calculate two-dimensional images and its corresponding generated image Manhattan distance as generation loss Therefore, the target loss of the current pre-training batch satisfies the following formula (7):

[0102] (7);

[0103] In formula (7), denotes the target loss of the current pre-training batch; denotes the number of two-dimensional images input in the current pre-training batch; denotes the generation loss corresponding to the i-th two-dimensional image in the current pre-training batch, i.e. ; ; denotes the diffusion loss corresponding to the i-th two-dimensional image in the current pre-training batch, i.e. ; ; denotes the weight of the generation loss, denotes the weight of the diffusion loss, both of which are preset values.

[0104] After that, the target loss of the current pre-training batch is used to perform back propagation on the parts of the diffusion model other than the quantization layer, so as to update the parameters of the parts of the diffusion model other than the quantization layer. Since the embedding vectors in the quantization module are discrete and non-differentiable, a straight-through estimator is used during back propagation to directly pass the gradient to the input end of the non-differentiable operation, i.e., the diffusion encoding module, so as to update the parameters of the parts of the diffusion model other than the quantization layer.

[0105] At the same time, the codebook of the quantization module is updated by using an Exponential Moving Average (EMA) mechanism to dynamically track the statistical characteristics of the diffusion encoding module output. Each embedding vector in the codebook is updated by using two statistics maintained respectively, one is a counter , used to record the number of times the embedding vector is selected, and the other is an accumulation vector , which accumulates the sum of the feature vectors output by the diffusion encoding module for all selected embedding vectors . In each pre-training batch update, the allocation results of the current pre-training batch are used to update the statistics, as shown in the following formula (8):

[0106] , (8);

[0107] In formula (8), denotes a preset decay rate, used to control the degree of retention of historical information; is the number of times the embedding vector is selected in the current pre-training batch; denotes the counter value of the embedding vector before the update in the current pre-training batch; embedding vectors in the current pre-training batch the counter value after the update; embedding vectors in the current pre-training batch the accumulated vector before the update; embedding vectors in the current pre-training batch the accumulated vector after the update.

[0108] Accordingly, in each pre-training batch, the codebook does not participate in gradient accumulation, and only part of the embedding vectors in the codebook can be selected for updating. The updated embedding vectors in the codebook satisfy the following formula (9):

[0109] (9);

[0110] In formula (9), embedding vectors in the current pre-training batch .

[0111] Optionally, in the process of self-supervised pre-training, the parameters of the diffusion encoding module are frozen in the first five pre-training batches, so that the parameters of the denoising generation module of the diffusion model are updated to an optimal value, and then the parameters of the diffusion encoding module are added to the back propagation and gradient update in the subsequent pre-training batches, thereby improving the efficiency and accuracy of pre-training.

[0112] Finally, it is determined whether the current pre-training batch is the last pre-training batch. If yes, the self-supervised pre-training is ended, and the pre-trained diffusion model is determined. If not, it indicates that the diffusion model has not been pre-trained sufficiently, and at this time, the next pre-training batch is jumped to, so as to realize the cyclic training.

[0113] Optionally, determining the pre-trained diffusion model can include: obtaining the parameters of the diffusion model under each pre-training batch, randomly selecting a parameter as a target parameter from the parameters of the diffusion model under each pre-training batch, and loading the target parameter to the diffusion model to obtain the pre-trained diffusion model. This way can quickly determine the corresponding model parameters for the diffusion model, thereby helping to improve the model output efficiency of self-supervised pre-training.

[0114] Optionally, the above method needs to evaluate the parameters of the diffusion model once in each pre-training batch, which not only produces errors due to not traversing all the training data, but also wastes computing resources and increases the cost of self-supervised pre-training. In this regard, determining the pre-trained diffusion model can include:

[0115] Before pre-training, the plurality of pre-training batches are divided into a plurality of pre-training rounds, each pre-training round including a plurality of pre-training batches. For example, the total number of pre-training batches is 80,000, and each 10 batches are divided into 1 round, so that 80,000 pre-training batches can be divided into 8,000 pre-training rounds. During pre-training, for each pre-training round, at the end of the current pre-training round, the parameters of the diffusion model of the last pre-training batch in the current pre-training round are saved as the model parameters of the current pre-training round, and the performance indicators of the diffusion model are verified.

[0116] The performance indicators of the diffusion model are selected from an initial score (Inception Score, IS), a Fréchet Inception Distance (FID), a Structural Similarity index (SSIM), and a Peak Signal-to-Noise Ratio (PSNR). Among them, the IS evaluates the quality of the generated image by measuring the classification accuracy and diversity of the generated image, the FID evaluates the quality of the generated image by comparing the distribution difference of the generated image and the real image in the feature space, the SSIM is used to measure the structural similarity of two images, considering three factors of brightness, contrast and structure, and the PSNR is an index for measuring signal quality, which can reflect the similarity between the generated image and the original image. In general, IS, SSIM, and PSNR are positively correlated with the quality of the generated image, that is, the higher the value of the indicator, the better the quality of the generated image, and the better the performance of the diffusion model; FID is negatively correlated with the quality of the generated image, that is, the lower the value of the indicator, the better the quality of the generated image, and the better the performance of the diffusion model. The performance indicators of the diffusion model are verified by using a pre-constructed verification set, and the verified performance indicators are saved as the model performance indicators of the current pre-training round. At the end of pre-training, the model parameters and model performance indicators of the plurality of pre-training rounds can be obtained.

[0117] Then, the pre-training rounds are taken as the abscissa, and the model performance indicators (i.e., IS, SSIM, PSNR, and FID) of the pre-training rounds are taken as the ordinate, respectively, to draw the corresponding curve graphs (i.e., the curve graphs corresponding to IS, SSIM, PSNR, and FID), and the optimal pre-training rounds corresponding to IS, SSIM, PSNR, and FID are selected through these curve graphs. Since the optimal pre-training rounds indicated by different indicators may be different, the optimal pre-training round needs to be selected from them, and the selection method can be random selection or manual selection, but is not limited thereto.

[0118] Finally, the model parameters of the optimal pre-training round are determined as target parameters, and the target parameters are loaded to the diffusion model to obtain the pre-trained diffusion model. In this way, by dividing the multiple pre-training batches into multiple pre-training rounds, and taking the pre-training rounds as the basis to evaluate the performance of the diffusion model, the optimal model parameters of the diffusion model are selected based on the model performance indicators of the multiple pre-training rounds. In this way, not only can all training data be traversed to reduce errors caused by not traversing all training data, but also the computing resources can be effectively reduced, and the cost of self-supervised pre-training can be reduced. In addition, by this way, the optimal model parameters of the diffusion model can be accurately determined, thereby improving the performance of the diffusion model.

[0119] (Three) Recognition model and its distillation learning:

[0120] In some embodiments, with reference to Figure 5 The above-mentioned recognition model can include:

[0121] The diffusion encoding module in the pre-trained diffusion model is used to map the two-dimensional image to the latent feature;

[0122] The quantization module in the pre-trained diffusion model is used to convert the latent feature into an embedding vector closest to the preset codebook;

[0123] The detection encoding module is used to encode the embedding vector to obtain the output of the detection encoding module;

[0124] The detection decoding module is used to decode the preset target query and the output of the detection decoding module to obtain the output of the detection decoding module;

[0125] The feedforward output layer is used to classify the output of the detection decoding module to obtain the two-dimensional candidate frame of the two-dimensional image.

[0126] In this embodiment, the diffusion encoding module and the quantization module of the pre-trained diffusion model are directly migrated to the recognition model, and the recognition model includes the detection encoding module, the detection decoding module, the feedforward output layer, and the diffusion encoding module and the quantization module in the previous pre-training stage. The functions and structures of the components of the recognition model will be further described below with reference to the accompanying drawings.

[0127] 1) The diffusion encoding module in the pre-trained diffusion model, which is denoted as "VQ-GAN encoder" in Figure 5 The description of this module can be referred to the description of the above-mentioned embodiment.

[0128] 2) The quantization module in the pre-trained diffusion model, which is denoted as "quantization" in Figure 5 The description of this module can be referred to the description of the above-mentioned embodiment.

[0129] Assume that each time input a two-dimensional image, the size of which is , is the number of channels of the two-dimensional image, is the height of the two-dimensional image, is the width of the two-dimensional image. The two-dimensional image is mapped from a high-dimensional pixel space to a low-dimensional discrete latent space (i.e., a discrete latent representation) by the diffusion encoding module and the quantization module, at this time the feature scale is , the number of channels of the feature is the same as the dimension of the embedding vector in the codebook, is the number of discrete embedding vectors. Since the diffusion encoding module and the quantization module are pre-trained in a large number of sample DBT images that do not contain breast cancer lesions in a self-supervised manner, they can more accurately compress the input two-dimensional image, reduce the resolution of the input image, while retaining the rich semantic information in the two-dimensional image, and ensure the feature extraction capability of the recognition model.

[0130] 3) a detection encoding module, which is denoted as “encoder” in Figure 5 , the module includes a plurality of sequentially connected detection encoding layers, and the detection encoding layer is a Multi-Head Deformable Attention (MDA) based Transformer block, which is denoted as “Deformable Transformer” in Figure 5 . The input of the first detection encoding layer is the embedding vector output by the quantization module, and the output of the last detection encoding layer is the output of the detection encoding module. Each detection encoding layer is used to encode the input of the detection encoding layer based on the multi-head deformable attention mechanism to obtain the output of the detection encoding layer. The multi-head deformable attention mechanism uses a small number of learnable reference points instead of global dense calculation, which not only ensures that the recognition model can focus on key areas, but also realizes adaptive sparse sampling of efficient calculation. Among them, the number of reference points of the multi-head deformable attention mechanism can be flexibly set according to actual conditions, for example, the reference points can be 4, but are not limited thereto. It can be understood that the construction of the detection encoding layer is similar to the existing Transformer block, and the difference lies in that the attention layer of the detection encoding layer is a multi-head deformable attention layer, which will not be described here.

[0131] 4) a detection decoding module, which is denoted as “decoder” in Figure 5The module denoted as "Decoder" includes a plurality of sequentially connected detection decoding layers. The input of the first detection decoding layer is object queries, which is a set of fixed number of learnable variables, and the initial value of each object query is a random value, and each object query represents a possible lesion. The input of the second detection decoding layer is the output of the first detection decoding layer and the output of the detection encoding module, and the input of the last two detection decoding layers is the output of the previous detection decoding layer. Among them, the second detection decoding layer is a Transformer block based on a multi-head cross-attention mechanism (MHCAM), which is denoted as "Transformer Block with Cross-Attention" in Figure 5 , and the second detection decoding layer is used to decode the output of the detection encoding module and the output of the first detection decoding layer to obtain the output of the second detection decoding layer. Except for the second detection decoding layer, the other detection decoding layers are Transformer blocks based on a multi-head self-attention mechanism, which are denoted as "Transformer Block" in Figure 5 , and the other detection decoding layers are used to decode the input of the detection decoding layer based on the multi-head self-attention mechanism to obtain the output of the detection decoding layer.

[0132] The features processed by the detection encoding module will be used as the input of the second detection decoding layer. The input of the first detection decoding layer is object queries, and the size of the object queries is , , which represents the maximum number of lesions that can be detected in each two-dimensional image, and the dimension of the maximum number of lesions is the same as the feature dimension of the output of the detection encoding module. Among them, the maximum number of lesions can be flexibly set according to actual conditions, for example, the maximum number of lesions can be 100, but is not limited thereto. The object queries are processed by the first detection decoding layer through multi-head self-attention, and the output of the first detection decoding layer is also used as the input of the second detection decoding layer. In the second detection decoding layer, the output of the first detection decoding layer is multiplied by the query weight to obtain the query , the output of the detection encoding module is multiplied by the key weight and the value weight to obtain the key and the value , the output of the second detection decoding layer is obtained by performing multi-head cross-attention calculation according to the above formula (6). The object queries processed by the first detection decoding layer are interacted with the output of the detection encoding module through the multi-head cross-attention mechanism, which can effectively realize the detection and positioning of the lesion area. In the last two detection decoding layers, the input of the detection encoding layer is processed through the multi-head self-attention mechanism to obtain the output of the detection encoding layer, and the output of the last detection decoding layer is the output of the detection decoding module. It can be understood that the construction of the second detection decoding layer is similar to the existing Transformer block, and the difference is that the attention layer of the second detection decoding layer is a multi-head cross-attention layer, which will not be described again.

[0133] 5) a feedforward output layer, which is essentially a feedforward neural network, denoted as "FFN" after "decoder" in Figure 5 . After the output of the detection decoding module is classified through the feedforward output layer, the prediction score of each lesion category predicted by the recognition model and the center position and height-width length of the corresponding two-dimensional candidate box can be obtained, and the dimensions are and , is a predetermined number of prediction boxes, i.e., the maximum number of lesions that can be detected in a two-dimensional image, is the number of lesion categories.

[0134] It should be noted that for a single two-dimensional image, the two-dimensional candidate box includes the two-dimensional candidate box corresponding to each lesion category in the two-dimensional image, and the lesion category and the two-dimensional candidate box are one-to-one. For a two-dimensional candidate box, the class with the maximum prediction score is selected as the final class.

[0135] In some embodiments, with reference to Figure 6 , the step S103 of training the recognition model based on knowledge distillation using the second sample set with the pre-trained diffusion model as the teacher network and the recognition model as the student network can include:

[0136] inputting the sample set of the current training batch into the recognition model to obtain the two-dimensional candidate box set and the first feature set of the current training batch; wherein the first feature set includes the outputs corresponding to the plurality of detection encoding layers of the detection encoding module in the recognition model respectively, and the sample set of the current training batch is obtained by batch dividing the second sample set;

[0137] inputting the sample set of the current training batch into the pre-trained diffusion model to obtain the second feature set of the current training batch; wherein the second feature set includes the outputs corresponding to the plurality of denoising decoding layers of the denoising generation module in the pre-trained diffusion model respectively, and the number of denoising decoding layers is the same as the number of detection encoding layers.

[0138] obtain a target loss of the current training batch according to the two-dimensional candidate box set, the first feature set and the second feature set of the current training batch;

[0139] update the recognition model by using the target loss of the current training batch;

[0140] if the current training batch is not the last training batch, jump to the next training batch, otherwise determine the trained recognition model.

[0141] In the distillation learning, the teacher network is a diffusion model based on latent space, and the parameters of all modules are obtained by self-supervised pre-training, which are not updated in the distillation process. The student network is a two-dimensional recognition model based on attention mechanism, wherein the parameters of the diffusion encoding module and the quantization module are obtained by pre-training, and the other modules are randomly initialized and need to be updated in the distillation process. In the distillation process, the second sample set is divided into multiple training batch sample sets, and for each training batch, the following operations are performed:

[0142] First, the time step of the current training batch is input the sample set of the current training batch into the recognition model as the student network, and obtain the two-dimensional candidate box set and the first feature set of the current training batch through the forward propagation of the recognition model. Specifically, for each two-dimensional image , the two-dimensional candidate box of the two-dimensional image and the output of each detection encoding layer in the detection encoding module can be obtained through the forward propagation of the recognition model, so the two-dimensional candidate box set of the current training batch can include the two-dimensional candidate boxes corresponding to the multiple two-dimensional images belonging to the current training batch respectively, and the first feature set of the current training batch can include the outputs corresponding to the multiple detection encoding layers of the detection encoding module in the current training batch respectively.

[0143] At the same time, input the sample set of the current training batch into the pre-trained diffusion model as the teacher network, and obtain the second feature set of the current training batch through the forward propagation of the diffusion model. Specifically, for each two-dimensional image , the two-dimensional image after adding noise and denoising can obtain the output of each denoising decoding layer in the denoising generation module, so the second feature set of the current training batch can include the outputs corresponding to the multiple denoising decoding layers of the denoising generation module in the current training batch respectively. It can be understood that the number of denoising decoding layers is the same as the number of detection encoding layers, which means that the multiple detection encoding layers of the detection encoding module in the recognition model and the multiple denoising decoding layers of the denoising generation module in the pre-trained diffusion model are aligned in feature representation.

[0144] Then, during the process of distillation learning, the corresponding task loss and distillation loss are calculated for multiple two-dimensional images belonging to the current training batch, and the target loss of the current training batch is obtained accordingly. Specifically, for the distillation loss, it is calculated by the Euclidean distance between the feature representation of the teacher network and the feature representation of the student network, as shown in the following formula (10):

[0145] (10);

[0146] In formula (10), denotes the distillation loss; denotes the output of the -th denoising decoding layer in the teacher network; denotes the output of the -th detection encoding layer in the student network; denotes the number of detection encoding layers or denoising decoding layers.

[0147] For the task loss, it is obtained by the gap between the output of the student network and the label. More specifically, for each two-dimensional image, the best matching combination can be obtained by matching the two-dimensional candidate box and the real box (i.e. the lesion annotation information) through the Hungarian algorithm, and then the classification loss , the bounding box loss and the intersection over union loss of the best matching combination are calculated, as shown in the following formula (11):

[0148] , ,

[0149] (11);

[0150] In formula (11), denotes the number of matched candidate boxes, denotes the number of categories, denotes the probability that the -th two-dimensional candidate box belongs to the -th category; when the real category of the -th two-dimensional candidate box is , , otherwise ; denotes the position information of the -th two-dimensional candidate box, denotes the position information of the corresponding real box of the -th two-dimensional candidate box, both of which contain the coordinates of the center point and the width and height of the box.

[0151] Further, the task loss can be expressed as the following formula (12):

[0152] (12);

[0153] In formula (12), denotes the task loss; denotes the weight of the classification loss; denotes the weight of the classification box loss; denotes the weight of the intersection over union loss, which are all preset values.

[0154] Accordingly, the target loss of the current training round satisfies formula (13) as follows:

[0155] (13);

[0156] In formula (13), denotes the target loss of the current training round; denotes the number of two-dimensional images input in the current training batch; denotes the task loss corresponding to the i-th two-dimensional image in the current training batch; denotes the distillation loss corresponding to the i-th two-dimensional image in the current training batch. Afterwards, the distillation process is essentially a fine-tuning process of the two-dimensional recognition model (i.e., the student network), which only updates the parameters of the student network, but not the parameters of the teacher network. Accordingly, the target loss of the current training batch is used for back propagation of all modules of the recognition model to update the parameters of all modules of the recognition model.

[0157] Finally, it is determined whether the current training batch is the last training batch. If yes, the distillation learning is ended, and the trained recognition model is determined. If no, it indicates that the recognition model is not pre-trained sufficiently, and at this time, the process jumps to the next training batch to realize the loop training.

[0158]

[0159] ​​Optionally, the determining the trained identification model can include: determining that a criterion for the two-dimensional candidate box being a true positive is that a center point of the two-dimensional candidate box being in the two-dimensional annotation box or a center point of the two-dimensional annotation box being in the two-dimensional candidate box, the two-dimensional annotation box being the aforementioned real box. Because each lesion can only have one true positive candidate box, the highest score is selected as the true positive candidate box, and the rest are false positives. Further, the precision refers to a ratio of a number of lesions predicted correctly by the identification model to a number of all lesions. According to the criterion for the two-dimensional candidate box being a true positive, the number of true positive two-dimensional candidate boxes and the number of false positive two-dimensional candidate boxes can be calculated. Further, in the verification and test stages of the identification model, according to different threshold scores (which are used to screen two-dimensional candidate boxes, which are described in the following embodiments), the corresponding precision and the average number of false positives of each two-dimensional image can be calculated. When the average number of false positives of each two-dimensional image is within a preset range, the corresponding precision is calculated and averaged to obtain the average precision of the average number of false positives of each two-dimensional image in the preset range, denoted as MTPF. The preset range can be flexibly set according to actual conditions, for example, the preset range can be [0.05, 2.0], but is not limited thereto. In the verification, according to the MTPF, the optimal parameters of the identification model can be selected, and the optimal parameters are loaded to the identification model, so as to obtain the trained identification model. In this way, the optimal parameters of the identification model are determined in full consideration of the precision and the average number of false positives, which can effectively improve the determination precision of the model parameters of the identification model, thereby ensuring the performance of the identification model.

[0160] In some embodiments, with reference to Figure 6 After updating the identification model by using the target loss of the identification model, the method further includes the following steps:

[0161] Taking the input of the identification model as a state and taking a time step as an action, a time step density distribution of the current training batch is determined through the policy network, and the time step is updated according to the time step density distribution of the current training round;

[0162] The sample set of the current training batch is input into the identification model to obtain a two-dimensional candidate box set of the current training batch;

[0163] According to the two-dimensional candidate box set of the current training batch, a task negative loss of the current training batch is obtained;

[0164] The policy network is updated according to the task negative loss of the current training batch and the time step density distribution.

[0165] In this embodiment, since the features extracted by the feature layer corresponding to the teacher network are different depending on the time step selected in the distillation learning, reinforcement learning is used to determine the optimal time step. This enables the selection of the optimal teacher network feature layer for the downstream recognition model, allowing the recognition model to adapt to different input images so that the recognition model can obtain the feature representation most relevant to the breast cancer lesion detection task from the diffusion model.

[0166] In the reinforcement learning of the current training batch, the input of the recognition model is used as the state. Time step As an action, each input can be output through the policy network. Corresponding time step density distribution This refers to the time step density distribution of the current training batch. Represents the policy network, These are the parameters of the policy network. After obtaining the time step density distribution, the policy network selects a time step through random sampling. Both high-probability and low-probability time steps can be selected, but high-probability time steps are more likely to be chosen. This approach helps the policy network avoid being limited to a single time step during training and allows it to discover better time steps. The policy network then replaces the current time step with the selected time step, thus updating the time step. Simultaneously, the sample set of the current training batch is input into the recognition model. The forward propagation of the recognition model yields a set of two-dimensional candidate boxes for the current training batch. Based on this set of candidate boxes, the task negative loss for the current training batch is calculated. The task negative loss is the inverse of the task loss, i.e., ... Then, based on the task negative loss and time step density distribution of the current training batch, the reward function value of the policy network in the current training batch is obtained, and the parameters of the policy network are updated using the reward function value of the policy network in the current training batch. The reward function value satisfies the following formula (14):

[0167] (14);

[0168] In equation (14), This represents the reward function value of the policy network in the current training batch; This results in a negative loss for the current training batch's tasks; The hyperparameter representing the entropy of the policy distribution is used to balance the relationship between the entropy of the policy distribution and the negative loss of the task; it is a preset value. The entropy representing the policy distribution is given by the following formula (15):

[0169] (15);

[0170] In formula (15), represents the optional maximum number of time steps; represents the input corresponding to the density distribution of the time step.

[0171] In this way, in the training process, the task negative loss and the entropy of the policy distribution are taken as rewards. The task negative loss can guide the policy network to select the time step that is most beneficial to the breast cancer lesion detection task, and the entropy of the policy distribution is equivalent to a regularization term, which can encourage the policy network to explore more possibilities of the time step. The rewards can encourage the policy network to gradually learn the optimal time step selection strategy, so as to dynamically select the most suitable time step for different input samples to extract the feature representation that is most helpful to the downstream breast cancer lesion detection task. This method not only improves the efficiency of knowledge transfer, but also enhances the performance and generalization ability of the student network.

[0172] (Four) Real-time recognition of breast cancer lesions:

[0173] In some embodiments, the recognition of the to-be-tested DBT image in step S104 above to obtain the breast cancer lesion recognition result can include:

[0174] Inputting the plurality of two-dimensional images of the to-be-tested DBT image into the trained recognition model to obtain the two-dimensional candidate boxes corresponding to the plurality of two-dimensional images in the to-be-tested DBT image respectively;

[0175] Performing screening processing on the two-dimensional candidate boxes corresponding to the plurality of two-dimensional images in the to-be-tested DBT image respectively to obtain a plurality of two-dimensional screening boxes;

[0176] Clustering the plurality of two-dimensional screening boxes based on the center points of the plurality of two-dimensional screening boxes to obtain a plurality of three-dimensional clustering clusters; wherein the difference between the depth coordinates of the center points of two adjacent two-dimensional screening boxes in a three-dimensional clustering cluster is less than or equal to a preset value, and the three-dimensional clustering cluster includes at least three two-dimensional screening boxes;

[0177] Determining the three-dimensional candidate boxes corresponding to the plurality of three-dimensional clustering clusters as the breast cancer lesion recognition result, wherein the three-dimensional candidate box is used to indicate the breast cancer lesion area of the to-be-tested DBT image.

[0178] In this embodiment, first, the to-be-tested DBT image is obtained and preprocessed to obtain a plurality of two-dimensional images of the to-be-tested DBT image. The preprocessing operation of the to-be-tested DBT image is the same as the preprocessing operation of the sample DBT image described above, and will not be described again. Inputting the plurality of two-dimensional images of the to-be-tested DBT image into the trained recognition model can obtain the prediction scores of each lesion class of each two-dimensional image, and the center point position and the height and width length of the corresponding two-dimensional candidate box.

[0179] Then, based on the acceptable number of false positive candidate boxes, a threshold score is selected to filter the two-dimensional candidate boxes corresponding to multiple two-dimensional images in the DBT image to be tested. When the category score is greater than the threshold score, the prediction result is retained; otherwise, it is discarded. In this way, multiple two-dimensional filtering boxes can be obtained, thereby filtering out low-scoring two-dimensional candidate boxes and reducing the impact of low-scoring two-dimensional candidate boxes on three-dimensional aggregation.

[0180] Next, clustering is performed on the center points of multiple two-dimensional filter boxes in a two-dimensional plane, where a two-dimensional plane refers to... Figure 2B shown In space Plane. Specifically, all two-dimensional filter boxes are located on a plane. When the axes are at the same position, only two-dimensional coordinates can be used. The center point of the two-dimensional filter box is represented by a density-based clustering of applications with noise (DBSCAN). This method combines the two-dimensional coordinates of the center point of each filter box to cluster all the two-dimensional filter boxes, resulting in multiple two-dimensional clusters, denoted as […]. Clusters, each two-dimensional cluster includes at least one two-dimensional filter box.

[0181] After that, The operation involves merging and discarding multiple two-dimensional clusters along the axis. Specifically, within each two-dimensional cluster, a secondary clustering is performed on all two-dimensional filter boxes based on the depth coordinates of their center points, resulting in multiple three-dimensional clusters, denoted as […]. Clustering. In quadratic clustering, if the difference in depth coordinates between two adjacent centroids is not greater than a preset value, they are grouped into one cluster, denoted as cluster 1. Clustering. If and only if There are three centroids in the cluster. At different positions of the axis, Only all two-dimensional filter boxes in a cluster can form a new three-dimensional filter box. Therefore, the difference in depth coordinates between the center points of two adjacent two-dimensional filter boxes in a three-dimensional cluster is limited to a preset value, and the three-dimensional cluster includes at least three two-dimensional filter boxes. The preset value can be flexibly set according to the actual situation; for example, the preset value can be 2, but it is not limited to this.

[0182] Finally, for each 3D cluster, the 2D filter boxes in the 3D cluster are aggregated into corresponding 3D candidate boxes to determine the... All two-dimensional filter boxes in a three-dimensional cluster in space axis, shaft and The maximum range corresponding to each axis is determined, and these maximum ranges are used to define the 3D candidate boxes. axis, shaft and The boundaries of the 3D candidate boxes are obtained by defining the ranges corresponding to the axes. The 3D candidate boxes corresponding to multiple 3D clusters are then used as the identification results for breast cancer lesions, thus achieving the identification and processing of breast cancer lesions.

[0183] It is understandable that the boundary of the 3D candidate box corresponding to the 3D cluster is the maximum and minimum x-coordinate, maximum and minimum y-coordinate, minimum and maximum depth coordinate of all 2D filter boxes in the 3D cluster. Furthermore, the prediction score of the 3D candidate box corresponding to the 3D cluster is the maximum prediction score of the 2D filter boxes in the 3D cluster.

[0184] like Figure 7 As shown, a to e are 5 items respectively. The two-dimensional filter box corresponding to the cluster is in In the view on the axis, gray represents two-dimensional filter boxes, and the numbers within them are the predicted scores for those two-dimensional filter boxes. During merging, All two-dimensional filter boxes whose axis values ​​differ by less than 2 are grouped into one. Clustering, to obtain Cluster the data into clusters a, c, d, and e, and discard those that do not meet the criteria. Cluster b. Then, to ensure that the 3D candidate boxes have at least three 2D cross-sections, discard those with fewer than three 2D cross-sections (i.e., fewer than three 2D filter boxes). Cluster c. Finally, in the figure... Clusters a, d, and e can all form corresponding three-dimensional candidate boxes. The maximum prediction score of the two-dimensional filter box is selected as the prediction score of the three-dimensional candidate box, which are 0.92, 0.95, and 0.96, respectively.

[0185] Optionally, in order to make the 3D candidate box in axis, shaft and The axes are continuous. In this implementation, the two-dimensional filter boxes of each two-dimensional cluster are filled during the merging and discarding operations. Specifically, for each two-dimensional cluster, the maximum, minimum, maximum, and minimum values ​​of the horizontal coordinates, vertical coordinates, and ordinates of the multiple two-dimensional filter boxes in the cluster are used as a reference to fill blank areas for each two-dimensional filter box, so that each two-dimensional filter box can achieve the maximum, minimum, maximum, and minimum values ​​of its horizontal coordinates, such as... Figure 7 The white areas a, c, d, and e are shown.

[0186] Optionally, the criterion for determining that the three-dimensional candidate box is a true positive is that the center point of the three-dimensional candidate box is in the three-dimensional labeled box or the center point of the three-dimensional labeled box is in the three-dimensional candidate box, the three-dimensional candidate box with the highest prediction score is selected as a true positive, and the rest are false positives. In the validation and test stages of the identification model, similar to the evaluation process of the two-dimensional candidate box, according to different threshold scores, the corresponding accuracy and the average number of false positives of each two-dimensional image can be obtained. The average number of false positives of each two-dimensional image is taken as the abscissa, and the accuracy is taken as the ordinate to draw a Free Receiver Operating Char (FROC) curve. The present embodiment evaluates the final identification result by drawing the FROC curve and calculating the MTPF, thereby optimizing the parameters of the identification model.

[0187] In addition, the present embodiment further provides a breast cancer lesion identification device, which can include an acquisition module, a pre-training module, a training module, and an identification module. The acquisition module is configured to acquire a first sample set and a second sample set. The first sample set includes a plurality of two-dimensional images corresponding to a plurality of sample DBT images without breast cancer lesions. The second sample set includes a plurality of two-dimensional images corresponding to a plurality of sample DBT images without breast cancer lesions and a plurality of two-dimensional images corresponding to a plurality of sample DBT images containing breast cancer lesions. Each sample DBT image containing a breast cancer lesion has corresponding lesion annotation information. The pre-training module is configured to pre-train a diffusion model using the first sample set to obtain a pre-trained diffusion model. The training module is configured to use the pre-trained diffusion model as a teacher network and an identification model as a student network, and perform knowledge distillation-based training processing on the identification model using the second sample set to obtain a trained identification model. The identification module is configured to perform identification processing on a to-be-tested DBT image using the trained identification model to obtain a breast cancer lesion identification result.

[0188] The content in the method embodiments described above is applicable to the device embodiments. The device embodiments specifically implement the same functions as the method embodiments described above, and achieve the same beneficial effects as the method embodiments described above.

[0189] In summary, the present embodiment provides a breast cancer lesion identification method and device, which has the following technical effects:

[0190] First, a large number of normal samples are used for self-supervised pre-training: the related technology has the problem that the recognition model is difficult to accurately capture the breast gland features due to the insufficient labeled samples of the DBT image, and the recognition model is under-fitted. To this end, the diffusion model is pre-trained on a large number of normal DBT images to enable the diffusion model to learn the law of breast gland distribution, and the feature extraction capability of the diffusion model is transferred to the downstream recognition model in subsequent distillation learning, which provides a good training starting point for the downstream recognition model, so that the model training does not depend on a large number of labeled samples, and reduces the influence of the inconsistency between the source domain and the target domain on the distillation learning.

[0191] Second, the diffusion model using a global attention as a basic framework is pre-trained: complex and variable lesion features often require the recognition model to comprehensively analyze and effectively extract the breast gland distribution features from multiple aspects. To this end, the diffusion model using a global attention as a basic framework is used as a pre-training model. On the one hand, different noise levels can obtain gland features of different scales, and on the other hand, the global attention mechanism can capture the global dependency of the gland, so that the diffusion model focuses on the key part and ignores the irrelevant part. These two points realize the multi-scale comprehensive use of gland information, and provide comprehensive, rich and diverse semantic information for the learning of the diffusion model and the recognition model.

[0192] Thirdly, a diffusion encoding module is introduced to process the high-resolution DBT image input: the training process of the high-resolution DBT image needs to be trained in a deeper and more complex network structure, and the requirement for computing resources is extremely high. At the same time, there is a large amount of redundant information in the high-resolution DBT image, which is easy to make the model pay too much attention to some unimportant features in the training process, and ignore the truly distinguishing features, and then cause the model to overfit. The related technology ignores the existence of a large amount of redundant information in the high-resolution DBT image, and achieves the purpose of processing the high-resolution DBT image by sacrificing part of the gland information. To this end, the VQ-GAN encoder, quantization layer and decoder are introduced in the diffusion model and the recognition model, which first compresses the high-resolution image into a low-dimensional continuous latent feature representation, so that the redundant high-dimensional data becomes a more compact low-dimensional continuous feature representation, which can reduce the requirement of high-resolution input data on computing resources; then the continuous latent feature representation is quantized into discrete latent feature representation, so that the model is more easy to capture the structured features of the data; then the discrete latent feature representation is multi-step noised to restore the continuous state for subsequent diffusion. In this way, the input image can be effectively mapped between the high-dimensional pixel space and the low-dimensional latent space, which not only removes the redundant information, but also reduces the computational amount of the model. Moreover, the diffusion encoding module also realizes the conversion between continuous feature representation and discrete feature representation (i.e. the redundant high-dimensional data is changed into a more compact low-dimensional continuous feature representation), so that the diffusion model and the recognition model can learn more structured gland features.

[0193] Fourthly, knowledge distillation is introduced for transfer learning: the transfer method used in the related art often requires the backbone network of the pre-training model to be basically consistent with the recognition model framework, and even needs to add an adaptation module to realize structure alignment. Specifically, the network structure of the pre-training model is more complex than that of the recognition model, and the task objectives of the two are also different, the former is diffusion and generation, and the latter is classification and positioning. This not only causes the general features learned by the pre-training model to be unable to be directly transferred to the breast cancer lesion detection task, but also causes the pre-training backbone network to be incompatible with the recognition model framework. Therefore, the present application introduces feature-based distillation learning for knowledge transfer, takes the diffusion model as the teacher network and the recognition model as the student network, the teacher network guides the student network to extract the gland distribution features of the DBT image, and the encoding layer of the recognition model is directly aligned with the decoding layer of the pre-training model with rich semantic information in the distillation learning, so that the breast cancer lesion detection task related feature representation learned by the pre-training model is simply, flexibly and effectively transferred to the recognition model. This not only enables the recognition model to adapt to different input images and learn the feature representation related to the breast cancer lesion detection task, but also avoids the complexity and incompatibility caused by embedding the pre-training model into the recognition model, successfully overcoming the differences between the pre-training model and the recognition model in network structure and task objective, and realizing the knowledge transfer from the complex model to the lightweight model.

[0194] Fifthly, reinforcement learning is introduced in the distillation learning to optimize the time step: in the training process, the task negative loss and the entropy of the policy distribution are taken as rewards, the task negative loss can guide the policy network to select the time step most beneficial to the breast cancer lesion detection task related features, and the entropy of the policy distribution is equivalent to a regularization term, which can encourage the policy network to explore more time step possibilities, the rewards can encourage the policy network to gradually learn the optimal time step selection strategy, so as to dynamically select the most suitable time step for different input samples to extract the feature representation most helpful to the downstream breast cancer lesion detection task. This method not only improves the efficiency of knowledge transfer, but also enhances the performance and generalization ability of the student network.

[0195] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

[0196] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A breast cancer lesion identification method, characterized by, The method comprises the following steps: obtaining a first sample set and a second sample set, the first sample set comprising a plurality of two-dimensional images respectively corresponding to a plurality of sample DBT images without breast cancer lesions, and the second sample set comprising a plurality of two-dimensional images respectively corresponding to a plurality of sample DBT images without breast cancer lesions and a plurality of two-dimensional images respectively corresponding to a plurality of sample DBT images containing breast cancer lesions, each sample DBT image containing a breast cancer lesion having corresponding lesion annotation information; pre-training a diffusion model using the first sample set to obtain a pre-trained diffusion model; using the pre-trained diffusion model as a teacher network and using an identification model as a student network, performing knowledge distillation-based training processing on the identification model using the second sample set to obtain a trained identification model; performing identification processing on a to-be-tested DBT image using the trained identification model to obtain a breast cancer lesion identification result; the diffusion model comprises: a diffusion encoding module configured to map the two-dimensional image to a latent feature; the diffusion encoding module is a VQ-GAN encoder; a quantization module configured to convert the latent feature into an embedded vector closest to a preset codebook; a noise-added diffusion module configured to add Gaussian noise to the embedded vector based on a preset time step to obtain a noise-added feature; a denoising generation module configured to perform denoising prediction processing according to the time step and the noise-added feature to obtain a predicted noise; an inverse decoding module configured to obtain a generated image of the two-dimensional image according to the noise-added feature and the predicted noise; the identification model comprises: the diffusion encoding module in the pre-trained diffusion model, configured to map the two-dimensional image to a latent feature; the quantization module in the pre-trained diffusion model, configured to convert the latent feature into an embedded vector closest to a preset codebook; a detection encoding module configured to perform encoding processing on the embedded vector to obtain an output of the detection encoding module; a detection decoding module configured to perform decoding processing on a preset target query and the output of the detection decoding module to obtain an output of the detection decoding module; a feedforward output layer configured to classify the output of the detection decoding module to obtain a two-dimensional candidate box of the two-dimensional image; the two-dimensional candidate box comprises a two-dimensional candidate box corresponding to each lesion class in the two-dimensional image, and the lesion class and the two-dimensional candidate box are in one-to-one correspondence; for one two-dimensional candidate box, the class with the maximum predicted score is selected as the final class thereof; the identification processing on the to-be-tested DBT image using the trained identification model to obtain the breast cancer lesion identification result comprises: inputting a plurality of two-dimensional images of the to-be-tested DBT image into the trained identification model to obtain a two-dimensional candidate box corresponding to each of the plurality of two-dimensional images in the to-be-tested DBT image; performing screening processing on the two-dimensional candidate boxes corresponding to the plurality of two-dimensional images in the to-be-tested DBT image to obtain a plurality of two-dimensional screening boxes; and The center points of the plurality of two-dimensional screening boxes are used to cluster the plurality of two-dimensional candidate boxes, to obtain a plurality of three-dimensional clustering clusters; wherein a difference between depth coordinates of center points of two adjacent two-dimensional screening boxes in the three-dimensional clustering cluster is less than or equal to a preset value, and the three-dimensional clustering cluster includes at least three two-dimensional screening boxes; The three-dimensional candidate boxes corresponding to the plurality of three-dimensional clustering clusters are determined as the breast cancer lesion identification result, wherein the three-dimensional candidate box is used to indicate a breast cancer lesion region of the to-be-detected DBT image.

2. The method of claim 1, wherein, The first sample set and the second sample set are obtained, including: A plurality of sample DBT images without breast cancer lesions and a plurality of sample DBT images containing breast cancer lesions are obtained, and for each sample DBT image, the following operations are performed: The sample DBT image is cropped to obtain a cropped image of the sample DBT image; The cropped image of the sample DBT image is two-dimensionally segmented to obtain a plurality of two-dimensional images of the sample DBT image.

3. The method of claim 1, wherein, The diffusion model is pre-trained using the first sample set to obtain a pre-trained diffusion model, including: The sample set of the current pre-training batch is input into the diffusion model to obtain a predicted noise set and a generated image set of the current pre-training batch; wherein the sample set of the current pre-training batch is obtained by batch dividing the first sample set; The target loss of the current pre-training batch is obtained according to the predicted noise set and the generated image set of the current pre-training batch; The quantization layer in the diffusion model is updated based on the exponential moving average method, and the target loss of the current pre-training batch is used to update other parts of the diffusion model except the quantization layer; If the current pre-training batch is not the last pre-training batch, jump to the next pre-training batch, otherwise determine the pre-trained diffusion model.

4. The method of claim 1, wherein, The detection decoding module includes a plurality of detection decoding layers, the input of the first detection decoding layer is the target query, the second detection decoding layer is used to decode the output of the detection encoding module and the output of the first detection decoding layer to obtain the output of the second detection decoding layer; The detection decoding layers other than the second detection decoding layer are used to decode the input of the detection decoding layer to obtain the output of the detection decoding layer.

5. The method of claim 1, wherein, The pre-trained diffusion model is used as a teacher network, and the identification model is used as a student network, and the identification model is trained based on knowledge distillation using the second sample set to obtain a trained identification model, including: The sample set of the current training batch is input into the identification model to obtain a two-dimensional candidate box set and a first feature set of the current training batch; wherein the first feature set includes outputs corresponding to a plurality of detection encoding layers of a detection encoding module in the identification model, and the sample set of the current training batch is obtained by batch dividing the second sample set; inputting the sample set of the current training batch into the pre-trained diffusion model to obtain a second feature set of the current training batch; wherein the second feature set comprises outputs corresponding to a plurality of denoising decoding layers of a denoising generation module in the pre-trained diffusion model, and the number of the denoising decoding layers is the same as the number of the detection encoding layers; obtaining a target loss of the current training batch according to the two-dimensional bounding box set, the first feature set and the second feature set of the current training batch; updating the recognition model by using the target loss of the current training batch; if the current training batch is not the last training batch, jumping to the next training batch, otherwise determining the trained recognition model.

6. The method of claim 5, wherein, After the step of updating the recognition model by using the target loss of the recognition model, the method further comprises the following steps: taking the input of the recognition model as a state and taking a time step as an action, determining a time step density distribution of the current training batch by a policy network, and updating the time step according to the time step density distribution of the current training round; inputting the sample set of the current training batch into the recognition model to obtain a two-dimensional bounding box set of the current training batch; obtaining a task negative loss of the current training batch according to the two-dimensional bounding box set of the current training batch; updating the policy network according to the task negative loss of the current training batch and the time step density distribution.

7. A breast cancer lesion identification apparatus characterized by comprising: comprise: an acquisition module configured to acquire a first sample set and a second sample set, the first sample set comprising a plurality of two-dimensional images corresponding to a plurality of sample DBT images without breast cancer lesions, and the second sample set comprising a plurality of two-dimensional images corresponding to a plurality of sample DBT images without breast cancer lesions and a plurality of two-dimensional images corresponding to a plurality of sample DBT images containing breast cancer lesions, each sample DBT image containing a breast cancer lesion having corresponding lesion annotation information; a pre-training module configured to pre-train a diffusion model by using the first sample set to obtain a pre-trained diffusion model; a training module configured to use the pre-trained diffusion model as a teacher network, use a recognition model as a student network, and perform knowledge distillation-based training processing on the recognition model by using the second sample set to obtain a trained recognition model; a recognition module configured to perform recognition processing on a to-be-tested DBT image by using the trained recognition model to obtain a breast cancer lesion recognition result; wherein the diffusion model comprises: a diffusion encoding module configured to map the two-dimensional image into a latent feature; the diffusion encoding module is a VQ-GAN encoder; a quantization module configured to convert the latent feature into an embedded vector closest to a preset codebook; a noisy diffusion module configured to add Gaussian noise to the embedded vector based on a preset time step to obtain a noisy feature; a denoising generation module configured to perform denoising prediction processing according to the time step and the noisy feature to obtain predicted noise; A reverse decoding module is configured to obtain a generated image of the two-dimensional image according to the noise-added feature and the predicted noise. The identification model comprises: A diffusion encoding module in the pre-trained diffusion model is configured to map the two-dimensional image into latent features. A quantization module in the pre-trained diffusion model is configured to convert the latent features into an embedding vector closest to a preset codebook. A detection encoding module is configured to perform encoding processing on the embedding vector to obtain an output of the detection encoding module. A detection decoding module is configured to perform decoding processing on a preset target query and the output of the detection decoding module to obtain an output of the detection decoding module. A feedforward output layer is configured to classify the output of the detection decoding module to obtain a two-dimensional bounding box of the two-dimensional image. The two-dimensional bounding box comprises a two-dimensional bounding box corresponding to each lesion category in the two-dimensional image. The lesion category is in one-to-one correspondence with the two-dimensional bounding box. For a two-dimensional bounding box, the category with the maximum predicted score is selected as the final category. The identification model is trained by using a plurality of training DBT images and a plurality of corresponding breast cancer lesion labels. The plurality of two-dimensional images of the to-be-tested DBT image are input into the trained identification model to obtain a two-dimensional bounding box corresponding to each of the plurality of two-dimensional images in the to-be-tested DBT image. The two-dimensional bounding boxes corresponding to the plurality of two-dimensional images in the to-be-tested DBT image are subjected to screening processing to obtain a plurality of two-dimensional screening boxes. The plurality of two-dimensional screening boxes are clustered based on the center points of the two-dimensional screening boxes to obtain a plurality of three-dimensional clustering clusters. The difference between the depth coordinates of the center points of two adjacent two-dimensional screening boxes in the three-dimensional clustering cluster is less than or equal to a preset value. The three-dimensional clustering cluster comprises at least three two-dimensional screening boxes. Three-dimensional bounding boxes corresponding to the plurality of three-dimensional clustering clusters are determined as the breast cancer lesion identification result. The three-dimensional bounding box is used to indicate a breast cancer lesion region of the to-be-tested DBT image.

Images