Medical image abnormal region identification method based on large model self-supervised learning

Abstract

The invention discloses a medical image abnormal region identification method based on large model self-supervised learning, and belongs to the technical field of image data processing, and the method comprises the steps: obtaining an abnormal sample data set D1 and a health sample data set D2 of a medical image; constructing and training a discrete mask auto-encoder; generating a pseudo-anomaly graph for the samples in the D2 to form a data set D3; automatically generating labels for the D2 and the D3; and constructing a medical image abnormal region identification network and total loss, and training the network by using D2 and D3 to obtain an abnormal region identification model for identifying the abnormal region in the sample in D1. The discrete mask auto-encoder can enhance the perception ability of the model for structure distortion and semantic mutation, alleviates the problem of abnormal region overfitting reconstruction, the pseudo-abnormal graph can improve the diversity and effectiveness of training samples, the abnormal region recognition model has the pixel-level anomaly positioning ability, and the recognition accuracy of the abnormal region is improved. And the structural design is adaptive to complex medical image features, the training mechanism is flexible, and end-to-end optimization can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method for identifying abnormal areas in medical images based on large-model self-supervised learning. Background Art

[0002] Although many studies have made significant progress in the field of unsupervised medical image anomaly detection, there are still several key technical bottlenecks that have not been effectively broken through. There are currently three commonly used medical image anomaly detection methods, namely (1) CCD method: see the paper Georgescu, M.-I. (2023). Masked autoencoders for unsupervised anomaly detection in medical images. arXiv; (2) single MAE reconstruction method, see the paper Tian, ​​Y., Pang, G., Liu, F., Chen, Y., Shin, SH, Verjans, JW, Singh, R., & Carneiro, G. (2021). Constrained contrastive distribution learning for unsupervised anomaly detection and localization in medical images. arXiv; (3) RealNet method, see the paper: Zhang, X., Xu, M., & Zhou, X. (2024). RealNet: A feature selection network with realistic synthetic anomaly for anomaly detection. arXiv.

[0003] (1) CCD method: It relies on complex contrastive learning and data augmentation techniques, which increases the complexity of training and the consumption of computational resources. This method, on the other hand, is trained through simple reconstruction tasks and has a lighter computational burden. Secondly, the CCD method focuses on minor anomalies and generates pseudo anomalies by enhancing data, which may cause the model to be overly sensitive to abnormal areas and require more preprocessing steps. This method, on the other hand, focuses on global structural information through reconstruction tasks and is more stable for large-scale anomaly detection. In general, the CCD method has certain limitations in terms of training complexity and excessive focus on minor anomalies.

[0004] (2) Single MAE reconstruction method (such as MAE-based strategy): Although the generative model is trained using only normal images to achieve anomaly detection, its modeling of abnormal areas usually lacks structural constraints, and the reconstruction results are more reflected in the image difference at the whole image level. MAE usually uses a continuous latent space, which may cause the model to over-recover abnormal areas when encountering abnormal areas, making it difficult to distinguish abnormalities from normal parts. At the same time, current evaluation strategies are mostly based on simulated abnormal samples, lack multi-scale feature extraction paths and local area contrast mechanisms, and have poor performance in identifying small lesions or low-contrast abnormalities. This type of method usually does not introduce explicit difference maps, residual maps or attention mechanisms, which inhibits the positioning accuracy and interpretability of the model.

[0005] (3) The RealNet method proposed for industrial vision tasks: Although it has shown good performance in texture defect detection, its design is more suitable for industrial scenarios with regular structures. In medical images, especially soft tissue or organ images, the structural complexity and abnormal manifestations are much higher than those in industrial backgrounds. Although the abnormality-aware feature selection and reconstruction residual selection in RealNet enhance feature discrimination capabilities, it still lacks an attention mechanism for spatial structure modeling. At the same time, this method generates synthetic abnormality maps through texture perturbation, which is difficult to cover structural abnormalities with clear boundaries and complex morphology such as tumors and hemorrhages, thus limiting its generalization ability in medical imaging scenarios.

[0006] Glossary: VQ-VAE, also known as Vector Quantized Variational Autoencoder, is a deep learning model that combines a variational autoencoder (VAE) with vector quantization to learn efficient data representation. By discretizing a continuous latent space, it addresses the problem of latent variables in traditional VAEs being too continuous and difficult to control.

[0007] A ViT-based encoder is an encoder based on the ViT (Vision Transformer) model, consisting of a block embedding layer (for image feature embedding), a multi-layer Transformer layer, and an MLP classification module. The Transformer layer uses a multi-head self-attention mechanism (MSA) for feature extraction. The Transformer attention map is a visualization of the self-attention mechanism in the Transformer architecture, showing the distribution of attention weights at different locations when processing input data.

[0008] The encoder of the VQ-VAE and the encoder of the multi-scale classifier in the present invention are both ViT-based encoders, and the two can be set up with independent encoders or share the same encoder. Summary of the Invention

[0009] The purpose of the present invention is to provide a method for identifying abnormal areas in medical images based on large-scale self-supervised learning, which can solve the above-mentioned defects such as certain limitations in training complexity and excessive attention to minor abnormalities, weak positioning accuracy and interpretability, and weak generalization ability, and can achieve accurate positioning of abnormal areas in medical images.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: a method for identifying abnormal areas in medical images based on large-scale model self-supervised learning, comprising the following steps: S1, obtain the abnormal sample dataset D1 and the healthy sample dataset D2 of medical images, where the samples in D1 are abnormal samples containing abnormal areas; the samples in D2 are healthy samples; S2, construct discrete mask autoencoder M VQ-MAE ; Get a VQ-VAE, set a random mask layer in the block embedding layer and encoding layer of its encoder to get VQ-MAE, and train VQ-MAE with D2 to get a discrete mask autoencoder M VQ-MAE The random mask layer is used to generate a mask for the output of the block embedding layer and then send it to the encoding layer. The encoding layer is an L-layer Transformer layer, and the M VQ-MAE Used to input natural images and output reconstructed images after reconstruction; S3, for each healthy sample in D2, a pseudo-abnormal map is generated by fusing the perturbation region. All pseudo-abnormal maps constitute the dataset D3; S4, automatically generates annotations for samples in D2 and D3; The sample in D3 is the first positive sample, with label y1=1. The sample in D2 is the first negative sample, with label y1=0. Each sample in D2 and D3 is divided into N blocks based on the block embedding layer. The block containing the perturbation area is the second positive sample, with label y2=1. The remaining blocks are the second negative samples, with label y2=0. S5, construct a medical image abnormal region recognition network, including M VQ-MAE , attention map localization unit, difference map generation unit, residual map localization unit, multi-scale classifier, anomaly score localization unit, fusion localization unit; The M VQ-MAE For input sample X, output reconstruction sample , the sample is a healthy sample in D2 or a pseudo-abnormal sample in D3; The attention map positioning unit is used to obtain X in M VQ-MAE The self-attention matrix of each Transformer layer in is weighted and fused to obtain the attention map ; The difference map generating unit is used to generate X and Normalized difference map of ; The difference map positioning unit is used to calculate the average residual of each block of the difference map, rearrange it into an image and then upsample it to the sample size X to obtain a residual map ; The multi-scale classifier includes an encoder, a first classification head connected to the encoder, a second classification head and a third classification head; The encoder is used to input the difference map , after being divided into N blocks, each block is subjected to L layers of Transformer layers to extract L block features. The block features extracted by the lth layer of Transformer layer for the nth block of the difference map are , 1≤n≤N, 1≤l≤L; The first classification head is used to output the predicted probability that X is the first positive sample , the second classification head is used to output the predicted probability that each block in X is the second positive sample , the third classification head is used to output is the predicted probability of the abnormal area , and based on the abnormal threshold get Abnormal label , among which, if ,but ,otherwise ; The anomaly score locating unit is used to locate the N blocks of the Lth layer. , spliced ​​into a matrix according to the block position, and then upsampled to the sample size X to obtain the abnormal score map ; The fusion positioning unit is used to 、 and Weighted fusion to obtain fusion graph , normalized to normalized fusion graph , then binarize it to get the fused abnormal area map; S6, use D2 and D3 to train the medical image abnormal region recognition network to obtain the abnormal region recognition model, and test it with D1; S7, input the samples in D1 into the abnormal region recognition model and output the corresponding fused abnormal region map.

[0011] As a preference, in S6, to minimize the total loss L total Training a network to identify abnormal regions in medical images; , Where, L UADis the normalized difference map loss, L VQ is the reconstruction loss of VQ-MAE, L align is the multi-scale alignment loss calculated based on the third classification head, L region is the regional supervision loss calculated based on the second classification head, L cls is the full-image classification loss calculated based on the first classification head, λ vq ,λ1,λ2,λ3 are L VQ , L align , L region , L cls The weight of .

[0012] As a preference, L UAD , L align , L region , L cls The following formulas are obtained respectively; , , , , , L UAD In the formula, X i,j 、 X and The pixel value in row i and column j, σ i,j For X and The pixel value in the i-th row and j-th column of the variance map of , M is the set of pixel positions in X, |M| is the total number of pixels in X, and ϵ is the minimum value; L align In the formula, λ l is the weight of the lth Transformer layer, is the square of the Euclidean distance; L region In the formula, L cls (n) is the classification loss of the nth block of the difference map.

[0013] Preferably, in S2, constructing the VQ-MAE specifically includes steps S21 to S24; S21, obtaining a VQ-VAE, including an encoder and a decoder, wherein the encoder includes a block embedding layer, a coding layer, and a vector quantization layer; S22, setting a random mask layer between the block embedding layer and the coding layer; The block embedding layer is used to divide the abnormal sample into N blocks, embed each block into a high-dimensional space to obtain an embedding vector, and form a Token sequence with the N embedding vectors. The embedding vector of the nth block in the Token sequence is z n ; The random mask layer is used to randomly mask the Token sequence according to the preset occlusion ratio r to obtain a mask sequence. The mask of the nth element in the Token sequence is Mask n ; The encoding layer is used to extract features from the mask sequence to obtain a continuous potential representation z; The vector quantization layer is used to quantize the potential representation z into N discrete codewords based on the codebook to obtain a discrete Token sequence; The decoder is used to reconstruct the abnormal sample according to the discrete Token sequence and output the reconstructed image; S23, construct the reconstruction loss L of VQ-MAE VQ , , Where e is the codebook vector generated for the potential representation z during training, sg[∙] is the stop gradient operation, is the square of the L2 norm, β is L VQ The second weight in S24, use D2 to train VQ-MAE according to the preset number of iterations, and calculate the number L in each iteration VQ , and update the network parameters of VQ-MAE and the value of the codebook vector e. The codebook vector after the iteration is used as the discrete Token sequence.

[0014] As a preferred embodiment, S3 generates a pseudo anomaly graph as follows: The healthy sample is input as a natural image P into M VQ-MAE , and obtain the reconstructed image ; Will Divide into sub-blocks of different sizes and randomly select W sub-blocks, and preset the w-th sub-block B w The perturbation intensity factor is β w , 1≤w≤W, for The pixel in row i and column j ,like Belongs to B w According to , generate perturbation pixels , all disturbed pixels constitute the disturbed area; Use Poisson fusion to fuse the disturbance area with the healthy sample to obtain the pseudo anomaly map .

[0015] Preferably, the attention map positioning unit obtains the attention map Specifically: For the N block features of the lth layer, the attention weights are obtained through the self-attention mechanism, and then spliced ​​into a matrix A according to the block position (l) , A (l) Upsample to sample size X to get the self-attention matrix of layer l ; Get the self-attention matrix of L layers in turn, and then calculate the attention map according to the following formula ; , Where, The weights of the lth Transformer layer.

[0016] As a preference, the normalized difference map of the difference map generating unit is According to the following formula: , Where, For X and The difference graph of is the variance map obtained based on the difference map, and ϵ is the minimum value.

[0017] As a preferred feature, in S5, the encoder of the multi-scale classifier is a shared M VQ-MAE encoder.

[0018] As a preference, normalize the fusion graph The fusion abnormal area map M is obtained according to the following formula final ; , Where M final (i,j), M final 、 The pixel in row i and column j in , τ is the preset fusion positioning threshold.

[0019] As a preference, S6 training process method S61~S63; S61, preset number of iterations; S62 randomly extracts a batch of samples from D2 and D3 and feeds them into the medical image abnormal region recognition network to generate a fused abnormal region map for each sample and calculate L total , and to minimize L total Adjust the network parameters of the multi-scale classifier; S63, repeat S62 until the iteration ends, and obtain the abnormal area recognition model.

[0020] In the present invention, VQ-MAE is constructed to introduce a vector quantization mechanism to build a discrete latent space, thereby enhancing the model's perception of structural distortion and semantic mutation, and alleviating the problem of overfitting reconstruction in abnormal areas.

[0021] The present invention proposes a new pseudo-anomaly map construction method, which naturally fuses reconstruction artifacts in healthy images to generate pseudo-anomaly maps. The purpose is to guide the model to learn abnormality perception ability in the absence of abnormal samples.

[0022] The multi-scale classifier of the present invention introduces a random mask to improve the information bottleneck effect of the masked area, and introduces a multi-scale alignment loss based on the multi-layer Transformer layer in the encoder, a full-image classification loss based on the complete image classification, and a regional supervision loss based on the classification calculation of each block of the image. This enables the multi-scale classifier to have the ability to jointly guide the location and discrimination of abnormal areas with feature layers of different depths.

[0023] Compared with the prior art, the advantages of the present invention are: (1) Unsupervised learning and high stability. This method relies on normal images for unsupervised learning, which makes the training process simple and consumes less computing resources. At the same time, this method introduces a vector quantization (VQ) mechanism to discretize the potential representation of the image, making the model more sensitive to structural distortion and semantic mutation. This can effectively improve the robustness of the model to abnormal areas and reduce the overfitting problem of abnormal areas during the reconstruction process.

[0024] (2) The pseudo-anomaly map construction is more realistic in medical structures. To address the problem that the MAE method lacks real structural simulation in pseudo-anomaly map generation, this paper designs a pseudo-anomaly map generation strategy that combines perturbation region control and Poisson fusion. This strategy not only simulates the spatial distribution of anomalies but also preserves the structural continuity and boundary features in medical images, thereby improving the diversity and effectiveness of training samples.

[0025] (3) It has the ability to locate abnormalities at the pixel level. The present invention completes the weighted integration of multi-source images in the fusion positioning unit through the collaborative design of the difference map generation unit, the residual map positioning unit, and the attention map positioning unit. Finally, a normalized fusion map is obtained and binarized. It can achieve high-resolution explicit identification of abnormal areas, making up for the shortcomings of the CCD method and RealNet method, which can only provide full-image-level abnormality scores and insufficient positioning accuracy. A large number of experiments on mainstream medical datasets have shown that the method of the present invention is significantly superior to existing mainstream self-supervised methods in both image-level recognition and pixel-level positioning tasks.

[0026] (4) Structural design adapts to complex medical image features. Unlike industrial image methods such as RealNet, this paper focuses on the complex structure, diverse textures, and blurred lesion boundaries in medical images. It introduces a ViT encoder and a multi-scale feature extraction mechanism, further aggregating the deep local representations of the Transformer layer in the multi-scale classifier, significantly enhancing the model's modeling and generalization capabilities for soft tissue abnormalities.

[0027] (5) Flexible training mechanism and end-to-end optimization. The present invention uniformly constructs a medical image abnormal region recognition network and constructs a total loss function based on the output of multiple modules, covering reconstruction loss, alignment loss, regional supervision loss, and classification loss, ensuring collaborative learning among modules. Through end-to-end training optimization, high-precision detection can be completed with only healthy samples and pseudo-abnormal samples, reducing data dependence. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the VQ-MAE structure diagram; Figure 2 Flowchart for generating pseudo-abnormal maps for healthy samples; Figure 3 Network structure diagram for abnormal region identification in medical images. DETAILED DESCRIPTION

[0029] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0030] Example 1: See Figures 1 to 3 A method for identifying abnormal regions in medical images based on large-scale self-supervised learning includes the following steps: S1, obtain the abnormal sample dataset D1 and the healthy sample dataset D2 of medical images, where the samples in D1 are abnormal samples containing abnormal areas; the samples in D2 are healthy samples; S2, construct discrete mask autoencoder M VQ-MAE ; Get a VQ-VAE, set a random mask layer in the block embedding layer and encoding layer of its encoder to get VQ-MAE, and train VQ-MAE with D2 to get a discrete mask autoencoder M VQ-MAE The random mask layer is used to generate a mask for the output of the block embedding layer and then send it to the encoding layer. The encoding layer is an L-layer Transformer layer, and the M VQ-MAE Used to input natural images and output reconstructed images after reconstruction; S3, for each healthy sample in D2, a pseudo-abnormal map is generated by fusing the perturbation region. All pseudo-abnormal maps constitute the dataset D3; S4, automatically generates annotations for samples in D2 and D3; The sample in D3 is the first positive sample, with label y1=1. The sample in D2 is the first negative sample, with label y1=0. Each sample in D2 and D3 is divided into N blocks based on the block embedding layer. The block containing the perturbation area is the second positive sample, with label y2=1. The remaining blocks are the second negative samples, with label y2=0. S5, construct a medical image abnormal region recognition network, including M VQ-MAE , attention map localization unit, difference map generation unit, residual map localization unit, multi-scale classifier, anomaly score localization unit, fusion localization unit; The M VQ-MAE For input sample X, output reconstruction sample , the sample is a healthy sample in D2 or a pseudo-abnormal sample in D3; The attention map positioning unit is used to obtain X in M VQ-MAE The self-attention matrix of each Transformer layer in is weighted and fused to obtain the attention map ; The difference map generating unit is used to generate X and Normalized difference map of ; The difference map positioning unit is used to calculate the average residual of each block of the difference map, rearrange it into an image and then upsample it to the sample size X to obtain a residual map ; The multi-scale classifier includes an encoder, a first classification head connected to the encoder, a second classification head and a third classification head; The encoder is used to input the difference map , after being divided into N blocks, each block is subjected to L layers of Transformer layers to extract L block features. The block features extracted by the lth layer of Transformer layer for the nth block of the difference map are , 1≤n≤N, 1≤l≤L; The first classification head is used to output the predicted probability that X is the first positive sample , the second classification head is used to output the predicted probability that each block in X is the second positive sample , the third classification head is used to output is the predicted probability of the abnormal area , and based on the abnormal threshold get Abnormal label , among which, if ,but ,otherwise ; The anomaly score locating unit is used to locate the N blocks of the Lth layer. , spliced ​​into a matrix according to the block position, and then upsampled to the sample size X to obtain the abnormal score map ; The fusion positioning unit is used to 、 and Weighted fusion to obtain fusion graph , normalized to normalized fusion graph , then binarize it to get the fused abnormal area map; S6, use D2 and D3 to train the medical image abnormal region recognition network to obtain the abnormal region recognition model, and test it with D1; S7, input the samples in D1 into the abnormal region recognition model and output the corresponding fused abnormal region map.

[0031] In this embodiment, the abnormal sample dataset D1 and the healthy sample dataset D2 can be constructed based on the BraTS2020 dataset or the LUNA16 dataset. Taking the BraTS2020 dataset as an example, the BraTS2020 training dataset contains 369 aligned quad-modal MRI data. The tumors presented in the dataset are glioblastomas and low-grade gliomas. One of the main tasks of this dataset is tumor segmentation. The present invention is used to solve a simpler problem: tumor detection. Tumors are regarded as abnormalities, samples containing tumors are regarded as abnormal samples and stored in D1, and samples without tumors are regarded as healthy samples and stored in D2, and only healthy samples in D2 are used to train VQ-MAE. D3 is constructed based on D2, and the medical image abnormal region recognition network is trained with D2 and D3 to obtain an abnormal region recognition model, and the abnormal region recognition model is tested with D1.

[0032] Example 2: See Figures 1 to 3 Based on Example 1, a training loss function design method is given, the purpose of which is to train the abnormal region recognition network of medical images to convergence and obtain an abnormal region recognition model so that it has the ability to recognize abnormal regions in samples. In S6, the total loss L is minimized. total Training a network to identify abnormal regions in medical images; , Where, L UAD is the normalized difference map loss, L VQ is the reconstruction loss of VQ-MAE, L align is the multi-scale alignment loss calculated based on the third classification head, L region is the regional supervision loss calculated based on the second classification head, L cls is the full-image classification loss calculated based on the first classification head, λ vq ,λ1,λ2,λ3 are L VQ , L align , L region , L cls The weight of .

[0033] L UAD , L align , L region , L cls The following formulas are obtained respectively; , , , , , X i,j 、 X and The pixel value in row i and column j, σ i,j For X and The pixel value in the i-th row and j-th column of the variance map is M, M is the set of pixel positions in X, |M| is the total number of pixels in X, ϵ is the minimum value, and λ l is the weight of the lth Transformer layer, is the square of the Euclidean distance, L cls (n) is the classification loss of the nth block of the difference map.

[0034] Example 3: See Figures 1 to 3 Based on Example 1, the construction of VQ-MAE in S2 specifically includes steps S21 to S24; S21, obtaining a VQ-VAE, including an encoder and a decoder, wherein the encoder includes a block embedding layer, a coding layer, and a vector quantization layer; S22, setting a random mask layer between the block embedding layer and the coding layer to obtain VQ-MAE; The block embedding layer is used to divide the abnormal sample into N blocks, embed each block into a high-dimensional space to obtain an embedding vector, and form a Token sequence with the N embedding vectors. The embedding vector of the nth block in the Token sequence is z n ; The random mask layer is used to randomly mask the Token sequence according to the preset occlusion ratio r to obtain a mask sequence. The mask of the nth element in the Token sequence is Mask n ; The encoding layer is used to extract features from the mask sequence to obtain a continuous potential representation z; The vector quantization layer is used to quantize the potential representation z into N discrete codewords based on the codebook to obtain a discrete Token sequence; The decoder is used to reconstruct the abnormal sample according to the discrete Token sequence and output the reconstructed image; S23, construct the reconstruction loss L of VQ-MAE VQ , , Where e is the codebook vector generated for the potential representation z during training, sg[∙] is the stop gradient operation, is the square of the L2 norm, β is L VQ The second weight in S24, use D2 to train VQ-MAE according to the preset number of iterations, and calculate the number L in each iteration VQ , and update the network parameters of VQ-MAE and the value of the codebook vector e. The codebook vector after the iteration is used as the discrete Token sequence.

[0035] Regarding training VQ-MAE: The encoder in VQ-MAE uses the ViT-Base encoder, with 8 Transformer blocks, 16 attention heads, and a decoder embedding dimension of 512. The abnormal sample dataset D1 can be either the BraTS2020 dataset or the LUNA16 dataset. Taking the BraTS2020 dataset as an example, the sample input size is set to 224×224×1. VQ-MAE is trained for 1600 epochs using samples from D1, with a mask ratio of 0.75. To obtain a reconstruction of the sample, we replace the unmasked tokens with the original tokens. To obtain the final reconstruction of the sample, we pass it through the VQ-MAE framework four times and average the resulting outputs.

[0036] Regarding training the network for identifying abnormal regions in medical images: The primary focus was on training a multi-scale classifier, whose encoder also used the ViT-Base encoder. This multi-scale classifier and the ViT-Base encoder used in VQ-MAE can be shared. When training the network for identifying abnormal regions in medical images, the training cycle (number of iterations) was 100. Optimization was performed using the AdamW optimizer, with a learning rate of 0.001 and weight decay of 0.05. The simulated experiments were conducted on a GeForce GTX 3090 GPU with 24 GB of VRAM.

[0037] Example 4: See Figures 1 to 3 ,S3 generates the pseudo anomaly graph specifically as follows; The healthy sample is input as a natural image P into M VQ-MAE , and obtain the reconstructed image ; Will Divide into sub-blocks of different sizes and randomly select W sub-blocks, and preset the w-th sub-block B w The perturbation intensity factor is β w , 1≤w≤W, for The pixel in row i and column j ,like Belongs to B w According to , generate perturbation pixels , all disturbed pixels constitute the disturbed area; Use Poisson fusion to fuse the disturbance area with the healthy sample to obtain the pseudo anomaly map .

[0038] The attention map positioning unit obtains the attention map Specifically: For the N block features of the lth layer, the attention weights are obtained through the self-attention mechanism, and then spliced ​​into a matrix A according to the block position (l) , A (l) Upsample to sample size X to get the self-attention matrix of layer l ; Get the self-attention matrix of L layers in turn, and then calculate the attention map according to the following formula ; , Where, The weights of the lth Transformer layer.

[0039] Normalized difference map of the difference map generation unit According to the following formula: , Where, For X and The difference graph of is the variance map obtained based on the difference map, and ϵ is the minimum value.

[0040] Normalized fusion graph The fusion abnormal area map M is obtained according to the following formula final ; , Where M final (i,j), M final 、 The pixel in row i and column j in , τ is the preset fusion positioning threshold.

[0041] S6 training process method S61~S63; S61, preset number of iterations; S62 randomly extracts a batch of samples from D2 and D3 and feeds them into the medical image abnormal region recognition network to generate a fused abnormal region map for each sample and calculate L total , and to minimize L total Adjust the network parameters of the multi-scale classifier; S63, repeat S62 until the iteration ends, and obtain the abnormal area recognition model.

[0042] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying abnormal regions in medical images based on large-scale model self-supervised learning, characterized in that: The following steps are involved: S1, obtain the abnormal sample dataset D1 and the healthy sample dataset D2 of medical images, where the samples in D1 are abnormal samples containing abnormal areas; The samples in D2 are healthy samples; S2, construct discrete mask autoencoder M VQ-MAE ; Get a VQ-VAE, set a random mask layer in the block embedding layer and encoding layer of its encoder to get VQ-MAE, and train VQ-MAE with D2 to get a discrete mask autoencoder M VQ-MAE The random mask layer is used to generate a mask for the output of the block embedding layer and then send it to the encoding layer. The encoding layer is an L-layer Transformer layer, and the M VQ-MAE Used to input natural images and output reconstructed images after reconstruction; S3, for each healthy sample in D2, a pseudo-abnormal map is generated by fusing the perturbation region. All pseudo-abnormal maps constitute the dataset D3; S4, automatically generates annotations for samples in D2 and D3; The sample in D3 is the first positive sample, with label y1=1. The sample in D2 is the first negative sample, with label y1=0. Each sample in D2 and D3 is divided into N blocks based on the block embedding layer. The block containing the perturbation area is the second positive sample, with label y2=1. The remaining blocks are the second negative samples, with label y2=0. S5, construct a medical image abnormal region recognition network, including M VQ-MAE , attention map localization unit, difference map generation unit, residual map localization unit, multi-scale classifier, anomaly score localization unit, fusion localization unit; The M VQ-MAE For input sample X, output reconstruction sample , the sample is a healthy sample in D2 or a pseudo-abnormal sample in D3; The attention map positioning unit is used to obtain X in M VQ-MAE The self-attention matrix of each Transformer layer in is weighted and fused to obtain the attention map ; The difference map generating unit is used to generate X and Normalized difference map of ; The difference map positioning unit is used to calculate the average residual of each block of the difference map, rearrange it into an image and then upsample it to the sample size X to obtain a residual map ; The multi-scale classifier includes an encoder, a first classification head connected to the encoder, a second classification head and a third classification head; The encoder is used to input the difference map , after being divided into N blocks, each block is subjected to L layers of Transformer layers to extract L block features. The block features extracted by the lth layer of Transformer layer for the nth block of the difference map are , 1≤n≤N, 1≤l≤L; The first classification head is used to output the predicted probability that X is the first positive sample , the second classification head is used to output the predicted probability that each block in X is the second positive sample , the third classification head is used to output is the predicted probability of the abnormal area , and based on the abnormal threshold get Abnormal label , among which, if ,but ,otherwise ; The anomaly score locating unit is used to locate the N blocks of the Lth layer. , spliced ​​into a matrix according to the block position, and then upsampled to the sample size X to obtain the abnormal score map ; The fusion positioning unit is used to 、 and Weighted fusion to obtain fusion graph , normalized to normalized fusion graph , then binarize it to get the fused abnormal area map; S6, use D2 and D3 to train the medical image abnormal region recognition network to obtain the abnormal region recognition model, and test it with D1; S7, input the samples in D1 into the abnormal region recognition model and output the corresponding fused abnormal region map.

2. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 1, characterized in that: In S6, to minimize the total loss L total Training a network to identify abnormal regions in medical images; ， Where, L UAD is the normalized difference map loss, L VQ is the reconstruction loss of VQ-MAE, L align is the multi-scale alignment loss calculated based on the third classification head, L region is the regional supervision loss calculated based on the second classification head, L cls is the full-image classification loss calculated based on the first classification head, λ vq ,λ1,λ2,λ3 are L VQ , L align , L region , L cls The weight of .

3. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 2, characterized in that: L UAD , L align , L region , L cls The following formulas are obtained respectively; ， ， ， ， ， L UAD In the formula, X i,j 、 X and The pixel value in row i and column j, σ i,j For X and The pixel value in the i-th row and j-th column of the variance map of , M is the set of pixel positions in X, |M| is the total number of pixels in X, and ϵ is the minimum value; L align In the formula, λ l is the weight of the lth Transformer layer, is the square of the Euclidean distance; L region In the formula, L cls (n) is the classification loss of the nth block of the difference map.

4. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 1, characterized in that: In S2, constructing VQ-MAE specifically includes steps S21 to S24; S21, obtaining a VQ-VAE, including an encoder and a decoder, wherein the encoder includes a block embedding layer, a coding layer, and a vector quantization layer; S22, setting a random mask layer between the block embedding layer and the coding layer; The block embedding layer is used to divide the abnormal sample into N blocks, embed each block into a high-dimensional space to obtain an embedding vector, and form a Token sequence with the N embedding vectors. The embedding vector of the nth block in the Token sequence is z n ; The random mask layer is used to randomly mask the Token sequence according to the preset occlusion ratio r to obtain a mask sequence. The mask of the nth element in the Token sequence is Mask n ; The encoding layer is used to extract features from the mask sequence to obtain a continuous potential representation z; The vector quantization layer is used to quantize the potential representation z into N discrete codewords based on the codebook to obtain a discrete Token sequence; The decoder is used to reconstruct the abnormal sample according to the discrete Token sequence and output the reconstructed image; S23, construct the reconstruction loss L of VQ-MAE VQ , ， Where e is the codebook vector generated for the potential representation z during training, sg[∙] is the stop gradient operation, is the square of the L2 norm, β is L VQ The second weight in S24, use D2 to train VQ-MAE according to the preset number of iterations, and calculate the number L in each iteration VQ , and update the network parameters of VQ-MAE and the value of the codebook vector e. The codebook vector after the iteration is used as the discrete Token sequence.

5. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 1, characterized in that: S3 generates a pseudo anomaly graph specifically as follows; The healthy sample is input as a natural image P into M VQ-MAE , and obtain the reconstructed image ; Will Divide into sub-blocks of different sizes and randomly select W sub-blocks, and preset the w-th sub-block B w The perturbation intensity factor is β w , 1≤w≤W, for The pixel in row i and column j ,like Belongs to B w According to , generate perturbation pixels , all disturbed pixels constitute the disturbed area; Use Poisson fusion to fuse the disturbance area with the healthy sample to obtain the pseudo anomaly map .

6. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 1, characterized in that: The attention map positioning unit obtains the attention map Specifically: For the N block features of the lth layer, the attention weights are obtained through the self-attention mechanism, and then spliced ​​into a matrix A according to the block position (l) , A (l) Upsample to sample size X to get the self-attention matrix of layer l ; Get the self-attention matrix of L layers in turn, and then calculate the attention map according to the following formula ; ， Where, The weights of the lth Transformer layer.

7. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 1, characterized in that: Normalized difference map of the difference map generation unit According to the following formula: ， Where, For X and The difference graph of is the variance map obtained based on the difference map, and ϵ is the minimum value.

8. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 1, characterized in that: In S5, the encoder of the multi-scale classifier is a shared M VQ-MAE encoder.

9. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 1, characterized in that: Normalized fusion graph The fusion abnormal area map M is obtained according to the following formula final ; ， Where M final (i,j), M final 、 The pixel in row i and column j in , τ is the preset fusion positioning threshold.

10. The method for identifying abnormal regions in medical images based on large-scale model self-supervised learning according to claim 2, characterized in that: S6 training process method S61~S63; S61, preset number of iterations; S62 randomly extracts a batch of samples from D2 and D3 and feeds them into the medical image abnormal region recognition network to generate a fused abnormal region map for each sample and calculate L total , and to minimize L total Adjust the network parameters of the multi-scale classifier; S63, repeat S62 until the iteration ends, and obtain the abnormal area recognition model.

Images