A self-supervised pre-training method based on diffusion model

Abstract

The application provides a self-supervised pre-training method based on a diffusion model, comprising the following steps: step 1, training a diffusion model based on a noise prediction manner on a pre-training data set, and taking the diffusion model as a teacher network; step 2, extracting a feature map of an up-sampling part in a U-Net network in the diffusion model trained in step 1, and splicing; step 3, aligning the feature map output by a student network with the feature map extracted in step 2, thereby training the student network; and step 4, obtaining a global feature map of an image through the student network trained in step 3. The method explicitly models the semantic correlation of different regions in the image, and adds a regularization constraint on the global feature of the image, thereby greatly improving the performance of the model pre-trained by the method in downstream tasks such as image classification, target detection and semantic segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to a self-supervised pre-training method in the field of computer vision, in particular to a self-supervised pre-training method based on a diffusion model. BACKGROUND

[0002] Computer vision is a technology that enables computers to understand and process image or video data. Computer vision has a wide range of applications in various fields, such as face recognition, autonomous driving, medical image analysis, etc. In order to improve the performance of computer vision tasks, it is necessary to learn effective visual features from a large amount of data, i.e., vectors that can represent the content and semantic information of images.

[0003] In recent years, with the development of deep neural networks in the field of computer vision, supervised learning has gradually become the mainstream for extracting visual features from images. However, in the framework of supervised learning, a large amount of manually labeled data is usually required to train the model. In practical applications, there is not always enough and high-quality labeled data available, because collecting and labeling data is both time-consuming and expensive, and there may be problems such as noise, bias, and inconsistency. In addition, in some fields or tasks, it is very difficult or impossible to obtain labeled data, such as medical image analysis, autonomous driving, etc. To solve this problem, self-supervised learning uses a large amount of unlabeled data to learn visual features of images. Self-supervised learning methods often generate pseudo-labels from data itself by designing some pretext tasks, and train neural networks with these pseudo-labels to learn general visual features. The process of training neural networks and extracting visual features from a large amount of unlabeled data through self-supervised learning is called pre-training.

[0004] Current self-supervised pre-training methods for computer vision still have the following problems: 1) existing pre-training methods lack modeling of semantic correlation between local regions in images; 2) existing pre-training methods either pay too much attention to pixel-level features in images or pay too much attention to global features of images while ignoring semantic structures in images, and these methods cannot balance the ability to learn global features and local details of images. The above problems will result in poor performance of the model pre-trained by these methods in linear classification, object detection, and other downstream tasks. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a self-supervised pre-training method based on a diffusion model to solve the problems of the prior art.

[0006] To solve the above technical problems, the application discloses a self-supervised pre-training method based on a diffusion model, comprising the following steps:

[0007] Step 1, training a diffusion model on a picture dataset by using a noise prediction method, and taking the trained diffusion model as a teacher network;

[0008] The method for training the diffusion model by using the noise prediction method comprises:

[0009] Step 1-1, selecting a picture dataset and scaling the pixel values of all pictures in the dataset to the interval [0, 1];

[0010] Step 1-2, for a picture x0 of the dataset, injecting noise into the picture x0 to obtain a picture x t at time t, the specific method being as follows:

[0011]

[0012] wherein, denotes the multiplication of the scaling factors at each time, α s denotes the scaling factor at time s, α t denotes the scaling factor at time t, α t :=1-β t , β t denotes the noise variance at time t, ∈ denotes the randomly sampled Gaussian noise injected, denotes a standard normal distribution.

[0013] Step 1-3, training the diffusion model by using a first loss function L(θ), the first loss function being as follows:

[0014]

[0015] wherein, denotes the expectation of the function in the square brackets, ∈ θ denotes a noise prediction network.

[0016] Step 2, extracting a feature vector by using the trained diffusion model obtained in step 1, that is, inputting a picture into the forward diffusion process in the diffusion model to obtain a picture corresponding to a time step, then inputting the obtained picture into the noise prediction network in the diffusion model, extracting a feature map in the up-sampling block in the noise prediction network, finally splicing the feature maps and sampling to obtain the feature vector of the input picture;

[0017] The method for extracting the feature vector comprises:

[0018] Step 2-1, the picture x at time t obtained in step 1-2 is input into the teacher network to obtain the feature map of the picture x at time t t Input the noise prediction network θ extract the intermediate layer feature maps of the up-sampling part of the noise prediction network, and splice the feature maps together;

[0019] Step 2-2, uniformly sample K points from the feature map obtained in step 2-1, and obtain the feature vector corresponding to the sampling point by bilinear interpolation represents the feature vector of the i-th image, that is, the feature vector generated by the teacher network, represents a real matrix of Kx D size, and D represents the dimension of the feature vector.

[0020] Step 3, taking the neural network to be trained as a student network, and training the student network, that is, sampling the feature map output by the student network to be trained, and then aligning the feature vector obtained by sampling in step 2, and finally training the student network by back propagation to obtain the trained student network;

[0021] The method for training the student network comprises:

[0022] Step 3-1, data augmentation is performed on the input image x0, and 50% of the image is masked using a binary mask;

[0023] The data augmentation method comprises:

[0024] Random cropping, horizontal flipping, color jittering and Gaussian blur are used to augment the image.

[0025] The method for masking 50% of the image using a binary mask comprises:

[0026] Step 3-1-1, the importance degree s of each block in the image is calculated, and the specific method is as follows:

[0027]

[0028] wherein q0 represents the query vector of the global feature of the picture, K 1: represents the index matrix of the picture except the global feature vector, and D represents the dimension of the feature vector, represents the mathematical expectation, and Softmax(·) represents the Softmax function.

[0029] Step 3-1-2, importance sampling is performed according to the importance degree s of each block, that is, blocks with an importance degree s greater than a threshold value are sampled, and are masked using a mask.

[0030] Step 3-2, input the remaining 50% of the image into the encoder in the student network to obtain the feature vector of the remaining image;

[0031] Step 3-3, splice the feature vector obtained in step 3-2 and the mask token, and input them into the decoder in the student network to obtain the feature map and global feature of the remaining image;

[0032] Step 3-4, obtain the feature vector of the remaining image by the same sampling method as step 2-2

[0033] Step 3-5, calculate the distance correlation coefficient between the feature vector generated by the student network and the feature vector generated by the teacher network Then, the second loss function is used to align the feature map, and the method is as follows:

[0034]

[0035] Wherein, N represents the number of pictures in the data set, and a represents a scaling factor;

[0036] Step 3-6, normalize the global feature of the input image, and then introduce a uniform regularization term to constrain the global feature of the image, and the specific method is as follows:

[0037]

[0038] Wherein, The third loss function is represented by L3, The global feature vector of the normalized picture is represented by I, and I represents the unit matrix, The Frobenius norm of the matrix is represented by ||.||F;

[0039] Step 3-7, combine the second loss function and the third loss function in step 3-5 and step 3-6 to obtain the total loss function to train the student network.

[0040] The total loss function is as follows:

[0041]

[0042] Wherein, λ represents a balance factor.

[0043] Step 4, use the student network trained in step 3 as a feature extractor to extract features, input the image into the trained student network to obtain the global feature map of the image, and complete the self-supervised pre-training based on the diffusion model;​

[0044] The method for feature extraction comprises:

[0045] The encoder in the student network obtained by training in step 3 is used as the network for feature extraction, and the feature map of the last layer of the encoder is reserved as the extracted feature after inputting the picture.

[0046] Step 5: The student network that has completed the self-supervised pre-training based on the diffusion model is applied to the task of object detection or semantic segmentation to identify and locate the objects in the image.

[0047] Beneficial effects:

[0048] The self-supervised pre-training method based on the diffusion model proposed in the present application aligns the output of the student network to be trained with the feature map extracted from the diffusion model, and uniformly constrains the global features of the image. Compared with other self-supervised pre-training methods, the present application explicitly models the semantic correlation of each region in the image, and enhances the ability of the model to extract global features of the image, thereby greatly improving the performance of the pre-trained model in various downstream tasks. BRIEF DESCRIPTION OF DRAWINGS

[0049] The above and / or other aspects of the present application will become more apparent by describing in detail the present application with reference to the attached drawings and specific embodiments.

[0050] Figure 1 A flowchart of the self-supervised pre-training method based on the diffusion model provided by the present application.

[0051] Figure 2 A flowchart of the feature map extraction method based on the diffusion model provided by the present application.

[0052] Figure 3 A flowchart of the student network training strategy provided by the present application.

[0053] Figure 4 A schematic diagram of the application of the present application in the tasks of object detection and semantic segmentation. DETAILED DESCRIPTION

[0054] The application provides a self-supervised pre-training method based on a diffusion model, which fully utilizes the feature maps with rich semantic information in the diffusion model, models the semantic correlation of each region in the image, and strengthens the ability of the model to extract global features through a regularization term, thereby improving the transfer performance of the model on various downstream tasks. The model pre-trained by the method provided in the application has greatly exceeded the performance of previous methods on tasks such as object detection and semantic segmentation that require detailed features in images, and has also achieved good results on tasks such as linear classification that require global features of images.

[0055] The application discloses a self-supervised pre-training method based on a diffusion model, comprising the following steps:

[0056] Step 1: training a diffusion model on a picture dataset by using a noise prediction method, and taking the diffusion model as a teacher network;

[0057] The diffusion model is a generative model, which gradually applies noise to the image in the forward process until the image is destroyed and becomes complete Gaussian noise, and then learns the process of restoring the original image from the Gaussian noise in the reverse process. In the training stage, the diffusion model predicts the noise in the image by training a noise prediction network, and in the generation stage, the diffusion model inputs the noisy image into the noise prediction network and removes the noise in the image by the predicted noise, and finally gradually obtains the image without noise. The noise prediction network adopts a U-Net architecture, and U-Net is a kind of neural network architecture widely used in image generation, image segmentation and other fields. U-Net includes two parts: a downsampling part and an upsampling part. The downsampling part gradually reduces the resolution of the input image while increasing the number of channels, while the upsampling part reverses the process by upsampling the low-resolution feature map to gradually restore the original image dimension. The upsampling part is composed of multiple layers of neural networks, and the middle layer specifically refers to the 6th, 7th and 8th layers of the upsampling part.

[0058] Step 2: extracting feature maps from the trained diffusion model obtained in step 1, that is, obtaining the corresponding time step picture after the input picture goes through the forward diffusion process, and then extracting and splicing the feature maps in the corresponding U-Net upsampling block;

[0059] Step 3: sampling the extracted feature maps in step 2, then aligning the feature maps output by the student network to be trained with the sampled feature maps, and finally training the student network by using backpropagation;

[0060] The student network generally adopts the ViT architecture (reference: [2010.11929] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (arxiv.org)), the student network and the teacher network are decoupled, the teacher network uses the diffusion model, and the student network can adopt various architectures, and generally adopts the ViT architecture.

[0061] Step 4: The trained student network obtained in step 3 is used as a feature extractor, and an image is input into the student network to obtain a global feature map of the image.

[0062] The noise prediction-based diffusion model training method in step 1 of the present application comprises:

[0063] Step 1-1: Select a picture data set and scale the pixel values of all pictures in it to the interval [0, 1];

[0064] Step 1-2: For a picture x0 of the data set, inject noise into the picture by the following formula and obtain a picture x at time t t :

[0065]

[0066] wherein β t represents the noise variance at time t;

[0067] Step 1-3: Train the diffusion model by using the following loss function:

[0068]

[0069] wherein ∈ θ represents a noise prediction network using U-Net as the basic architecture (reference: [1505.04597] U-Net: Convolutional Networks for Biomedical Image Segmentation (arxiv.org)), and ∈ represents the sampled Gaussian noise. The model obtained after training is called the teacher network.

[0070] The feature map splicing method in step 2 of the present application comprises:

[0071] Step 2-1: Given a training set Take image x0 in the image, inject randomly sampled Gaussian noise into it, and obtain image x at time t according to the formula in steps 1-2. t ;

[0072] Step 2-2, move image x t Input Teacher Network ∈ θ Then, the intermediate layer feature maps of the upsampled part of U-Net are extracted and these feature maps are stitched together.

[0073] Step 2-3: Uniformly sample K points from the feature map obtained in Step 2-2, and then use bilinear interpolation to obtain the feature vectors corresponding to these sampled points.

[0074] The training strategy for the student network in step 3 of this invention includes:

[0075] Step 3-1: Perform data augmentation on the input image x0 and use a binary mask to cover 50% of the image;

[0076] Step 3-2: Input the remaining visible part of the image into the encoder in the student network to obtain the feature vector of this part of the image;

[0077] Step 3-3: The feature vector obtained in step 3-2 and the learnable mask token (a token used to represent the masked region in the image, which can be regarded as a set of learnable parameters) are concatenated and input into the decoder in the student network to obtain the feature map and global features of the image.

[0078] Step 3-4: Obtain the image feature vector by applying the same sampling strategy as in Step 2-3 to the feature map obtained in Step 3-3.

[0079] Steps 3-5: Calculate the feature vectors generated by the student network. and feature vectors generated by the teacher network The distance correlation coefficient between Then, the following loss function is used for feature map alignment:

[0080]

[0081] Where N represents the number of images, and α represents the scaling factor.

[0082] Steps 3-6 involve normalizing the global features of the image, and then introducing the following uniform regularization term to constrain the global features of the image:

[0083]

[0084] in A global feature vector representing the normalized picture.

[0085] Step 3-7, combining the loss functions in step 3-5 and step 3-6, obtains a loss function as follows to train the student network:

[0086]

[0087] Wherein λ represents a balance factor used to weigh the importance of the first term and the second term.

[0088] The data enhancement and mask method in step 3-1 of the application comprises:

[0089] Step 3-1-1, using random cropping, horizontal flipping, color jittering, Gaussian blur and other strategies to enhance the picture.

[0090] Step 3-1-2, the importance of each patch in the picture is calculated by the following formula:

[0091]

[0092] Wherein q0 represents the query vector of the global token of the picture, K 1: represents an index matrix except the global feature vector.

[0093] Step 3-1-3, importance sampling is performed according to the importance of each patch, and the sampled patch is hidden using the mask.

[0094] The feature extraction method in step 4 of the application comprises:

[0095] The encoder in the student network trained by step 3-5 is used as the feature extraction network, and the feature map of the last layer of the encoder is reserved as the extracted feature after inputting the picture.

[0096] Embodiment:

[0097] The application discloses a self-supervised pre-training method based on a diffusion model, as shown in Figure 1 The method comprises the following steps:

[0098] Step 1, training a diffusion model using a noise prediction method on a picture dataset, and using it as a teacher network;

[0099] Step 2, extracting the feature map of the trained diffusion model obtained in step 1, that is, obtaining the picture corresponding to the time step after the input picture goes through the forward diffusion process, and then extracting and splicing the feature map in the up-sampling block in the corresponding U-Net;

[0100] Step 3, sample the feature map extracted in step 2, then align the feature map output by the student network to be trained with the sampled feature map, and finally train the student network by back propagation;

[0101] Step 4, take the student network trained in step 3 as a feature extractor, and input an image into the student network to obtain the global feature map of the image.

[0102] Step 1 includes the following steps:

[0103] Step 1-1, select a picture data set For example, ImageNet, and scale the pixel values of all pictures in it to the interval [0, 1];

[0104] Step 1-2, for a picture x0 of the data set, inject noise into the picture by the following formula and obtain the picture x at time t t :

[0105]

[0106] Wherein β t represents the noise variance at time t;

[0107] Step 1-3, train the diffusion model using the following loss function:

[0108]

[0109] Wherein ∈ θ represents a noise prediction network using U-Net as the basic architecture, and ∈ represents the sampled Gaussian noise. The model obtained after training is called the teacher network.

[0110] The input of the trained diffusion model is the picture x t with noise at time t and the time t, and the output is the picture x t contains noise, then the picture x t-1 at time t-1 can be predicted.

[0111] As shown in Figure 2 , step 2 includes the following steps:

[0112] Step 2-1, given a picture x0 in the training set , inject randomly sampled Gaussian noise into it, and obtain the picture x t at time t according to the formula in step 1-2;

[0113] Step 2-2, input the picture x t into the teacher network ∈ θThen the intermediate layer feature maps of the up-sampling part of the U-Net are extracted and spliced together;

[0114] Step 2-3, uniformly sample K points from the feature map obtained in step 2-2, and then sample bilinear interpolation to obtain the feature vectors corresponding to the sampling points

[0115] Since the feature map extracted in the diffusion model contains rich semantic information, the feature vectors corresponding to these sampling points contain the semantic information of the regions around these points.

[0116] As shown in Figure 3 , step 3 includes the following steps:

[0117] Step 3-1, data augmentation is performed on the input image x0, and 50% of the image is masked using a binary mask;

[0118] Step 3-2, input the visible part of the image into the encoder in the student network to obtain the feature vector of this part of the image. The network structure of the encoder is consistent with the traditional ViT, that is, the input image is divided into several patches of equal size, and a linear transformation is used to convert the original pixel block into a vector, and then input into L standard TransformerBlock, each Block is composed of self-attention module and multi-layer perception.

[0119] Step 3-3, splice the feature vector obtained in step 3-2 and the mask token, and input it into the decoder in the student network, so as to obtain the feature map and global feature of the image. Unlike the encoder, the decoder is composed of a series of mutual attention modules. The input h m of the module is composed of M mask tokens plus the corresponding position encoding, each token is the same and is a learnable parameter. K and V in the mutual attention module are composed of the features z v of the visible part of the image. The calculation formula of the mutual attention module is as follows:

[0120]

[0121] Where Q m = h m W Q , K = z v W K , V = z v W V , W Q , W K , W V are learnable parameters.

[0122] Step 3-4, the feature vector of the image is obtained by the feature map of step 3-3 through the same sampling strategy as step 2-3

[0123] Step 3-5, calculate the distance correlation coefficient between the feature vector generated by the student network and the feature vector generated by the teacher network Then the feature map alignment is performed using the following loss function:

[0124]

[0125] Where N represents the number of pictures, and a represents the scaling factor.

[0126] Step 3-6, normalize the global feature of the picture, and then introduce the following uniform regularization term to constrain the global feature of the picture:

[0127]

[0128] Where The normalized global feature vector of the picture is represented by During training, for N pictures in a batch, the global features of these pictures are composed of the [cls] token output by the decoder. Then further batch normalization is performed on these feature vectors, so that each feature dimension has a mean of 0 and a standard deviation of

[0129]

[0130] Thus, the normalized global feature vector of the picture is

[0131] Step 3-7, combine the loss functions in step 3-5 and step 3-6 to obtain the following loss function to train the student network:

[0132]

[0133] Where λ represents a balance factor used to weigh the importance of the first and second terms.

[0134] Step 3-1 includes the following steps:

[0135] Step 3-1-1, use random cropping, horizontal flipping, color jittering, and Gaussian blur strategies to enhance the picture.

[0136] Step 3-1-2, calculate the importance of each patch in the picture using the following formula:

[0137] ​

[0138] wherein q0 represents a picture global feature vector, K 1: represents an index matrix except the global feature vector.

[0139] Step 3-1-3, importance sampling is performed according to the importance degree of each patch, and the sampled patch is hidden using a mask. In the actual sampling process, first, the distribution p(x) is obtained according to the importance of each patch, then the random distribution p(x) is obtained through a random number generator, and finally, the value of each patch is sorted from large to small according to The first 50% of the patches are hidden, and the remaining patches are used as the input of the encoder in the student network.

[0140] Step 4 includes: using the encoder in the student network trained by step 3-5 as a feature extraction network, inputting a picture, and retaining the feature map of the last layer of the encoder as the extracted feature.

[0141] In order to verify the effectiveness of the present application, the present application uses ImageNet as a pre-training data set, and compares the performance of other excellent models on the target detection and semantic segmentation tasks. For the target detection task, the present application uses the commonly used Mask R-CNN structure, uses the pre-trained model as the backbone network, and then fine tunes on the MS-COCO data. The final experimental results are shown in Table 1:

[0142] Table 1: Performance comparison table of the target detection task provided by the present application

[0143]

[0144] As can be seen from Table 1, the pre-training method proposed by the present application achieves an average precision (AP) of 52.8% on the target detection task, and an AP of 46.7% on the instance segmentation task, which is improved by 3.0% and 2.8% respectively compared with the previous best method CAE.

[0145] For the semantic segmentation task, the present application uses the UperNet structure, and then fine tunes on the ADE20K data set. The final experimental results are shown in Table 2:

[0146] Table 2: Performance comparison table of the semantic segmentation task provided by the present application

[0147]

[0148] As can be seen from Table 2, the pre-training method proposed in the application achieves a mean Intersection Over Union (mIOU) of 52.4%, an all pixel Accuracy (aAcc) of 62.8%, and a mean class Accuracy (mAcc) of 86.1% on the semantic segmentation task, which are respectively improved by 2.1%, 2.5% and 1.3% compared with the best method CAE before.

[0149] The application is applied to target detection (up) and semantic segmentation task (down) as shown in the following. Figure 4 Figure 4 Figure 4

[0150] In summary, according to the experimental results, the multiple test indicators of the application on the target detection and semantic segmentation task are all greatly superior to the previous models.

[0151] In the specific implementation, the application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and the computer program can run the invention content of the self-supervised pre-training method based on the diffusion model and part or all steps in each embodiment when executed by the data processing unit. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0152] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the application can be realized by means of a computer program and its corresponding general hardware platform. Based on such understanding, the technical solutions in the embodiments of the application can be embodied in the form of a computer program, i.e. a software product, which can be stored in a storage medium, including a plurality of instructions for causing a device (which can be a personal computer, a server, a single-chip microcomputer, a MUU or a network device, etc.) containing a data processing unit to execute the method described in each embodiment or some parts of the embodiments of the application.

[0153] The application provides a self-supervised pre-training method based on a diffusion model, and there are many methods and approaches to realize the technical solutions, and the above description is only the preferred embodiments of the application. It should be pointed out that for ordinary skilled persons in the technical field, without departing from the principles of the application, some improvements and refinements can be made, which should also be regarded as the protection scope of the application. The components not explicitly described in the embodiments can be realized by using the existing technology.​​​

Claims

1. A self-supervised pre-training method based on a diffusion model, characterized in that, The method comprises the following steps: Step 1, training a diffusion model on a picture dataset using a noise prediction method, and taking the trained diffusion model as a teacher network; Step 2, extracting a feature vector using the trained diffusion model obtained in step 1, that is, inputting a picture into a forward diffusion process in the diffusion model to obtain a picture at a corresponding time step, then inputting the obtained picture into a noise prediction network in the diffusion model, extracting a feature map of an intermediate layer of an upsampling part in the noise prediction network, finally splicing the feature maps and sampling to obtain a feature vector of the input picture; Step 3, taking a neural network to be trained as a student network, and training the student network, that is, sampling a feature map output by the student network to be trained, then aligning the feature map with the feature vector sampled in step 2, and finally training the student network using back propagation to obtain a trained student network; Step 4, taking the trained student network obtained in step 3 as a feature extractor to perform feature extraction, inputting an image into the trained student network to obtain a global feature map of the image, and completing self-supervised pre-training based on the diffusion model; Step 5, applying the student network that has completed self-supervised pre-training based on the diffusion model to a target detection or semantic segmentation task to identify and locate objects in the image; In step 3, the method for training the student network comprises: Step 3-1, performing data enhancement on an input image x0, and using a binary mask to mask 50% of the image; Step 3-2, inputting the remaining 50% of the image into an encoder in the student network to obtain a feature vector of the remaining image; Step 3-3, splicing the feature vector obtained in step 3-2 and a learnable mask token, and inputting the spliced feature vector into a decoder in the student network to obtain a feature map of the remaining image and a global feature; Step 3-4, the feature map obtained in step 3-3 is sampled by the same sampling method as step 2-2 to obtain the feature vector of the remaining image Steps 3-5: Calculate the feature vectors generated by the student network. and feature vectors generated by the teacher network The distance correlation coefficient between Then the second loss function is used. Feature map alignment is performed as follows: Wherein, N represents the number of pictures in the dataset, and a represents a scaling factor; Step 3-6, normalizing the global feature of the input image, then introducing a uniform regularization term to constrain the global feature of the image, and the specific method is as follows: wherein, denotes a third loss function, denotes a global feature vector of the normalized picture, I denotes an identity matrix, denotes the Frobenius norm of a matrix; Step 3-7, combine the second loss function and the third loss function in step 3-5 and step 3-6 to obtain a total loss function to train the student network.

2. The self-supervised pre-training method based on diffusion model according to claim 1, wherein, The method for training the diffusion model in step 1 comprises: Step 1-1, Selecting the picture dataset and scale the pixel values of all pictures therein to the interval [0, 1]; Steps 1-2, for a picture x0 of the dataset, inject noise into the picture x0 and obtain a picture x at time t t ; Step 1-3, training the diffusion model using a first loss function L(θ), and the first loss function is as follows: wherein, denotes the expectation of the function in square brackets, ∈ θ denotes the noise prediction network.

3. The self-supervised pre-training method based on diffusion model according to claim 2, characterized in that , injecting noise into the picture x0 and obtaining the picture x at time t as described in steps 1-2 t The specific method is as follows: where denotes the multiplication of the scaling factors at each time instant, α s denotes the scaling factor at time s, α t denotes the scaling factor at time t, α t : = 1 - β t , β t denotes the noise variance at time t, ∈ denotes the injected random sampled Gaussian noise, denotes the standard normal distribution.

4. The self-supervised pre-training method based on diffusion model according to claim 3, wherein, The method for extracting the feature vector in step 2 comprises: Step 2-1, picture x at time t obtained in step 1-2 is input into the noise prediction network t Inputting the noise prediction network ∈ θ The intermediate layer feature maps of the up-sampling part of the noise prediction network are extracted and spliced together Step 2-2, uniformly sample K points from the feature map obtained in step 2-1, and obtain the feature vector corresponding to the sampling point by bilinear interpolation represents the feature vector of the i-th image, i.e., the feature vector generated by the teacher network, represents a real matrix of size KxD, and D represents the dimension of the feature vector.

5. The self-supervised pre-training method based on diffusion model according to claim 4, wherein, The total loss function described in steps 3-7 As follows: Wherein, λ represents a balance factor.

6. The self-supervised pre-training method based on diffusion model according to claim 5, wherein, The method for performing feature extraction in step 4 comprises: Taking the encoder in the student network trained in step 3 as a feature extraction network, and retaining the feature map of the last layer of the encoder as the extracted feature after inputting an image.

7. The self-supervised pre-training method based on diffusion model according to claim 6, wherein, The method for data enhancement in step 3-1 comprises: Random cropping, horizontal flipping, color jittering and Gaussian blur are used to enhance the image.

8. The self-supervised pre-training method based on diffusion model according to claim 7, wherein, The method for masking 50% of the image using a binary mask in step 3-1 comprises: Step 3-1-1, calculating the importance degree s of each block in the image; Step 3-1-2, importance sampling is performed according to the importance degree s of each block, that is, blocks with an importance degree s greater than a threshold value are sampled out, and the blocks are hidden using a mask.

9. The self-supervised pre-training method based on diffusion model according to claim 8, wherein, The importance degree s of each block in the image is calculated in step 3-1-1, and the specific method is as follows: wherein q0represents a query vector of the picture global feature, K 1: represents an index matrix of the picture except the global feature vector, D represents a dimension of the feature vector, represents mathematical expectation, and Softmax(·) represents a Softmax function.

Images