A self-supervised model training method, device, equipment and storage medium

Abstract

The application discloses a self-supervised model training method and device, equipment and a storage medium. The obtained sample image is subjected to data augmentation transformation processing to obtain a first transformed image and a second transformed image, a first network model and a second network model with the same structure and parameters are used to extract a first feature vector and a second feature vector representing a target region of interest in the sample image from the first transformed image and the second transformed image respectively, the probability of the sample image belonging to each category is predicted based on the first feature vector to obtain a probability vector, a first loss value is calculated based on the probability vector and the second feature vector, and the parameters of the first network model are updated based on the first loss value. The self-supervised training method of contrast learning is used for model training, thereby improving the performance of the model. Since no negative sample is required when calculating the loss value, the training efficiency is improved, and the sample collection cost is saved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of machine learning technology, and in particular to a self-supervised model training method, apparatus, device and storage medium. Background Technology

[0002] With the development of artificial intelligence, deep neural networks have achieved remarkable results in many fields due to their powerful fitting ability.

[0003] Applications of deep neural networks typically require pre-building a network model and then training it to obtain a satisfactory model. Self-supervised learning has recently gained significant attention for training these models because it avoids extensive labeling of datasets. It uses self-defined pseudo-labels as training signals and then applies the learned representations to downstream tasks. Contrastive learning is considered a crucial part of self-supervised learning. Its goal is to make different augmented versions of a given sample as close as possible in the embedding space, while keeping different samples as far apart as possible. This requires that after different augmentations, the high-level semantic features extracted by the feature extraction network for the same sample have high similarity, while the high-level semantic features between different samples have low similarity.

[0004] However, in certain specific fields, such as medical imaging, the high-level semantic features of medical images of the same part of different individuals have high similarity, while the high-level semantic features of the same image may have low similarity after different data augmentations. Therefore, it is difficult to directly use the idea of ​​contrastive learning for self-supervised pre-training, and the performance of the obtained model is poor. Summary of the Invention

[0005] This invention provides a self-supervised model training method, apparatus, device, and storage medium to improve model performance.

[0006] In a first aspect, embodiments of the present invention provide a self-supervised model training method, comprising:

[0007] Obtain the sample images required for model training;

[0008] The sample image is subjected to data amplification and transformation processing to obtain a first transformed image and a second transformed image;

[0009] A first feature vector and a second feature vector representing the target region of interest in the sample image are extracted from the first transformed image and the second transformed image, respectively, using a first network model and a second network model with the same structure and parameters.

[0010] Based on the first feature vector, predict the probability that the sample image belongs to each category to obtain a probability vector;

[0011] Calculate the first loss value based on the probability vector and the second feature vector;

[0012] The parameters of the first network model are updated based on the first loss value.

[0013] Secondly, embodiments of the present invention also provide a self-supervised model training apparatus, comprising:

[0014] The image sample acquisition module is used to acquire sample images required for model training.

[0015] The amplification and transformation module is used to perform data amplification and transformation processing on the sample image to obtain a first transformed image and a second transformed image.

[0016] The feature extraction module is used to extract a first feature vector and a second feature vector representing the target region of interest in the sample image from the first transformed image and the second transformed image, respectively, using a first network model and a second network model with the same structure and parameters;

[0017] The probability prediction module is used to predict the probability that the sample image belongs to each category based on the first feature vector, and obtain a probability vector;

[0018] The first loss value calculation module is used to calculate the first loss value based on the probability vector and the second feature vector;

[0019] The parameter update module is used to update the parameters of the first network model based on the first loss value.

[0020] Thirdly, embodiments of the present invention also provide a computer device, comprising:

[0021] One or more processors;

[0022] Storage device for storing one or more programs;

[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the self-supervised model training method provided in the first aspect of the present invention.

[0024] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the self-supervised model training method as provided in the first aspect of the present invention.

[0025] The self-supervised model training method provided in this invention performs data augmentation and transformation on acquired sample images to obtain transformed first and second transformed images. A first network model and a second network model with identical structure and parameters are used to extract a first feature vector and a second feature vector representing the target region of interest in the sample images from the first and second transformed images, respectively. Based on the first feature vector, the probability of the sample image belonging to each category is predicted to obtain a probability vector. A first loss value is calculated based on the probability vector and the second feature vector, and the parameters of the first network model are updated based on the first loss value. The similarity between the features of the target regions in the first and second transformed images of the sample images is high, while the similarity between the features of the target regions in different sample images is low. Therefore, a contrastive learning self-supervised training method can be used for model training, thereby improving model performance. Since negative samples are not required when calculating the loss value, training efficiency is improved and sample collection costs are saved. Attached Figure Description

[0026] Figure 1 This is a flowchart of a self-supervised model training method provided in Embodiment 1 of the present invention;

[0027] Figure 2A This is a flowchart of a self-supervised model training method provided in Embodiment 2 of the present invention;

[0028] Figure 2B This is a data processing flowchart of the self-supervised model training method provided in an embodiment of the present invention;

[0029] Figure 2C This is a schematic diagram of the network model provided in an embodiment of the present invention;

[0030] Figure 2D The convergence curve for the first loss value;

[0031] Figure 2E The convergence curve for the second loss value;

[0032] Figure 3 This is a self-supervised model training device provided in Embodiment 3 of the present invention;

[0033] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0034] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0035] Example 1

[0036] Figure 1 This is a flowchart of a self-supervised model training method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where the high-level semantic feature similarity of the same sample is low after different data amplifications, while the high-level semantic feature similarity between different samples is high. This method can be executed by the self-supervised model training device provided in this embodiment of the present invention. This device can be implemented in software and / or hardware, and is typically configured in a computer device, such as... Figure 1 As shown, the method specifically includes the following steps:

[0037] S101. Obtain the sample images required for model training.

[0038] In this embodiment of the invention, the sample images show low similarity in high-level semantic features after different data amplification processes for the same sample, while showing high similarity in high-level semantic features between different samples. For example, this embodiment uses a 3D chest CT image as a sample image for illustrative purposes. A CT (Computed Tomography) image is a scanned image obtained by using precisely collimated X-ray beams, gamma rays, ultrasound, etc., along with a highly sensitive detector, to perform a series of cross-sectional scans around a specific part of the human body. The detector receives the X-rays passing through the slice, converts them into visible light, then converts them into electrical signals via photoelectric conversion, and finally converts them into digital signals via an analog-to-digital converter. These digital signals are then input into a computer for processing to obtain the CT image.

[0039] In other embodiments of the present invention, the sample image may also be other medical images, such as magnetic resonance images (MRI), and the embodiments of the present invention are not limited thereto.

[0040] S102. Perform data amplification and transformation on the sample image to obtain the first transformed image and the second transformed image.

[0041] Data augmentation transformation refers to creating more data by transforming the original data without actually adding to it. In this embodiment of the invention, to maintain the algorithm's invariance to translation, rotation, and size, and to have a certain degree of anti-interference capability, the data augmentation transformation of the sample image can include: random flipping, random cropping, random scaling, Gaussian blurring, and random erasing, etc. After the data augmentation transformation, the first transformed image and the second transformed image are obtained.

[0042] S103. Using a first network model and a second network model with the same structure and parameters, extract the first feature vector and the second feature vector representing the target region of interest in the sample image from the first transformed image and the second transformed image, respectively.

[0043] In this embodiment of the invention, a first network model is used to extract a first feature vector representing the target region of interest in the sample image from a first transformed image, and a second network model is used to extract a second feature vector representing the target region of interest in the sample image from a second transformed image. The first and second network models have the same network structure and initial parameters. The target region of interest is an information-rich local region in the sample image, or a local region with a high rank in the sample image.

[0044] The first and second transformed images obtained by performing two different data augmentation transformations on the same sample image show high similarity in the features of their target regions (i.e., the first and second feature vectors), while the features of the target regions of different sample images show low similarity. Therefore, a self-supervised training method based on contrastive learning can be used for model training.

[0045] S104. Based on the first feature vector, predict the probability that the sample image belongs to each category to obtain the probability vector.

[0046] In this embodiment of the invention, a prediction mapping transformation is performed on the first feature vector to map it to the sample label space to obtain a probability vector, wherein the elements in the probability vector represent the probability that the sample image belongs to each category.

[0047] S105. Calculate the first loss value based on the probability vector and the second feature vector.

[0048] In this embodiment of the invention, the similarity between the probability vector and the second feature vector can be calculated, and then a first loss value can be determined based on this similarity. The similarity between the probability vector and the second feature vector can be represented by Euclidean distance, cosine similarity, Pearson correlation coefficient, etc., and this embodiment of the invention does not limit the specific representation.

[0049] S106. Update the parameters of the first network model based on the first loss value.

[0050] In this embodiment of the invention, only positive samples (i.e., images after data augmentation and transformation) are used to calculate similarity as the loss function, without the need for negative samples (images that do not belong to the same person as the sample images). By stopping the gradient update of one branch (i.e. stopping the update of the parameters of the second network model), the training collapse caused by the lack of negative sample pairs can be prevented (the network outputs a fixed value).

[0051] The self-supervised model training method provided in this invention performs data augmentation and transformation on acquired sample images to obtain transformed first and second transformed images. A first network model and a second network model with identical structure and parameters are used to extract a first feature vector and a second feature vector representing the target region of interest in the sample images from the first and second transformed images, respectively. Based on the first feature vector, the probability of the sample image belonging to each category is predicted to obtain a probability vector. A first loss value is calculated based on the probability vector and the second feature vector, and the parameters of the first network model are updated based on the first loss value. The similarity between the features of the target regions in the first and second transformed images of the sample images is high, while the similarity between the features of the target regions in different sample images is low. Therefore, a contrastive learning self-supervised training method can be used for model training, thereby improving model performance. Since negative samples are not required when calculating the loss value, training efficiency is improved and sample collection costs are saved.

[0052] Example 2

[0053] Figure 2A This is a flowchart of a self-supervised model training method provided in Embodiment 2 of the present invention. Figure 2B This is a data processing flowchart of the self-supervised model training method provided in this embodiment of the invention. This embodiment refines the above-described embodiment one, detailing the specific implementation process of each step in the self-supervised model training method, such as... Figure 2A , 2B As shown, the method includes:

[0054] S201. Obtain the sample images required for model training.

[0055] In this embodiment of the invention, the sample images show low similarity in high-level semantic features after different data amplification processes for the same sample, while high similarity in high-level semantic features exists between different samples. For example, this embodiment uses a 3D chest CT image as the sample image for illustrative purposes.

[0056] S202. Extract the target region of interest from the sample image.

[0057] The target region of interest is the information-rich local region in the sample image, or the local region with a large rank in the sample image.

[0058] In this embodiment of the invention, to make the learning and prediction of the target region by the algorithm network meaningful, several regions are randomly cropped from the sample image according to a certain proportion. Then, the variance of each region is calculated, and the variance is used to measure the information content of the image region. The region with the largest variance is taken as the target region of interest in the sample image. Figure 2B(The region outlined in the box). Of course, in other embodiments of the present invention, the rank of each region image can also be calculated, and the region image with the highest rank can be used as the target region of interest in the sample image.

[0059] S203. Perform data amplification and transformation on the sample image to obtain the first transformed image and the second transformed image.

[0060] In this embodiment of the invention, to maintain the translation, rotation, and size invariance of the designed algorithm and to have a certain degree of anti-interference capability, the data augmentation transformation of the sample image can include: random flipping, random cropping, random scaling, Gaussian blurring, and random erasing, etc. After the data augmentation transformation process, the transformed first and second transformed images are obtained. Figure 2B In this context, t1 and t2 represent two different data amplification transformations, respectively.

[0061] S204. Extract multiple global feature maps of different scales from the first transformed image and the second transformed image respectively.

[0062] In this embodiment of the invention, the first transformed image and the second transformed image are respectively input into a first network model and a second network model with identical network structure and initial parameters. Figure 2C This is a schematic diagram of the network model provided in an embodiment of the present invention, such as... Figure 2C As shown, both network models include a backbone network and a feature pyramid network (FPN). Figure 2B The processing procedure of the network model is replaced by f.

[0063] For example, in this embodiment of the invention, the backbone network is a 3D ResNet34 network, which includes five convolutional blocks connected sequentially, with the output of the previous convolutional block serving as the input of the next. The output features of the five convolutional blocks are C1, C2, C3, C4, and C5, respectively, with the resolution of output features C1, C2, C3, C4, and C5 decreasing sequentially. For example, in this embodiment of the invention, the parameters of each convolutional block are shown in the table below.

[0064]

[0065] S205. Fuse global feature maps of different scales from the first transformed image and global feature maps of different scales from the second transformed image to obtain a first fused feature map and a second fused feature map corresponding to the first transformed image and the second transformed image.

[0066] In this embodiment of the invention, a feature pyramid network is used to fuse global feature maps of different scales from a first transformed image, and to fuse global feature maps of different scales from a second transformed image. For example, as shown... Figure 2C As shown, firstly, a one-dimensional convolution is performed on the output feature C5 of the backbone network to obtain feature P5. Then, feature P5 is upsampled to obtain an upsampled feature with the same resolution as the output feature C4. Next, the upsampled feature is added to the output feature C4 to obtain feature P4. Finally, after a 3*3*3 convolution, a first fusion feature map and a second fusion feature map with the same resolution as feature C4 are output.

[0067] In deep neural networks, feature maps output by lower-level networks have higher resolution and contain more location and detail information, but due to fewer convolutions, they have lower semantic meaning and more noise. Feature maps output by higher-level networks have stronger semantic information, but their resolution is very low, and their ability to perceive details is poor. In this embodiment of the invention, multiple feature maps of different scales are extracted from sample images and fused, taking into account location, detail, and semantic information, thereby improving feature representation and ultimately improving model performance.

[0068] It should be noted that in the embodiments of the present invention, the backbone network can be replaced with other deep learning networks that require pre-training, such as changing the number of each convolutional block to form a 3D convolutional network with different numbers of convolutional layers; the FPN network can modify the number of feature layers from the current 2 layers to other numbers (such as 3 or 4 layers), and the feature fusion method can be changed from the current addition to channel concatenation. The embodiments of the present invention do not limit this.

[0069] S206. Extract the first feature vector and the second feature vector representing the target region from the target region of the first fused feature map and the second fused feature map, respectively.

[0070] For example, in this embodiment of the invention, center pooling is performed on the target regions of the first fused feature map and the second fused feature map to obtain a first local feature h1 and a second local feature h2 representing the target region. Then, the first local feature h1 and the second local feature h2 are transformed by projection mapping to obtain a first feature vector p1 and a second feature vector p2 of the target region.

[0071] Existing methods for extracting local features from target regions typically employ ROI Pooling or ROI Align, but these methods are computationally expensive, resulting in slow training speeds, and lack mature 3D implementations. This invention directly performs average pooling on the central region of the target region within the fused feature map, offering a simple and fast implementation.

[0072] The first and second transformed images obtained by performing two different data augmentation transformations on the same sample image show high similarity in the features of their target regions (i.e., the first and second feature vectors), while the features of the target regions of different sample images show low similarity. Therefore, a self-supervised training method based on contrastive learning can be used for model training.

[0073] For example, in this embodiment of the invention, in order to isolate the influence of self-supervised training on the feature vectors, a multilayer perceptron (MLP) is used to perform a projection mapping transformation (proj) on the first local feature h1 and the second local feature h2. The vectors of each layer of the multilayer perceptron have the same dimension, and the same dimension spatial mapping transformation is performed, that is, the channel dimension is 256.

[0074] S207. Based on the first feature vector, predict the probability that the sample image belongs to each category to obtain the probability vector.

[0075] In this embodiment of the invention, a multilayer perceptron is used to perform a predictive mapping transformation (pred) on the first feature vector p1, mapping it to the sample label space to obtain the probability vector z1. The multilayer perceptron first performs a dimensionality compression mapping and then a restoration mapping, that is, first mapping to 64 and then restoring the mapping to 256, thereby reducing the influence of redundant features.

[0076] It should be noted that, in this embodiment of the invention, the multilayer perceptron may contain only one hidden layer or multiple hidden layers, and this embodiment of the invention is not limited thereto. The structures of the multilayer perceptrons used for projection mapping and prediction mapping may be the same or different. The multilayer perceptron may also be replaced by a fully connected network, and this embodiment of the invention is not limited thereto.

[0077] S208. Calculate the first loss value based on the probability vector and the second feature vector.

[0078] In this embodiment of the invention, the similarity between the probability vector and the second feature vector can be calculated, and then a first loss value can be determined based on this similarity. For example, in this embodiment, the cosine similarity between the probability vector z1 and the second feature vector p2 is calculated as the similarity between the probability vector z1 and the second feature vector p2. Then, the negative of the cosine similarity is taken as the first loss value. Specifically, the formula for calculating the first loss value is as follows:

[0079]

[0080] S209. Reconstruct the target region of the sample image based on the first feature vector or probability vector to obtain the reconstructed image.

[0081] In this embodiment of the invention, to ensure that the learned feature vectors are diverse and contain approximate structural information of the target region, after obtaining the first feature vector p1 or probability vector z1, the target region of the sample image can be reconstructed based on the first feature vector p1 or probability vector z1 to obtain a reconstructed image. Then, the loss value of the target region between the reconstructed image and the original sample image is calculated.

[0082] For example, such as Figure 2C As shown, the probability vector z1 is reconstructed at low resolution to generate a low-resolution 3D local image (i.e., the reconstructed image). For example, image reconstruction can be performed using upsampling, deconvolution, etc., and this embodiment of the invention is not limited thereto.

[0083] S210. Perform downsampling processing on the sample image to obtain a downsampled image with the same resolution as the reconstructed image.

[0084] For example, such as Figure 2C As shown, the original sample image is downsampled to obtain a downsampled image with the same resolution as the reconstructed image.

[0085] S211. Calculate the mean square error of the target region in the reconstructed image and the downsampled image as the second loss value.

[0086] The mean squared error (MES) is the mean of the sum of squares of the errors at corresponding points in the predicted data and the original data. In this embodiment of the invention, the mean squared error of the pixel values ​​of corresponding pixels in the reconstructed image and the downsampled image is calculated as the second loss value L. reconstruct .

[0087] S212. Update the parameters of the first network model based on the first loss value and the second loss value.

[0088] In this embodiment of the invention, to balance the weights of the first loss value and the second loss value, a balance coefficient is set for the second loss value. The product of the second loss value and the balance parameter is calculated to obtain the third loss value. Then, the sum of the first loss value and the third loss value is calculated to obtain the total loss value. The specific calculation formula is as follows:

[0089] Loss = L contrast +α*L reconstruct

[0090] Where α is the balance coefficient.

[0091] When the total loss value exceeds a preset loss threshold, the parameters of the first network model are updated, and the process returns to obtaining the sample images required for model training. This training is iterated until the total loss value is less than or equal to the preset loss threshold, at which point the model training is considered complete.

[0092] Low-resolution reconstruction makes the learned feature representations more targeted, enabling the inverse reconstruction of the approximate structure of the target region. Low-resolution reconstruction is an important aid and supplement to contrastive learning, allowing the learned local features to have specific representational meanings.

[0093] In this embodiment of the invention, only positive samples (i.e., images after data augmentation and transformation) are used to calculate similarity as the loss function, without the need for negative samples (images that do not belong to the same person as the sample images). By stopping the gradient update of one branch (i.e. stopping the update of the parameters of the second network model), the training collapse caused by the lack of negative sample pairs can be prevented (the network outputs a fixed value).

[0094] In this embodiment of the invention, a trained first network model is used to verify the effectiveness of the self-supervised learning scheme designed in this invention. Figure 2D The convergence curve for the first loss value. Figure 2E The convergence curve for the second loss value is shown. Self-supervised training was performed using over 20,000 MR image datasets, with a balance coefficient α set to 1.0. Figure 2D and Figure 2E As shown, the designed self-supervised algorithm converges during training, and the first loss value decreases more significantly (initially 0).

[0095] A segmentation model for nasopharyngeal carcinoma was designed using the first network model as the backbone model. It was trained with more than 400 segmentation labeled data. The performance of the self-supervised pre-trained model was compared and verified. As shown in the table below, after using the pre-trained model of this invention, the segmentation algorithm performance improved by 2.1 and 5.9 percentage points on the validation set and the test set, respectively (the improvement was more obvious on the test set, and the generalization ability was stronger). However, when using the SimSiam self-supervised method for pre-training, the model performance slightly decreased on the validation set and only improved by 1.9 percentage points on the test set.

[0096]

[0097]

[0098] The self-supervised model training method provided in this invention exhibits high similarity between the features of the target regions in the first and second transformed images of the sample images, while low similarity exists between the features of the target regions in different sample images. Therefore, a contrastive learning-based self-supervised training method can be used for model training, thereby improving model performance. Since negative samples are not required when calculating the loss value, training efficiency is improved, and sample collection costs are saved. By fusing feature maps of multiple scales, location, detail, and semantic information are considered, enhancing feature representation and thus improving model performance. Low-resolution reconstruction makes the learned feature representation more directional.

[0099] Example 3

[0100] Figure 3 This invention provides a self-supervised model training device according to Embodiment 3 of the present invention, such as... Figure 3 As shown, the device includes:

[0101] Image sample acquisition module 301 is used to acquire sample images required for model training;

[0102] The amplification and transformation module 302 is used to perform data amplification and transformation processing on the sample image to obtain a first transformed image and a second transformed image.

[0103] Feature extraction module 303 is used to extract a first feature vector and a second feature vector representing the target region of interest in the sample image from the first transformed image and the second transformed image respectively using a first network model and a second network model with the same structure and parameters;

[0104] The probability prediction module 304 is used to predict the probability that the sample image belongs to each category based on the first feature vector, and obtain a probability vector.

[0105] The first loss value calculation module 305 is used to calculate the first loss value based on the probability vector and the second feature vector;

[0106] The parameter update module 306 is used to update the parameters of the first network model based on the first loss value.

[0107] In some embodiments of the present invention, the self-supervised model training apparatus further includes:

[0108] The region cropping module is used to randomly crop several regions from the sample images according to a proportional size after obtaining the sample images required for model training.

[0109] A variance calculation module is used to calculate the variance of each of the aforementioned region images;

[0110] The target region determination module is used to identify the region with the largest variance as the target region of interest in the sample image.

[0111] In some embodiments of the present invention, the feature extraction module 303 includes:

[0112] The global feature extraction submodule is used to extract multiple global feature maps of different scales from the first transformed image and the second transformed image, respectively.

[0113] The feature fusion submodule is used to fuse multiple global feature maps of different scales of the first transformed image and multiple global feature maps of different scales of the second transformed image to obtain a first fused feature map and a second fused feature map corresponding to the first transformed image and the second transformed image.

[0114] The feature vector extraction submodule is used to extract a first feature vector and a second feature vector representing the target region from the target region of the first fused feature map and the second fused feature map, respectively.

[0115] In some embodiments of the present invention, the feature vector extraction submodule includes:

[0116] A central pooling unit is used to perform central average pooling on the target regions of the first fused feature map and the second fused feature map respectively to obtain a first local feature and a second local feature representing the target region;

[0117] The projection unit is used to perform projection mapping transformation on the first local feature and the second local feature respectively to obtain the first feature vector and the second feature vector of the target region.

[0118] In some embodiments of the present invention, the first loss value calculation module 305 includes:

[0119] The matching degree calculation submodule is used to calculate the cosine similarity between the probability vector and the second feature vector as the matching degree between the probability vector and the second feature vector;

[0120] The first loss value calculation submodule is used to take the opposite of the cosine similarity as the first loss value.

[0121] In some embodiments of the present invention, the self-supervised model training apparatus further includes:

[0122] The target region reconstruction module is used to reconstruct the target region of the sample image based on the first feature vector or the probability vector after predicting the probability vector of the sample image belonging to each category based on the first feature vector, so as to obtain a reconstructed image.

[0123] The downsampling module is used to downsample the sample image to obtain a downsampled image with the same resolution as the reconstructed image;

[0124] The second loss value calculation module is used to calculate the mean square error of the target region in the reconstructed image and the downsampled image as the second loss value.

[0125] In some embodiments of the present invention, the parameter update module 306 includes:

[0126] The third loss value calculation submodule is used to calculate the product of the second loss value and the balance parameter to obtain the third loss value;

[0127] The total loss calculation submodule is used to calculate the sum of the first loss value and the third loss value to obtain the total loss value;

[0128] The parameter update submodule is used to update the parameters of the first network model when the total loss value is greater than a preset loss value threshold, and return to the step of obtaining the sample images required for model training.

[0129] The training completion determination submodule is used to determine that the model training is complete when the total loss value is less than or equal to a preset loss value threshold.

[0130] The above-described self-supervised model training device can execute the self-supervised model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the self-supervised model training method.

[0131] Example 4

[0132] Embodiment 4 of the present invention provides a computer device, Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention, as shown below. Figure 4 As shown, the computer device includes:

[0133] Processor 401, memory 402, communication module 403, input device 404, and output device 405; the number of processors 401 in the computer device can be one or more. Figure 4 Taking a processor 401 as an example; the processor 401, memory 402, communication module 403, input device 404, and output device 405 in a computer device can be connected via a bus or other means. Figure 4 Taking a bus connection as an example, the processor 401, memory 402, communication module 403, input device 404, and output device 405 mentioned above can be integrated into a computer device.

[0134] The memory 402, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the modules corresponding to the self-supervised model training method in the above embodiments. The processor 401 executes various functional applications and data processing of the computer device by running the software programs, instructions, and modules stored in the memory 402, thereby realizing the self-supervised model training method described above.

[0135] Memory 402 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the microcomputer. Furthermore, memory 402 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, memory 402 may further include memory remotely located relative to processor 401, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0136] The communication module 403 is used to establish a connection with external devices (such as smart terminals) and to realize data interaction with external devices. The input device 404 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the computer device.

[0137] The computer device provided in this embodiment can execute the self-supervised model training method provided in any of the above embodiments of the present invention, and has corresponding functions and beneficial effects.

[0138] Example 5

[0139] Embodiment 5 of the present invention provides a storage medium containing computer-executable instructions, on which a computer program is stored. When executed by a processor, the program implements the loop closure detection self-supervised model training method provided in any of the above embodiments of the present invention. The method includes:

[0140] Obtain the sample images required for model training;

[0141] The sample image is subjected to data amplification and transformation processing to obtain a first transformed image and a second transformed image;

[0142] A first feature vector and a second feature vector representing the target region of interest in the sample image are extracted from the first transformed image and the second transformed image, respectively, using a first network model and a second network model with the same structure and parameters.

[0143] Based on the first feature vector, predict the probability that the sample image belongs to each category to obtain a probability vector;

[0144] Calculate the first loss value based on the probability vector and the second feature vector;

[0145] The parameters of the first network model are updated based on the first loss value.

[0146] It should be noted that the embodiments of the apparatus, computer equipment, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple. For relevant details, please refer to the descriptions of the method embodiments.

[0147] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a robot, personal computer, server, or network device, etc.) to execute the self-supervised model training method described in any embodiment of the present invention.

[0148] It is worth noting that the various modules, sub-modules, and units included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as they can achieve the corresponding functions; in addition, the specific names of each functional module are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0149] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0150] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0151] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A self-supervised model training method, characterized in that, include: Obtain the sample images required for model training; the sample images are medical images. Several regions of the image are randomly cropped from the sample image according to a proportional size. Calculate the variance of the image for each of the aforementioned regions; The region with the largest variance is selected as the target region of interest in the sample image. The sample image is subjected to data amplification and transformation processing to obtain a first transformed image and a second transformed image; A first feature vector and a second feature vector representing the target region of interest in the sample image are extracted from the first transformed image and the second transformed image respectively using a first network model and a second network model with the same structure and parameters. The target region of interest is an information-rich local region in the sample image. Based on the first feature vector, predict the probability that the sample image belongs to each category to obtain a probability vector; Calculate the first loss value based on the probability vector and the second feature vector; The parameters of the first network model are updated based on the first loss value.

2. The self-supervised model training method according to claim 1, characterized in that, Using a first network model and a second network model with identical structure and parameters, a first feature vector and a second feature vector representing the target region of interest in the sample image are extracted from the first transformed image and the second transformed image, respectively, including: Multiple global feature maps of different scales are extracted from the first transformed image and the second transformed image, respectively; By fusing global feature maps of different scales from the first transformed image and global feature maps of different scales from the second transformed image, a first fused feature map and a second fused feature map corresponding to the first transformed image and the second transformed image are obtained. A first feature vector and a second feature vector representing the target region are extracted from the target region of the first fused feature map and the second fused feature map, respectively.

3. The self-supervised model training method according to claim 2, characterized in that, Extracting a first feature vector and a second feature vector representing the target region from the target region of the first fused feature map and the second fused feature map, respectively, includes: The target regions of the first fused feature map and the second fused feature map are respectively subjected to center average pooling to obtain a first local feature and a second local feature representing the target region; The first local feature and the second local feature are respectively subjected to projection mapping transformation to obtain the first feature vector and the second feature vector of the target region.

4. The self-supervised model training method according to claim 1, characterized in that, The first loss value is calculated based on the probability vector and the second feature vector, including: The cosine similarity between the probability vector and the second feature vector is calculated as the matching degree between the probability vector and the second feature vector; The negative of the cosine similarity is taken as the first loss value.

5. The self-supervised model training method according to claim 1, characterized in that, After predicting the probability vectors of the sample image belonging to each category based on the first feature vector, the method further includes: The target region of the sample image is reconstructed based on the first feature vector or the probability vector to obtain a reconstructed image; The sample image is downsampled to obtain a downsampled image with the same resolution as the reconstructed image; The mean square error of the target region in the reconstructed image and the downsampled image is calculated as the second loss value.

6. The self-supervised model training method according to claim 5, characterized in that, Updating the parameters of the first network model based on the first loss value includes: The third loss value is obtained by multiplying the second loss value by the balance parameter. Calculate the sum of the first loss value and the third loss value to obtain the total loss value; When the total loss value is greater than a preset loss value threshold, the parameters of the first network model are updated, and the process returns to the step of obtaining the sample images required for model training. The model training is considered complete when the total loss value is less than or equal to a preset loss value threshold.

7. A self-supervised model training device, characterized in that, include: An image sample acquisition module is used to acquire sample images required for model training, wherein the sample images are medical images; The region cropping module is used to randomly crop several regions from the sample images according to a proportional size after obtaining the sample images required for model training. A variance calculation module is used to calculate the variance of each of the aforementioned region images; The target region determination module is used to identify the region image with the largest variance as the target region of interest in the sample image. The amplification and transformation module is used to perform data amplification and transformation processing on the sample image to obtain a first transformed image and a second transformed image. The feature extraction module is used to extract a first feature vector and a second feature vector representing the target region of interest in the sample image from the first transformed image and the second transformed image, respectively, using a first network model and a second network model with the same structure and parameters, wherein the target region of interest is an information-rich local region in the sample image; The probability prediction module is used to predict the probability that the sample image belongs to each category based on the first feature vector, and obtain a probability vector; The first loss value calculation module is used to calculate the first loss value based on the probability vector and the second feature vector; The parameter update module is used to update the parameters of the first network model based on the first loss value.

8. A computer device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the self-supervised model training method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the self-supervised model training method as described in any one of claims 1-6.

Images