Supervised learning method, device and equipment of image feature and storage medium

Abstract

The application discloses a supervised learning method and device of image features, equipment and a storage medium, and relates to the field of artificial intelligence. The method comprises the following steps: performing data enhancement on an original medical image to obtain a first enhanced image and a second enhanced image, the first enhanced image and the second enhanced image being positive samples of each other; performing feature extraction on the first enhanced image and the second enhanced image by using a feature extraction model to obtain a first image feature of the first enhanced image and a second image feature of the second enhanced image; determining a model loss of the feature extraction model based on the first image feature, the second image feature and a negative sample image feature, the negative sample image feature being an image feature of an enhanced image corresponding to other original medical images; and training the feature extraction model based on the model loss. In the embodiment of the application, the self-supervised learning mode is used to enable the feature extraction model to learn the image features of the medical images, manual image labeling is not required, and the model training efficiency is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence, and in particular to a supervised learning method and device for image features, an apparatus, and a storage medium. BACKGROUND

[0002] Computer vision (CV) technology, as an important branch of artificial intelligence, is widely used in medical image recognition (recognizing the category of tissue organs in medical images), medical image retrieval (retrieving detailed medical images from a database), medical image segmentation (segmenting tissue structures in medical images), and other medical image processing scenarios.

[0003] Image feature extraction, as an important link in the image processing process, directly affects the final image processing result. In related technologies, a feature extraction model is trained to extract features from medical images, and then subsequent image processing procedures are performed based on the extracted image features. In one model training method, when supervised learning is used for model training, the annotation information of sample medical images is usually used as supervision for model training.

[0004] However, a large number of samples are required in the model training process, and manual annotation of sample medical images requires a lot of time, resulting in low model training efficiency. SUMMARY

[0005] Embodiments of the present application provide a supervised learning method, device, apparatus, and storage medium for image features, which can realize self-supervised learning of image features without manual annotation, thereby improving the model training efficiency. The technical solution is as follows:

[0006] On the one hand, the present application provides a supervised learning method for image features, which comprises:

[0007] performing data augmentation on the original medical images to obtain a first augmented image and a second augmented image, the first augmented image and the second augmented image being positive samples for each other;

[0008] extracting features from the first augmented image and the second augmented image by a feature extraction model to obtain a first image feature of the first augmented image and a second image feature of the second augmented image;

[0009] determining a model loss of the feature extraction model based on the first image feature, the second image feature, and a negative sample image feature, the negative sample image feature being an image feature of an augmented image corresponding to another original medical image;

[0010] training the feature extraction model based on the model loss.

[0011] In another aspect, an embodiment of the present application provides a device for supervised learning of image features, the device comprising:

[0012] a data enhancement module configured to perform data enhancement on an original medical image to obtain a first enhanced image and a second enhanced image, the first enhanced image and the second enhanced image being positive samples of each other;

[0013] a feature extraction module configured to perform feature extraction on the first enhanced image and the second enhanced image by using a feature extraction model to obtain a first image feature of the first enhanced image and a second image feature of the second enhanced image;

[0014] a loss determination module configured to determine a model loss of the feature extraction model based on the first image feature, the second image feature, and a negative sample image feature, the negative sample image feature being an image feature of an enhanced image corresponding to another original medical image;

[0015] a first training module configured to train the feature extraction model based on the model loss.

[0016] In another aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the supervised learning method of image features according to the above aspect.

[0017] In another aspect, an embodiment of the present application provides a computer readable storage medium, the readable storage medium storing at least one instruction, the at least one instruction being loaded and executed by a processor to implement the supervised learning method of image features according to the above aspect.

[0018] In another aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the supervised learning method of image features according to the above aspect.

[0019] In the embodiment of the present application, the first enhanced image and the second enhanced image which are positive samples of each other are obtained by data enhancement on the original medical image, and the first image feature and the second image feature are obtained by feature extraction through the feature extraction model. Then, other original medical images different from the original medical image are taken as negative samples, and the model loss of the feature extraction model is determined based on the first image feature, the second image feature and the negative sample image feature. Finally, the feature extraction model is trained by using the model loss. In the whole process, the self-supervised learning method is used to make the feature extraction model learn the image features of the medical image, without manual medical image labeling, thereby reducing the manual labeling cost in the model training process and improving the training efficiency of the feature extraction model. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a schematic diagram of the image feature supervised learning method according to an example embodiment of the present application;

[0022] Figure 2 is an implementation schematic diagram of the medical image classification scene according to an example embodiment of the present application;

[0023] Figure 3 is an implementation schematic diagram of the medical image retrieval scene according to an example embodiment of the present application;

[0024] Figure 4 shows a flowchart of the image feature supervised learning method according to an example embodiment of the present application;

[0025] Figure 5 is a medical image which is a positive sample according to an example embodiment of the present application;

[0026] Figure 6 shows a flowchart of the image feature supervised learning method according to another example embodiment of the present application;

[0027] Figure 7 is an implementation schematic diagram of the image feature self-supervised learning process according to an example embodiment of the present application;

[0028] Figure 8 is a schematic diagram of the multi-global descriptor network according to an example embodiment of the present application;

[0029] Figure 9is a flow chart of a model loss determination process according to an example embodiment of the present application;

[0030] Figure 10 is a flow chart of a supervised learning method of image features according to another example embodiment of the present application;

[0031] Figure 11 is a schematic diagram of valid samples and invalid samples according to an example embodiment of the present application;

[0032] Figure 12 is an implementation schematic diagram of a multi-image feature weighted summation process according to an example embodiment of the present application;

[0033] Figure 13 is a structural schematic diagram of a computer device according to an example embodiment of the present application;

[0034] Figure 14 is a structural block diagram of a supervised learning device of image features according to an example embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0036] Computer vision is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further to do image processing, so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, intelligent transportation, etc. It also includes common face recognition, fingerprint recognition and other biometric recognition technologies.

[0037] Image feature extraction is an important link to realize specific functions, and the quality of the extracted image features will directly affect the quality of the function implementation. For example, when performing image recognition, high-quality image features extracted can help improve the accuracy of subsequent image recognition; when performing image retrieval, high-quality image features extracted can help improve the comprehensiveness of the retrieval results and reduce the probability of retrieving irrelevant results.

[0038] In the related art, a supervised model training method is usually used to train a feature extraction model, so as to perform image feature extraction by using the trained feature extraction model. Before the supervised model training, a large number of sample images containing annotation information need to be prepared in advance, so as to subsequently perform model training under the supervision of the annotation information. For example, when a feature extraction model is combined with a classification model to realize image classification, the sample images used for model training need to contain type labels; when a feature extraction model is combined with a segmentation model to realize image segmentation, the sample images used for model training need to contain object segmentation information. However, manual annotation of the sample images needs to consume a large amount of time and has a high annotation cost, resulting in low training efficiency of the feature extraction model.

[0039] To reduce the dependence on manual annotation and improve the model training efficiency under the premise of ensuring the feature extraction quality, an image feature supervised learning method is provided in the embodiments of the present application, as shown in Figure 1 As shown in the figure, the computer device obtains a first enhanced image 12 and a second enhanced image 13 that are positive samples of each other based on the original medical image 11 by using a data enhancement technology, and obtains other original medical images 14 different from the original medical image 11 as negative samples, so as to determine a model loss 18 based on the image features (including a first image feature 15, a second image feature 16, and a negative sample image feature 17) of the first enhanced image 12, the second enhanced image 13, and the negative samples, and then train a feature extraction model 19 by using the model loss 18. In the whole model training process, without the aid of manual annotation, the computer device only needs to use the original medical image to realize self-supervised feature learning, which helps to reduce the sample preparation cost in the early stage of model training and improve the model training efficiency.

[0040] The feature extraction model trained by using the scheme provided in the embodiments of the present application can be used to extract image features of a medical image, and the extracted image features can be used for medical image classification, similar medical image retrieval, and the like.

[0041] In a possible application scenario, as shown in Figure 2 After the medical image to be classified 21 is input into the pre-trained feature extraction model 22, the image feature 23 of the medical image to be classified 21 can be obtained, and the image feature 23 is input into the pre-trained classifier 24. The classifier 24 performs image classification according to the image feature 23, and finally outputs the classification label 25 corresponding to the medical image to be classified 21.

[0042] In another possible application scenario, as shown in Figure 3As shown, in the offline data processing stage, the computer device first divides the Whole Slide Image (WSI) 301 to obtain medical images 302 of small sizes, and uses a pre-trained feature extraction model 303 to extract features of each medical image 302 to obtain image features of each medical image 302, and constructs a medical image feature database based on the image features 304. In the online retrieval stage, the user selects a retrieval area for a WSI 305 to be retrieved to obtain a medical image 306 to be retrieved, and uses the pre-trained feature extraction model 303 to extract features of the medical image 306 to be retrieved to obtain image features 307 to be retrieved, so as to perform feature matching between the image features 307 to be retrieved and the image features 304 in the medical image feature database, and then determine the medical images 302 with a feature matching degree higher than a threshold as similar images 308.

[0043] It should be noted that the above application scenarios are only exemplary, and the feature extraction model trained by the scheme provided in the embodiments of the present application can also be used in other scenarios using image features, such as division of abnormal tissue regions in medical images, and the like, which are not limited by the embodiments of the present application.

[0044] In addition, the supervised learning method of image features provided in the embodiments of the present application can be used for a computer device for training a feature extraction model, which can be a personal computer, a workstation, a physical server, or a cloud server, and the like. For the convenience of description, the following embodiments are described by taking the method applied to the computer device as an example.

[0045] Figure 4 A flowchart of the supervised learning method of image features provided in an exemplary embodiment of the present application is shown. The present embodiment takes the method applied to the computer device as an example, which includes the following steps.

[0046] Step 401, data augmentation is performed on the original medical image to obtain a first augmented image and a second augmented image, and the first augmented image and the second augmented image are positive samples of each other.

[0047] The goal of using the self-supervised learning method to train the feature extraction model is to reduce the distance of similar medical images in the feature encoding space, and to increase the distance of dissimilar images in the feature encoding space, so that the model has the ability to distinguish the similarity of images. Therefore, how to determine the similarity between the input images in the model training process, and based on the correct guidance of the model, becomes the key to self-supervised learning. In the embodiments of the present application, different degrees or ways of data augmentation are performed on the original medical image to obtain similar but not completely identical first and second augmented images. Correspondingly, the image features of the first and second augmented images have high similarity, but are not completely consistent.

[0048] In one possible implementation, taking into account the characteristics of medical images, computer devices can enhance data in two aspects: color (because medical images are stained microscopic tissue sections, the degree of staining may vary) and orientation (because tissue sections can be at any angle under a microscope, medical images are not sensitive to display orientation). Color enhancement is used to change the brightness of the image, thereby increasing its robustness across the color gamut, while orientation enhancement is used to change the angle or orientation of the image, thereby reducing sensitivity to the image's display orientation.

[0049] In some embodiments, a method for a computer device to enhance the color of an image can be described as follows: ,in, This represents the brightness of each pixel in the original medical image. and All are adjustment coefficients. This represents the numerical range of the adjustment coefficient.

[0050] When computer equipment performs directional enhancement on images, it can perform random angle rotation, random mirroring, and other processing on the original medical images. This embodiment does not limit this.

[0051] Optionally, the computer performs color enhancement and orientation enhancement on the original medical image based on the first enhancement parameter to obtain a first enhanced image, and performs color enhancement and orientation enhancement on the original medical image based on the second enhancement parameter to obtain a second enhanced image, wherein the first enhancement parameter and the second enhancement parameter are different.

[0052] For example, in the first enhancement parameter , The rotation angle is +25°, in the second enhancement parameter, , The rotation angle is -25°.

[0053] Of course, in addition to data augmentation of images from the above dimensions, computer devices can also perform data augmentation of images from other dimensions, and the embodiments of this application do not constitute a limitation.

[0054] Furthermore, considering the strong distance correlation of medical images on a physical scale—that is, medical images that are physically close to each other are similar—in other possible implementations, the computer device determines two medical images that are less than a distance threshold as positive samples, further increasing the number of positive samples. This distance threshold is related to the resolution of the medical images; for example, at a 10x magnification, this distance is consistently 100 pixels.

[0055] Indicative, such as Figure 5As shown, the computer device determines that the first medical image 51 of the first region and the second medical image 52 of the second region in the same WSI are positive samples of each other.

[0056] At step 402, the first enhanced image and the second enhanced image are subjected to feature extraction by a feature extraction model to obtain first image features of the first enhanced image and second image features of the second enhanced image.

[0057] Further, the computer device inputs the first enhanced image and the second enhanced image into the feature extraction model respectively, and the feature extraction model is subjected to feature extraction to obtain the first image features and the second image features. The first image features and the second image features are represented by feature maps, and the feature extraction model can be a model with a residual network (ResNet), ResNeXt, or a vision transformer (ViT) as a backbone network, and the embodiments of the present application do not limit the backbone network used by the feature extraction model.

[0058] At step 403, based on the first image features, the second image features, and negative sample image features, a model loss of the feature extraction model is determined, and the negative sample image features are image features of enhanced images corresponding to other original medical images.

[0059] During the model training process, in addition to the first enhanced image and the second enhanced image that are positive samples of each other, the computer device also needs to introduce negative samples that are not similar to the first enhanced image and the second enhanced image, so that the feature extraction model can only learn the difference between the image features of the dissimilar images.

[0060] In a possible implementation, the computer device takes other original medical images different from the current original medical image as negative samples of the current original medical image, and then takes enhanced images corresponding to the other original medical images as negative samples of the first enhanced image and the second enhanced image. Optionally, the current original medical image and the other original medical image are different images belonging to the same training batch, and the other original medical image is subjected to data enhancement and feature extraction before the current original medical image.

[0061] Optionally, the computer device determines the loss of the feature extraction model based on the feature difference between the first image features and the second image features, and the feature difference between the first image features (or the second image features) and the negative sample image features. The feature difference between the image features can be represented by a feature distance, which can be an Euclidean distance, a Manhattan distance, a cosine distance, etc., and the embodiments of the present application do not limit the feature distance.

[0062] At step 404, the feature extraction model is trained based on the model loss.

[0063] Further, the computer device takes minimizing the model loss as a training objective, i.e., reducing the feature difference between the first image features and the second image features, and increasing the feature difference between the first image features (or the second image features) and the negative sample image features, and trains the feature extraction model until a training completion condition is met.

[0064] To sum up, in the embodiments of the present application, the original medical images are subjected to data augmentation to obtain the first and second augmented images which are positive samples to each other, and the feature extraction model is used to extract features to obtain the first and second image features. Then, other original medical images different from the original medical images are taken as negative samples, and the model loss of the feature extraction model is determined based on the first and second image features and the negative sample image features. Finally, the feature extraction model is trained by using the model loss. In the whole process, the self-supervised learning manner is used to enable the feature extraction model to learn the image features of the medical images, without manual medical image labeling, thereby reducing the manual labeling cost in the model training process and improving the training efficiency of the feature extraction model.

[0065] In a possible implementation, to avoid using the same feature extraction network to extract features from the first and second augmented images and output the same feature extraction result, thereby leading to collapse, the feature extraction model in the embodiments of the present application includes two feature extraction branches, so that different feature extraction branches are used to extract features from different augmented images. Different feature extraction branches use different parameter feature extraction networks (i.e., the feature extraction network weights are not shared).

[0066] Figure 6 A flowchart of an image feature supervised learning method provided by another exemplary embodiment of the present application is shown. The present embodiment takes the method used for a computer device as an example for illustration, and the method includes the following steps.

[0067] Step 601, data augmentation is performed on the original medical images to obtain the first and second augmented images, and the first and second augmented images are positive samples to each other.

[0068] The implementation of the present step can refer to the above-mentioned step 401, and the present embodiment will not be described here again.

[0069] Illustratively, as shown in Figure 7 , the computer device obtains the first and second augmented images 702 and 703, respectively, after data augmentation on the original image 701.

[0070] Step 602: Extract features from the first enhanced image through the first feature extraction branch to obtain the first image features. The first feature extraction branch includes a first feature extraction network.

[0071] In one possible implementation, the first enhanced image input to the first feature extraction branch is processed by the first feature extraction network to extract features and obtain the first image features.

[0072] Typically, after feature extraction from an enhanced image, computer devices perform pooling on the extracted image features to reduce feature dimensionality and computational load in subsequent inference processes. Common pooling methods include max pooling and average pooling. However, different pooling methods focus on different aspects; for example, max pooling focuses on the maximum value within the pooling region, while average pooling focuses on the average value. Therefore, to improve the feature representation of the image features, in one possible implementation, a Multiple Global Descriptor (MGD) network is connected after the feature extraction network. This MGD network is used to aggregate and output image features (corresponding to different pooling methods) under different descriptors. This step may include the following steps:

[0073] 1. The first enhanced image is subjected to feature extraction through a first feature extraction network to obtain the features of the first intermediate image.

[0074] The computer device inputs the first enhanced image into the first feature extraction network to obtain the first intermediate image features output by the network.

[0075] Indicative, such as Figure 7 As shown, the computer device inputs the first enhanced image 702 into the first feature extraction branch, where the first feature extraction network 704 performs feature extraction to obtain the first intermediate image features.

[0076] 2. At least two pooling processes are applied to the first intermediate image features through a multi-global descriptor network to obtain at least two first global descriptors.

[0077] In some embodiments, the multi-global descriptor network consists of at least two pooling layers, and different pooling layers correspond to different pooling processing methods. After feature extraction is completed, the computer device performs pooling processing on the first intermediate image features through at least two pooling layers to obtain at least two first global descriptors.

[0078] Optionally, the pooling layer may include at least two of the following: Global Average Pooling (GAP) layer, Global Maximum Pooling (GMP) layer, and General Average Pooling (GeAP) layer. Of course, the computer device may also use other pooling methods to perform pooling processing on intermediate image features; this embodiment does not limit this.

[0079] Indicative, such as Figure 8 As shown, the multi-global descriptor network includes a GAP layer 801, a GMP layer 802, and a GeAP layer 803. The intermediate image features 804 output from the feature extraction network are input into the GAP layer 801, GMP layer 802, and GeAP layer 803, respectively, to obtain three global descriptors 805 after different pooling processes. The dimensions of the intermediate image features are (N, C, H, W), and the dimensions of the global descriptors are (N, C, 1, 1), where N is the number of enhanced images, C is the number of channels, H is the feature map height, and W is the feature map width.

[0080] Third, at least two first global descriptors are cascaded through a multi-global descriptor network, and first image features are generated based on the cascaded first global descriptors.

[0081] In some embodiments, a multilayer perceptron (MLP) is connected after each pooling layer. The computer device further processes the first global descriptor obtained after pooling through the MLP, and performs cascade processing on at least two types of the processed first global descriptors. Finally, the cascaded first global descriptor is input into the MLP to obtain the first image features of the first enhanced image.

[0082] Indicative, such as Figure 8 As shown, the computer device inputs the global descriptor 805 into the MLP 806 and performs cascade processing on the outputs of each MLP 806 to obtain the cascaded global descriptor 807. Finally, the cascaded global descriptor 807 is processed by the MLP 808 to obtain the first image feature 809.

[0083] Step 603: Extract features from the second enhanced image through the second feature extraction branch to obtain the second image features. The second feature extraction branch includes the second feature extraction network.

[0084] Similar to the first feature extraction branch, the second feature extraction branch consists of a second feature extraction network and multiple global descriptor subnetworks. The weights of the second feature extraction network and the first feature extraction network are not shared, while the multiple global descriptor subnetworks in both feature extraction branches are identical. In one possible implementation, this step may include the following steps:

[0085] I. performing feature extraction on the second enhanced image by a second feature extraction network to obtain a second intermediate image feature.

[0086] The computer device inputs the second enhanced image into the second feature extraction network to obtain the second intermediate image feature output by the network.

[0087] II. performing at least two pooling processes on the second intermediate image feature by a multi-global descriptor network to obtain at least two global descriptors.

[0088] After the feature extraction is completed, the computer device performs pooling processes on the second intermediate image feature through at least two pooling layers to obtain at least two second global descriptors.

[0089] III. performing cascade processing on the at least two second global descriptors by the multi-global descriptor network, and generating a second image feature based on the second global descriptors after the cascade processing.

[0090] The computer device further processes the second global descriptors obtained after the pooling processes by the MLP, and performs cascade processing on the at least two second global descriptors after the processing. Finally, the computer device inputs the second global descriptors after the cascade processing into the MLP to obtain the second image feature of the second enhanced image.

[0091] The process of performing feature extraction by the second feature extraction branch can refer to step 602, which will not be described herein.

[0092] As shown in FIG. 7, the computer device performs feature extraction on the first enhanced image 702 by the first feature extraction network 704 and the multi-global descriptor network 705 in the first feature extraction branch to obtain the first image feature, and performs feature extraction on the second enhanced image 703 by the second feature extraction network 706 and the multi-global descriptor network 705 in the second feature extraction branch to obtain the second image feature. Figure 7

[0093] Step 604: determining a model loss of the feature extraction model based on the first image feature, the second image feature, and the negative sample image feature.

[0094] ​Since the feature extraction model aims to reduce the feature distance between similar images and expand the feature distance between dissimilar images, in this embodiment, the model loss of the feature extraction model includes a distance loss, which is determined by a positive sample feature distance and a negative sample feature distance. The positive sample feature distance is the feature distance between the first image feature and the second image feature, and the negative sample feature distance is the feature distance between the first image feature (or the second image feature) and the negative sample image feature. The positive sample feature distance is positively correlated with the distance loss, and the negative sample feature distance is negatively correlated with the distance loss.

[0095] Although model training based on the model loss including the distance loss can reduce the feature distance between similar images and expand the feature distance between dissimilar images, there is a problem of too few positive samples, that is, each original medical image is regarded as an independent class, and model training based on the distance loss expands the distance between all classes. However, only expanding the distance between samples will cause learning difficulty and even the problem of false negative samples. In order to avoid the above problems, in the embodiment of the present application, the model loss of the feature extraction model includes not only the distance loss but also a clustering loss, so as to produce better similarity image cohesion.

[0096] In one way of determining the distance loss, the computer device clusters the first image features corresponding to each original medical image in the current training batch to obtain k first-class cluster centroids, k being an integer greater than or equal to 2; clusters the second image features corresponding to each original medical image in the current training batch to obtain k second-class cluster centroids; and determines the clustering loss based on the distances between the first image features and the k second-class cluster centroids, and the distances between the second image features and the k first-class cluster centroids.

[0097] The computer device can use a clustering algorithm such as K-Means clustering or mean shift clustering to determine the cluster centroids, and the cluster centroids can be represented by the average features of the image features of the same cluster, which is not limited in this embodiment.

[0098] However, since there is a certain antagonistic relationship between the distance loss and the clustering loss, directly clustering the first image features and the second image features may cause learning difficulty in the subsequent training process. In order to avoid the above problems, in another possible implementation, the computer device generates first target features and second target features corresponding to the original medical image based on the first image features, and generates third target features and fourth target features corresponding to the original medical image based on the second image features, so as to determine the distance loss and the clustering loss by using different target features.

[0099] In some embodiments, each feature extraction branch further comprises a first MLP and a second MLP. The computer device inputs the first image feature into the first MLP and the second MLP respectively to obtain a first target feature and a second target feature; and inputs the second image feature into the first MLP and the second MLP respectively to obtain a third target feature and a fourth target feature.

[0100] As shown in FIG. 7, the computer device processes the first image feature through the first MLP 707 to output a first target feature 709, and processes the first image feature through the second MLP 708 to output a second target feature 710. The computer device processes the second image feature through the first MLP 707 to output a third target feature 711, and processes the second image feature through the second MLP 708 to output a fourth target feature 712. Figure 7

[0101] As shown in FIG. 7, the computer device processes the first image feature through the first MLP 707 to output a first target feature 709, and processes the first image feature through the second MLP 708 to output a second target feature 710. The computer device processes the second image feature through the first MLP 707 to output a third target feature 711, and processes the second image feature through the second MLP 708 to output a fourth target feature 712. Figure 9

[0102] Step 604A, determining a distance loss based on the feature distance between the first image feature and the second image feature, and the feature distance between the first image feature and the negative sample image feature.

[0103] In a possible implementation, the computer device determines the distance loss by using the target features output by the same MLP. Optionally, the computer device determines the distance loss based on the feature distance between the first target feature and the third target feature, and the feature distance between the first target feature (or the second target feature) and the negative sample image feature.

[0104] Optionally, the computer device is provided with a negative sample image feature queue, which contains the image features of the l original images input most recently, and the negative sample image feature queue can be represented as Since each original image appears only once in each training round, the image features in the negative sample image feature queue are data augmentations from different input images.

[0105] In some embodiments, the computer device calculates the distance loss by using infoNCE, and the distance loss can be represented as:

[0106]

[0107] wherein, is the number of negative sample image features, represents the first target feature, represents the third target feature, the i-th negative sample image feature,​​ This refers to the hyperparameters used to control the smoothness of the loss function.

[0108] Indicative, such as Figure 7 As shown, the computer device calculates the distance loss 714 based on the first target feature 709, the third target feature 711, and the negative sample image feature 713.

[0109] Step 604B: Cluster the first image features and the second image features corresponding to each original medical image in the current training batch, and determine the clustering loss based on the clustering results.

[0110] To avoid learning difficulties, when performing feature clustering on image features, the computer device also clusters the target features output by another MLP, and determines the clustering loss based on the clustering results. In one possible implementation, determining the clustering loss may include the following steps:

[0111] 1. Cluster the second target features corresponding to each original medical image in the current training batch to obtain k centroids of the third cluster.

[0112] Optionally, the computer device clusters the second target features corresponding to each original medical image in the current training batch to obtain k clusters, and determines the centroid of the third cluster based on the second target features in each cluster.

[0113] Indicative, such as Figure 7 As shown, the computer device clusters the second target features 710 corresponding to N original medical images to obtain k third cluster centroids 715.

[0114] 2. Cluster the fourth target features corresponding to each original medical image in the current training batch to obtain k fourth cluster centroids.

[0115] Optionally, the computer device clusters the fourth target features corresponding to each original medical image in the current training batch to obtain k clusters, and determines the centroid of the fourth cluster based on the fourth target features in each cluster.

[0116] Indicative, such as Figure 7 As shown, the computer device clusters the fourth target features 712 corresponding to N original medical images to obtain k fourth cluster centroids 716.

[0117] 3. Determine the clustering loss based on the distance between the first target feature and the centroids of the k fourth-class clusters, and the distance between the third target feature and the centroids of the k third-class clusters.

[0118] Optionally, the clustering loss is composed of infoNCE between the class cluster center corresponding to the first enhanced image and the target feature corresponding to the second enhanced image, and infoNCE between the class cluster center corresponding to the second enhanced image and the target feature corresponding to the first enhanced image (i.e. symmetric loss). Wherein, the computer device takes the class cluster center of the class cluster to which the target feature belongs as a positive sample, takes the class cluster center of other class clusters as a negative sample, determines the distance between the target feature and the class cluster center, and then obtains the clustering loss.

[0119] Illustratively, the clustering loss can be represented as:

[0120]

[0121] Wherein, is the number of fourth class cluster centers, denotes the first target feature, denotes the fourth class cluster center, denotes that the first target feature belongs to the class cluster corresponding to the jth fourth class cluster center, is a hyperparameter for controlling the smoothness of the loss function.

[0122] Illustratively, as shown in Figure 7 , the computer device calculates the clustering loss 717 based on the first target feature 709, the third target feature 711, the third class cluster center 715, and the fourth class cluster center 716.

[0123] Step 604C, determining the distance loss and the clustering loss as the model loss.

[0124] Illustratively, as shown in Figure 7 , the computer device determines the model loss 718 as the sum of the distance loss 714 and the clustering loss 717.

[0125] It should be noted that after completing the above training process, the computer device updates the negative sample image feature based on the first image feature and the second image feature, so as to ensure that the negative sample image feature queue contains the image features of the last l original images input. Illustratively, as shown in Figure 7 , the computer device updates the negative sample image feature 713 based on the first target feature 709 and the third target feature 711.

[0126] Step 605, training the first feature extraction network based on the model loss through a back propagation algorithm.

[0127] In one possible implementation, during model training, the network parameters of the first feature extraction network participate in gradient backpropagation, while the network parameters of the second feature extraction network do not participate in gradient backpropagation but are updated from the network parameters of the first feature extraction network. Therefore, when training the model based on the model loss, the computer device adjusts the network parameters of the first feature extraction network through the backpropagation algorithm to complete one round of training for the feature extraction network.

[0128] Indicative, such as Figure 7 As shown, the computer device updates the parameters of the first feature extraction network 704 based on the model loss 718.

[0129] Step 606: Update the network parameters of the second feature extraction network based on the network parameters of the first feature extraction network after training.

[0130] After training the first feature extraction network, the computer device further updates the network parameters of the second feature extraction network based on the network parameters of the trained first feature extraction network. In one possible implementation, the computer device can use a moving average method to update the network parameters of the second feature extraction network based on the network parameters of the first feature extraction network, wherein the moving average process can be expressed as:

[0131] )

[0132] in, The network parameters of the network are extracted for the second feature. The network parameters of the network are extracted for the first feature. These are control parameters.

[0133] Indicative, such as Figure 4 As shown, the computer device updates the network parameters of the second feature extraction network 706 by moving average based on the updated network parameters of the first feature extraction network 704.

[0134] In this embodiment, the computer device determines the distance loss based on the feature distance between positive sample image features and the feature distance between positive sample image features and negative sample image features. This enables the feature extraction network to learn the similarity of features between similar images and the differences of features between dissimilar images during training. At the same time, by clustering image features and determining the clustering loss based on the distance between image features and the centroids of each cluster, it helps to improve the cohesion between similar images, thereby improving the feature extraction quality of the trained feature extraction network.

[0135] In addition, in the embodiment, the multiple global descriptor networks are arranged, and the multiple global descriptors are aggregated by using the multiple global descriptor networks, so that the feature expression of the image features is improved, and the subsequent training quality is improved.

[0136] In addition, in the embodiment, the image features are processed by using two MLPs to obtain two target features of the same enhanced image, and then the target features are clustered and the clustering loss is determined, so that the problem of difficult training caused by the antagonism between the clustering loss and the distance loss when the clustering loss is directly determined by using the image features is avoided.

[0137] Under different microscope magnifications, the same medical image often contains different semantic information, which leads to completely different similarities. Therefore, when training the feature extraction model, the computer device needs to train the model based on the medical image under the same magnification, and correspondingly, the feature extraction model obtained by training is used for feature extraction of the image under the target magnification. In a possible implementation manner, based on the above, Figure 10 as shown in Figure 11 , before data augmentation of the original medical image, the method further includes the following steps:

[0138] Step 4001, the WSI under the target magnification is segmented to obtain segmented images.

[0139] Generally, the medical image is usually in the WSI format, which contains images of the same picture under different resolutions. Since the data amount of the WSI is too large, the WSI needs to be segmented under different microscope magnifications (such as 10 times, 20 times, and 40 times) to obtain a plurality of segmented images. The image size of each segmented image is the same, and meets the image output size of the feature extraction model.

[0140] Optionally, when the feature extraction model corresponding to the target magnification needs to be trained, the computer device segments the WSI under the target magnification to obtain segmented images.

[0141] Step 4002, the segmented images are screened based on the image information amount to obtain the original medical image.

[0142] For the segmented images located at the edge, since the image information amount contained in the images is small, which is not conducive to subsequent model training, after segmentation, the computer device also needs to screen the segmented images according to the image information amount, filter the segmented images with low image data amount, and finally obtain the original medical image.

[0143] Illustratively, as shown in Figure 12As shown, the first split image 1101 is an invalid sample that needs to be filtered, and the second split image 1102 is a valid sample that needs to be retained.

[0144] In addition, in actual application, if the magnification of the medical image cannot be known, the image features extracted by the computer device using the trained feature extraction network may not be accurate. In order to improve the feature extraction quality in this case, in a possible implementation, the computer device trains a magnification prediction model based on original medical images of different magnifications, and the magnification prediction model is used to predict the magnification of an input image.

[0145] Optionally, the computer device trains the magnification prediction model with the magnification corresponding to the original medical image as supervision, and the trained magnification prediction model is used to output the probabilities of various magnifications. For example, when the magnifications of the medical image include 10 times, 20 times and 40 times, if the magnification prediction model outputs 0.01, 0.95 and 0.04, it means that the probability of the input medical image being magnified by 10 times is 0.01, the probability of the input medical image being magnified by 20 times is 0.95, and the probability of the input medical image being magnified by 40 times is 0.04.

[0146] In application, the prediction result of the magnification prediction model is used for feature fusion of the image features extracted by different feature extraction models. In a possible implementation, after the computer device obtains the probabilities of the magnifications of the medical image through the magnification prediction model, the computer device extracts features of the medical image through the feature extraction models corresponding to different magnifications, and then performs feature fusion (such as feature weighted summation) on the image features extracted by different feature extraction models based on the predicted probabilities, so as to perform subsequent processing based on the fused image features.

[0147] As shown in the schematic diagram, Figure 13 As shown, the computer device extracts features of the medical image 1201 through the first feature extraction model 1202, the second feature extraction model 1203 and the third feature extraction model 1204 (corresponding to different magnifications) respectively, and predicts the magnification of the medical image 1201 through the magnification prediction model 1205, so as to perform weighted summation on the image features output by the three feature extraction models based on the magnification prediction result, and obtain the target image feature 1206.

[0148] Please refer to Figure 14FIG. 13 shows a diagram of an example computer device that can be used to implement a system according to the present application. In particular, the computer device 1300 includes a central processing unit (CPU) 1301, a system memory 1304, including a random access memory 1302 and a read-only memory 1303, and a system bus 1305 that couples the system memory 1304 to the central processing unit 1301. The computer device 1300 also includes an input / output (I / O) system 1306 that helps transfer information between various devices within the computer, and a mass storage device 1307 for storing an operating system 1313, application programs 1314, and other program modules 1315.

[0149] The I / O system 1306 includes a display 1308 for displaying information and an input device 1309, such as a mouse, keyboard, or the like, for inputting information into the computer. Both the display 1308 and the input device 1309 are connected to the central processing unit 1301 through an input / output controller 1310 that is connected to the system bus 1305. The I / O system 1306 can also include the input / output controller 1310 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1310 provides output to a display screen, a printer, or other type of output device.

[0150] The mass storage device 1307 is connected to the central processing unit 1301 through a mass storage controller (not shown) that is connected to the system bus 1305. The mass storage device 1307 and its associated computer-readable media provide non-volatile storage for the computer device 1300. That is, the mass storage device 1307 can include a computer-readable medium (not shown), such as a hard disk or drive.

[0151] Without loss of generality, the computer readable medium can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes Random Access Memory (RAM), Read Only Memory (ROM), flash memory or other solid state memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. It should be understood by those skilled in the art that the computer storage media is not limited to the above-mentioned several. The system memory 1304 and the mass storage device 1307 mentioned above can be collectively referred to as memory.

[0152] The memory stores one or more programs configured to be executed by the one or more central processing units 1301, and the one or more programs contain instructions for implementing the above method. The central processing unit 1301 executes the one or more programs to implement the method provided by each method embodiment.

[0153] According to various embodiments of the present application, the computer device 1300 can also be connected to a remote computer operating on a network such as the Internet. That is, the computer device 1300 can be connected to a network 1312 through a network interface unit 1311 connected to the system bus 1305, or can be connected to other types of networks or remote computer systems (not shown) using the network interface unit 1311.

[0154] The memory also includes one or more programs stored in the memory, and the one or more programs contain steps performed by the computer device in the method provided by the embodiments of the present application.

[0155] ​ is a structural block diagram of an image feature supervised learning device provided by an exemplary embodiment of the present application, which includes:

[0156] The data enhancement module 1401 is configured to perform data enhancement on the original medical image to obtain a first enhanced image and a second enhanced image, and the first enhanced image and the second enhanced image are positive samples of each other.

[0157] The feature extraction module 1402 is configured to perform feature extraction on the first enhanced image and the second enhanced image by using a feature extraction model to obtain first image features of the first enhanced image and second image features of the second enhanced image.

[0158] The loss determination module 1403 is configured to determine a model loss of the feature extraction model based on the first image features, the second image features, and negative sample image features, the negative sample image features being image features of enhanced images corresponding to other original medical images.

[0159] The first training module 1404 is configured to train the feature extraction model based on the model loss.

[0160] Optionally, the feature extraction model includes a first feature extraction branch and a second feature extraction branch, and the first feature extraction branch and the second feature extraction branch use feature extraction networks with different parameters.

[0161] The feature extraction module 1402 includes:

[0162] The first extraction unit is configured to perform feature extraction on the first enhanced image by using the first feature extraction branch to obtain the first image features.

[0163] The second extraction unit is configured to perform feature extraction on the second enhanced image by using the second feature extraction branch to obtain the second image features.

[0164] Optionally, the first feature extraction branch includes a first feature extraction network and a multi-global descriptor network, the second feature extraction branch includes a second feature extraction network and the multi-global descriptor network, and the multi-global descriptor network is configured to aggregate image features under different descriptors and output.

[0165] The first extraction unit is specifically configured to:

[0166] perform feature extraction on the first enhanced image by using the first feature extraction network to obtain first intermediate image features;

[0167] perform at least two kinds of pooling processing on the first intermediate image features by using the multi-global descriptor network to obtain at least two kinds of first global descriptors;

[0168] perform cascade processing on the at least two kinds of first global descriptors by using the multi-global descriptor network, and generate the first image features based on the first global descriptors after the cascade processing;

[0169] The second extraction unit is specifically configured to:

[0170] extracting features of the second enhanced image through the second feature extraction network to obtain a second intermediate image feature;

[0171] performing at least two pooling processes on the second intermediate image feature through the multi-global descriptor network to obtain at least two global descriptors;

[0172] performing cascade processing on the at least two second global descriptors through the multi-global descriptor network, and generating the second image feature based on the second global descriptors after the cascade processing.

[0173] Optionally, the first training module 1404 comprises:

[0174] a first training unit configured to train the first feature extraction network through a back propagation algorithm based on the model loss;

[0175] a second training unit configured to update network parameters of the second feature extraction network based on network parameters of the first feature extraction network after training.

[0176] Optionally, the loss determination module 1403 comprises:

[0177] a first loss determination unit configured to determine a distance loss based on a feature distance between the first image feature and the second image feature and a feature distance between the first image feature and the negative sample image feature;

[0178] a second loss determination unit configured to respectively cluster the first image features and the second image features corresponding to each of the original medical images in a current training batch, and determine a clustering loss based on clustering results;

[0179] a total loss determination unit configured to determine the distance loss and the clustering loss as the model loss.

[0180] Optionally, the second loss determination unit is configured to:

[0181] cluster the first image features corresponding to each of the original medical images in the current training batch to obtain k first-class cluster centers, k being an integer greater than or equal to 2;

[0182] cluster the second image features corresponding to each of the original medical images in the current training batch to obtain k second-class cluster centers;

[0183] determine the clustering loss based on distances between the first image features and the k second-class cluster centers and distances between the second image features and the k first-class cluster centers.

[0184] Optionally, the apparatus further comprises:

[0185] a first generation module configured to generate a first target feature and a second target feature corresponding to the original medical image based on the first image feature;

[0186] a second generation module configured to generate a third target feature and a fourth target feature corresponding to the original medical image based on the second image feature;

[0187] The first loss determination unit is specifically configured to:

[0188] determine the distance loss based on a feature distance between the first target feature and the third target feature, and a feature distance between the first target feature and the negative sample image feature;

[0189] The second loss determination unit is specifically configured to:

[0190] cluster the second target features corresponding to the original medical images in the current training batch to obtain k third-class cluster centers;

[0191] cluster the fourth target features corresponding to the original medical images in the current training batch to obtain k fourth-class cluster centers;

[0192] determine the clustering loss based on distances between the first target feature and the k fourth-class cluster centers, and distances between the third target feature and the k third-class cluster centers.

[0193] Optionally, the first generation module is configured to:

[0194] input the first image feature into a first multi-layer perception (MLP) and a second MLP respectively to obtain the first target feature and the second target feature;

[0195] Optionally, the second generation module is configured to:

[0196] input the second image feature into the first MLP and the second MLP respectively to obtain the third target feature and the fourth target feature.

[0197] Optionally, the data enhancement module 1401 comprises:

[0198] a first enhancement unit configured to perform color enhancement and direction enhancement on the original medical image based on first enhancement parameters to obtain the first enhanced image;

[0199] a second enhancement unit, configured to perform color enhancement and direction enhancement on the original medical image by using a second enhancement parameter to obtain a second enhanced image, wherein the first enhancement parameter and the second enhancement parameter are different.

[0200] Optionally, the apparatus and the method further comprise:

[0201] an updating module, configured to update the negative sample image feature based on the first image feature and the second image feature.

[0202] Optionally, the feature extraction model is used to extract features of an image under a target magnification.

[0203] The apparatus comprises:

[0204] a segmentation module, configured to segment a whole slide digital image (WSI) under the target magnification to obtain a segmented image.

[0205] a screening module, configured to screen the segmented image based on image information quantity to obtain the original medical image.

[0206] Optionally, the apparatus further comprises:

[0207] a second training module, configured to train a magnification prediction model based on original medical images under different magnifications, wherein the magnification prediction model is used to predict a magnification of an input image, and a prediction result of the magnification prediction model is used to perform feature fusion on image features extracted by different feature extraction models.

[0208] To sum up, in the embodiments of the present application, the first enhanced image and the second enhanced image, which are positive samples of each other, are obtained by performing data enhancement on the original medical image, and the first image feature and the second image feature are obtained by performing feature extraction on the original medical image by using the feature extraction model. Then, other original medical images different from the original medical image are taken as negative samples, and a model loss of the feature extraction model is determined based on the first image feature, the second image feature, and the negative sample image feature. Finally, the feature extraction model is trained by using the model loss. In the whole process, the feature extraction model learns the image features of the medical image by using the self-supervised learning method, and manual medical image labeling is not required, thereby reducing the manual labeling cost in the model training process and improving the training efficiency of the feature extraction model.

[0209] In the embodiment, the computer device determines the distance loss based on the feature distance between the positive sample image features and the feature distance between the positive sample image features and the negative sample image features, so that the feature extraction network can learn the similarity of features between similar images and the difference of features between dissimilar images in the training process. Meanwhile, by clustering the image features and determining the clustering loss based on the distance between the image features and the cluster centers after clustering, the cohesion between similar images is improved, and thus the feature extraction quality of the trained feature extraction network is improved.

[0210] In addition, in the embodiment, multiple global descriptor networks are set to aggregate and represent multiple global descriptors using the multiple global descriptor networks, thereby improving the feature expression of image features and helping to improve the subsequent training quality.

[0211] Furthermore, in the embodiment, two MLPs are used to process the image features to obtain two target features for the same augmented image, and then the target features are clustered and the clustering loss is determined, thereby avoiding the problem of training difficulty caused by the antagonism between the clustering loss and the distance loss when the clustering loss is directly determined using the image features.

[0212] It should be noted that the device provided in the above embodiments is only exemplified by the division of the above functional modules. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be described here.

[0213] The embodiment of the present application further provides a computer readable storage medium, which stores at least one instruction. The at least one instruction is loaded and executed by a processor to implement the supervised learning method of image features according to any one of the above embodiments.

[0214] Optionally, the computer readable storage medium can include ROM, RAM, solid state disk (SSD, Solid State Drives) or optical disk, etc. Among them, the RAM can include resistance random access memory (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic Random Access Memory).

[0215] The embodiment of the present application provides a computer program product or computer program, the computer program product or computer program comprises computer instructions, the computer instructions are stored in a computer readable storage medium. The processor of the computer equipment reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer equipment executes the supervised learning method of the image feature described in the above embodiment.

[0216] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program to instruct relevant hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0217] The above is only an optional embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of supervised learning of image features, characterized in that, The method comprises: performing data enhancement on the original medical images to obtain first enhanced images and second enhanced images, the first enhanced images and the second enhanced images being positive samples of each other; performing feature extraction on the first enhanced images and the second enhanced images by a feature extraction model to obtain first image features of the first enhanced images and second image features of the second enhanced images; determining a distance loss based on a feature distance between the first image features and the second image features and a feature distance between the first image features and negative sample image features, the negative sample image features being image features of enhanced images corresponding to other original medical images; performing clustering on the first image features corresponding to each of the original medical images in a current training batch to obtain k first-class cluster centroids, k being an integer greater than or equal to 2; performing clustering on the second image features corresponding to each of the original medical images in the current training batch to obtain k second-class cluster centroids; determining a clustering loss based on distances between the first image features and the k second-class cluster centroids and distances between the second image features and the k first-class cluster centroids; determining the distance loss and the clustering loss as a model loss of the feature extraction model; training the feature extraction model based on the model loss.

2. The method of claim 1, wherein, The feature extraction model comprises a first feature extraction branch and a second feature extraction branch, and the first feature extraction branch and the second feature extraction branch adopt feature extraction networks with different parameters; The feature extraction on the first enhanced images and the second enhanced images by the feature extraction model to obtain the first image features of the first enhanced images and the second image features of the second enhanced images comprises: performing feature extraction on the first enhanced images by the first feature extraction branch to obtain the first image features; performing feature extraction on the second enhanced images by the second feature extraction branch to obtain the second image features.

3. The method of claim 2, wherein, The first feature extraction branch comprises a first feature extraction network and a multi-global descriptor network, and the second feature extraction branch comprises a second feature extraction network and the multi-global descriptor network, and the multi-global descriptor network is used for aggregating image features under different descriptors and outputting; The feature extraction on the first enhanced images by the first feature extraction branch to obtain the first image features comprises: performing feature extraction on the first enhanced images by the first feature extraction network to obtain first intermediate image features; performing at least two kinds of pooling processing on the first intermediate image features by the multi-global descriptor network to obtain at least two kinds of first global descriptors; performing cascade processing on the at least two kinds of first global descriptors by the multi-global descriptor network, and generating the first image features based on the first global descriptors after the cascade processing; The feature extraction on the second enhanced images by the second feature extraction branch to obtain the second image features comprises: extracting features of the second enhanced image through the second feature extraction network to obtain a second intermediate image feature; performing at least two kinds of pooling processing on the second intermediate image feature through the multi-global descriptor network to obtain at least two kinds of global descriptors; performing cascade processing on the at least two kinds of second global descriptors through the multi-global descriptor network, and generating the second image feature based on the second global descriptors after the cascade processing.

4. The method of claim 3, wherein, The training of the feature extraction model based on the model loss comprises: training the first feature extraction network based on the model loss through a back propagation algorithm; updating network parameters of the second feature extraction network based on the network parameters of the first feature extraction network after the training.

5. The method of claim 1, wherein, The method further comprises: generating first target features and second target features corresponding to the original medical image based on the first image feature; generating third target features and fourth target features corresponding to the original medical image based on the second image feature; The distance loss is determined based on a feature distance between the first image feature and the second image feature, and a feature distance between the first image feature and the negative sample image feature, comprising: The distance loss is determined based on a feature distance between the first target features and the third target features, and a feature distance between the first target features and the negative sample image feature. The method further comprises: performing clustering on the second target features corresponding to each of the original medical images in the current training batch to obtain k third-class cluster centers; performing clustering on the fourth target features corresponding to each of the original medical images in the current training batch to obtain k fourth-class cluster centers; The clustering loss is determined based on distances between the first target features and the k fourth-class cluster centers, and distances between the third target features and the k third-class cluster centers.

6. The method of claim 5, wherein, The generation of the first target features and the second target features corresponding to the original medical image based on the first image feature comprises: inputting the first image feature into a first multi-layer perception (MLP) and a second MLP respectively to obtain the first target features and the second target features; The generation of the third target features and the fourth target features corresponding to the original medical image based on the second image feature comprises: inputting the second image feature into a first MLP and a second MLP respectively to obtain the third target features and the fourth target features.

7. The method according to any one of claims 1 to 4, characterized in that, The data augmentation of the original medical image to obtain the first enhanced image and the second enhanced image comprises: performing color enhancement and direction enhancement on the original medical image based on a first enhancement parameter to obtain the first enhanced image; performing color enhancement and direction enhancement on the original medical image through a second enhancement parameter to obtain the second enhanced image, the first enhancement parameter and the second enhancement parameter being different.

8. The method according to any one of claims 1 to 4, characterized in that, After the training of the feature extraction model based on the model loss, the method further comprises: updating the negative sample image feature based on the first image feature and the second image feature.

9. The method according to any one of claims 1 to 4, characterized in that, The feature extraction model is used for feature extraction on an image under a target magnification; Before the data augmentation on the original medical image to obtain the first augmented image and the second augmented image, the method comprises: segmenting a full field of view digital slice WSI under the target magnification to obtain a segmented image; screening the segmented image based on image information quantity to obtain the original medical image.

10. The method of claim 9, wherein, The method further comprises: training a magnification prediction model based on original medical images under different magnifications, the magnification prediction model being used for predicting a magnification of an input image, and a prediction result of the magnification prediction model being used for feature fusion on image features extracted by different feature extraction models.

11. An apparatus for supervised learning of image features, characterized by The device comprises: a data augmentation module configured to perform data augmentation on an original medical image to obtain a first augmented image and a second augmented image, the first augmented image and the second augmented image being positive samples of each other; a feature extraction module configured to perform feature extraction on the first augmented image and the second augmented image by a feature extraction model to obtain first image features of the first augmented image and second image features of the second augmented image; a loss determination module configured to determine a distance loss based on a feature distance between the first image features and the second image features and a feature distance between the first image features and negative sample image features, the negative sample image features being image features of augmented images corresponding to other original medical images; performing clustering on the first image features corresponding to each of the original medical images in a current training batch to obtain k first-class cluster centroids, k being an integer greater than or equal to 2; performing clustering on the second image features corresponding to each of the original medical images in the current training batch to obtain k second-class cluster centroids; determining a clustering loss based on distances between the first image features and the k second-class cluster centroids and distances between the second image features and the k first-class cluster centroids; determining the distance loss and the clustering loss as a model loss of the feature extraction model; a first training module configured to train the feature extraction model based on the model loss.

12. A computer device, comprising: The computer device comprises a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the supervised learning method of image features according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, the at least one instruction being loaded and executed by the processor to implement the supervised learning method of image features according to any one of claims 1 to 10.

Images