Image processing method based on classification uncertainty

Abstract

The application discloses an image processing method based on classification uncertainty. The method comprises the following steps: obtaining a to-be-labeled image, performing cutting processing on the to-be-labeled image to obtain a plurality of to-be-processed images; inputting the to-be-processed image into an image classification model to obtain an attention attribute of the to-be-processed image; determining an uncertainty attribute of the to-be-processed image for a preset classification result based on the attention attribute of the to-be-processed image, a first feature vector of the to-be-labeled image and a second feature vector of a reference sample image; and performing region labeling on the to-be-labeled image based on the uncertainty attributes of all the to-be-processed images. The method solves the problem that region labeling is wrong in the prior art through the attention mechanism for image labeling, and achieves the effect of improving the image classification accuracy and the accuracy of labeling the region of interest in the image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and medium based on classification uncertainty. Background Technology

[0002] With the continuous development of computer vision, image classification models based on attention mechanisms are commonly used to handle image classification tasks. For example, an image is typically divided into multiple image patches, and the feature sequences of each image patch are used as input to an image classification model with a self-attention mechanism. Then, regions with high attention between image patches are used to extract and label regions of interest in the image.

[0003] However, because different features in an image have similarities, the attention region calculated in this way differs from the manually labeled region of interest, leading to errors in region of interest determination. Summary of the Invention

[0004] This invention provides an image processing method, apparatus, device, and medium based on classification uncertainty, so as to improve the accuracy of image classification and the accuracy of region of interest labeling in the image.

[0005] According to one aspect of the present invention, an image processing method based on classification uncertainty is provided, the method comprising:

[0006] Obtain the image to be labeled, and perform segmentation processing on the image to be labeled to obtain multiple images to be processed;

[0007] The image to be processed is input into an image classification model to obtain the attention attributes of the image to be processed; wherein, the image classification model is trained based on a pre-trained initial classification model;

[0008] For each image to be processed, based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image, the uncertainty attribute of the image to be processed as a preset classification result is determined;

[0009] Regions of interest (ROIs) are labeled in the images to be processed based on all the uncertainty attributes of the images to be processed.

[0010] According to another aspect of the present invention, an image processing apparatus based on classification uncertainty is provided, the apparatus comprising:

[0011] The cutting module is used to acquire the image to be labeled and to cut the image to be labeled to obtain multiple images to be processed.

[0012] An attention attribute determination module is used to input the image to be processed into an image classification model to obtain the attention attributes of the image to be processed; wherein, the image classification model is obtained based on a pre-trained initial classification model;

[0013] The uncertainty attribute determination module is used to determine the uncertainty attribute of the image to be processed as a preset classification result for each image to be processed based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image.

[0014] The annotation module is used to annotate the regions of interest in the image to be processed based on all the uncertainty attributes of the image.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the image processing method based on classification uncertainty as described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the image processing method based on classification uncertainty as described in any embodiment of the present invention.

[0020] The technical solution of this invention involves acquiring an image to be labeled, segmenting the image to be labeled to obtain multiple images to be processed; inputting the images to be processed into an image classification model to obtain the attention attributes of the images to be processed; determining the uncertainty attribute of the images to be processed as a preset classification result based on the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image; and labeling the regions of interest (ROIs) of the images to be labeled based on the uncertainty attributes of all the images to be processed. This solves the problem of region labeling errors caused by image labeling through attention mechanisms in the prior art. It achieves the goal of inputting multiple images to be processed from the segmented image to be labeled into an image classification model, processing them through the model to obtain the attention attributes of each image to be processed, and then combining the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image to determine the uncertainty of the classification result determined by mining image feature information based on the attention mechanism, i.e., the uncertainty attribute. This improves the accuracy of the classification result determination and also allows for the labeling of ROIs of the images to be labeled based on the magnitude of the uncertainty attribute, thereby improving the accuracy of ROI labeling in the image.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an image processing method based on classification uncertainty provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of an image processing method based on classification uncertainty provided in Embodiment 2 of the present invention;

[0025] Figure 3 This is a flowchart of an image processing method based on classification uncertainty provided in Embodiment 3 of the present invention;

[0026] Figure 4 This is a schematic diagram of an image processing method provided in Embodiment 4 of the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of an image processing device based on classification uncertainty according to Embodiment 5 of the present invention;

[0028] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the image processing method based on classification uncertainty according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Example 1

[0032] Figure 1 This is a flowchart of an image processing method based on classification uncertainty according to Embodiment 1 of the present invention. This embodiment is applicable to the case of annotating regions of interest in an image. The method can be executed by an image processing device based on classification uncertainty, which can be implemented in hardware and / or software and can be configured in a computing device. Figure 1 As shown, the method includes:

[0033] S110. Obtain the image to be labeled, and perform segmentation processing on the image to be labeled to obtain multiple images to be processed.

[0034] The image to be labeled can be an image that requires annotation of information such as image category, location of interest, or region. For example, medical images, such as whole slide images (WSI) of histopathology.

[0035] In this embodiment, uploaded images can be received via an interface or network and used as images to be labeled. To improve the accuracy of labeling, the image to be labeled can be divided into several smaller images, each of which can be used as an image to be processed. For example, the image to be labeled can be divided into smaller images of a preset size, and a preset number of smaller images can be extracted as images to be processed based on conditions.

[0036] For example, the entire WSI image (i.e. the image to be labeled) is cut into several 256×256 patch images (i.e. the images to be processed).

[0037] S120. Input the image to be processed into the image classification model to obtain the attention attributes of the image to be processed.

[0038] The image classification model can be trained on a pre-trained initial classification model to determine the image annotation results. The initial classification model can be trained on a dataset from ImageNet (a large visualization database used for research on visual object recognition software). The attention attribute can be used to characterize the degree of attention paid to useful information (i.e., information of interest) in the image, such as the degree of attention paid to lesion information in the image.

[0039] In this embodiment, the image to be processed can be used as input to an image classification model, and an attention mechanism can be used to effectively select useful information in the image to obtain the attention attribute of the image.

[0040] It should be noted that, considering the structural similarity between image features of different classification results (such as true positives and false positives), graph convolutional neural networks can be used to avoid misclassification. This involves leveraging graph convolutional networks to fully mine the latent geometric information and similarity relationships between images, better learn image features, and extract high-level geometric features. Furthermore, attention mechanisms can be used to effectively select information based on image features, improving the effectiveness and accuracy of information mining, thereby enhancing the precision of image classification and annotation.

[0041] Optionally, the image to be processed is input into an image classification model to obtain the attention attributes of the image to be processed, including: performing feature extraction processing on the image to be processed based on the feature extraction model in the image classification model to obtain the third feature vector of the image to be processed; and determining the attention attributes of the image to be processed based on the third feature vector.

[0042] The feature extraction model can be a convolutional neural network. For example, a feature extractor can be constructed based on the ResNet 50 convolutional neural network, and the network can be pre-trained using image data from ImageNet to obtain a trained feature extraction model.

[0043] In this embodiment, a feature extraction model can be used to extract feature information from each image to be processed, obtaining an image feature vector for each image to be processed, which serves as the third feature vector. Furthermore, an attention mechanism can be used to mine relevant information in the image to obtain the attention attributes of the image to be processed.

[0044] For example, a model pre-trained on ImageNet is used to extract the feature vectors of each segmented image to be processed from the image to be labeled, i.e., H = {h1, ..., h...} K}, where K is the number of images to be processed, and h1 is the feature vector extracted from image 1 to be processed, and the size of the vector can be 1×1024.

[0045] In this embodiment, determining the attention attribute of the image to be processed based on the third feature vector includes: processing the third feature vector based on the attention determination model in the image classification model to obtain the attention attribute of the image to be processed.

[0046] Specifically, the third feature vector of the image to be processed can be input into the attention determination model, and the target calculation function of the attention determination model can be used to process the third feature vector to obtain the attention attribute of the image to be processed.

[0047] The objective function can be: a k Let π = {m, V1, V2} represent the attention attributes of the k-th image to be processed, where π = {m, V1, V2} are the neural network parameters, and h k Let be the third feature vector of the k-th image to be processed, and exp be an exponential function with the natural constant e as the base.

[0048] It should be noted that, in the process of determining the annotation result of the image to be labeled, the classification result of the image to be labeled, such as whether it is healthy or unhealthy, can also be determined by comprehensively considering the attention attributes of each image to be processed after the image to be labeled is divided.

[0049] In this embodiment, after inputting the image to be processed into the image classification model and obtaining the attention attribute of the image to be processed, the method further includes: determining the classification attribute of the image to be labeled based on the category determination model in the image classification model according to the third feature vector and attention attribute of all the images to be processed; and determining the classification result of the image to be labeled based on the classification attribute.

[0050] Among them, categorical attributes can be used to characterize the probability of a categorization result, and can be expressed in the form of decimals, fractions, percentages, etc.

[0051] In practical applications, the third feature vectors and attention attributes of all images to be processed can be input into the class determination model. The model can then aggregate the information from all images to obtain aggregated information that effectively represents the feature information of the unlabeled images. This aggregated information can be used as a classification attribute. For example, it can use... To determine the classification attribute, the third feature vector and attention attribute of the same image to be processed are multiplied to obtain the product value for each image. All product values ​​are then summed, and the sum is used as the classification attribute. This classification attribute can correspond to a preset classification result, such as "unhealthy". If the classification attribute is greater than a preset threshold, the image to be labeled is considered to have the preset classification result. Alternatively, in binary classification, a classification attribute distinct from the preset classification result can be determined based on its classification attribute; for example, "healthy" corresponds to 80%, and "unhealthy" corresponds to 20%. The classification result corresponding to the highest classification attribute can be used as the classification result for the image to be labeled.

[0052] To further improve classification accuracy and ensure the stability of classification attribute determination, in the process of determining the classification attribute corresponding to the image to be labeled based on the third feature vector and attention attribute of all images to be processed, a first intermediate value can be determined based on the attention attribute of all images to be processed; a second intermediate value corresponding to the image to be processed can be determined based on the third feature vector and attention attribute of the same image to be processed; and the classification attribute can be determined based on the first intermediate value and the second intermediate values ​​corresponding to all images to be processed.

[0053] In this embodiment, it can be based on Determine the classification attribute. Specifically, sum the attention attributes of all images to be processed to obtain a sum value. Divide this sum value by a preset value (e.g., 1), and use the quotient as the first intermediate value. Multiply the third feature vector and attention attribute of the same image to be processed, and use the product value as the second intermediate value corresponding to that image. Sum all the second intermediate values, and then multiply the sum value by the first intermediate value. Use the product value as the classification attribute of the image to be labeled.

[0054] S130. For each image to be processed, based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image, determine the uncertainty attribute of the image to be processed as a preset classification result.

[0055] The reference sample image can be an image of the same type as the image to be labeled, such as a pathological image. The preset classification result can be the first classification result, such as medical examination results like "not healthy," "positive," or "abnormal." The uncertainty attribute can characterize the uncertainty of the classification result determined by mining image feature information based on the attention mechanism, that is, the uncertainty of the classification result determined based on the attention attribute. It should be noted that the method for determining the uncertainty attribute corresponding to each image to be processed is the same, and the determination of the uncertainty attribute of any one of the images to be processed can be used as an example for introduction.

[0056] In this embodiment, feature information of the reference sample image can be extracted based on a feature extraction model to obtain the feature vector of the reference sample image as the second feature vector. Alternatively, the reference sample image can be segmented to obtain multiple sub-sample images, and feature information of each sub-sample image can be extracted based on the feature extraction model to obtain the feature vector of each sub-sample image. All feature vectors are then summarized to obtain the second feature vector of the reference sample image. Correspondingly, the third feature vectors of all images to be processed can also be summarized to obtain the first feature vector of the image to be labeled. Furthermore, the attention attribute of the image to be processed can be corrected by the relationship between the first feature vector and the second feature vector, and the corrected attention attribute is used as an uncertainty attribute. For example, if the classification result of the image to which the second feature vector belongs is the second classification result, such as health, and the similarity between the first feature vector and the second feature vector is greater than a preset value, then the attention attribute of the image to be processed can be reduced. Alternatively, the third feature vector and the second feature vector of the image to be processed can be compared to correct the attention attribute of the image to be processed. Alternatively, the attention attribute of the image to be processed can be obtained from the attention attribute itself, for example, based on the attention function f. π The parameters are used to construct a prior distribution p(π), and then the model is used to learn the posterior distribution p(π|D), or the empirical posterior distribution p(π|D) is directly learned using model ensemble. This posterior distribution can then be used to calculate the uncertainty attribute based on the attention attribute using the following formula (1):

[0057]

[0058] Where a is an uncertain attribute, D = {H} i} S Let H represent the set of images to be labeled, where S is the number of images to be labeled, and H = {h1, ..., h2}. K Let H be the set of feature vectors of all images to be processed under the image to be labeled. * π is the second feature vector of the reference sample image. s~p(π|D). The feature vector set of the image to be labeled, the second feature vector of the reference sample image, and the attention attribute of the image to be processed can be input into formula (1) to obtain the uncertainty attribute p(a|H) of the image to be processed. * (D).

[0059] It should be noted that, in order to extract the uncertainty of the image at the instance level, the attention probability distribution of each image to be processed can be calculated, and further, the uncertainty attribute of the image to be processed can be determined based on the attention probability distribution.

[0060] In this embodiment, the uncertainty attribute of the image to be processed as a preset classification result is determined based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image. This includes: determining the first feature vector of the image to be labeled based on the third feature vectors of all the images to be processed; and determining the uncertainty attribute of the image to be processed as a preset classification result based on the uncertainty determination model in the image classification model, according to the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image.

[0061] In practical applications, the third feature vectors of all images to be processed can be aggregated, and this aggregated feature vector set can be used as the first feature vector of the image to be labeled. Furthermore, the first feature vector, the attention attribute of the image to be processed, and the second feature vector of the reference sample image can be input into the uncertainty determination model. Then, based on the uncertainty determination function in the uncertainty determination model, the uncertainty attribute of the image to be processed as a preset classification result can be calculated. The uncertainty determination function can be: in, Let p(π|D) represent the expectation of p(π|D), where p(π|D) represents the probability of π occurring given that D has occurred. k ,π|H * ) indicates in H * Under what conditions does a occur k The probability of π occurring. I[a k ,π|H * [,D] is used to characterize the uncertainty properties under the model. That is, a global uncertainty function. This refers to the data uncertainty function. Optionally, the uncertainty determination model includes a global uncertainty function and a data uncertainty function. Accordingly, based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image, the uncertainty attribute of the image to be processed as a preset classification result is determined, including: determining the global uncertainty attribute based on the global uncertainty function, the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image; determining the data uncertainty attribute based on the data uncertainty function, the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image; and determining the uncertainty attribute based on the global uncertainty attribute and the data uncertainty attribute.

[0062] Specifically, the global uncertainty attribute of the image to be processed can be determined based on the global uncertainty function; the data uncertainty attribute of the image to be processed can be determined based on the data uncertainty function. Furthermore, the difference between the global uncertainty attribute and the data uncertainty attribute can be processed, and the difference can be used as the uncertainty attribute I[a]. k ,π|H * ,D].

[0063] S140. Based on the uncertainty attributes of all the images to be processed, label the regions of interest in the image to be labeled.

[0064] In practical applications, after determining the uncertainty attribute of each image to be processed, the region of interest in the image to be labeled can be visualized based on the magnitude of the uncertainty attribute value. For example, the larger the value, the more attention the image region receives, and the higher the probability that the image is a pre-defined classification result (e.g., positive). The region of the image to be processed within the labeled image can be highlighted. Conversely, the smaller the value, the more the image region is ignored, and the lower the probability that the image is a pre-defined classification result. The region of the image to be processed within the labeled image can be highlighted less. For example, the larger the uncertainty attribute value, the darker the red; the smaller the value, the darker the blue.

[0065] In this embodiment, the classification result of the image to be labeled can also be determined based on the uncertainty attributes of each image to be processed. For example, if the number of uncertainty attributes exceeding a set value is greater than a preset number, the classification result of the image to be labeled is considered to be the preset classification result. Alternatively, weights can be set for the uncertainty attributes of each image to be processed, and the uncertainty attributes and their corresponding weights can be weighted and then averaged. The classification result of the image to be labeled can be determined based on the mean. For example, if the mean is greater than a preset value, the classification result of the image to be labeled is considered to be the preset classification result.

[0066] Alternatively, the classification attribute of the image to be labeled can be determined based on the uncertainty attribute and the third feature vector of each image to be processed. This can be achieved by: multiplying the uncertainty attribute and the third feature vector of the same image to obtain the third intermediate value corresponding to that image; summing all the third intermediate values, and using the sum as the fourth intermediate value; summing all the uncertainty attributes of the images to be processed, and using the sum as the fifth intermediate value; dividing the fifth intermediate value by a preset threshold (e.g., 1), and using the quotient as the sixth intermediate value; and multiplying the sixth intermediate value and the fourth intermediate value, and using the product as the classification attribute of the image to be labeled, thus determining the classification result based on the classification attribute. For example, it can be based on... Determine the classification attributes of the image to be labeled, I k This represents the uncertainty attribute of the k-th image to be processed. Another way to determine the classification attribute of the image to be labeled based on the uncertainty attribute and the third feature vector of each image to be processed can be: obtaining the probability values ​​of different classification results for the same image to be processed based on its uncertainty attribute and the third feature vector; determining the classification attribute (maximum probability value) of the image to be processed as the target classification result based on the probability values ​​of different classification results; and determining the classification attribute of the image to be labeled based on the classification attributes of all images to be processed under the image to be labeled. For example, the classification attributes of all images to be processed are weighted and averaged, and the resulting average is used as the classification attribute of the image to be labeled. Further, the classification result is determined based on the classification attribute of the image to be labeled. For example, if the classification attribute is greater than a preset value, the classification result is the first classification result; if the classification attribute is not greater than the preset value, the classification result is the second classification result.

[0067] The technical solution of this embodiment involves acquiring an image to be labeled, segmenting the image to be labeled to obtain multiple images to be processed; inputting the images to be processed into an image classification model to obtain the attention attributes of the images to be processed; determining the uncertainty attribute of the images to be processed as a preset classification result based on the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image; and labeling the regions of interest (ROIs) of the images to be labeled based on the uncertainty attributes of all the images to be processed. This solves the problem of region labeling errors caused by image labeling through attention mechanisms in the prior art. It realizes that by inputting multiple images to be processed from the segmented images to be labeled into an image classification model, the model processes them to obtain the attention attributes of each image to be processed, and then combining the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image, the uncertainty attribute of the classification result determined by mining image feature information based on the attention mechanism is determined. This improves the accuracy of the classification result determination and also improves the accuracy of labeling regions of interest in the images by using the magnitude of the uncertainty attribute.

[0068] Example 2

[0069] Figure 2 This is a flowchart of an image processing method based on classification uncertainty according to Embodiment 2 of the present invention. Building upon the foregoing embodiments, multiple training samples can be acquired, and an image classification model can be trained based on these samples to improve classification accuracy. Specific implementation details can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0070] like Figure 2 As shown, the method specifically includes the following steps:

[0071] S210. Obtain multiple initial sample images.

[0072] To improve the accuracy of the model, as many and varied training samples as possible can be obtained. The initial sample images can be whole-section images of histopathological tissues that have not been segmented, which can be images captured by an imaging device or images pre-stored in storage space.

[0073] S220. Perform segmentation processing on each initial sample image to obtain multiple sample sub-images corresponding to each initial sample image.

[0074] In this embodiment, the initial sample image can be preprocessed to cut it into several smaller images of the same size (e.g., 256×256), with each smaller image serving as a sample sub-image. Accordingly, multiple sample sub-images can be obtained for each initial sample image.

[0075] S230. Based on the theoretical labels of multiple sample sub-images corresponding to each initial sample image and the initial sample image, determine the training samples.

[0076] Theoretical labels refer to the classification labels assigned to the initial sample images, such as positive or negative.

[0077] In this embodiment, a set of sample sub-images under an initial sample image and the theoretical label of that initial sample image can be used as a training sample. Accordingly, multiple training samples can be obtained.

[0078] It should be noted that the dataset can also be divided into a training sample set and a test sample set. This allows the image classification model to be trained using the training sample set and tested using the test sample set, thus evaluating the model's performance and improving its accuracy while preventing the model from overfitting the training data.

[0079] S240. Train the initial classification model based on multiple training samples to obtain the image classification model.

[0080] In this embodiment, the initial classification model can be a model pre-trained on the ImageNet public dataset. The parameters in this model can be fixed. To improve classification accuracy, after obtaining multiple training samples, a set of sample sub-images from the initial sample image in the training samples can be input into the initial classification model. The initial classification model performs classification processing and outputs the classification label of the initial sample image. By comparing the classification label of the initial sample image with the theoretical label, the loss is calculated, and the model parameters of the initial classification model are adjusted based on the loss value to obtain the final image classification model. The advantage of this setup is that it can improve both model training efficiency and model classification accuracy.

[0081] The technical solution of this embodiment acquires multiple initial sample images, and then segments each initial sample image to obtain multiple sample sub-images corresponding to each initial sample image. Based on the multiple sample sub-images corresponding to each initial sample image and the theoretical labels of the initial sample images, training samples are determined. An image classification model is trained based on multiple training samples. By preprocessing the initial sample images, the size of the processed images is reduced, thereby improving the image classification accuracy.

[0082] Example 3

[0083] Figure 3 This is a flowchart of an image processing method based on classification uncertainty according to Embodiment 3 of the present invention. Based on the foregoing embodiments, S240 is further refined. Specific implementation details can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0084] like Figure 3 As shown, the method specifically includes the following steps:

[0085] S310. For multiple sample sub-images, the sample sub-images are processed based on the feature extraction model in the initial classification model to obtain the fourth feature vector of the sample sub-images.

[0086] It should be noted that the processing method for each training sample is the same; therefore, we will use the processing of one training sample as an example. This training sample contains multiple sample sub-images. The feature extraction model can be a pre-trained convolutional neural network. For example, if the model parameters in this convolutional neural network are set to default values, it needs to be trained and corrected to obtain the desired feature extraction model. For instance, to obtain a high-precision feature extraction model, the pre-trained convolutional neural network can be further trained based on the training samples, so that after the convolutional neural network is trained, a final applicable feature extraction model can be obtained. The feature extraction method for each sample sub-image is the same.

[0087] Specifically, the set of sample sub-images from the initial sample images in the training samples can be input into the initial classification model. The feature extraction model in the initial classification model processes each sample sub-image to extract its feature information, obtaining a feature vector for each sample sub-image, which serves as the fourth feature vector. For example, the feature vectors extracted from the sample sub-images using a pre-trained convolutional neural network can be used as input to the attention determination model to output the attention value of the sample sub-images.

[0088] S320. Based on the attention determination model in the initial classification model, the fourth feature vector is processed to obtain the attention value of the sample sub-image.

[0089] In this embodiment, the fourth feature vector can be input into the attention determination model in the initial classification model. The attention calculation function of the attention determination model is used to process the fourth feature vector to obtain the attention value of the sample sub-image. For example, the attention calculation function can be:

[0090] Values, π = {m, V1, V2} are the neural network parameters, h kIt is the fourth feature vector of the k-th sample sub-image.

[0091] S330. Based on the uncertainty determination model in the initial classification model, the attention value of the sample sub-image, the fifth feature vector of the initial sample image to which the sample sub-image belongs, and the sixth feature vector of the test sample image are processed to obtain the uncertainty value of the sample sub-image.

[0092] The test sample image can be any sample image from the test sample set. The sixth feature vector is the set of feature vectors for all smaller images after segmenting the test sample image. The fifth feature vector is the set of feature vectors for all sample sub-images under the initial sample image.

[0093] In this embodiment, the attention value of the sample sub-image, the fifth feature vector of the initial sample image to which the sample sub-image belongs, and the sixth feature vector of the test sample image can be input into the uncertainty determination model in the initial classification model. The uncertainty value determination function in the uncertainty determination model then processes the data to obtain the uncertainty value of the sample sub-image. For example, the uncertainty value determination function can be: Where, D={H i ,y i} n H represents the training samples in the training sample set, where n is the number of training samples. i Let y be the set of feature vectors of all sample sub-images under the initial sample image in the i-th training sample. i Let be the theoretical label of the initial sample image in the i-th training sample. Let p(π|D) represent the expectation of p(π|D), where p(π|D) represents the probability of π occurring given that D has occurred. k ,π|H * ) indicates in H * Under what conditions does a occur k The probability of π occurring. H * This is the sixth feature vector of the test sample image. I[a] k ,π|H * [D] represents the uncertainty value under the model. Characterizing global uncertainty, Characterize the uncertainty of the data.

[0094] S340. Based on the fourth feature vector and uncertainty value of all sample sub-images under the initial sample image, and the theoretical label of the initial sample image, determine the predicted label corresponding to the initial sample image.

[0095] In this embodiment, the fourth feature vectors and uncertainty values ​​of all sample sub-images under the initial sample image can be input into the category determination model in the initial classification model. Further, based on the fourth feature vectors and uncertainty values ​​of the sample sub-images, a first probability of the sample sub-image being a different prediction result is obtained; based on the first probability of the sample sub-image being a different prediction result, a second probability of the sample sub-image being associated with the theoretical label of its corresponding initial sample image is determined; based on the second probabilities of all sample sub-images under the initial sample image, a third probability of the initial sample image being associated with the theoretical label is determined. This third probability can be used as the fourth probability of the predicted label of the initial sample image, and loss processing is performed based on the fourth probability of the predicted label and the theoretical label to correct the model parameters in the initial classification model.

[0096] For example, in the case of binary classification, the third probability that the i-th initial sample image is the theoretical label is: Where K represents the number of sample sub-images under the initial sample image, y ik The label y represents the k-th sample sub-image, a positive class (i.e., consistent with the theoretical label of its initial sample image). i A value of 1 indicates a negative class (i.e., one whose label does not match the theoretical label of its initial sample image) 1-y i The value is 0. ik This represents the probability (i.e., the second probability) that the k-th sample sub-image is predicted as positive.

[0097] S350, based on the predicted label, theoretical label, and pre-defined cross-entropy loss function, corrects the model parameters in the initial classification model.

[0098] It should be noted that, in order to improve the accuracy of model training, the KL divergence KL[q] can be minimized. φ The optimization objective is determined by approximating the posterior distribution p(π|D) using the method [π|D], which can be used as max. φ L φ =L D (φ)-KL[q φ (π)||p(π)]. Where φ is the variational parameter,

[0099] p(y i |H i Let p(π) represent the probability of predicting the theoretical label. p(π) is the actual probability distribution, and q... φ (ρ) is the predicted probability distribution.

[0100] In actual encoding, a negative sign can be added before maximizing (max) to transform the maximization problem into a minimization problem, thus constructing the cross-entropy loss function. The cross-entropy loss function can be -max. φ L φ =

[0101] -(L D (φ)-KL[q φ Specifically, the loss value between the predicted label and the theoretical label is calculated based on the cross-entropy loss function, and the model parameters in the initial classification model are corrected with the goal of minimizing the loss value.

[0102] In this embodiment, a loss function can also be used to process the predicted and theoretical labels of each sample sub-image under the initial sample image, thereby calculating the corresponding loss value. All calculated loss values ​​are then fused to obtain a fused loss value, which can be used as the target loss value. The model parameters in the model can then be corrected based on the target loss value.

[0103] S360. Using the convergence of the cross-entropy loss function as the training objective, an image classification model is obtained.

[0104] It should be noted that since the model parameters in the initial classification model are uncorrected, the predicted labels output by the model at this time will differ from the theoretical labels of the initial sample images corresponding to the training samples. The model parameters in the initial classification model can be continuously corrected so that the classification model can output the expected theoretical labels.

[0105] Specifically, the training error of the loss function, i.e., the loss parameter, can be used as a condition to detect whether the loss function has reached convergence. For example, whether the training error is less than a preset error, whether the error trend is stable, or whether the current number of iterations equals a preset number. If convergence is detected, such as the training error of the loss function being less than the preset error or the error trend being stable, it indicates that the initial classification model has been trained successfully, and iterative training can be stopped. If the loss function has not converged, further training samples can be obtained to continue training the initial classification model until the loss function converges. At this point, the initial classification model can be considered successfully trained and used as an image classification model. When an image to be labeled is input into the trained image classification model, the model can label the regions of interest in the image and obtain the classification result.

[0106] The technical solution of this embodiment processes sample sub-images based on the feature extraction model in the initial classification model to fully mine image feature information and obtain the fourth feature vector of the sample sub-image, thereby improving image recognition accuracy. Further, the fourth feature vector is processed based on the attention determination model in the initial classification model to obtain the attention value of the sample sub-image. Then, based on the uncertainty determination model in the initial classification model, the attention value of the sample sub-image, the fifth feature vector of the initial sample image to which the sample sub-image belongs, and the sixth feature vector of the test sample image are processed to obtain the uncertainty value of the sample sub-image. The uncertainty value fully expresses the classification result of the initial sample image, improving image classification accuracy. Furthermore, based on the fourth feature vectors and uncertainty values ​​of all sample sub-images under the initial sample image, and the theoretical label of the initial sample image, a predicted label corresponding to the initial sample image is determined. Based on the predicted label, theoretical label, and cross-entropy loss function, the model parameters in the initial classification model are corrected to optimize the model output result, thereby improving the effectiveness of parameter correction, reducing time costs, and improving the accuracy of the image classification model.

[0107] Example 4

[0108] As an optional embodiment of the above embodiments, Figure 4 This is a schematic diagram of an image processing method according to Embodiment 4 of the present invention. For details, please refer to the following specific content.

[0109] See Figure 4 First, the entire WSI image (i.e., the image to be labeled) is cut into several 256×256 patch images (i.e., images to be processed). Then, the feature vector of each patch image is extracted using a feature extraction model in the image classification model, resulting in a set of feature vectors X = {x1,…,x...} for each patch image under the initial sample image. K} where K is the number of patch images, and x1 represents the feature vector of the first patch image. Furthermore, an attention determination model is used to determine the attention value 'a' for the k-th patch image. k ,

[0110]

[0111] Where ρ={m,V1,V2} are the neural network parameters, h k Let z be the feature vector of the k-th patch image. Based on each patch image, the classification attribute z of the entire WSI image can be determined.

[0112]

[0113] Furthermore, the uncertainty attribute I[a] of the attention value of the patch image is calculated using an uncertainty determination model. k ,π|H * ,D],

[0114]

[0115] in, This represents the global uncertainty value. This represents the uncertainty value of the data. Let p(π|D) represent the expectation of p(π|D), where p(π|D) represents the probability of π occurring given that D has occurred. k ,π|H * ) indicates in H * Under what conditions does a occur k The probability of π occurring.

[0116] Furthermore, based on the uncertainty attribute of the attention value of each patch image, the entire WSI image is labeled with lesion regions (i.e., regions of interest). For example, the larger the value of the uncertainty attribute, the darker the color of the label.

[0117] Building upon the above approach, a training sample set can be obtained, and an image classification model can be trained based on this set. This can be achieved by first constructing a feature extraction model based on a ResNet 50 convolutional neural network, and then using this model to extract the feature vectors of the segmented patch images in the WSI, i.e., H = {h1, ..., h...}. K}, where h1 is the feature vector extracted from patch image 1, and the size of this vector is 1×1024, resulting in the training sample set D={H i ,y i} n n is the number of training samples, H i Let y be the set of feature vectors for all sample sub-images under the initial sample image in the i-th training sample. i ∈{0,1} represents the theoretical label of the initial sample image in the i-th training sample. To extract uncertainty at the patch instance level, the attention probability distribution of each patch image needs to be calculated. The attention value is calculated as follows: Furthermore, the uncertainty value for each patch image is determined based on the attention value. The uncertainty value is calculated as follows: The uncertainty value is input into the category determination model, and variational inference is used to approximate the posterior. Specifically, q is used. φ (π) to minimize the KL divergence KL[q φThe posterior distribution p(π|D) is approximated using the formula [(π)||p(π|D)], where φ is a variational parameter. This method is equivalent to maximizing the optimization objective as follows: max φ L φ =L D (φ)-KL[q φ [(ρ)||p(π)], In actual coding, a negative sign can be added before maximizing `max` to transform the maximization problem into a minimization problem, i.e., -max. φ L φ =-(L D (φ)-KL[q φ (π)||p(π)]). Based on the value of the optimization objective (loss value, -max) φ L φ This is used to determine whether to stop training, correct the model parameters in the image classification model, and obtain the trained image classification model.

[0118] The technical solution of this embodiment involves acquiring an image to be labeled, segmenting the image to be labeled to obtain multiple images to be processed; inputting the images to be processed into an image classification model to obtain the attention attributes of the images to be processed; determining the uncertainty attribute of the images to be processed as a preset classification result based on the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image; and labeling the regions of interest (ROIs) of the images to be labeled based on the uncertainty attributes of all the images to be processed. This solves the problem of region labeling errors caused by image labeling through attention mechanisms in the prior art. It realizes that by inputting multiple images to be processed from the segmented images to be labeled into an image classification model, the model processes them to obtain the attention attributes of each image to be processed, and then combining the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image, the uncertainty attribute of the classification result determined by mining image feature information based on the attention mechanism is determined. This improves the accuracy of the classification result determination and also improves the accuracy of labeling regions of interest in the images by using the magnitude of the uncertainty attribute.

[0119] Example 5

[0120] Figure 5 This is a schematic diagram of the structure of an image processing device based on classification uncertainty according to Embodiment 5 of the present invention. Figure 5 As shown, the device includes: a cutting module 410, an attention attribute determination module 420, an uncertainty attribute determination module 430, and a labeling module 440.

[0121] The system includes a cutting module 410 for acquiring an image to be labeled and cutting it to obtain multiple images to be processed; an attention attribute determination module 420 for inputting the images to be processed into an image classification model to obtain the attention attributes of the images to be processed; wherein the image classification model is trained based on a pre-trained initial classification model; an uncertainty attribute determination module 430 for determining the uncertainty attribute of each image to be processed as a preset classification result based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of a reference sample image; and an annotation module 440 for annotating the regions of interest of the images to be labeled based on the uncertainty attributes of all the images to be processed.

[0122] The technical solution of this embodiment involves acquiring an image to be labeled, segmenting the image to be labeled to obtain multiple images to be processed; inputting the images to be processed into an image classification model to obtain the attention attributes of the images to be processed; determining the uncertainty attribute of the images to be processed as a preset classification result based on the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image; and labeling the regions of interest (ROIs) of the images to be labeled based on the uncertainty attributes of all the images to be processed. This solves the problem of region labeling errors caused by image labeling through attention mechanisms in the prior art. It realizes that by inputting multiple images to be processed from the segmented images to be labeled into an image classification model, the model processes them to obtain the attention attributes of each image to be processed, and then combining the attention attributes of the images to be processed, the first feature vector of the images to be labeled, and the second feature vector of the reference sample image, the uncertainty attribute of the classification result determined by mining image feature information based on the attention mechanism is determined. This improves the accuracy of the classification result determination and also improves the accuracy of labeling regions of interest in the images by using the magnitude of the uncertainty attribute.

[0123] Optionally, based on the above-mentioned device, the attention attribute determination module 420 includes a third feature vector determination unit and an attention attribute determination unit.

[0124] The third feature vector determination unit is used to perform feature extraction processing on the image to be processed based on the feature extraction model in the image classification model to obtain the third feature vector of the image to be processed.

[0125] An attention attribute determination unit is used to determine the attention attributes of the image to be processed based on the third feature vector.

[0126] Optionally, based on the above-mentioned device, the attention attribute determination unit is specifically used to process the third feature vector based on the attention determination model in the image classification model to obtain the attention attribute of the image to be processed.

[0127] Optionally, based on the above-described apparatus, the apparatus may further include a classification module. The classification module includes a classification attribute determination unit and a classification result determination unit.

[0128] The classification attribute determination unit is used to determine the classification attribute corresponding to the image to be labeled based on the category determination model in the image classification model according to the third feature vector and attention attribute of all the images to be processed; the classification result determination unit is used to determine the classification result of the image to be labeled based on the classification attribute.

[0129] Based on the above-mentioned device, optionally, the classification attribute determination unit includes a first intermediate value determination subunit, a second intermediate value determination subunit, and a classification attribute determination subunit.

[0130] The first intermediate value determination subunit is used to determine a first intermediate value based on the attention attributes of all the images to be processed;

[0131] The second intermediate value determination subunit is used to determine the second intermediate value corresponding to the image to be processed based on the third feature vector and attention attribute of the same image to be processed;

[0132] The classification attribute determination subunit is used to determine the classification attribute based on the first intermediate value and the second intermediate values ​​corresponding to all the images to be processed.

[0133] Based on the above-mentioned device, optionally, the uncertainty attribute determination module 430 includes a first feature vector determination unit and an uncertainty attribute determination unit.

[0134] The first feature vector determination unit is used to determine the first feature vector of the image to be labeled based on the third feature vectors of all the images to be processed;

[0135] The uncertainty attribute determination unit is used to determine the uncertainty attribute of the image to be processed as a preset classification result based on the uncertainty determination model in the image classification model, according to the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image.

[0136] Based on the above-mentioned device, optionally, the uncertainty determination model includes a global uncertainty function and a data uncertainty function, and the uncertainty attribute determination unit includes a global uncertainty attribute determination subunit, a data uncertainty attribute determination subunit, and an uncertainty attribute determination subunit.

[0137] The global uncertainty attribute determination subunit is used to determine the global uncertainty attribute based on the global uncertainty function according to the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image;

[0138] The data uncertainty attribute determination subunit is used to determine the data uncertainty attribute based on the data uncertainty function according to the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image;

[0139] The uncertainty attribute determination subunit is used to determine the uncertainty attribute based on the global uncertainty attribute and the data uncertainty attribute.

[0140] Optionally, based on the above-described apparatus, the apparatus may further include a model training module, which includes: an initial sample image determination unit, a sample sub-image determination unit, a training sample determination unit, and a model training unit.

[0141] An initial sample image determination unit is used to acquire multiple initial sample images;

[0142] The sample sub-image determination unit is used to perform segmentation processing on each of the initial sample images to obtain multiple sample sub-images corresponding to each of the initial sample images;

[0143] The training sample determination unit is used to determine training samples based on multiple sample sub-images corresponding to each of the initial sample images and the theoretical labels of the initial sample images;

[0144] The model training unit is used to train the initial classification model based on multiple training samples to obtain the image classification model.

[0145] Based on the above-mentioned device, optionally, the model training determination unit includes: a fourth feature vector determination subunit, an attention value determination subunit, an uncertainty value determination subunit, a prediction label determination subunit, a parameter adjustment subunit, and a model training subunit.

[0146] The fourth feature vector determination subunit is used to process the sample sub-images based on the feature extraction model in the initial classification model for the multiple sample sub-images to obtain the fourth feature vector of the sample sub-images;

[0147] The attention value determination subunit is used to process the fourth feature vector based on the attention determination model in the initial classification model to obtain the attention value of the sample sub-image;

[0148] The uncertainty value determination subunit is used to process the attention value of the sample sub-image, the fifth feature vector of the initial sample image to which the sample sub-image belongs, and the sixth feature vector of the test sample image based on the uncertainty determination model in the initial classification model to obtain the uncertainty value of the sample sub-image.

[0149] The prediction label determination subunit is used to determine the prediction label corresponding to the initial sample image based on the fourth feature vector and uncertainty value of all the sample sub-images under the initial sample image, and the theoretical label of the initial sample image;

[0150] The parameter adjustment subunit is used to correct the model parameters in the initial classification model based on the predicted label, the theoretical label, and the pre-set cross-entropy loss function.

[0151] The model training subunit is used to converge the cross-entropy loss function as the training objective to obtain the image classification model.

[0152] The image processing apparatus based on classification uncertainty provided in the embodiments of the present invention can execute the image processing method based on classification uncertainty provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0153] Example 6

[0154] Figure 6 This is a schematic diagram of an electronic device implementing the image processing method based on classification uncertainty according to embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0155] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0156] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0157] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image processing methods.

[0158] In some embodiments, the image processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image processing method by any other suitable means (e.g., by means of firmware).

[0159] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0160] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0161] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0163] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0164] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0165] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0166] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image processing method based on classification uncertainty, characterized in that, include: Obtain the image to be labeled, and perform segmentation processing on the image to be labeled to obtain multiple images to be processed; The image to be processed is input into an image classification model to obtain the attention attributes of the image to be processed; wherein, the image classification model is trained based on a pre-trained initial classification model; For each image to be processed, based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image, the uncertainty attribute of the image to be processed as a preset classification result is determined; Regions of interest are labeled in the images to be labeled based on all the uncertainty attributes of the images to be processed; The determination of the uncertainty attribute of the image to be processed as a preset classification result based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image includes: Based on the third feature vectors of all the images to be processed, the first feature vector of the image to be labeled is determined; Based on the uncertainty determination model in the image classification model, the uncertainty attribute of the image to be processed is determined according to the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image. The step of determining the first feature vector of the image to be labeled based on the third feature vectors of all the images to be processed includes: The third feature vectors of all the images to be processed are summarized, and the summarized feature vector set is used as the first feature vector of the image to be labeled. The third feature vector is the image feature vector of each image to be processed; Specifically, the uncertainty attribute of the image to be processed as a preset classification result is calculated based on the uncertainty determination function in the uncertainty determination model. The uncertainty determination function is: ； in, express Expectations Indicates in Under the conditions of occurrence The probability of occurrence; Indicates in Under the conditions of occurrence The probability of occurrence; Used to characterize the uncertainty properties under the model That is, a global uncertainty function. This is the data uncertainty function; Indicates the first Attention properties of the images to be processed; For neural network parameters; This represents the set of images to be labeled; This is the second feature vector of the reference sample image.

2. The method according to claim 1, characterized in that, The step of inputting the image to be processed into an image classification model to obtain the attention attributes of the image to be processed includes: Based on the feature extraction model in the image classification model, feature extraction processing is performed on the image to be processed to obtain the third feature vector of the image to be processed; The attention attributes of the image to be processed are determined based on the third feature vector.

3. The method according to claim 2, characterized in that, Determining the attention attributes of the image to be processed based on the third feature vector includes: The third feature vector is processed based on the attention determination model in the image classification model to obtain the attention attribute of the image to be processed.

4. The method according to claim 2, characterized in that, After inputting the image to be processed into the image classification model to obtain the attention attributes of the image to be processed, the method further includes: The category determination model in the image classification model determines the classification attribute of the image to be labeled based on the third feature vector and attention attribute of all the images to be processed. The classification result of the image to be labeled is determined based on the classification attributes.

5. The method according to claim 4, characterized in that, The step of determining the classification attribute corresponding to the image to be labeled based on the third feature vector and attention attribute of all the images to be processed includes: Based on the attention attributes of all the images to be processed, a first intermediate value is determined; Based on the third feature vector and attention attribute of the same image to be processed, a second intermediate value corresponding to the image to be processed is determined; The classification attribute is determined based on the first intermediate value and the second intermediate values ​​corresponding to all the images to be processed.

6. The method according to claim 1, characterized in that, The uncertainty determination model includes a global uncertainty function and a data uncertainty function. The step of determining the uncertainty attribute of the image to be processed as a preset classification result based on the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image includes: Based on the global uncertainty function, the global uncertainty attribute is determined according to the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image; Based on the data uncertainty function, the data uncertainty attribute is determined according to the attention attribute of the image to be processed, the first feature vector of the image to be labeled, and the second feature vector of the reference sample image; The uncertainty attribute is determined based on the global uncertainty attribute and the data uncertainty attribute.

7. The method according to claim 1, characterized in that, Also includes: Acquire multiple initial sample images; Each initial sample image is segmented to obtain multiple sample sub-images corresponding to each initial sample image; Training samples are determined based on multiple sample sub-images corresponding to each initial sample image and the theoretical labels of the initial sample images; The initial classification model is trained based on multiple training samples to obtain the image classification model.

8. The method according to claim 7, characterized in that, The step of training the initial classification model based on multiple training samples to obtain the image classification model includes: For the multiple sample sub-images, the sample sub-images are processed based on the feature extraction model in the initial classification model to obtain the fourth feature vector of the sample sub-images; The fourth feature vector is processed based on the attention determination model in the initial classification model to obtain the attention value of the sample sub-image; Based on the uncertainty determination model in the initial classification model, the attention value of the sample sub-image, the fifth feature vector of the initial sample image to which the sample sub-image belongs, and the sixth feature vector of the test sample image are processed to obtain the uncertainty value of the sample sub-image. Based on the fourth feature vector and uncertainty value of all the sample sub-images under the initial sample image, and the theoretical label of the initial sample image, the predicted label corresponding to the initial sample image is determined; Based on the predicted label, the theoretical label, and the pre-set cross-entropy loss function, the model parameters in the initial classification model are corrected; The convergence of the cross-entropy loss function is used as the training objective to obtain the image classification model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image processing method based on classification uncertainty as described in any one of claims 1-8.

Images