Classification model generation method and device, electronic equipment and storage medium

Abstract

The invention provides a classification model generation method and device, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of artificial intelligence such as computer vision and deep learning. According to the specific implementation scheme, a first data set is obtained, and the first data set comprises a first sample image and a first category label corresponding to the first sample image; inputting the first sample image into a target detection model to obtain a first probability graph corresponding to each object type; inputting the first sample image into an initial classification model to obtain a category activation graph corresponding to each image category and a first prediction category corresponding to the first sample image; and according to the difference between the first prediction category and the first category label and the difference between the first probability graph and the category activation graph, correcting the initial classification model to obtain a target classification model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technology such as computer vision and deep learning, and specifically to a method, device, electronic device and storage medium for generating a classification model. Background Art

[0002] Image classification is an important research direction in the field of computer vision. With the development of deep learning, image classification models have been widely used in many fields such as industry, medicine, and military. Therefore, how to train image classification models and improve the accuracy of image classification models has become a key research direction. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device and storage medium for generating a classification model.

[0004] According to a first aspect of the present disclosure, a method for generating a classification model is provided, comprising:

[0005] Acquire a first data set, wherein the first data set includes a first sample image and a first category label corresponding to the first sample image;

[0006] Inputting the first sample image into an object detection model to obtain a first probability map corresponding to each object type;

[0007] Inputting the first sample image into an initial classification model to obtain a class activation map corresponding to each image class and a first predicted class corresponding to the first sample image;

[0008] According to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, the initial classification model is modified to obtain a target classification model.

[0009] According to a second aspect of the present disclosure, there is provided an image classification method, comprising:

[0010] Get the image to be classified;

[0011] The image to be classified is input into a target classification model to obtain a target category corresponding to the image to be classified; wherein the target classification model is generated by training according to the method described in the first aspect.

[0012] According to a third aspect of the present disclosure, a medical image classification method is provided, comprising:

[0013] Obtaining a medical image to be classified;

[0014] The medical image is input into a target classification model to obtain a target category corresponding to the medical image; wherein the target classification model is generated by training according to the method described in the first aspect.

[0015] According to a fourth aspect of the present disclosure, there is provided a device for generating a classification model, comprising:

[0016] An acquisition module, configured to acquire a first data set, wherein the first data set includes a first sample image and a first category label corresponding to the first sample image;

[0017] A first processing module, configured to input the first sample image into a target detection model to obtain a first probability map corresponding to each object type;

[0018] A second processing module, used for inputting the first sample image into an initial classification model to obtain a class activation map corresponding to each image class and a first predicted class corresponding to the first sample image;

[0019] A correction module is used to correct the initial classification model according to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, so as to obtain a target classification model.

[0020] According to a fifth aspect of the present disclosure, there is provided an image classification device, comprising:

[0021] An acquisition module, used for acquiring images to be classified;

[0022] A classification module is used to input the image to be classified into a target classification model to obtain a target category corresponding to the image to be classified; wherein the target classification model is generated by training according to the device described in the fourth aspect.

[0023] According to a sixth aspect of the present disclosure, there is provided a medical image classification device, comprising:

[0024] An acquisition module, used for acquiring medical images to be classified;

[0025] A classification module is used to input the medical image into a target classification model to obtain a target category corresponding to the medical image; wherein the target classification model is generated by training according to the device described in the fourth aspect.

[0026] According to a seventh aspect of the present disclosure, there is provided an electronic device, including:

[0027] at least one processor; and

[0028] a memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a classification model as described in the first aspect, or can execute the image classification method as described in the second aspect, or can execute the medical image classification method as described in the third aspect.

[0030] According to an eighth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method for generating a classification model as described in the first aspect, or to execute the image classification method as described in the second aspect, or to execute the medical image classification method as described in the third aspect.

[0031] According to the ninth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the method for generating a classification model as described in the first aspect, or are capable of executing the steps of the image classification method as described in the second aspect, or are capable of executing the steps of the medical image classification method as described in the third aspect.

[0032] The classification model generation method, device, electronic device and storage medium provided by the present disclosure have the following beneficial effects:

[0033] In the disclosed embodiment, a first sample image and a first category label corresponding to the first sample image are first obtained, and then the first sample image is input into the target detection model to obtain a first probability map corresponding to each object type, and the first sample image is input into the initial classification model to obtain a category activation map corresponding to each image category and a first predicted category corresponding to the first sample image, and finally, the initial classification model is corrected according to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, to obtain the target classification model. Thus, in the process of training the target classification model, the location information of the object in the first sample image can be learned based on the difference between the category activation map and the first probability map, so that the target classification model has the ability to locate the object, and then the image can be classified more accurately based on the location information of the object, thereby improving the accuracy of the target classification model in classifying images.

[0034] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0036] Figure 1 It is a flowchart of a method for generating a classification model provided according to an embodiment of the present disclosure;

[0037] Figure 2 A schematic diagram of determining a category activation map provided by an embodiment of the present disclosure;

[0038] Figure 3 is a flow chart of a method for generating a classification model according to another embodiment of the present disclosure;

[0039] Figure 4 is a flow chart of a method for generating a classification model according to another embodiment of the present disclosure;

[0040] Figure 5 is a flowchart of an image classification method provided according to an embodiment of the present disclosure;

[0041] Figure 6 is a flowchart of a medical image classification method provided according to another embodiment of the present disclosure;

[0042] Figure 7 is a structural schematic diagram of a classification model generation device provided according to an embodiment of the present disclosure;

[0043] Figure 8 is a structural schematic diagram of an image classification device provided according to an embodiment of the present disclosure;

[0044] Fig. 9 is a structural schematic diagram of a medical image classification device provided according to an embodiment of the present disclosure;

[0045] Fig.10 The block diagram is a block diagram of an electronic device used to implement the method for generating a classification model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0047] The embodiments of the present disclosure relate to the fields of artificial intelligence technologies such as computer vision and deep learning.

[0048] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0049] Deep learning is the process of learning the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in interpreting data such as text, images, and sounds. The ultimate goal of deep learning is to enable machines to have analytical learning capabilities like humans and to be able to recognize data such as text, images, and sounds.

[0050] Computer vision refers to the use of cameras and computers to replace the human eye to identify, track and measure targets, and further perform graphic processing so that the computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection.

[0051] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0052] The following describes a method, device, electronic device, and storage medium for generating a classification model according to an embodiment of the present disclosure with reference to the accompanying drawings.

[0053] It should be noted that the executor of the classification model generation method of this embodiment is a classification model generation device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, and the electronic device may include but is not limited to a terminal, a server, etc.

[0054] Figure 1 It is a flowchart of a method for generating a classification model provided according to an embodiment of the present disclosure.

[0055] like Figure 1 As shown, the generation method of the classification model includes:

[0056] S101, obtaining a first data set, wherein the first data set includes a first sample image and a first category label corresponding to the first sample image.

[0057] The first sample image may be an image associated with a target classification task. For example, the target classification task is a face image age classification task. Then the first sample image may be a face image, and the corresponding first category label may be an age category corresponding to the face image. For example, the age category may be 0-10 years old, 10-20 years old, 20-30 years old, 30-40 years old, 40-50 years old, etc. This disclosure does not limit this.

[0058] S102: Input the first sample image into the target detection model to obtain a first probability map corresponding to each object type.

[0059] The object type may be a pre-set type of object associated with each category in the target classification task.

[0060] For example, the target classification task is the age classification task of face images. The classification categories can be 0-10 years old, 10-20 years old, 20-30 years old, 30-40 years old, 40-50 years old, etc. The object types can be nasolabial folds, eye wrinkles, forehead wrinkles, etc.

[0061] The first probability map corresponding to each object type includes the probability that the object in each area of ​​the first sample image belongs to the object type.

[0062] In some embodiments, the target detection model can be a fully convolutional neural network, including units such as convolution, activation function, pooling, and normalization. After convolution layer by layer, the resolution of the feature map is gradually reduced, so that each pixel on the feature map corresponds to a block area on the original input image. For example, the target detection model output is in the form of H out ,W out , C’s feature map, and use the sigmoid function to activate the feature map to get the first probability map. Assume that the resolution of the first sample image is H in ,W in , then the value of each point on the first probability map corresponds to the original image The probability that the object in the block area belongs to each object category.

[0063] Among them, the target detection model can be pre-trained based on the target task.

[0064] In some embodiments, a second data set may be first obtained, wherein the second data set includes a second sample image and label data corresponding to the second sample image, wherein the label data includes a type label and a position label corresponding to the sample object in the second sample image, and then the second sample image is input into the initial target detection model to obtain the probability that the object at each position in the second sample image belongs to each object type, and finally, based on the difference between the label data and the probability that the object at each position in the second sample image belongs to each object type, the initial target detection model is trained to obtain the target detection model. Thus, the initial target detection model may be trained based on the second data set, thereby improving the performance of the target detection model, and further improving the accuracy of the first probability map corresponding to each object type determined.

[0065] The second sample images include images associated with the target task, for example, the target task is a face image age classification task, and the second sample images may be face images.

[0066] In some embodiments, the second sample image may be the same as or different from the first sample image, which is not limited in the present disclosure.

[0067] The sample objects may be objects corresponding to the object type associated with the target classification task. For example, if the target classification task is a face image age classification task, the objects may include nasolabial fold 1, nasolabial fold 2, eye wrinkle 1, and the like.

[0068] The category label may be the object category to which the sample object belongs.

[0069] The position label may be the position of the sample object in the second sample image.

[0070] In some embodiments, the loss value between the label data and the probability that the object at each position in the second sample image belongs to each object category can be determined based on a cross entropy loss function or a logarithmic loss function, and then the initial target detection model can be modified based on the loss value.

[0071] S103: Input the first sample image into an initial classification model to obtain a category activation map corresponding to each image category and a first predicted category corresponding to the first sample image.

[0072] In some embodiments, the initial classification model may be a convolutional neural network, for example, a deep residual network (Residual Network, ResNet), a dense convolutional network (Densely Connected Convolutional Networks, DenseNet), etc. This disclosure does not limit this.

[0073] Figure 2 A schematic diagram of determining a class activation map provided by an embodiment of the present disclosure. Figure 2 As shown in the figure, the first sample image is first input into the convolution layer (ConvBN) in the initial classification model to obtain a feature map. The feature map output by the previous convolution layer will be used as the input feature map of the next convolution layer. The resolution in the width and height directions of the image gradually decreases, and the number of channels gradually increases. Finally, after transformation by the global pooling layer and the fully connected layer, the probability of the output image belonging to each image category is obtained.

[0074] Secondly, for each image category, the probability of each image category is calculated based on cross entropy to the gradient map of the feature map output by any convolution layer. For example, the first convolution layer outputs a feature map of the shape H, W, C, and its gradient shape is also H, W, C. Then, the gradient map is globally average pooled and transformed into 1, 1, C. The feature map is dot-multiplied (equivalent to weighting the feature map along the channel direction after the average gradient) and the feature map after dot multiplication is averaged along the channel direction to obtain the category activation map (Grad-CAM) corresponding to each image category.

[0075] In some embodiments, the probability of each image category and the category activation area between multiple feature maps may also be calculated, and the multiple category activation maps may be weighted to obtain the category activation map corresponding to each image category.

[0076] In some embodiments, step S102 and step S103 may be performed in parallel, thereby further improving the training efficiency of the classification model.

[0077] S104: According to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, the initial classification model is modified to obtain a target classification model.

[0078] The training goal in the disclosed embodiment is to make the difference between the first predicted category and the second category label smaller and smaller, and the difference between the first probability map and the category activation map smaller and smaller. Thus, the target classification model can learn the location information of the object in the first sample image through the first probability map and the category activation map, thereby improving the accuracy of the target classification model in classifying the image.

[0079] In the disclosed embodiment, a first sample image and a first category label corresponding to the first sample image are first obtained, and then the first sample image is input into the target detection model to obtain a first probability map corresponding to each object type, and the first sample image is input into the initial classification model to obtain a category activation map corresponding to each image category and a first predicted category corresponding to the first sample image, and finally, the initial classification model is corrected according to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, to obtain the target classification model. Thus, in the process of training the target classification model, the location information of the object in the first sample image can be learned based on the difference between the category activation map and the first probability map, so that the target classification model has the ability to locate the object, and then the image can be classified more accurately based on the location information of the object, thereby improving the accuracy of the target classification model in classifying images.

[0080] Figure 3 is a flow chart of a method for generating a classification model according to another embodiment of the present disclosure;

[0081] like Figure 3 As shown, the generation method of the classification model includes:

[0082] S301, obtaining a first data set, wherein the first data set includes a first sample image and a first category label corresponding to the first sample image.

[0083] S302: Input the first sample image into the target detection model to obtain a first probability map corresponding to each object type.

[0084] S303: Input the first sample image into the initial classification model to obtain a category activation map corresponding to each image category and a first predicted category corresponding to the first sample image.

[0085] The specific implementation forms of steps S301 to S303 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0086] S304: Determine a first loss value according to a difference between the first predicted category and the first category label.

[0087] In some embodiments, a first loss value between the first predicted category and the first category label may be determined using a cross entropy loss function or a logarithmic loss function.

[0088] S305: Determine a second loss value according to the difference between the first probability map and the class activation map.

[0089] In some embodiments, the first probability map corresponding to each class activation map is determined according to the object type associated with each image category, the third loss value corresponding to each class activation map is determined according to the difference between each class activation map and the corresponding first probability map, and the sum of at least two third loss values ​​is determined as the second loss value. Thus, the second loss value between the first probability map and the class activation map can be accurately determined according to the association between the image category and the object type, thereby improving the training efficiency.

[0090] The object type used to determine each image category is the object type associated with the image category.

[0091] For example, object types include type 1, type 2, type 3, and type 4, and image categories include category A, category B, and category C, wherein the object corresponding to type 1 is used to determine whether the image belongs to category A, and the object type associated with category A is type 1; the objects corresponding to type 2 and type 3 are used to determine whether the image belongs to category B, and the object types associated with category B are type 2 and type 3; the object corresponding to type 4 is used to determine whether the image belongs to category C, and the object type associated with category C is type 4.

[0092] For example, if the object category associated with image category A is category 1, then the first probability map corresponding to the category activation map of image category A is the first probability map of category 1.

[0093] Alternatively, if the object categories associated with image category B are category 2 and category 3, the first probability map corresponding to the category activation map of image category B is the first probability map of category 2 and the first probability map of category 3.

[0094] In some embodiments, when the class activation map corresponds to a first probability map, the similarity between the class activation map and the first probability map can be directly determined as the third loss value corresponding to the class probability map.

[0095] In some embodiments, when any class activation map corresponds to at least two first probability maps, the similarity between any class activation map and each corresponding first probability map is determined, and at least two similarities are merged to determine the third loss value corresponding to any class activation map. Thus, when any class activation map corresponds to at least two first probability maps, the third loss value corresponding to the class activation map can be accurately determined.

[0096] In some embodiments, a distance function such as bulldozer distance, mean square error, KL divergence, or cosine similarity may be used to measure the similarity between the class activation map and the first probability map, which is not limited in the present disclosure.

[0097] It should be noted that the size between the category activation area and the first probability map may be different. Therefore, before calculating the similarity, the size between the first probability map and the category activation area can be unified to facilitate the calculation of the similarity between the first probability map and the category activation area.

[0098] In some embodiments, when the first size of the class activation map is larger than the second size of the first probability map, the first probability map is upsampled based on the first size, or the class activation map is downsampled based on the second size.

[0099] Wherein, when the first size of the class activation map is larger than the second size of the first probability map, upsampling the first probability map may make the size of the upsampled first probability map the same as the first size of the class activation map. Downsampling the class activation map may make the downsampled class activation map the same as the second size of the first probability map.

[0100] In some embodiments, when the first size of the class activation map is smaller than the second size of the first probability map, the class activation map is upsampled based on the second size, or the first probability map is downsampled based on the first size.

[0101] Wherein, when the first size of the class activation map is smaller than the second size of the first probability map, downsampling the first probability map can make the size of the downsampled first probability map the same as the first size of the class activation map. Upsampling the class activation map can make the upsampled class activation map the same as the second size of the first probability map.

[0102] S306: Determine a target loss value according to the first loss value and the second loss value.

[0103] In some embodiments, the first loss value and the second loss value may be added to obtain a target loss value.

[0104] In some embodiments, a first weight corresponding to the first loss value and a second weight corresponding to the second loss value may also be determined, and then the first loss value and the second loss value may be weighted based on the first weight and the second weight to obtain a target loss value.

[0105] In some embodiments, the first weight is less than or equal to the second weight. For example, the first weight is 0-1, and the second weight is 1. The present disclosure does not limit the values ​​of the first weight and the second weight.

[0106] S307, modifying the initial classification model according to the target loss value to obtain a target classification model.

[0107] In the disclosed embodiment, the first sample image is first input into the target detection model to obtain the first probability map corresponding to each object type, and the first sample image is input into the initial classification model to obtain the category activation map corresponding to each image category and the first predicted category corresponding to the first sample image. Then, the first loss value is determined based on the difference between the first predicted category and the first category label, and the second loss value is determined based on the difference between the first probability map and the category activation map. Finally, the first loss value and the second loss value are fused, and the initial classification model is corrected based on the fused target loss value to determine the target classification model. In this way, the first loss value corresponding to the image category and the second loss value corresponding to the object position can be determined respectively, and then the first loss value and the second loss value are fused to train the target classification model, which can improve the training efficiency of the target classification model and the accuracy of image classification.

[0108] Figure 4 is a flow chart of a method for generating a classification model according to another embodiment of the present disclosure; Figure 4 As shown, the generation method of the classification model includes:

[0109] S401, obtaining a third data set, wherein the third data set includes a third sample image and a second category label corresponding to the third sample image.

[0110] The third sample image is an image associated with a target classification task. For example, the target classification task is a face image age classification task. Then the third sample image may be a face image, and the corresponding second category label may be an age category corresponding to the face image. For example, the age category may be 0-10 years old, 10-20 years old, 20-30 years old, 30-40 years old, 40-50 years old, etc. This disclosure does not limit this.

[0111] In some embodiments, the third sample image may be the same as or different from the first sample image and the second sample image, and this disclosure does not limit this.

[0112] S402: Input the third sample image into a preset classification model to obtain a second predicted category corresponding to the third sample image.

[0113] In some embodiments, the preset classification model may be a convolutional neural network, for example, a deep residual network (Residual Network, ResNet), a dense convolutional network (Densely Connected Convolutional Networks, DenseNet), etc. This disclosure does not limit this.

[0114] S403: Based on the difference between the second classification label and the second predicted category, a preset classification model is trained to obtain an initial classification model.

[0115] In some embodiments, the difference between the second predicted category and the second category label can be determined based on a cross entropy loss function or a logarithmic loss function, and then the preset classification model is modified based on the calculated loss value to obtain an initial classification model.

[0116] In the disclosed embodiment, before training the initial classification model, the preset classification model can also be pre-trained to obtain the initial classification model, so that the initial classification model can first learn a part of the image classification information, thereby reducing the difficulty of training the initial classification model and the amount of sample data required for training, thereby improving the training efficiency of the target classification model.

[0117] S404: Acquire a first data set, where the first data set includes a first sample image and a first category label corresponding to the first sample image.

[0118] S405: Input the first sample image into the target detection model to obtain a first probability map corresponding to each object type.

[0119] S406: Input the first sample image into the initial classification model to obtain a category activation map corresponding to each image category and a first predicted category corresponding to the first sample image.

[0120] S407: According to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, the initial classification model is modified to obtain a target classification model.

[0121] The specific implementation forms of steps S404 to S407 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0122] In the disclosed embodiment, before training the initial classification model, a third data set is first obtained, wherein the third data set includes a third sample image and a second category label corresponding to the third sample image, and the third sample image is input into the preset classification model to obtain the second predicted category corresponding to the third sample image, and based on the difference between the second classification label and the second predicted category, the preset classification model is trained to obtain the initial classification model, and then the initial classification model is trained again to obtain the target classification model. Thus, the preset classification model can be pre-trained to obtain the initial classification model, so that the initial classification model can first learn a part of the image classification information, thereby reducing the difficulty of training the initial classification model and the amount of sample data required for training, thereby improving the training efficiency of the target classification model.

[0123] Figure 5 is a flowchart of an image classification method provided according to another embodiment of the present disclosure; Figure 5 As shown, the image classification method includes:

[0124] S501, obtaining an image to be classified.

[0125] The image to be classified may be an image to be classified by a target classification model.

[0126] The type of the image to be classified is the same as the type of the first sample image. For example, if the first sample image is a face image, then the image to be classified is also a classification image.

[0127] S502: Input the image to be classified into the target classification model to obtain the target category corresponding to the image to be classified.

[0128] Among them, the target classification model is generated by training according to the generation method of the classification model.

[0129] In some embodiments, the image to be classified is input into a target classification model to obtain the target category corresponding to the image to be classified and the position of the target object associated with the target category in the image to be classified, wherein the type of the target object is the same as the type of object associated with the target type.

[0130] For example, if the target category is category A and the object type associated with category A is category 1, then the object type to which the target object belongs is category 1.

[0131] It should be noted that, during the training process, the target classification model learns the position information of the object associated with the image category in the first sample image through the category activation map corresponding to each image category. Therefore, in the embodiment of the present disclosure, after determining the target category corresponding to the image to be classified through the target classification model, the position of the target object associated with the target category in the image to be classified can also be determined based on the category activation map corresponding to the target category. In this way, not only the target category corresponding to the image to be classified can be determined, but also the position of the target object associated with the target category in the image to be classified can be determined, which can improve the accuracy of image classification and provide an intuitive classification basis.

[0132] In the disclosed embodiment, an image to be classified is obtained, and then the image to be classified is input into a target classification model to obtain a target category corresponding to the image to be classified. Thus, the image to be classified can be classified by a target classification model with object positioning capability, thereby improving the accuracy of classifying the image to be classified.

[0133] Figure 6 is a flowchart of a medical image classification method provided according to an embodiment of the present disclosure; Figure 6 The image classification method shown includes:

[0134] S601, obtaining a medical image to be classified.

[0135] The medical image to be classified may be a medical image to be classified by a target classification model.

[0136] The type of the medical image to be classified is the same as the type of the first sample image. For example, if the first sample image is a retinal image, then the image to be classified is also a retinal image.

[0137] S602, inputting the medical image into the target classification model to obtain the target category corresponding to the medical image.

[0138] Among them, the target classification model is generated by training according to the generation method of the classification model.

[0139] In some embodiments, a medical image is input into a target classification model to obtain a target category corresponding to the medical image and a position of a target object associated with the target category in the image to be classified, wherein the type of the target object is the same as the type of object associated with the target type.

[0140] For example, if the target category is category A and the object type associated with category A is category 1, then the object type to which the target object belongs is category 1.

[0141] It should be noted that, during the training process, the target classification model learns the position information of the object associated with the image category in the first sample image through the category activation map corresponding to each image category. Therefore, in the embodiment of the present disclosure, after determining the target category corresponding to the pharmaceutical image through the target classification model, the position of the target object associated with the target category in the medical image can also be determined according to the category activation map corresponding to the target category. In this way, not only the target category corresponding to the medical image can be determined, but also the position of the target object associated with the target category in the medical image can be determined, which can improve the accuracy of medical image classification and provide an intuitive classification basis.

[0142] In the disclosed embodiment, a medical image to be classified is obtained, and then the medical image is input into a target classification model to obtain a target category corresponding to the medical image. Thus, the medical image can be classified by a target classification model with object positioning capability, which provides support for image classification in the medical field, thereby improving the accuracy of classifying medical images.

[0143] Figure 7 is a structural diagram of a classification model generation device provided according to another embodiment of the present disclosure

[0144] like Figure 7 As shown, the classification model generation device 700 includes:

[0145] An acquisition module 701 is used to acquire a first data set, wherein the first data set includes a first sample image and a first category label corresponding to the first sample image;

[0146] A first processing module 702 is used to input the first sample image into the object detection model to obtain a first probability map corresponding to each object type;

[0147] The second processing module 703 is used to input the first sample image into the initial classification model to obtain a class activation map corresponding to each image class and a first predicted class corresponding to the first sample image;

[0148] The correction module 704 is used to correct the initial classification model according to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, so as to obtain a target classification model.

[0149] In some embodiments of the present disclosure, the correction module 704 is used to:

[0150] Determining a first loss value based on a difference between the first predicted category and the first category label;

[0151] Determining a second loss value based on a difference between the first probability map and the class activation map;

[0152] Determining a target loss value according to the first loss value and the second loss value;

[0153] According to the target loss value, the initial classification model is modified to obtain the target classification model.

[0154] In some embodiments of the present disclosure, the correction module 704 is used to:

[0155] Determining a first probability map corresponding to each class activation map based on the object type associated with each image class;

[0156] Determine a third loss value corresponding to each class activation map according to a difference between each class activation map and the corresponding first probability map;

[0157] The sum of at least two third loss values ​​is determined as the second loss value.

[0158] In some embodiments of the present disclosure, the correction module 704 is used to:

[0159] In the case where any class activation map corresponds to at least two first probability maps, determining the similarity between any class activation map and each corresponding first probability map;

[0160] At least two similarities are fused to determine a third loss value corresponding to any category activation map.

[0161] In some embodiments of the present disclosure, a sampling module is further included, which is used to:

[0162] In a case where the first size of the class activation map is larger than the second size of the first probability map, upsampling the first probability map based on the first size, or downsampling the class activation map based on the second size; or,

[0163] In a case where the first size of the class activation map is smaller than the second size of the first probability map, the class activation map is upsampled based on the second size, or the first probability map is downsampled based on the first size.

[0164] In some embodiments of the present disclosure, a first training module is further included for:

[0165] Acquire a second data set, wherein the second data set includes a second sample image and an object label corresponding to the second sample image, and the object label includes a location and a type of the object;

[0166] Inputting the second sample image into the initial object detection model to obtain the predicted object position and the predicted object type;

[0167] Based on the difference between the object label and the predicted object position and the predicted corresponding category, an initial object detection model is trained to obtain an object detection model.

[0168] In some embodiments of the present disclosure, a second training module is further included for:

[0169] Acquire a third data set, wherein the third data set includes a third sample image and a second category label corresponding to the third sample image;

[0170] Inputting the third sample image into a preset classification model to obtain a second predicted category corresponding to the third sample image;

[0171] Based on the difference between the second classification label and the second predicted category, the preset classification model is trained to obtain an initial classification model.

[0172] It should be noted that the aforementioned explanation of the method for generating a classification model is also applicable to the device for generating a classification model of this embodiment, and will not be repeated here.

[0173] In the disclosed embodiment, a first sample image and a first category label corresponding to the first sample image are first obtained, and then the first sample image is input into the target detection model to obtain a first probability map corresponding to each object type, and the first sample image is input into the initial classification model to obtain a category activation map corresponding to each image category and a first predicted category corresponding to the first sample image, and finally, the initial classification model is corrected according to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, to obtain the target classification model. Thus, in the process of training the target classification model, the location information of the object in the first sample image can be learned based on the difference between the category activation map and the first probability map, so that the target classification model has the ability to locate the object, and then the image can be classified more accurately based on the location information of the object, thereby improving the accuracy of the target classification model in classifying images.

[0174] Figure 8 is a structural diagram of an image classification device provided according to another embodiment of the present disclosure

[0175] like Figure 8 As shown, the image classification device 800 includes:

[0176] An acquisition module 801 is used to acquire an image to be classified;

[0177] The classification module 802 is used to input the image to be classified into the target classification model to obtain the target category corresponding to the image to be classified; wherein the target classification model is generated by training according to the training device of the classification model.

[0178] In some embodiments of the present disclosure, the classification model 802 is used to:

[0179] The image to be classified is input into the target classification model to obtain the target category corresponding to the image to be classified and the position of the target object associated with the target category in the image to be classified, wherein the type of the target object is the same as the type of the object associated with the target type.

[0180] It should be noted that the aforementioned explanation of the method for generating a classification model is also applicable to the device for generating a classification model of this embodiment, and will not be repeated here.

[0181] In the disclosed embodiment, an image to be classified is obtained, and then the image to be classified is input into a target classification model to obtain a target category corresponding to the image to be classified. Thus, the image to be classified can be classified by a target classification model with object positioning capability, thereby improving the accuracy of classifying the image to be classified.

[0182] Fig. 9 is a schematic diagram of the structure of a medical image classification device provided according to another embodiment of the present disclosure; Fig. 9 As shown, the medical image classification device 900 includes:

[0183] An acquisition module 901 is used to acquire a medical image to be classified;

[0184] The classification module 902 is used to input the medical image into the target classification model to obtain the target category corresponding to the medical image; wherein the target classification model is generated by training according to the training device of the classification model.

[0185] It should be noted that the aforementioned explanation of the method for generating a classification model is also applicable to the device for generating a classification model of this embodiment, and will not be repeated here.

[0186] In the disclosed embodiment, a medical image to be classified is obtained, and then the medical image is input into a target classification model to obtain a target category corresponding to the medical image. Thus, the medical image can be classified by a target classification model with object positioning capability, which provides support for image classification in the medical field, thereby improving the accuracy of classifying medical images.

[0187] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0188] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0189] like Fig.10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0190] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0191] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as a method for generating a classification model, or an image classification method. For example, in some embodiments, a method for generating a classification model, or an image classification method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, the method for generating a classification model described above, or one or more steps of the image classification method may be executed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the classification model generation method or the image classification method in any other appropriate manner (eg, by means of firmware).

[0192] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0193] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0194] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0195] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0196] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0197] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0198] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0199] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the description of the present disclosure, the words "if" and "if" used can be interpreted as "at the time of" or "when" or "in response to determination" or "under the circumstances of".

[0200] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a classification model, wherein: include: Acquire a first data set, wherein the first data set includes a first sample image and a first category label corresponding to the first sample image; Inputting the first sample image into an object detection model to obtain a first probability map corresponding to each object type; Inputting the first sample image into an initial classification model to obtain a class activation map corresponding to each image class and a first predicted class corresponding to the first sample image; According to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, the initial classification model is modified to obtain a target classification model.

2. The method according to claim 1, wherein: The step of modifying the initial classification model according to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, to obtain a target classification model, includes: Determining a first loss value based on a difference between the first predicted category and the first category label; Determining a second loss value based on a difference between the first probability map and the class activation map; Determining a target loss value according to the first loss value and the second loss value; According to the target loss value, the initial classification model is modified to obtain the target classification model.

3. The method according to claim 2, wherein: The determining a second loss value according to the difference between the first probability map and the class activation map comprises: Determining a first probability map corresponding to each of the category activation maps according to the object type associated with each of the image categories; Determining a third loss value corresponding to each of the class activation maps according to a difference between each of the class activation maps and the corresponding first probability map; The sum of at least two of the third loss values ​​is determined as the second loss value.

4. The method according to claim 3, wherein: Determining the third loss value corresponding to each of the class activation maps according to the difference between each of the class activation maps and the corresponding first probability map includes: In the case where any class activation map corresponds to at least two first probability maps, determining the similarity between the any class activation map and each corresponding first probability map; At least two of the similarities are merged to determine a third loss value corresponding to any one of the category activation maps.

5. The method according to claim 3, wherein: Before determining the third loss value corresponding to each of the class activation maps according to the difference between each of the class activation maps and the corresponding first probability map, the method further includes: In a case where the first size of the class activation map is larger than the second size of the first probability map, upsampling the first probability map based on the first size, or downsampling the class activation map based on the second size; or, When the first size of the class activation map is smaller than the second size of the first probability map, the class activation map is upsampled based on the second size, or the first probability map is downsampled based on the first size.

6. The method according to claim 1, wherein: Before inputting the first sample image into the target detection model to obtain a first probability map corresponding to each object type, the method further includes: Acquire a second data set, wherein the second data set includes a second sample image and an object label corresponding to the second sample image, and the object label includes a location and a type of the object; Inputting the second sample image into the initial object detection model to obtain the predicted object position and the predicted object type; Based on the difference between the object label and the predicted object position and the predicted corresponding category, the initial object detection model is trained to obtain the object detection model.

7. The method according to claim 1, wherein: Before inputting the first sample image into the initial classification model to obtain a category activation map corresponding to each image category and a first predicted category corresponding to the first sample image, the method further includes: Acquire a third data set, wherein the third data set includes a third sample image and a second category label corresponding to the third sample image; Inputting the third sample image into a preset classification model to obtain a second predicted category corresponding to the third sample image; Based on the difference between the second classification label and the second predicted category, the preset classification model is trained to obtain the initial classification model.

8. A method for image classification, wherein: include: Get the image to be classified; The image to be classified is input into a target classification model to obtain a target category corresponding to the image to be classified; wherein the target classification model is generated by training according to any one of the methods of claims 1-7.

9. The method according to claim 8, wherein: Also includes: The image to be classified is input into a target classification model to obtain a target category corresponding to the image to be classified and a position of a target object associated with the target category in the image to be classified, wherein the type of the target object is the same as the type of object associated with the target type.

10. A medical image classification method, wherein: include: Obtaining a medical image to be classified; The medical image is input into a target classification model to obtain a target category corresponding to the medical image; wherein the target classification model is generated by training according to any one of the methods of claims 1-7.

11. A device for generating a classification model, wherein: include: An acquisition module, configured to acquire a first data set, wherein the first data set includes a first sample image and a first category label corresponding to the first sample image; A first processing module, configured to input the first sample image into a target detection model to obtain a first probability map corresponding to each object type; A second processing module, used for inputting the first sample image into an initial classification model to obtain a class activation map corresponding to each image class and a first predicted class corresponding to the first sample image; A correction module is used to correct the initial classification model according to the difference between the first predicted category and the first category label, and the difference between the first probability map and the category activation map, so as to obtain a target classification model.

12. The device according to claim 11, wherein: The correction module is used to: Determining a first loss value based on a difference between the first predicted category and the first category label; Determining a second loss value based on a difference between the first probability map and the class activation map; Determining a target loss value according to the first loss value and the second loss value; According to the target loss value, the initial classification model is modified to obtain the target classification model.

13. The device according to claim 12, wherein: The correction module is used to: Determining a first probability map corresponding to each of the category activation maps according to the object type associated with each of the image categories; Determining a third loss value corresponding to each of the class activation maps according to a difference between each of the class activation maps and the corresponding first probability map; The sum of at least two of the third loss values ​​is determined as the second loss value.

14. The device according to claim 13, wherein: The correction module is used to: In the case where any class activation map corresponds to at least two first probability maps, determining the similarity between the any class activation map and each corresponding first probability map; At least two of the similarities are merged to determine a third loss value corresponding to any one of the category activation maps.

15. The device according to claim 13, wherein: Also included are sampling modules for: In a case where the first size of the class activation map is larger than the second size of the first probability map, upsampling the first probability map based on the first size, or downsampling the class activation map based on the second size; or, When the first size of the class activation map is smaller than the second size of the first probability map, the class activation map is upsampled based on the second size, or the first probability map is downsampled based on the first size.

16. The device according to claim 11, wherein Also included is a first training module for: Acquire a second data set, wherein the second data set includes a second sample image and an object label corresponding to the second sample image, and the object label includes a location and a type of the object; Inputting the second sample image into the initial object detection model to obtain the predicted object position and the predicted object type; Based on the difference between the object label and the predicted object position and the predicted corresponding category, the initial object detection model is trained to obtain the object detection model.

17. The device according to claim 11, wherein: Also included is a second training module for: Acquire a third data set, wherein the third data set includes a third sample image and a second category label corresponding to the third sample image; Inputting the third sample image into a preset classification model to obtain a second predicted category corresponding to the third sample image; Based on the difference between the second classification label and the second predicted category, the preset classification model is trained to obtain the initial classification model.

18. An image classification device, wherein: include: An acquisition module, used for acquiring images to be classified; A classification module is used to input the image to be classified into a target classification model to obtain a target category corresponding to the image to be classified; wherein the target classification model is generated by training according to any one of claims 11-17.

19. The device according to claim 18, wherein: The classification module is used to: The image to be classified is input into a target classification model to obtain a target category corresponding to the image to be classified and a position of a target object associated with the target category in the image to be classified, wherein the type of the target object is the same as the type of object associated with the target type.

20. A medical image classification device, wherein: include: An acquisition module, used for acquiring medical images to be classified; A classification module is used to input the medical image into a target classification model to obtain a target category corresponding to the medical image; wherein the target classification model is generated by training according to any one of the methods described in claims 1-7.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 7, or the method described in any one of claims 8 to 9, or the method described in claim 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method of any one of claims 1 to 7, or to execute the method of any one of claims 8 to 9, or to execute the steps of the method described in claim 10.

23. A computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in any one of claims 1 to 7, or can execute the steps of the method described in any one of claims 8 to 9, or can execute the steps of the method described in claim 10.

Images