A training method and device of an image processing model and an electronic device

Abstract

The application discloses a training method and device of an image processing model and electronic equipment. A sample image containing lesion annotation region information and a target disease category label corresponding to the sample image are acquired; the sample image is input into a preset image processing model for image processing to obtain a prediction probability value and a predicted lesion heat map; a disease probability loss value is determined based on the prediction probability value and the target disease category label; an annotation mask corresponding to the lesion annotation region is generated, and a foreground heat map region and a background heat map region are determined; a foreground loss value corresponding to the sample image is determined based on the foreground heat map region and the annotation mask; a background loss value corresponding to the sample image is determined based on the background heat map region and the annotation mask; and the preset image processing model is trained based on the disease probability loss value, the foreground loss value and the background loss value until a preset condition is met, so that the training accuracy of the image processing model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a training method and device of an image processing model and an electronic device. BACKGROUND

[0002] With the rapid development of computer technology, an image processing model based on deep learning can be used for image classification.

[0003] At present, before using an image processing model, the image processing model needs to be trained using sample images, so that the image processing model after training can accurately classify images.

[0004] However, in the existing image processing model training process, a large amount of non-key information is contained in the sample images, and training only using the probability loss value of the classification result will result in low accuracy of the image processing model training. SUMMARY

[0005] The embodiments of the present application provide a training method and device of an image processing model and an electronic device to improve the training accuracy of the image processing model and ensure the high performance of the image processing model.

[0006] In a first aspect, the present application provides a training method of an image processing model, which comprises:

[0007] obtaining a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image;

[0008] inputting the sample image into a preset image processing model to be trained for image processing, to obtain a predicted probability value of the sample image being a target disease category and a predicted lesion heat map;

[0009] determining a disease probability loss value corresponding to the sample image based on the predicted probability value and the target disease category label;

[0010] generating a label mask corresponding to the lesion annotation region based on the lesion annotation region information in the sample image, and determining a foreground heat map region and a background heat map region in the predicted lesion heat map based on the label mask;

[0011] determining a foreground loss value corresponding to the sample image based on the foreground heat map region and the label mask;

[0012] determining a background loss value corresponding to the sample image based on the background heat map region and the label mask;

[0013] The preset image processing model is trained based on the disease probability loss value, the foreground loss value and the background loss value until a preset condition is met.

[0014] In a second aspect, the present application provides a training device of an image processing model, which comprises:

[0015] A sample acquisition module is configured to acquire a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image.

[0016] A prediction result determination module is configured to input the sample image into a preset image processing model to be trained for image processing, so as to obtain a prediction probability value of the sample image as a target disease category and a predicted lesion heat map.

[0017] A probability loss value determination module is configured to determine a disease probability loss value corresponding to the sample image based on the prediction probability value and the target disease category label.

[0018] A heat map region determination module is configured to generate an annotation mask corresponding to the lesion annotation region based on the lesion annotation region information in the sample image, and determine a foreground heat map region and a background heat map region in the predicted lesion heat map based on the annotation mask.

[0019] A foreground loss value determination module is configured to determine a foreground loss value corresponding to the sample image based on the foreground heat map region and the annotation mask.

[0020] A background loss value determination module is configured to determine a background loss value corresponding to the sample image based on the background heat map region and the annotation mask.

[0021] A model training end module is configured to train the preset image processing model based on the disease probability loss value, the foreground loss value and the background loss value until a preset condition is met.

[0022] In a third aspect, the present application provides an electronic device, which comprises:

[0023] at least one processor; and

[0024] a memory in communication with the at least one processor; wherein

[0025] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the training method of the image processing model according to any one of the embodiments of the present application.

[0026] In a fourth aspect, the present application provides a computer readable storage medium storing computer instructions for causing a processor to implement the training method of the image processing model according to any of the embodiments of the present application.

[0027] The technical scheme provided by the embodiment of the present application is to obtain a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image, then input the sample image into a preset image processing model to be trained for image processing, obtain a prediction probability value and a predicted lesion heat map, and further determine a disease probability loss value corresponding to the sample image based on the prediction probability value and the disease category label. Based on the lesion annotation region information in the sample image, a label mask corresponding to the lesion annotation region is generated, and based on the label mask, a foreground heat map region and a background heat map region in the predicted lesion heat map are determined, and then based on the foreground heat map region and the label mask, a foreground loss value corresponding to the sample image is determined; based on the background heat map region and the label mask, a background loss value corresponding to the sample image is determined. Based on the disease probability loss value, the foreground loss value and the background loss value, the preset image processing model is trained until the preset condition is met, so as to realize the model training of the image processing model. By respectively determining the disease probability loss value, the foreground loss value and the background loss value, the image processing model parameters are optimized based on multiple loss values, the problem of low training accuracy of the image processing model is solved, the training accuracy of the image processing model is improved, and the high performance of the image processing model is ensured.

[0028] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0030] Figure 1 A flowchart of the training method of the image processing model provided for the first embodiment of the present application;

[0031] Figure 2 A schematic diagram of the lesion annotation region information related to the first embodiment of the present application;

[0032] Figure 3 A structure schematic diagram of ResNet related to the first embodiment of the present application;

[0033] Figure 4 is a residual block structure diagram related to the embodiment one of the present application;

[0034] Figure 5 is a label mask schematic diagram related to the embodiment one of the present application;

[0035] Figure 6 is a ROC curve schematic diagram for evaluating the image processing model related to the embodiment one of the present application;

[0036] Figure 7 is a flowchart of the training method of the image processing model provided by the embodiment two of the present application;

[0037] Figure 8 is a schematic diagram of the sample feature map corresponding to the sample image related to the embodiment two of the present application;

[0038] Figure 9 is a schematic diagram of the sample feature map obtaining a logical value related to the embodiment two of the present application;

[0039] Figure 10 is a schematic diagram of the sample feature map obtaining a sample logical map related to the embodiment two of the present application;

[0040] Figure 11 is a flowchart of the training method of the image processing model provided by the embodiment three of the present application;

[0041] Figure 12 is a structure schematic diagram of the training device of the image processing model provided by the embodiment four of the present application;

[0042] Figure 13 is a structure schematic diagram of the device provided by the embodiment five of the present application. DETAILED DESCRIPTION

[0043] In order to make the personnel in the art better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0044] It should be noted that the terms "first preset condition", "second preset condition" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0045] Embodiment one

[0046] Figure 1 A flowchart of a training method of an image processing model provided for the first embodiment of the present application, the present embodiment can be applicable to the case of training an image processing model based on sample images. The method can be performed by a training device of the image processing model, which can be realized in the form of hardware and / or software, and can be configured on a computer device, which can be a notebook, a desktop computer, and a smart tablet, etc. As shown in Figure 1 the method comprises:

[0047] S110, obtaining a sample image containing lesion annotation region information and a target disease category annotation corresponding to the sample image.

[0048] Among them, the sample image is an image containing a lesion. The number of sample images is one or more. Illustratively, the sample image can be a previously scanned patient chest frontal image.

[0049] Among them, the lesion annotation region information is the information of annotating the lesion part in the sample image. Illustratively, the lesion annotation region information schematic diagram is shown in Figure 2 As shown in Figure 2 Region A is the lesion region, and the doctor annotates the corresponding lesion region in the sample image. The lesion annotation region information can be JSON format information (specifically, a JSON file) that determines whether there is a lesion region (such as pulmonary tuberculosis) in the sample image. The JSON file contains a polygon lesion frame that outlines and annotates the lesion region in the image, which is composed of multiple sets of coordinate point lists, that is, the coordinate point information of the frame.

[0050] The target disease category is one of the pre-set image categories. For example, the pre-set image categories include a region with a lesion and a region without a lesion, and the target disease category is the region with a lesion. The disease category label is a probability value of the sample image being the target disease category. Generally, the disease category label of the sample image corresponding to the region with a lesion is set to 1, which can be understood as the probability of the pre-set sample image belonging to the target disease category being 100%. Correspondingly, the disease category label of the sample image corresponding to the region without a lesion is set to 0.

[0051] Specifically, the sample image can be an image of a historical patient obtained from an image database of a hospital or other channels. The lesion annotation region information can be information annotated by a doctor on the sample image. For example, the doctor can use a circle selection tool to frame the lesion part on the sample image, and the coordinates corresponding to the framed region are stored in the format of a JSON table. The target disease category label corresponding to the sample image is a pre-set value, which can be stored and retrieved in the form of a numerical value.

[0052] In this embodiment, after obtaining the sample image, the sample image needs to be pre-processed. Specifically, first, the data format of the sample image is converted to a target data format required by the image processing model, then the sample image after format conversion is subjected to histogram equalization, and the sample image after equalization is scaled to a target size. Further, the number of channels of the scaled sample image is converted to a target number of channels required by the image processing model, and finally the pixel value of each pixel point of the sample image after channel conversion is subtracted by a pre-set pixel value, thereby obtaining a pre-processed sample image.

[0053] The target data format can be a data format required by a pre-set image processing model for inputting an image inside it. The target size can be a size of an image required by a pre-set image processing model for inputting an image inside it. The target number of channels can be a number of channels required by a pre-set image processing model for inputting an image inside it. The pre-set pixel value can be a pre-set pixel value, which can be in the middle value range of the pixel value. For example, the pixel value of the image ranges from 0 to 255, and the pre-set pixel value can be 125.

[0054] In the embodiment, the sample image is subjected to histogram equalization to adjust the contrast of the sample image after format conversion, so that the contrast is clear and the image is clear. It should be noted that whether the histogram equalization is required or not can be set by the user, because the contrast of some sample images is good and does not require histogram equalization. Therefore, the sample image after format conversion can also not be subjected to histogram equalization. The specific selection can be made according to the user's demand, which is not limited here. The pixel value of each pixel point on the sample image after channel conversion is subtracted by the preset pixel value, so that the mean value of the pixels of the sample image after channel conversion is near 0, preventing the problem of excessive gradient of the parameters during deep learning optimization due to the excessive mean value of the image pixels.

[0055] It should be noted that a general sample image is rectangular and has a long side and a short side. Therefore, the sample image input into the preset image processing model needs to be square, and the short side needs to be padded. After the padding operation is completed, the sample image needs to be scaled to ensure that the size of all images input into the preset image processing model is consistent.

[0056] In the embodiment, the sample image is preprocessed, so that better image data meeting the requirements of the preset image processing model is obtained, thereby ensuring the training quality of the image processing model.

[0057] S120, inputting the sample image into the preset image processing model to be trained for image processing, to obtain a prediction probability value and a prediction lesion heat map of the sample image for the target disease category.

[0058] The preset image processing model can be any neural network model, and the model parameters in the preset image processing model are initial values, which need to be corrected in the model training process. For example, the preset image processing model can be a classic classification convolutional neural network model (Deep Residual Network, ResNet34) with residual learning capability and a FPN model in the CNN model, such as Figure 3 the structure diagram of the ResNet, and Figure 4 the residual block structure diagram. In the ResNet network structure, Figure 3 it is mainly composed of 4 stages, each stage is composed of a plurality of residual blocks, and the residual block structure is as shown in Figure 4 .

[0059] The prediction probability value is the model operation result output by the preset image processing model after processing the sample image, and the prediction probability value is a value between 0 and 1. The prediction lesion heat map is the lesion region heat map output by the preset image processing model after processing the sample image.

[0060] Specifically, the preset image processing model comprises a feature map extraction sub-model, a probability value prediction sub-model, and a heat map generation sub-model. The sample image is input into the preset image processing model, and first, the feature map extraction sub-model processes the sample image to obtain a feature map corresponding to the sample image. Then, the feature map is input into the probability value prediction sub-model and the heat map generation sub-model. The probability value prediction sub-model processes the feature map to output a predicted probability value of the sample image belonging to a target disease category. The heat map generation sub-model processes the feature map to output a predicted lesion heat map corresponding to the sample image.

[0061] S130, determining a disease probability loss value corresponding to the sample image based on the predicted probability value and the target disease category label.

[0062] The disease probability loss value is used to represent the difference between the predicted probability value and the target disease category label. For example, the disease probability loss value can be a cross-entropy loss between the predicted probability value and the target disease category label.

[0063] Specifically, the preset image processing model processes the sample image to obtain the predicted probability value, and the target disease category label is a predefined value. Based on this, the cross-entropy loss between the predicted probability value and the target disease category label is calculated, and the disease probability loss value corresponding to the sample image can be determined. For example, the disease category label is 1, and the predicted probability value is 0.8. The probability loss value can be 0.2231.

[0064] S140, generating a label mask corresponding to the lesion label region based on the lesion label region information in the sample image, and determining a foreground heat map region and a background heat map region in the predicted lesion heat map based on the label mask.

[0065] The label mask is a binary image generated based on a preset algorithm and the lesion label region information in the sample image.

[0066] In this embodiment, the doctor can determine the lesion label region information of the sample image after labeling the lesion region. Through a preset mask conversion algorithm, a label mask corresponding to the sample image containing the doctor's label information can be generated (for example, the region A in FIG. 6 is the lesion region). Figure 5 , Figure 5 The region A in FIG. 6 is the lesion region.

[0067] Optionally, the determining the foreground heat map region and the background heat map region in the predicted lesion heat map specifically comprises: regarding a region composed of each pixel point in the lesion annotation region in the annotation mask as a foreground mask region, and regarding a region composed of each pixel point outside the lesion annotation region in the annotation mask as a background mask region; in the predicted lesion heat map, a region composed of pixel points corresponding to the foreground mask region is determined as the foreground heat map region, and a region composed of pixel points corresponding to the background mask region is determined as the background heat map region.

[0068] The predicted lesion heat map and the annotation mask have the same size.

[0069] In the annotation mask, the pixel value corresponding to each pixel point in the lesion annotation region is marked as 1, which is called a foreground mask region; the pixel value corresponding to each pixel point in all other regions except the lesion annotation region is marked as 0, which is called a foreground mask region. Since the size of the annotation mask and the predicted lesion heat map is completely the same, the pixel points contained in the annotation mask and the predicted lesion heat map are also the same. For each pixel point in the annotation mask, there is a pixel point corresponding to it in the predicted lesion heat map. Based on this, in the predicted lesion heat map, the region composed of pixel points corresponding to the foreground mask region can be determined as the foreground heat map region, and the region composed of pixel points corresponding to the background mask region can be determined as the background heat map region.

[0070] S150, determining a foreground loss value corresponding to the sample image based on the foreground heat map region and the annotation mask.

[0071] The foreground loss value is used to represent the difference between the heat point prediction probability value in the foreground heat map region and the marked value in the lesion annotation region in the mask.

[0072] Specifically, when determining the foreground loss value corresponding to the sample image, a preset number of maximum heat point prediction probability values are selected from the multiple heat point prediction probability values in the foreground heat map region as target heat point prediction probability values. Subsequently, each target heat point prediction probability value and the marked value 1 of the lesion annotation region in the annotation mask are cross-entropy loss calculated to obtain a preset number of candidate loss values. Subsequently, the preset number of candidate loss values are averaged to obtain the foreground loss value corresponding to the sample image.

[0073] S160, determining a background loss value corresponding to the sample image based on the background heat map region and the annotation mask.

[0074] On the basis of the above embodiments, the background loss value represents the difference between the heat point prediction probability value in the background heat map region and the marked value in the non-lesion position in the annotation mask.

[0075] Specifically, when determining the background loss value corresponding to the sample image, a preset number of maximum heat point prediction probability values are selected from the plurality of heat point prediction probability values in the background heat map region as target heat point prediction probability values. Subsequently, each target heat point prediction probability value is cross-entropy loss calculated with the label value 0 of the non-lesion label region in the label mask to obtain a preset number of candidate loss values. Subsequently, the preset number of candidate loss values are averaged to obtain the background loss value corresponding to the sample image.

[0076] S170, based on the disease probability loss value, the foreground loss value and the background loss value, the preset image processing model is trained until the preset condition is met.

[0077] The preset condition is a condition preset in advance. When the training result meets the preset condition during the training of the preset image processing model, it indicates that the preset image processing model is trained.

[0078] Optionally, the preset condition includes at least one of the following situations: the number of training iterations reaches a preset iteration threshold; the disease probability loss value, the foreground loss value and the background loss value are respectively less than the corresponding loss threshold; a comprehensive loss value is determined by the disease probability loss value, the foreground loss value and the background loss value, and the comprehensive loss value is less than a comprehensive loss threshold.

[0079] In the embodiment of the present application, the disease probability loss value, the foreground loss value and the background loss value jointly optimize the parameters of the preset image processing model. Each time the sample image is input into the preset image processing model, a set of disease probability loss value, foreground loss value and background loss value can be determined, and each time the determined disease probability loss value, foreground loss value and background loss value can be fed back to the preset image processing model to optimize the parameters in the preset image processing model. In addition, a comprehensive loss value can be determined by the disease probability loss value, the foreground loss value and the background loss value, and the comprehensive loss value is fed back to the preset image processing model to optimize the parameters in the preset image processing model. Subsequently, the preset image processing model after model optimization processes the next set of sample images, and a preset iteration threshold can be set. When the number of times the preset image processing model is trained based on the sample image reaches the iteration threshold, and the disease probability loss value, the foreground loss value and the background loss value are respectively less than the corresponding loss threshold, it can be proved that the image processing model is trained. In another embodiment, when the number of times the preset image processing model is trained based on the sample image reaches the iteration threshold, and the comprehensive loss value is less than the loss threshold, it can be proved that the image processing model is trained.

[0080] In the embodiment of the present application, after the preset image processing model is trained, the trained image processing model can also be evaluated to determine whether it is feasible.

[0081] Specifically, when evaluating the trained image processing model, the evaluation indicators that can be used include accuracy, sensitivity, and specificity. When the sensitivity is greater than a preset sensitivity threshold, and the specificity is greater than a preset specificity threshold, for example, the sensitivity is greater than or equal to 90%, and the specificity is greater than 80%, it can be considered that the evaluation effect of the image processing model is good, and the image processing model can be well used for subsequent image processing. The training method is as follows: The historical images are divided into a training set and a verification set in a ratio of 9:1, and the verification set is used to verify the image processing model. Specifically, the ROC curve shown in the figure can be used to evaluate the image processing model. Figure 6

[0082] The technical scheme provided by the embodiment of the application obtains a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image, then inputs the sample image into a preset image processing model to be trained for image processing, obtains a prediction probability value and a predicted lesion heat map, and further determines a disease probability loss value corresponding to the sample image based on the prediction probability value and the disease category label. Based on the lesion annotation region information in the sample image, a label mask corresponding to the lesion annotation region is generated, the foreground heat map region and the background heat map region in the predicted lesion heat map are determined based on the label mask, and then the foreground loss value corresponding to the sample image is determined based on the foreground heat map region and the label mask; the background loss value corresponding to the sample image is determined based on the background heat map region and the label mask. The preset image processing model is trained based on the disease probability loss value, the foreground loss value, and the background loss value until the training ends when the preset condition is met, thereby realizing the model training of the image processing model. By respectively determining the disease probability loss value, the foreground loss value, and the background loss value, the image processing model parameters are optimized based on multiple loss values, the problem of low training accuracy of the image processing model is solved, the training accuracy of the image processing model is improved, and the high performance of the image processing model is ensured.

[0083] On the basis of the above embodiment, S170 specifically includes the following steps: determining a comprehensive loss value based on the disease probability loss value, the foreground loss value, and the background loss value; and propagating the comprehensive loss value to the preset image processing model in reverse to adjust the network parameters in the preset image processing model until the training ends when the preset condition is met.

[0084] The comprehensive loss value is the sum of the disease probability loss value, the foreground loss value, and the background loss value.

[0085] ​Specifically, the preset image processing model can determine a set of disease probability loss values, foreground loss values and background loss values after processing a set of sample images. First, it is judged whether the preset image processing model at this time meets the preset condition. If the preset condition is met, the training is ended. If the preset condition is not met, the determined disease probability loss values, foreground loss values and background loss values are summed to determine a comprehensive loss value, and the comprehensive loss value is fed back to the preset image processing model. The preset image processing model optimizes the model parameters based on the comprehensive loss value, and then processes the next set of sample images based on the preset image processing model after parameter optimization. In this way, the training is iterated until the preset condition is met and the training is ended.

[0086] Embodiment Two

[0087] Figure 7 A flowchart of a training method of an image processing model provided for embodiment two of the present application. Based on the above-mentioned embodiments, the preset image processing model can include a feature map extraction sub-model, a probability value prediction sub-model and a heat map generation sub-model. And on this basis, the step of "inputting the sample image into the preset image processing model to be trained for image processing to obtain the predicted probability value of the sample image for the target disease category and the predicted lesion heat map" is further optimized. The present application can be combined with one or more of the above-mentioned embodiments. As shown in Figure 7 , the method comprises:

[0088] S210, obtaining a sample image containing lesion annotation region information and a target disease category annotation corresponding to the sample image.

[0089] S220, inputting the sample image into the feature map extraction sub-model for feature extraction to obtain a sample feature map corresponding to the sample image.

[0090] The feature map extraction sub-model is a deep learning model. The sample feature map (featmap) can be an image obtained by feature extraction on the sample image.

[0091] In the present embodiment, the preset image processing model includes a feature map extraction sub-model, a probability value prediction sub-model and a heat map generation sub-model. Here, feature extraction on the sample image can be realized based on the feature map extraction sub-model, so that the feature map corresponding to the sample image can be quickly obtained.

[0092] Specifically, the schematic diagram of obtaining the sample feature map corresponding to the sample image is shown in Figure 8 . As shown in Figure 8 , the feature map extraction sub-model is a deep learning model. Figure 8The leftmost image in the diagram is a sample image. The specific feature extraction process can be as follows: The sample image is a single-channel image of a certain size, i.e., a two-dimensional matrix image with a shape of H*W, where H and W represent the dimensions of the sample image. After inputting the sample image into the feature map extraction sub-model, a multi-dimensional matrix image is obtained, i.e., the sample feature map. The sample feature map is a multi-dimensional matrix with a shape of C*h*w, where C represents the number of channels, and h and w represent the dimensions of the feature map. Figure 8 As can be seen, the number of channels increases with the increase of model depth, and the dimensions corresponding to h and w decrease (generally h == w, and they are also equal in the feature map extraction sub-model in this embodiment of the invention).

[0093] It should be noted that, as Figure 8 As shown, the values ​​H and W of the acquired sample image are not consistent with the values ​​h and w of the features obtained after feature extraction. Generally, the size of the sample feature map is smaller than the size of the sample. In this embodiment of the invention, h = H / 32, w = W / 32, that is, the size of the sample feature map is reduced by a factor of 32 compared to the sample image.

[0094] S230. Input the sample feature map into the probability value prediction sub-model for classification prediction to obtain the predicted probability value of the sample image belonging to the target disease category.

[0095] The probability value prediction sub-model determines the predicted probability of a sample image belonging to a target disease category based on the feature values ​​of each channel in the sample feature map. The probability value prediction sub-model includes a fully pooled layer and a fully connected layer.

[0096] In this embodiment, after obtaining the sample feature map corresponding to the sample image, the sample feature map is input into the probability value prediction sub-model. The probability value prediction sub-model calculates the feature values ​​of each channel in the sample feature map to obtain the target feature value of each channel. Then, the target feature values ​​are weighted and summed to obtain the logical value corresponding to the sample feature map. Based on this logical value, the predicted probability value of the sample image belonging to the target disease category is determined.

[0097] Specifically, see the diagram illustrating the process of obtaining logical values ​​from sample feature maps. Figure 9 . Figure 9 The example below uses four channels. Figure 9 The topmost image in the image is the sample feature map (this image is...). Figure 8 A simplified diagram of the sample feature map generated in the process, used to explain the calculation of the logical value, but it is not a complete representation. Figure 9 (in the form of) Figure 9 The sample feature map in the image uses rectangles from right to left to represent one channel, such as... Figure 9The feature values on the rightmost channel are calculated through the global pooling layer to obtain the maximum value (or average value) of the feature values on the channel, that is, the value of block 1 (the target feature value of the rightmost channel), the feature values on the second channel from the right are calculated through the global pooling layer to obtain the value of block 2 (the target feature value of the second channel from the right), and so on, to obtain the target feature values of the channels (such as Figure 9 The values of block 1, block 2, block 3 and block 4 in the global pooling layer), and the target feature values of the channels are respectively weighted and summed to obtain the logical value corresponding to the sample feature map (that is, the value of block 5).

[0098] It should be noted that whether the maximum value or the average value of the feature values on each channel is obtained when the global pooling layer is calculated is determined according to the structure of the global pooling layer. If the structure of the global pooling layer is the maximum value structure, the maximum value of the feature values on each channel is obtained, and if the structure of the global pooling layer is the average value structure, the average value of the feature values on each channel is obtained. The structure of the global pooling layer can be set by the user according to the user's needs, which is not limited here.

[0099] After obtaining the logical value corresponding to the sample feature map, the logical value is normalized by using a sigmoid function, and a value between 0 and 1 is obtained, which is the prediction probability value of the sample image belonging to the target disease category.

[0100] S240, input the sample feature map into the heat map generation sub-model to generate a heat map, and obtain a prediction lesion heat map corresponding to the sample image.

[0101] The heat map generation sub-model is used to determine the prediction lesion heat map corresponding to the sample image according to the feature values of each channel of the sample feature map.

[0102] In this embodiment, the heat map generation sub-model includes a logical map extraction unit and a heat map generation unit. The main structure in the logical map extraction unit is a fully connected layer. The heat map generation unit includes a heat map rendering module.

[0103] In this embodiment, after obtaining the sample feature map corresponding to the sample image, the sample feature map is input into the heat map generation sub-model, the logical map extraction unit in the heat map generation sub-model performs channel compression on the sample feature map to obtain a channel-compressed sample logical map. Then, the sample logical map is input into the heat map generation unit to further determine the logical point prediction probability value corresponding to each logical point in the sample logical map, and generate a prediction lesion heat map corresponding to the sample image according to the logical point prediction probability values.

[0104] On the basis of the above-mentioned embodiments, S240 specifically comprises the following steps: inputting the sample feature map into a logical graph extraction unit for logical graph generation to obtain a sample logical graph corresponding to the sample feature map; inputting the sample logical graph into a heat map generation unit for probability prediction to obtain a logical point prediction probability value corresponding to each logical point in the sample logical graph, and generating a predicted lesion heat map corresponding to the sample image according to the logical point prediction probability values.

[0105] The sample logical graph can be an image formed after channel compression of the sample feature map. The logical point prediction probability value represents a prediction probability value of a region in the sample image corresponding to the logical point belonging to the target disease category.

[0106] Specifically, the sample feature map is input into the logical graph extraction unit for logical graph generation to obtain a sample logical graph corresponding to the sample feature map. For a schematic diagram of obtaining the sample logical graph according to the sample feature map, refer to Figure 10 . The leftmost image in Figure 10 is taken as an example of four channels, Figure 10 The sample feature map in Figure 10 is taken as an example of four channels, Figure 10 The first channel, the second channel, the third channel and the fourth channel. The first data on each channel (i.e. the data at the upper left corner of each channel, such as the data of the dashed line in Figure 10 ) is calculated by weighted summation to obtain the first block (block M) in the subsequent sample feature map. By analogy, the corresponding data of each channel is calculated by weighted summation to obtain the subsequent sample logical graph. In the embodiment of the present application, each small block in the sample logical graph represents a small region in the sample image.

[0107] When the sample logical graph is obtained, the sample logical graph is input into the heat map generation unit for probability prediction to obtain a logical point prediction probability value corresponding to each logical point in the sample logical graph. The sigmoid function is calculated for each logical point (i.e. each small block in Figure 10 ) in the sample logical graph to obtain the logical point prediction probability value. Further, the predicted lesion heat map corresponding to the sample image is generated according to the logical point prediction probability value.

[0108] In the embodiment of the present application, the predicted lesion heat map can be drawn by the heat map rendering module according to the logical point prediction probability value. In the process of drawing the predicted lesion heat map, the higher the probability value of the region, the redder the color of the region.

[0109] In another embodiment, in addition to obtaining the logical value by the above-mentioned Figure 9 , the logical value can also be obtained by the sample logical graph. Specifically, theFigure 10 The maximum value in the sample logical graph obtained in the above manner is taken as a logical value.

[0110] The above two ways of obtaining a logical value can be selected according to user requirements, and are not limited here.

[0111] S250, determining a disease probability loss value corresponding to the sample image based on the prediction probability value and the target disease category label.

[0112] S260, generating a label mask corresponding to the lesion label region based on the lesion label region information in the sample image, and determining a foreground heat map region and a background heat map region in the predicted lesion heat map based on the label mask.

[0113] S270, determining a foreground loss value corresponding to the sample image based on the foreground heat map region and the label mask.

[0114] S280, determining a background loss value corresponding to the sample image based on the background heat map region and the label mask.

[0115] S290, training the preset image processing model based on the disease probability loss value, the foreground loss value, and the background loss value, and ending the training when a preset condition is met.

[0116] The technical scheme provided by the embodiment of the application, when obtaining the prediction probability value and the predicted lesion heat map, first inputs the sample image into the feature map extraction sub-model for feature extraction to obtain a sample feature map corresponding to the sample image, and then inputs the sample image into the preset image processing model, and the feature map extraction sub-model processes the sample image to obtain a feature map corresponding to the sample image. Subsequently, the feature map is input into the probability value prediction sub-model and the heat map generation sub-model, respectively. After the probability value prediction sub-model processes the feature map, a prediction probability value of the sample image belonging to the target disease category is output. After the heat map generation sub-model processes the feature map, a predicted lesion heat map corresponding to the sample image is output. The preset image processing model includes multiple processing sub-units, and the prediction probability value and the predicted lesion heat map corresponding to the sample image can be quickly and accurately obtained.

[0117] Embodiment three

[0118] Figure 11 The flowchart of the training method of the image processing model provided by the third embodiment of the application, the embodiment of the application further optimizes the steps of "determining a foreground loss value corresponding to the sample image based on the foreground heat map region and the label mask" and "determining a background loss value corresponding to the sample image based on the background heat map region and the label mask" based on the above embodiments, and the embodiment of the application can be combined with one or more optional schemes in the above embodiments. For example, Figure 11As shown, the method comprises:

[0119] S310, acquire a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image.

[0120] S320, input the sample image into a preset image processing model to be trained for image processing, and obtain a prediction probability value of the sample image for the target disease category and a predicted lesion heat map.

[0121] S330, based on the prediction probability value and the target disease category label, determine a disease probability loss value corresponding to the sample image.

[0122] S340, based on the lesion annotation region information in the sample image, generate an annotation mask corresponding to the lesion annotation region, and determine a foreground heat map region and a background heat map region in the predicted lesion heat map based on the annotation mask.

[0123] S350, determine a foreground marker value corresponding to a foreground mask region in the annotation mask.

[0124] In the annotation mask, the pixel value corresponding to the pixel point located in the lesion annotation region is set to 1, and the region composed of each pixel point in the lesion annotation region is called the foreground mask region; the pixel value corresponding to the pixel point located outside the lesion annotation region is set to 0, and the region composed of each pixel point outside the lesion annotation region is called the background mask region. Based on this, the marker value 1 corresponding to the pixel point in the foreground mask region in the annotation mask is the foreground marker value.

[0125] S360, based on the heat point prediction probability value corresponding to each heat point in the foreground heat map region and the foreground marker value, determine a foreground loss value corresponding to the sample image.

[0126] In this embodiment, after determining the heat point prediction probability value corresponding to each heat point in the foreground heat map region, a preset number of heat point prediction probability values with large values can be determined from a plurality of heat point prediction probability values, and based on the preset number of heat point prediction probability values and the foreground marker value, the foreground loss value corresponding to the sample image is determined.

[0127] Optionally, S360 can specifically include the following steps: arranging the heat point prediction probability value corresponding to each heat point in the foreground heat map region in descending order, determining the first preset number of heat point prediction probability values after arrangement as target heat point prediction probability values; based on the target heat point prediction probability value and the foreground marker value, determine a candidate loss value corresponding to each target heat point; take the average of each said candidate loss value to determine the foreground loss value corresponding to the sample image.

[0128] The to-be-selected loss value is a loss determined by cross-entropy loss calculation of the heat point prediction probability value corresponding to each target heat point and the foreground label value.

[0129] In a specific implementation, the loss calculation of each target heat point adopts a BCE loss function, and the BCE loss function is as follows:

[0130] CE(p t )=-log(p t )wherein,

[0131] CE is cross entropy, y is the doctor's annotation information (the foreground label value is 1, and the background label value is 0), and p is the heat point prediction probability value corresponding to the target heat point.

[0132] For example, the foreground heat map region includes 10 heat points, and the heat point prediction probability values corresponding to the heat points in the foreground heat map region are arranged in descending order, which can be represented as: 0.99, 0.98, 0.96, 0.95, 0.95, 0.94, 0.93, 0.91, 0.91, and 0.9. If the preset number is 3, the target heat point prediction probability values are 0.99, 0.98, and 0.96. Further, the target heat point prediction probability values are respectively subjected to cross-entropy loss calculation with the foreground label value 1 to determine the to-be-processed loss values corresponding to each target heat point. Finally, the average of the to-be-processed loss values is obtained, and the foreground loss value corresponding to the sample image is obtained.

[0133] S370, determining the background label value corresponding to the background mask region in the annotation mask.

[0134] In the annotation mask, the pixel value of a pixel point located outside the lesion annotation region is set to 0, and a region composed of the pixel points outside the lesion annotation region is referred to as a background mask region. Based on this, the label value 0 of the pixel point in the background mask region in the annotation mask is a background label value.

[0135] S380, determining the background loss value corresponding to the sample image based on the heat point prediction probability value corresponding to each heat point in the background heat map region and the background label value.

[0136] In this embodiment, the manner of determining the background loss value is the same as the manner of determining the foreground loss value in S360, and the difference lies in that the foreground label value is 1, and the background label value is 0.

[0137] Optionally, S380 can specifically include the following steps: arranging the heat point prediction probability values corresponding to each heat point in the background heat map region in descending order, determining the target heat point prediction probability values as the top pre-set number of heat point prediction probability values after arrangement; determining the candidate loss values corresponding to each target heat point based on the target heat point prediction probability values and the background label value; and determining the background loss value corresponding to the sample image by averaging the candidate loss values.

[0138] S390, training the preset image processing model based on the disease probability loss value, the foreground loss value and the background loss value, and ending the training when a preset condition is met.

[0139] The technical scheme provided by the embodiment of the present application determines the foreground loss value and the background loss value respectively when determining the loss value of the heat map of the lesion region. When determining the foreground loss value corresponding to the sample image, the foreground label value 1 at the lesion position in the annotation mask is used as the training label; when determining the background loss value corresponding to the sample image, the background label value 0 at the non-lesion position in the annotation mask is used as the training label. In this way, the image processing model after training can reduce the false positive rate, further improve the training accuracy of the image processing model, and ensure the high performance of the image processing model.

[0140] Embodiment four

[0141] Figure 12 A structural schematic diagram of an image processing model training device provided by the fourth embodiment of the present application is provided. The device can execute the image processing model training method provided by the embodiment of the present application. The device includes a sample acquisition module 410, a prediction result determination module 420, a probability loss value determination module 430, a heat map region determination module 440, a foreground loss value determination module 450, a background loss value determination module 460 and a model training end module 470.

[0142] The sample acquisition module 410 is configured to acquire a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image.

[0143] The prediction result determination module 420 is configured to input the sample image into a preset image processing model to be trained for image processing, and obtain a prediction probability value of the sample image being a target disease category and a predicted lesion heat map.

[0144] The probability loss value determination module 430 is configured to determine a disease probability loss value corresponding to the sample image based on the prediction probability value and the target disease category label.

[0145] The heat map region determination module 440 is configured to generate a label mask corresponding to the lesion label region based on the lesion label region information in the sample image, and determine a foreground heat map region and a background heat map region in the predicted lesion heat map based on the label mask.

[0146] The foreground loss value determination module 450 is configured to determine a foreground loss value corresponding to the sample image based on the foreground heat map region and the label mask.

[0147] The background loss value determination module 460 is configured to determine a background loss value corresponding to the sample image based on the background heat map region and the label mask.

[0148] The model training end module 470 is configured to train the preset image processing model based on the disease probability loss value, the foreground loss value and the background loss value, and the training ends when a preset condition is met.

[0149] On the basis of the above technical solutions, the prediction result determination module 420 comprises:

[0150] The feature map extraction unit is configured to input the sample image into the feature map extraction sub-model to perform feature extraction, and obtain a sample feature map corresponding to the sample image.

[0151] The probability value prediction unit is configured to input the sample feature map into the probability value prediction sub-model to perform classification prediction, and obtain a prediction probability value of the sample image belonging to the target disease category.

[0152] The heat map generation unit is configured to input the sample feature map into the heat map generation sub-model to perform heat map generation, and obtain a predicted lesion heat map corresponding to the sample image.

[0153] On the basis of the above technical solutions, the heat map generation unit comprises:

[0154] The logical graph extraction sub-unit is configured to input the sample feature map into the logical graph extraction unit to perform logical graph generation, and obtain a sample logical graph corresponding to the sample feature map.

[0155] The heat map generation sub-unit is configured to input the sample logical graph into the heat map generation unit to perform probability prediction, and obtain a logical point prediction probability value corresponding to each logical point in the sample logical graph; and generate a predicted lesion heat map corresponding to the sample image according to the logical point prediction probability values.

[0156] On the basis of each of the technical solutions above, the heat map region determination module 440 is further configured to: regard a region composed of each pixel point in a lesion annotation region in the annotation mask as a foreground mask region, and regard a region composed of each pixel point outside the lesion annotation region in the annotation mask as a background mask region; regard a region composed of pixel points corresponding to the foreground mask region in the predicted lesion heat map as a foreground heat map region, and regard a region composed of pixel points corresponding to the background mask region in the predicted lesion heat map as a background heat map region; wherein the predicted lesion heat map and the annotation mask have the same size.

[0157] On the basis of each of the technical solutions above, the foreground loss value determination module 450 includes:

[0158] a foreground label value determination unit configured to determine a foreground label value corresponding to the foreground mask region in the annotation mask;

[0159] a foreground loss value determination unit configured to determine a foreground loss value corresponding to the sample image based on a heat point prediction probability value corresponding to each heat point in the foreground heat map region and the foreground label value.

[0160] On the basis of each of the technical solutions above, the foreground loss value determination unit includes:

[0161] a target prediction probability value determination sub-unit configured to arrange heat point prediction probability values corresponding to each heat point in the foreground heat map region in descending order, and determine a target heat point prediction probability value from the top pre-set number of arranged heat point prediction probability values;

[0162] a candidate loss value determination sub-unit configured to determine a candidate loss value corresponding to each target heat point based on the target heat point prediction probability value and the foreground label value;

[0163] a foreground loss value configured to determine a foreground loss value corresponding to the sample image by averaging each candidate loss value.

[0164] On the basis of each of the technical solutions above, the background loss value determination module 460 includes:

[0165] a background label value determination unit configured to determine a background label value corresponding to the background mask region in the annotation mask;

[0166] a background loss value determination unit configured to determine a background loss value corresponding to the sample image based on a heat point prediction probability value corresponding to each heat point in the background heat map region and the background label value.

[0167] On the basis of each of the technical solutions above, the model training end module 470 includes:

[0168] The comprehensive loss value determination unit is configured to determine a comprehensive loss value based on the disease probability loss value, the foreground loss value, and the background loss value.

[0169] The model training end unit is configured to back-propagate the comprehensive loss value to the preset image processing model, adjust network parameters in the preset image processing model, and end the training when a preset condition is met.

[0170] The technical scheme provided by the embodiment of the application comprises the following steps: obtaining a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image, inputting the sample image into a preset image processing model to be trained for image processing, obtaining a prediction probability value and a predicted lesion heat map, determining a disease probability loss value corresponding to the sample image based on the prediction probability value and the disease category label, generating a label mask corresponding to the lesion annotation region based on the lesion annotation region information in the sample image, determining a foreground heat map region and a background heat map region in the predicted lesion heat map based on the label mask, determining a foreground loss value corresponding to the sample image based on the foreground heat map region and the label mask, determining a background loss value corresponding to the sample image based on the background heat map region and the label mask, and training the preset image processing model based on the disease probability loss value, the foreground loss value, and the background loss value until the training ends when a preset condition is met, thereby realizing model training of the image processing model, optimizing the image processing model parameters based on multiple loss values by respectively determining the disease probability loss value, the foreground loss value, and the background loss value, solving the problem of low training accuracy of the image processing model, improving the training accuracy of the image processing model, and ensuring high performance of the image processing model.

[0171] The training device of the image processing model provided by the embodiment of the present disclosure can perform the training method of the image processing model provided by any embodiment of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method.

[0172] It should be noted that each unit and module included in the above device is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for convenient mutual distinction, and does not limit the protection scope of the embodiment of the present disclosure.

[0173] Embodiment five

[0174] Figure 13A block diagram of a device for Embodiment Five of the present application is provided. The device 10 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The device can also represent various forms of mobile devices such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headgear, eyewear, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0175] As shown in Figure 13 The device 10 includes at least one processor 11, and memory, such as read-only memory (ROM) 12, random access memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 13. An input / output (I / O) interface 15 is also connected to the bus 13.

[0176] Various components in the device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0177] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the training method of the image processing model.

[0178] In some embodiments, the training method of the image processing model can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, portions or all of the computer program can be loaded and / or installed onto device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the training method of the image processing model described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the training method of the image processing model by other means, e.g., with the aid of firmware.

[0179] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0180] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable image processing model training device to produce a machine, such that the computer program, when executed, enables the system specified in the flow diagrams and / or the block diagrams to implement the functions / acts specified in the flow diagrams and / or the block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, or entirely on a remote machine or server.

[0181] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0182] To provide for interaction with a user, the systems and techniques described here can be implemented on a device having a display (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0183] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0184] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service. It should be understood that the various forms of flow shown above can be reordered, added, or deleted steps. For example, each step described in the present application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein. The above specific embodiments do not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for training an image processing model, characterized in that, The method comprises: obtaining a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image; inputting the sample image into a preset image processing model to be trained for image processing, to obtain a prediction probability value of the sample image belonging to the target disease category and a prediction lesion heat map; based on the prediction probability value and the target disease category label, determining a disease probability loss value corresponding to the sample image; based on the lesion annotation region information in the sample image, generating an annotation mask corresponding to the lesion annotation region, and based on the annotation mask, determining a foreground heat map region and a background heat map region in the prediction lesion heat map; based on the foreground heat map region and the annotation mask, determining a foreground loss value corresponding to the sample image; based on the background heat map region and the annotation mask, determining a background loss value corresponding to the sample image; based on the disease probability loss value, the foreground loss value, and the background loss value, training the preset image processing model until a preset condition is met; wherein, based on the foreground heat map region and the annotation mask, determining the foreground loss value corresponding to the sample image comprises: determining a foreground label value corresponding to a foreground mask region in the annotation mask; arranging the prediction probability values of each heat point in the foreground heat map region in descending order, and determining the top pre-set number of prediction probability values of heat points as target heat point prediction probability values; based on the target heat point prediction probability values and the foreground label value, determining a candidate loss value corresponding to each target heat point; wherein the candidate loss value is a loss value determined by cross-entropy loss calculation of the heat point prediction probability value corresponding to each target heat point and the foreground label value; determining the foreground loss value corresponding to the sample image by averaging each candidate loss value.

2. The method of claim 1, wherein, The preset image processing model comprises: a feature map extraction sub-model, a probability value prediction sub-model, and a heat map generation sub-model; inputting the sample image into the feature map extraction sub-model for feature extraction to obtain a sample feature map corresponding to the sample image; inputting the sample feature map into the probability value prediction sub-model for classification prediction to obtain a prediction probability value of the sample image belonging to the target disease category; inputting the sample feature map into the heat map generation sub-model for heat map generation to obtain a prediction lesion heat map corresponding to the sample image. The heat map generation sub-model comprises a logical graph extraction unit and a heat map generation unit; 3. The method of claim 2, wherein, inputting the sample feature map into the logical graph extraction unit for logical graph generation to obtain a sample logical graph corresponding to the sample feature map; ​ ​ inputting the sample logical graph into the heat map generation unit for probability prediction to obtain a logical point prediction probability value corresponding to each logical point in the sample logical graph; generating a predicted lesion heat map corresponding to the sample image according to the logical point prediction probability values.

4. The method of claim 1, wherein, The determination of the foreground heat map region and the background heat map region in the predicted lesion heat map based on the annotation mask comprises: regarding a region formed by each pixel point in the lesion annotation region in the annotation mask as a foreground mask region, and regarding a region formed by each pixel point outside the lesion annotation region in the annotation mask as a background mask region; in the predicted lesion heat map, a region formed by pixel points corresponding to the foreground mask region is determined as a foreground heat map region, and a region formed by pixel points corresponding to the background mask region is determined as a background heat map region; wherein the predicted lesion heat map and the annotation mask are of the same size.

5. The method of claim 1, wherein, The determination of the background loss value corresponding to the sample image based on the background heat map region and the annotation mask comprises: determining a background label value corresponding to the background mask region in the annotation mask; determining the background loss value corresponding to the sample image based on a heat point prediction probability value corresponding to each heat point in the background heat map region and the background label value.

6. The method of claim 1, wherein, The training of the preset image processing model based on the disease probability loss value, the foreground loss value and the background loss value until the training is ended when a preset condition is met comprises: determining a comprehensive loss value based on the disease probability loss value, the foreground loss value and the background loss value; propagating the comprehensive loss value to the preset image processing model to adjust network parameters in the preset image processing model until the training is ended when a preset condition is met.

7. A training device for an image processing model, characterized in that, Comprise: a sample acquisition module configured to acquire a sample image containing lesion annotation region information and a target disease category label corresponding to the sample image; a prediction result determination module configured to input the sample image into a preset image processing model to be trained for image processing to obtain a prediction probability value of the sample image being a target disease category and a predicted lesion heat map; a probability loss value determination module configured to determine a disease probability loss value corresponding to the sample image based on the prediction probability value and the target disease category label; a heat map region determination module configured to generate an annotation mask corresponding to a lesion annotation region based on the lesion annotation region information in the sample image, and determine a foreground heat map region and a background heat map region in the predicted lesion heat map based on the annotation mask; a foreground loss value determination module configured to determine a foreground loss value corresponding to the sample image based on the foreground heat map region and the annotation mask; a background loss value determination module configured to determine a background loss value corresponding to the sample image based on the background heat map region and the annotation mask; a model training end module configured to train the preset image processing model based on the disease probability loss value, the foreground loss value and the background loss value until the training is ended when a preset condition is met. The foreground loss value determination module comprises: The foreground label value determination unit is configured to determine a foreground label value corresponding to a foreground mask region in the annotation mask; The foreground loss value determination unit comprises: The target prediction probability value determination subunit is configured to arrange the prediction probability values of the heat points in descending order, and determine the target prediction probability values of the top pre-set number of heat points in the arranged order. The candidate loss value determination subunit is configured to determine a candidate loss value corresponding to each target heat point based on the target prediction probability value and the foreground label value. The foreground loss value determination subunit is configured to determine the foreground loss value corresponding to the sample image by averaging the candidate loss values.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the training method of the image processing model in any one of claims 1-6.

Images