Image detection method and device

Abstract

The present disclosure provides an image detection method, which includes: obtaining an image to be detected; inputting the image to be detected into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category and indicates a predicted probability of at least one image region in the image to be detected based on the predicted disease category; displaying the disease region in the image to be detected according to the predicted probability associated with each image region, and outputting the image detection result. The present disclosure also provides an image detection device, an electronic device, and a computer-readable storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to an image detection method, an image detection device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the increasing maturity of computer technology, image detection technology has developed rapidly. Image detection has a wide range of applications in the field of medical images, and computer-aided diagnosis achieved by image detection plays an important role in the current field of medical images.

[0003] In the process of realizing the inventive concept of the present disclosure, the inventors found that in the related art, when using image detection technology to identify diseases, a positioning frame is often used to mark the disease area in the image to be detected. This has the defects of weak readability of the marking of the disease area and poor marking effect. Summary of the invention

[0004] In view of this, the present disclosure provides an image detection method and device with strong marking readability of disease areas and effectively improved marking effect.

[0005] One aspect of the present disclosure provides an image detection method, comprising: acquiring an image to be detected; inputting the image to be detected into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category and indicates a predicted probability of at least one image region in the image to be detected based on the predicted disease category; displaying the disease region in the image to be detected according to the predicted probability associated with each of the image regions, and outputting the image detection result.

[0006] Optionally, the above-mentioned displaying the disease area in the above-mentioned image to be detected based on the predicted probability associated with each of the above-mentioned image areas includes: determining display parameters for each of the above-mentioned image areas based on the predicted probability associated with each of the above-mentioned image areas; and displaying each of the above-mentioned image areas based on the display parameters for each of the above-mentioned image areas, so as to realize the display of the disease area in the above-mentioned image to be detected.

[0007] Optionally, the above-mentioned determination of display parameters for each of the above-mentioned image regions based on the prediction probability associated with each of the above-mentioned image regions includes: determining the display brightness for each of the above-mentioned image regions based on the prediction probability associated with each of the above-mentioned image regions, wherein the higher the prediction probability associated with the above-mentioned image region, the greater the display brightness for the above-mentioned image region.

[0008] Optionally, the above-mentioned image to be detected is input into a preset detection model to obtain an image detection result, including: performing feature extraction on the above-mentioned image to be detected through the separable convolutional layer of the above-mentioned detection model to obtain at least one image feature associated with the above-mentioned image to be detected; performing splicing processing on the above-mentioned at least one image feature to obtain a spliced ​​image feature; performing recognition processing on the above-mentioned spliced ​​image feature to obtain the above-mentioned image detection result.

[0009] Optionally, the method further comprises: smoothing the above-mentioned disease area using different smoothing coefficients to realize display of predicted diseases of different disease levels.

[0010] Optionally, the training method of the above-mentioned detection model includes: obtaining original image samples with disease category labels; determining sparse disease categories with a sample number lower than a preset threshold based on the above-mentioned disease category labels; performing enhancement processing on the original image samples associated with the above-mentioned sparse disease categories to obtain enhanced image samples; and performing model training using the enhanced image samples associated with the sparse disease categories and the original image samples associated with the non-sparse disease categories to obtain the above-mentioned detection model.

[0011] Optionally, the above-mentioned enhancement processing of the original image samples associated with the above-mentioned sparse disease categories includes at least one of the following: performing random value filling processing on the disease areas associated with the above-mentioned sparse disease categories; performing affine transformation processing on the disease areas associated with the above-mentioned sparse disease categories; and fusing the original image samples associated with the above-mentioned sparse disease categories with other original image samples.

[0012] Another aspect of the present disclosure provides an image detection device, including: an acquisition module, used to acquire an image to be detected; a first processing module, used to input the above-mentioned image to be detected into a preset detection model to obtain an image detection result, wherein the above-mentioned image detection result indicates a predicted disease category, and indicates a predicted probability of at least one image area in the above-mentioned image to be detected based on the above-mentioned predicted disease category; a second processing module, used to display the disease area in the above-mentioned image to be detected according to the predicted probability associated with each of the above-mentioned image areas, and output the above-mentioned image detection result.

[0013] Optionally, the second processing module includes: a first processing submodule, used to determine display parameters for each of the above-mentioned image regions based on the predicted probability associated with each of the above-mentioned image regions; and a second processing submodule, used to display each of the above-mentioned image regions based on the display parameters for each of the above-mentioned image regions, so as to display the diseased area in the above-mentioned image to be detected.

[0014] Optionally, the first processing submodule includes: a first processing unit, used to determine the display brightness for each of the above-mentioned image regions based on the prediction probability associated with each of the above-mentioned image regions, wherein the higher the prediction probability associated with the above-mentioned image region, the greater the display brightness for the above-mentioned image region.

[0015] Optionally, the above-mentioned first processing module includes: a third processing sub-module, used to perform feature extraction on the above-mentioned image to be detected through the separable convolution layer of the above-mentioned detection model, and obtain at least one image feature associated with the above-mentioned image to be detected; a fourth processing sub-module, used to perform splicing processing on the above-mentioned at least one image feature to obtain the spliced ​​image feature; and a fifth processing sub-module, used to perform recognition processing on the above-mentioned spliced ​​image feature to obtain the above-mentioned image detection result.

[0016] Optionally, the second processing module package further includes: a sixth processing submodule, configured to perform smoothing processing on the symptom region using different smoothing coefficients, so as to realize display of predicted symptoms of different symptom levels.

[0017] Optionally, the training process of the above-mentioned detection model includes: obtaining original image samples with disease category labels; determining sparse disease categories with a sample number lower than a preset threshold based on the above-mentioned disease category labels; enhancing the original image samples associated with the above-mentioned sparse disease categories to obtain enhanced image samples; using the enhanced image samples associated with the sparse disease categories and the original image samples associated with the non-sparse disease categories to perform model training to obtain the above-mentioned detection model.

[0018] Optionally, the original image samples associated with the above-mentioned sparse disease categories are enhanced, including at least one of the following: random value filling processing is performed on the disease area associated with the above-mentioned sparse disease categories; affine transformation processing is performed on the disease area associated with the above-mentioned sparse disease categories; the original image samples associated with the above-mentioned sparse disease categories are fused with other original image samples.

[0019] Another aspect of the present disclosure provides an electronic device. The electronic device includes at least one processor and a memory connected to the at least one processor. The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor implements the method of the embodiment of the present disclosure.

[0020] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method of the embodiment of the present disclosure when executed.

[0021] Another aspect of the present disclosure provides a computer program, wherein the computer program includes computer executable instructions, and the instructions are used to implement the method of the embodiment of the present disclosure when executed.

[0022] Through the embodiments of the present disclosure, the technical problems of weak readability of marking of disease areas and poor marking effect in related technologies are at least partially overcome, thereby achieving the technical effect of effectively improving the readability of marking of disease areas and effectively improving the confidence of image detection results as auxiliary diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] For a more complete understanding of the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0024] Figure 1 The image detection system architecture according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 2 A flowchart of an image detection method according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 3 A schematic diagram schematically shows a method of extracting image features using a detection model according to an embodiment of the present disclosure;

[0027] Figure 4 A flowchart of another image detection method according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 5 A block diagram of an image detection device according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 6 A block diagram of an electronic device suitable for implementing the image detection method and apparatus according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0032] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0033] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] Various embodiments of the present disclosure provide an image detection method and a detection device capable of applying the method. The method may include obtaining an image to be detected, and inputting the image to be detected into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category, and indicates a predicted probability of at least one image region in the image to be detected based on the predicted disease category, and then displaying the disease region in the image to be detected according to the predicted probability associated with each image region, and outputting the image detection result.

[0035] like Figure 1As shown, the system architecture 100 includes at least one terminal (a plurality of terminals are shown in the figure, such as terminals 101, 102, and 103, and the terminals may include, for example, user terminals and diagnostic equipment) and a server 104 (specifically, it may be a business server for performing image detection, or it may be a business server cluster, which is not shown in the figure). In the system architecture 100, the server 104 obtains the image to be detected from the terminal (such as terminals 101, 102, and 103), and inputs the image to be detected into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category, and indicates a predicted probability of at least one image region in the image to be detected based on the predicted disease category, and then, according to the predicted probability associated with each image region, the disease region in the image to be detected is displayed in the terminal (such as terminals 101, 102, and 103), and the image detection result is output.

[0036] The present disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] Figure 2 A flow chart of an image detection method according to an embodiment of the present disclosure is schematically shown, and the method applies a test platform.

[0038] like Figure 2 As shown, the method may include, for example, operations S210 to S230.

[0039] In operation S210, an image to be detected is acquired.

[0040] In the disclosed embodiment, specifically, an image to be detected is obtained from a terminal, and the terminal may include a user terminal and a diagnosis and treatment device. The user terminal may include, for example, a terminal device with data processing and display functions, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, etc. The image to be detected may be one or more images, and the image to be detected may include the user's skin lesion area. The user can use the application in the terminal to take a picture of a body part as an image to be detected, and then upload the image to be detected through the image upload page, so as to upload the image to be detected to the business server (the server for image detection).

[0041] The business server obtains and collects the images to be detected uploaded by the user through the backend server corresponding to the user terminal or the diagnosis and treatment device. The business server determines the image detection result for the image to be detected, and returns the image detection result to the backend server, so that the user can view the predicted disease category indicated by the image detection result through the display page of the user terminal or the diagnosis and treatment device, and view the disease area associated with the predicted disease category.

[0042] Next, in operation S220, the image to be detected is input into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category and a predicted probability of at least one image region in the image to be detected based on the predicted disease category.

[0043] In the disclosed embodiment, specifically, a preset detection model is used to perform recognition processing on the image to be detected, specifically to perform recognition and positioning processing on the skin lesion area in the image to be detected, and obtain an image detection result for the image to be detected, the image detection result indicates the predicted disease category, and indicates the predicted probability of at least one image area in the image to be detected based on the predicted disease category. The detection model can be a convolutional neural network model, for example, it can be an Xception neural network model, which replaces the convolution layer of the residual learning neural network model (ResidualNeural Network, ResNet) with a separable convolution layer, so as to improve the classification accuracy and model performance of the detection model.

[0044] Specifically, the image to be detected is input into a preset detection model, and the detection model is used to perform feature extraction and feature recognition processing on the image to be detected. The extracted image features can specifically be initial skin lesion features of the skin lesion area. The initial skin lesion features are converted into skin lesion feature information of different granularities, and the predicted disease category for the skin lesion area is determined according to the skin lesion feature information of different granularities, and the predicted probability of different image areas in the detection image based on the predicted disease category is determined.

[0045] Optionally, feature extraction is performed on the image to be detected through a separable convolutional layer of the detection model to obtain at least one image feature associated with the image to be detected. Figure 3 The schematic diagram of using the detection model to extract image features according to the embodiment of the present disclosure is schematically shown. Figure 3 In the example 300 shown in FIG. 300 , the input image features are convolved through a 1×1 convolution kernel (such as Figure 3 301 in ), and each channel in the 1×1 convolution kernel output result is processed by a 3×3 convolution kernel (such as Figure 3 302~306) in the above figure are used for feature extraction, and finally, the image features output by a series of 3×3 convolution kernels are spliced, and the spliced ​​image features constitute the output image features.

[0046] In addition, the output image features obtained by the above operation can be used as input image features and input into the 1×1 convolution kernel again (such as Figure 3301 in the process), repeat the process in 300 to obtain the spliced ​​image features again. Then, the spliced ​​image features obtained again are downsampled by the pooling layer, and the downsampling results are superimposed on the input image features in the previous round of 300 operations to obtain the output image features in this round of 300 operations. Optionally, the output image features obtained in the previous round of 300 operations are used as the input image features of the next round of 300 operations, and feature extraction is repeated. Then, recognition processing is performed based on the image features finally obtained to obtain image detection results.

[0047] The image detection result includes the predicted disease category indicated by the image to be detected, and the predicted probability of different image regions based on the predicted disease category. The predicted probability of different image regions based on the predicted disease category can be determined by the weight values ​​of different image features based on the predicted disease category. The weight values ​​of different image features based on the predicted disease category can be obtained by reverse derivation of the classification result of the detection model. The weight value represents the contribution of the image feature to the identification of the predicted disease category, that is, the predicted probability of the image feature for the predicted disease category. Since the image features of different image regions in the image to be detected may be different, the predicted probabilities associated with different image regions can be determined according to the weight values ​​of different image features for the predicted disease category.

[0048] Next, in operation S230, the diseased region in the image to be detected is displayed according to the prediction probability associated with each image region, and the image detection result is output.

[0049] In the disclosed embodiment, specifically, display parameters for each image region are determined based on the predicted probability associated with each image region, so as to display different image regions based on the predicted probability. The greater the predicted probability based on the predicted disease category, the higher the similarity between the image region and the lesion region of the predicted disease category, and the higher the possibility that the image region is a lesion region. Display parameters for different image regions are determined based on the predicted probability associated with each image region, so as to display the disease region in the image to be detected, which is conducive to improving the accuracy and readability of the display of the disease region, and is conducive to improving the confidence of image detection as an auxiliary diagnosis.

[0050] In the disclosed embodiment, an image to be detected is obtained; the image to be detected is input into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category and indicates a predicted probability of at least one image region in the image to be detected based on the predicted disease category; the disease region in the image to be detected is displayed according to the predicted probability associated with each image region, and the image detection result is output. Displaying different image regions in the image to be detected based on the predicted probability is, on the one hand, conducive to intuitively presenting the degree of correlation between different image regions and the predicted disease category through display parameters, and is conducive to enhancing the readability of the disease region marking; on the other hand, compared with the related art of selecting the disease region in the image to be detected by positioning the frame, this solution can effectively realize the display of irregular disease regions, which is conducive to improving the accuracy of the display of the disease region and the confidence of auxiliary diagnosis.

[0051] The training method of the above-mentioned detection model includes: obtaining original image samples with disease category labels, and determining sparse disease categories with a sample number lower than a preset threshold based on the disease category labels, and then enhancing the original image samples associated with the sparse disease categories to obtain enhanced image samples, and finally using the enhanced image samples associated with the sparse disease categories and the original image samples associated with the non-sparse disease categories to perform model training to obtain a detection model. Due to the different incidence rates of different disease categories, image samples associated with different disease categories may be unevenly distributed. In order to alleviate the problem of model prediction bias caused by uneven sample distribution, the original image samples associated with the sparse disease categories can be enhanced to increase the number of image samples for the sparse disease categories.

[0052] Among them, the enhancement processing of the original image samples associated with the sparse disease category may include, for example: random value filling processing of the disease area associated with the sparse disease category, affine transformation processing of the disease area associated with the sparse disease category, fusion processing of the original image samples associated with the sparse disease category with other original image samples, etc. Specifically, by filling the disease area associated with the sparse disease category with random values, the ability of the model to identify the disease category through different image features is increased. Alternatively, by flipping, scaling, rotating, radial transformation, light and dark changes, projection changes, etc., the disease area associated with the sparse disease category is processed to increase the number of image samples associated with the sparse disease category.

[0053] Alternatively, different image samples are fused, and the fused image is used as a new training sample. The different image samples to be fused may contain the same or different disease categories, and different disease categories may include sparse disease categories and non-sparse disease categories. Different image samples before fusion respectively contain lesion areas of corresponding disease categories, and the weight of the image sample for its corresponding disease category is 1. After the different image samples are fused, the fused image samples may contain lesion areas of different disease categories. The weights of the fused image samples for different disease categories change, but the sum of the weights for all disease categories is 1. Fusion of different image samples is beneficial to improving the diversity of image samples on the one hand, and on the other hand, it is beneficial to increase image samples of different disease degrees, which is beneficial to improving the robustness of the trained detection model. The diversified enhancement processing method for image samples effectively increases the sample size of image samples, especially greatly increases the number of image samples associated with sparse disease categories. A sufficient number of image samples for different disease categories makes it possible to train multiple disease classification models.

[0054] Figure 4 The flowchart of another image detection method according to an embodiment of the present disclosure is schematically shown.

[0055] like Figure 4 As shown, operation S230 may include operations S410 to S420, for example.

[0056] In operation S410, display parameters for each image region are determined according to the prediction probability associated with each image region.

[0057] In the disclosed embodiment, specifically, according to the predicted probability of the predicted disease category of at least one image region in the image to be detected, the display parameters for each image region are determined to realize the display of the disease region in the image to be detected. Different display parameters result in different display effects of the image region. This design is conducive to associating the display effects of different image regions with the predicted probability, which is conducive to improving the readability of the disease region display on the one hand, and on the other hand, it is conducive to improving the accuracy and confidence of the disease region display.

[0058] Optionally, the display brightness for each image region can be determined according to the prediction probability associated with each image region, wherein the higher the prediction probability associated with the image region, the greater the display brightness for the image region. By brightening the display of the diseased region and dimming the display of the non-disease region, it is helpful to achieve the extraction of significant effects on irregular skin disease regions while accurately locating the diseased region.

[0059] Different image regions may have different image features. When determining the prediction probability associated with each image region, it can be determined according to the prediction probability associated with different image features, that is, according to the weight values ​​of different image features for predicting disease categories. The weight values ​​of different image features for predicting disease categories are obtained by reverse derivation of the classification results of the detection model, which is used to characterize the contribution ratio of each image feature to obtaining the prediction results. The reverse derivation of the classification results of the detection model can be implemented using existing methods (for example, the Grad-CAM++ algorithm can be used), which is not described in detail in this application. Different image regions are fused according to the corresponding weight values ​​to display the disease areas in the image to be detected in the form of a heat map.

[0060] Next, in operation S420, each image region is displayed according to the display parameters for each image region, so as to realize display of the disease region in the image to be detected.

[0061] In the disclosed embodiment, specifically, according to the display parameters determined for each image area, each image area is displayed to realize the display of the disease area in the image to be detected. Optionally, when displaying the disease area in the image to be detected, the disease area is smoothed using different smoothing coefficients to realize the display of predicted diseases of different disease levels. Specifically, when it is necessary to display the changes in the disease area for the predicted improvement of the disease, the image smoothing technology is used to gradually fade the displayed disease area. Specifically, the display of the disease area in the direction of improvement can be realized by increasing the smoothing coefficient over time. When it is necessary to display the changes in the disease area for the predicted deterioration of the disease, the image filtering technology is used to gradually sharpen the displayed disease area. Specifically, the display of the disease area in the direction of deterioration can be realized by increasing the sharpening coefficient over time. This design is conducive to improving the readability of the disease area display by visualizing the evolution process of the disease area, and helps to enhance the reference value of auxiliary diagnosis.

[0062] Figure 5 The block diagram of the image detection device according to the embodiment of the present disclosure is schematically shown.

[0063] like Figure 5 As shown, the device may include an acquisition module 501 , a first processing module 502 , and a second processing module 503 .

[0064] Specifically, the acquisition module 501 is used to acquire the image to be detected; the first processing module 502 is used to input the image to be detected into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category and a predicted probability of at least one image area in the image to be detected based on the predicted disease category; the second processing module 503 is used to display the disease area in the image to be detected according to the predicted probability associated with each image area, and output the image detection result.

[0065] In the disclosed embodiment, an image to be detected is obtained; the image to be detected is input into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category and indicates a predicted probability of at least one image region in the image to be detected based on the predicted disease category; the disease region in the image to be detected is displayed according to the predicted probability associated with each image region, and the image detection result is output. Displaying different image regions in the image to be detected based on the predicted probability is, on the one hand, conducive to intuitively presenting the degree of correlation between different image regions and the predicted disease category through display parameters, and is conducive to enhancing the readability of the disease region marking; on the other hand, compared with the related art of selecting the disease region in the image to be detected by positioning the frame, this solution can effectively realize the display of irregular disease regions, which is conducive to improving the accuracy of the display of the disease region and the confidence of auxiliary diagnosis.

[0066] As an optional embodiment, the second processing module includes: a first processing submodule, used to determine the display parameters for each image area based on the predicted probability associated with each image area; a second processing submodule, used to display each image area according to the display parameters for each image area, so as to display the disease area in the image to be detected.

[0067] As an optional embodiment, the first processing submodule includes: a first processing unit, used to determine the display brightness for each image area based on the prediction probability associated with each image area, wherein the higher the prediction probability associated with the image area, the greater the display brightness for the image area.

[0068] As an optional embodiment, the first processing module includes: a third processing sub-module, used to extract features of the image to be detected through a separable convolutional layer of the detection model, and obtain at least one image feature associated with the image to be detected; a fourth processing sub-module, used to perform splicing processing on at least one image feature to obtain a spliced ​​image feature; and a fifth processing sub-module, used to perform recognition processing on the spliced ​​image features to obtain an image detection result.

[0069] As an optional embodiment, the second processing module package further includes: a sixth processing submodule, which is used to perform smoothing processing on the disease area using different smoothing coefficients to realize display of predicted diseases of different disease levels.

[0070] As an optional embodiment, the training process of the detection model includes: obtaining original image samples with disease category labels; determining sparse disease categories with a sample number lower than a preset threshold based on the disease category labels; performing enhancement processing on the original image samples associated with the sparse disease categories to obtain enhanced image samples; and performing model training using the enhanced image samples associated with the sparse disease categories and the original image samples associated with the non-sparse disease categories to obtain a detection model.

[0071] As an optional embodiment, the original image samples associated with the sparse disease category are enhanced, including at least one of the following: random value filling processing is performed on the disease area associated with the sparse disease category; affine transformation processing is performed on the disease area associated with the sparse disease category; and the original image samples associated with the sparse disease category are fused with other original image samples.

[0072] Optionally, any one of the modules or submodules in the acquisition module 501, the first processing module 502, and the second processing module 503, or at least part of the functions of any one of them can be implemented in one module. According to any one or more of the modules of the embodiments of the present disclosure, it can be split into multiple modules for implementation. According to any one or more of the modules of the embodiments of the present disclosure, it can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in any appropriate combination of any of them. Or according to one or more of the modules of the embodiments of the present disclosure, it can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0073] For example, any multiple of the acquisition module 501, the first processing module 502, and the second processing module 503 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. Optionally, at least one of the acquisition module 501, the first processing module 502, and the second processing module 503 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 501, the first processing module 502, and the second processing module 503 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0074] Figure 6 A block diagram of an electronic device suitable for implementing the image detection method and apparatus according to an embodiment of the present disclosure is schematically shown. Figure 6 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0075] like Figure 6 As shown, the computer system 600 according to the embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0076] In RAM 603, various programs and data required for the operation of system 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other through bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0077] Optionally, the system 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The system 600 may further include one or more of the following components connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 606 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0078] Optionally, the method flow according to the embodiment of the present disclosure can be implemented as a computer software program. For example, the embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. Optionally, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0079] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0080] Alternatively, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. For example, optionally, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0081] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0082] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0083] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An image detection method, comprising: Acquire an image to be detected, where the image to be detected includes a skin lesion area of ​​a user; Inputting the image to be detected into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category for the skin lesion area, and indicates a predicted probability of at least one image area in the image to be detected based on the predicted disease category, the predicted disease category includes a sparse disease category and a non-sparse disease category, the predicted probability is determined based on weight values ​​of different image features for the predicted disease category, the image features represent initial skin lesion features of the skin lesion area, and the weight values ​​represent contributions made by the image features to identifying the predicted disease category; Based on the predicted probability associated with each of the image areas, display parameters for each of the image areas are determined to display the diseased area in the image to be detected, and the image detection result is output. The display parameters include display brightness. The higher the predicted probability associated with the image area, the greater the display brightness for the image area.

2. The method according to claim 1, wherein: The step of determining display parameters for each image region according to the prediction probability associated with each image region to display the disease region in the image to be detected includes: Determining display parameters for each of the image regions according to the prediction probability associated with each of the image regions; Each of the image regions is displayed according to the display parameters for each of the image regions, so as to realize display of the disease region in the image to be detected.

3. The method according to claim 2, wherein: The step of determining a display parameter for each of the image regions according to the prediction probability associated with each of the image regions comprises: The display brightness for each of the image regions is determined according to the prediction probability associated with each of the image regions.

4. The method according to claim 1, wherein the step of inputting the image to be detected into a preset detection model to obtain an image detection result comprises: Performing feature extraction on the image to be detected through the separable convolutional layer of the detection model to obtain at least one image feature associated with the image to be detected; Performing splicing processing on the at least one image feature to obtain a spliced ​​image feature; The features of the spliced ​​images are identified and processed to obtain the image detection result.

5. The method according to claim 1, further comprising: The disease area is smoothed using different smoothing coefficients to achieve the display of predicted diseases of different disease levels.

6. The method according to any one of claims 1 to 5, wherein: The training method of the detection model includes: Obtaining original image samples with disease category labels; According to the disease category labels, determining sparse disease categories whose sample numbers are lower than a preset threshold; Performing enhancement processing on the original image samples associated with the sparse disease category to obtain enhanced image samples; The enhanced image samples associated with the sparse disease category and the original image samples associated with the non-sparse disease category are used to perform model training to obtain the detection model.

7. The method according to claim 6, wherein: The enhancing process of the original image samples associated with the sparse disease category comprises at least one of the following: Performing random value filling processing on the disease area associated with the sparse disease category; performing affine transformation processing on the disease area associated with the sparse disease category; The original image samples associated with the sparse disease category are fused with other original image samples.

8. An image detection device, comprising: An acquisition module, used for acquiring an image to be detected, wherein the image to be detected includes a skin lesion area of ​​a user; A first processing module is used to input the image to be detected into a preset detection model to obtain an image detection result, wherein the image detection result indicates a predicted disease category for the skin lesion area, and indicates a predicted probability of at least one image area in the image to be detected based on the predicted disease category, the predicted disease category includes a sparse disease category and a non-sparse disease category, the predicted probability is determined based on weight values ​​of different image features for the predicted disease category, the image features represent initial skin lesion features of the skin lesion area, and the weight values ​​represent contributions made by the image features to identifying the predicted disease category; The second processing module is used to determine the display parameters for each image area according to the predicted probability associated with each image area, so as to display the disease area in the image to be detected and output the image detection result. The display parameters include display brightness. The higher the predicted probability associated with the image area, the greater the display brightness for the image area.

9. An electronic device, comprising: one or more processors; as well as a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 7.

Images