Image processing method, electronic device, and storage medium

Abstract

The application provides an image processing method, which comprises: processing a first picture to obtain a first matrix; performing Gaussian blur processing on the first picture to obtain a first denoising picture; fusing the first picture and the first denoising picture based on the first matrix and a preset threshold to generate a second picture; processing the second picture to obtain a second matrix; performing Gaussian blur processing on the second picture to obtain a second denoising picture; and fusing the second picture and the second denoising picture based on the second matrix and the preset threshold to generate a third picture. The application can be applied to medical image classification, and the heat map obtained based on the explainable neural network is processed, so that the negative and positive regions of the original input picture can be effectively divided, the positive region is mainly retained, the influence of the negative region features on the neural network is reduced, and the model accuracy and trustworthiness are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to an image processing method, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, deep learning has become increasingly popular in the medical field. In particular, the classic image classification task in the image field has been widely used in the medical field, such as using an image classification network to determine whether a lung picture in an X-ray film has a lesion, etc. to assist doctors in diagnosing practical applications. However, when using deep learning for image classification, there may be accuracy and trust issues. For example, when classifying a lesion in a certain part of a medical image, other parts of the image may interfere with image classification, i.e. the accuracy of image classification may be affected by negative regions of the image, and because people do not have an intuitive explanation for neural networks, the results obtained by using neural networks for image classification are not trusted.

[0003] Therefore, in view of the "trust" crisis of deep learning, it is necessary to provide a method for processing a classification image to reduce the influence of negative regions on image classification, thereby improving the accuracy and trust of medical image classification. SUMMARY

[0004] In view of the above reasons, it is necessary to provide an image processing method, an electronic device and a storage medium, so that the image classification result can achieve higher accuracy under the premise that the neural network used for image processing is interpretable, and at the same time solve the two basic problems of low accuracy of image classification result and lack of explanation of neural network used for image processing, so that deep learning can better play an important role in the medical field.

[0005] To achieve the above-mentioned purpose, the present application provides an image processing method, which comprises the following steps:

[0006] S1, obtaining a first picture, processing the first picture based on a preset neural network to obtain a first heat map; using a preset program to perform a closed operation on the first heat map to obtain a first closed operation heat map;

[0007] S2, obtaining a first matrix by processing the first closed operation heat map; performing Gaussian blur processing on the first picture to obtain a first denoising picture; fusing the first picture and the first denoising picture based on the first matrix and a preset threshold to generate a second picture;

[0008] S3, processing the second picture as the first picture in steps S1 and S2 to obtain a third picture.

[0009] Preferably, the first picture is a picture to be classified included in a test set to be input into an image classification network; the neural network is an explainable neural network, and the first heat map is an explainable heat map; the first heat map and the first closed operation heat map have the same size as the first picture; and the kernel size of the closed operation is 7.

[0010] Preferably, the first matrix obtained by processing the first closed operation heat map comprises:

[0011] extracting pixel point values of a red channel from the first closed operation heat map;

[0012] performing normalization calculation on the pixel point values of the red channel by using a preset program of Python;

[0013] performing Softmax processing on the data obtained by the normalization calculation by row to obtain a new first matrix having the same size as the first closed operation heat map.

[0014] Preferably, the normalization calculation on the pixel point values of the red channel by using the preset program of Python comprises:

[0015] dividing the extracted pixel point values of the red channel by a maximum value 255 to normalize the pixel point values to a range of [0, 1] in decimal form.

[0016] Preferably, in the Softmax processing on the data obtained by the normalization calculation by row to obtain a new first matrix having the same size as the first closed operation heat map,

[0017]

[0018] wherein x is a value of a pixel point, exp represents an exponential function with a natural constant e as a base, i is an arrangement serial number of a red channel pixel value in a matrix, and j is a total number of red channel pixel values in the matrix.

[0019] Preferably, the Gaussian blur processing on the first picture comprises:

[0020] performing convolution calculation on the first picture and a probability density function of a two-dimensional Gaussian distribution, and a calculation formula is as follows:

[0021]

[0022] wherein (x, y) represents coordinates of a pixel point of a two-dimensional image, g(x, y) represents a pixel value of the pixel point after processing, r represents a convolution kernel radius, s(x, y) represents an original pixel value of the pixel point, and f(u, v) is also commonly referred to as a filter function.

[0023] Preferably, the filter function is a probability density function of a two-dimensional Gaussian distribution, and the formula is

[0024]

[0025] dx, dy respectively correspond to the distance from the current horizontal and vertical coordinates to the center of the convolution kernel, sigma is the standard deviation of the Gaussian distribution, and e is the natural constant.

[0026] Preferably, the kernel size used in the Gaussian blur process is 7*7.

[0027] Preferably, the fusion of the first picture and the first denoising picture based on the first matrix and a preset threshold to generate a second picture comprises:

[0028] The pixel points corresponding to the values less than the preset threshold in the first matrix are taken as the marked pixel points.

[0029] The region where the marked pixel points in the first picture are located is identified based on the marked pixel points, and is recorded as a first pixel marked region. The region where the marked pixel points in the first denoising picture are located is identified based on the marked pixel points, and is recorded as a second pixel marked region.

[0030] The pixel point values in the first pixel marked region are replaced with the corresponding pixel point values in the second pixel marked region to obtain a second picture.

[0031] Preferably, the preset threshold is 0.004.

[0032] The application further provides an image processing device, which comprises:

[0033] An acquisition module is configured to acquire a first picture, wherein the first picture is a picture to be classified in a test set of an image classification network.

[0034] A neural network module is configured to receive the acquired first picture, process the first picture based on a preset neural network, and obtain a first heat map.

[0035] A closing operation module is configured to receive the first heat map, perform a closing operation on the first heat map using a preset program, and obtain a first closing operation heat map.

[0036] A normalization module is configured to receive the first closing operation heat map, extract the pixel point values of a red channel of the first closing operation heat map, and perform normalization calculation on the pixel point values of the red channel using a preset program.

[0037] The Softmax module receives the data calculated by the normalization module, and performs Softmax processing on the data by row to obtain a new first matrix with the same size as the first closed operation heat map.

[0038] The Gaussian blur module receives the first picture and performs Gaussian blur processing on the first picture to obtain a first denoising picture.

[0039] The fusion module fuses the first picture and the first denoising picture based on the first matrix and a preset threshold to generate a second picture.

[0040] The present application also provides an electronic device including a memory and a processor, wherein the memory includes an image processing program, and the image processing program is executed by the processor to implement any step of the image processing method.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a computer readable storage medium, wherein the computer readable storage medium includes an image processing program, and the image processing program is executed by a processor to implement any step of the image processing method.

[0042] The image processing method, the electronic device and the computer readable storage medium provided by the present application can output the picture through the preset interpretable neural network, only process the original picture according to the output result of the interpretable network, and will not be limited by the interpretable process, so that the subsequent step results based on the picture are all reliable. The present application obtains a numerical matrix similar to the weight contribution by combining the basic numerical calculation with the traditional image processing method, and the calculation process does not need any artificial operation and is fast, so that the negative and positive regions of the original input picture can be effectively divided, the positive region is mainly retained, the influence of the negative region features on the neural network is reduced, and the model accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is an optional hardware architecture schematic diagram of the electronic device of the present application;

[0044] Figure 2 is a module schematic diagram in an embodiment of the image processing device of the present application (which can also be a program module schematic diagram of the image processing program in an embodiment of the electronic device of the present application);

[0045] Figure 3 is an implementation flowchart of the preferred embodiment of the image processing method of the present application.

[0046] The implementation of the present application, the functional characteristics and the advantages will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0047] The principles and spirits of the present application will be described below with reference to several specific embodiments. It should be understood that the specific embodiments described herein are merely used to explain the present application and should not be used to limit the present application.

[0048] Referring to Figure 1 Fig. 1 shows a schematic diagram of an optional hardware architecture of an electronic device according to the present application. In this embodiment, the electronic device 10 can include, but is not limited to, a memory 11, a processor 12, and a network interface 13, which are communicatively connected via a communication bus 14. It should be noted that, Figure 1 Only the electronic device 10 with components 11-14 is shown, but it should be understood that all the shown components are not required to be implemented, and more or less components can be alternatively implemented.

[0049] The memory 11 includes at least one type of computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 10, such as a hard disk or a memory of the electronic device 10. In other embodiments, the memory 11 can also be an external storage device of the electronic device 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. installed on the electronic device 10. Of course, the memory 11 can include both the internal storage unit and the external storage device of the electronic device 10. In this embodiment, the memory 11 is generally used to store an operating system and various application software, such as an image processing program, etc., installed on the electronic device 10. In addition, the memory 11 can also be used to temporarily store various data that have been output or will be output.

[0050] The processor 12 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 12 is generally used to control the overall operation of the electronic device 10. In this embodiment, the processor 12 is used to run program codes or process data stored in the memory 11, such as an image processing program, etc.

[0051] The network interface 13 can include a wireless network interface or a wired network interface, and the network interface 13 is generally used to establish a communication connection between the electronic device 10 and other electronic devices.

[0052] The communication bus 14 is used to realize a communication connection between the components 11-13.

[0053] Optionally, the electronic device 10 can further include a user interface (not shown in the figure), which can include a display, an input unit such as a keyboard, and the user interface can further include a standard wired interface, a wireless interface, etc. Figure 1

[0054] Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED touch, etc. Further, the display can also be referred to as a display screen or a display unit, which is used to display processed information in the electronic device 10 and to display a visualized user interface.

[0055] Optionally, in some embodiments, the electronic device 10 can further include an audio unit (the audio unit is not shown in the figure), which can convert received or stored audio data into an audio signal when the electronic device 10 is in a call signal receiving mode, a talking mode, a recording mode, a voice recognition mode, a broadcast receiving mode, etc. Figure 1

[0056] Optionally, in some embodiments, the electronic device 10 can further include an alarm unit (not shown in the figure), which can provide an output to notify the electronic device 10 of the occurrence of an event. Typical events can include call reception, message reception, key signal input, touch input, etc. In addition to audio or video output, the alarm unit can provide output in different ways to notify the occurrence of an event. For example, the alarm unit can provide output in the form of vibration when a call, a message or some other event that can make the electronic device 10 enter a communication mode is received, and the alarm unit can provide a tactile output (i.e., vibration) to notify the user of the same.

[0057] In the above embodiments, the processor 12 realizes the following steps when executing the image processing program stored in the memory 11:

[0058] ​​S1, obtaining a first picture, processing the first picture based on a preset neural network to obtain a first heat map, performing a close operation on the first heat map using a preset program to obtain a first close operation heat map;

[0059] S2, obtaining a first matrix by processing the first close operation heat map, performing Gaussian blur processing on the first picture to obtain a first denoising picture, and fusing the first picture and the first denoising picture based on the first matrix and a preset threshold to generate a second picture;

[0060] S3, performing processing on the second picture as described in steps S1 and S2 on the first picture to obtain a third picture.

[0061] For detailed introduction of the above steps, please refer to the following Figure 2 For the program module diagram of the image processing program and Figure 3 the implementation flowchart of the preferred embodiment of the image processing method.

[0062] In this embodiment, the picture is converted and output by the preset interpretable neural network, and only the output result of the interpretable network is processed on the original picture, which will not be limited by the interpretable process, and the subsequent step results based on the picture will be reliable.

[0063] Referring to Figure 2 Fig. 1 shows a module schematic diagram in an embodiment of the image processing device of the present application (which can also be a program module schematic diagram of the image processing program in an embodiment of the electronic device of the present application), in which multiple modules are stored in the memory 11 and executed by the processor 12 to complete the present application. The module referred to in the present application refers to a series of computer program instructions that can complete a specific function.

[0064] The image processing program 10 can be divided into: an acquisition module 201, a neural network module 202, a close operation module 203, a normalization module 204, a Softmax module 205, a Gaussian blur module 206, and a fusion module 207.

[0065] The acquisition module 201 is configured to acquire a first picture, which is a to-be-classified picture (CT picture, X-ray picture, etc.) included in a test set to be input into an image classification network. The first picture can be acquired from a memory 11 in the electronic device 10 or acquired from another electronic device storing the first picture through a network interface 13. In detail, if the first picture is stored in the memory 11 of the electronic device 10, the acquisition module 201 acquires the first picture from the memory 11; if the first picture is stored in another electronic device, the acquisition module 201 acquires the first picture by connecting to the other electronic device through the network interface 13. For example, assuming that the electronic device 10 is a computer, if the first picture is stored in the local disk of the computer, the acquisition module can acquire the first picture from the local disk of the computer; if the first picture is stored in another computer, the acquisition module can acquire the first picture by connecting to the computer storing the first picture through the network interface; similarly, the first picture can also be acquired from the Internet.

[0066] The neural network module 202 is configured to receive the acquired first picture (CT picture, X-ray picture, etc.) and process the first picture based on a preset neural network to obtain a first heat map. The neural network is an interpretable neural network, and the first heat map is an interpretable heat map; the size of the first heat map is the same as that of the first picture. The first heat map represents the degree of attention of the neural network to different regions of the first picture in different colors, and a red region is a positive region with a high degree of attention, that is, an important region containing features for image classification; a blue region is a negative region with a low degree of attention, that is, an important region not containing features for image classification.

[0067] For example, when classifying the cause categories of the abdomen of a human CT picture into categories such as the stomach and the appendix, the collected CT picture is processed as the first picture, and in the obtained first heat map, the heat map corresponding to the stomach lesion region of the stomach lesion displays red, and other regions display corresponding blue.

[0068] The present application retains the positive region features of the first picture to strengthen the features for image classification in the picture and eliminates the negative region features of the first picture to reduce the interference of the negative region on image classification, and then re-enters the picture into the image classification network, so as to achieve better recognition effect. The picture is converted and output by the preset interpretable network, and only the output result of the interpretable network is processed, so that the subsequent steps based on the picture are not limited by the interpretable process and can be trusted.

[0069] The closing operation module 203 is configured to receive the first heat map and perform a closing operation on the first heat map using a preset program to obtain a first closing operation heat map. The first closing operation heat map has the same size as the first picture, and the kernel size of the closing operation is 7. Image closing operation is a process in which an image is sequentially subjected to expansion and corrosion processing, which is helpful to process small holes or black spots in the foreground, connect adjacent objects and smooth the boundary without changing the area. Through closing operation, adjacent but not connected positive regions in the heat map can be connected into a whole and the boundary can be smoothed, so as to retain the features of the positive regions of the original image as much as possible.

[0070] The normalization module 204 is configured to receive the first closing operation heat map, extract the pixel point value of the red channel of the first closing operation heat map, and perform normalization calculation on the pixel point value of the red channel by using a preset program of Python. Specifically, the extracted pixel point value of the red channel is divided by the maximum value 255, and is normalized to the range of [0, 1] in decimal form. For example, the pixel point of the first closing operation heat map is (255, 100, 20), the extracted pixel point value of the red channel is 255, and the normalized value is 1. The normalized value is arranged in the corresponding pixel point position. For example, the arrangement of some pixel points is The pixel point value of the red channel extracted from the pixel point A is divided by the maximum value 255, and is normalized to the range of [0, 1] in decimal form, and the normalized value is a. Correspondingly, the pixel point values of the red channel extracted from the pixel points B, C and D are divided by the maximum value 255, and are normalized to the range of [0, 1] in decimal form, and the normalized values are b, c and d respectively. The arrangement of a, b, c and d is The Softmax module 205 receives the data obtained by the normalization calculation and performs Softmax processing on the data by row to obtain a new first matrix having the same size as the first closing operation heat map. Specifically, the Softmax operation formula is:

[0071]

[0072] Wherein, x is the value of the red channel, exp represents the exponential function with the natural constant e as the base, i is the arrangement number of the red channel pixel value in the matrix, and j is the total number of the red channel pixel values in the matrix.

[0073] The Gaussian blur module 206 receives the first picture and performs Gaussian blur processing on the first picture to obtain a first denoising picture. The Gaussian blur processing on the first picture includes convolution of the first picture and a two-dimensional Gaussian distribution probability density function, and the convolution formula is

[0074]

[0075] wherein (x, y) represents the coordinate of a pixel point in a two-dimensional image, g(x, y) represents the pixel value of the pixel point after processing, r represents the radius of the convolution kernel, s(x, y) represents the original pixel value of the pixel point, and f(u, v) is usually also referred to as a filter function. Gaussian blur is to use a Gaussian distribution as a filter function. Since the image is two-dimensional, a two-dimensional Gaussian distribution is needed, and the corresponding probability density function is

[0076]

[0077] dx, dy respectively correspond to the distance from the current horizontal and vertical coordinates to the center of the convolution kernel, σ is the standard deviation of the Gaussian distribution, and e is the natural constant. Preferably, the kernel size used in the Gaussian blur processing process is 7*7. It is proved by comparison test that when the kernel size is set to 7*7, and the subsequent steps are completed, the highest accuracy can be finally achieved. Replacing the related pixel points after using Gaussian blur to reduce the noise of the picture is beneficial to enhance the features used for picture classification.

[0078] The fusion module 207 fuses the first picture and the first denoised picture based on the first matrix and a preset threshold to generate a second picture. The fusion of the first picture and the first denoised picture based on the preset threshold of the first matrix to generate the second picture includes the following steps:

[0079] The pixel points corresponding to the values less than the preset threshold in the first matrix are taken as marked pixel points. Preferably, the preset threshold is 0.004.

[0080] The region where the marked pixel points in the first picture are located is identified based on the marked pixel points, and is recorded as a first pixel marked region. The region where the marked pixel points in the first denoised picture are located is identified based on the marked pixel points, and is recorded as a second pixel marked region.

[0081] The pixel point values in the first pixel marked region are replaced by the corresponding pixel point values in the second pixel marked region to obtain the second picture.

[0082] For example, the first matrix is If the value of B is less than 0.004, the corresponding pixel in the first picture is wherein the first pixel marked region has β, and the corresponding pixel in the first denoised picture is wherein the second pixel marked region has b, and thus the pixel of the second picture is

[0083] In this embodiment, a numerical matrix similar to weight contribution is obtained by combining the basic numerical calculation with the traditional image processing method. The calculation process does not have complex gradient calculation, does not require any manual operation, and is fast. The calculation process can effectively divide the negative and positive regions of the first picture, focus on retaining the positive region, reduce the influence of the negative region features on the neural network, and thus improve the model accuracy.

[0084] Referring to Figure 3 As shown in FIG. 1, the processor 12 of the electronic device 10 executes the image processing program in the memory 11 to implement the following steps of the image processing method:

[0085] In step S301, a first picture is obtained, and a first heat map is obtained by processing the first picture based on a preset neural network. A first closed operation heat map is obtained by performing a closed operation on the first heat map using a preset program.

[0086] In step S302, a first matrix is obtained by processing the first closed operation heat map. A first denoising picture is obtained by performing Gaussian blur processing on the first picture. A second picture is generated by fusing the first picture and the first denoising picture based on the first matrix and a preset threshold.

[0087] In step S303, the second picture is processed as the first picture in steps S301 and S302 to obtain a third picture. In detail, the second picture is input into the preset neural network to generate a second heat map. A second closed operation heat map is obtained by performing a closed operation on the second heat map. A second matrix is obtained by processing the second closed operation heat map. A second denoising picture is obtained by performing Gaussian blur processing on the second picture. A third picture is generated by fusing the second picture and the second denoising picture based on the second matrix and the preset threshold.

[0088] In this embodiment, the explainable network is used to obtain a negative region label (labeled pixel point) through a series of processing. Then, the corresponding negative region (second pixel label region) of the first picture obtained by Gaussian blur processing is used to replace the corresponding negative region (first pixel label region) of the first picture, to obtain a picture that can effectively divide the negative and positive regions of the first picture, focus on retaining the positive region, and reduce the influence of the negative region features on the neural network. Compared with directly inputting the first picture into the same image classification network, the accuracy is obviously improved.

[0089] Further, the embodiment of the present application also provides a computer readable storage medium, which can be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a compact disc read-only memory (CD-ROM), a USB memory, etc. The computer readable storage medium includes an image processing program, and the image processing program, when executed by a processor, implements the following operations:

[0090] S1, obtaining a first picture, processing the first picture based on a preset neural network to obtain a first heat map, and performing a close operation on the first heat map using a preset program to obtain a first close operation heat map;

[0091] S2, obtaining a first matrix by processing the first close operation heat map, performing Gaussian blur processing on the first picture to obtain a first denoising picture, and fusing the first picture and the first denoising picture based on the first matrix and a preset threshold to generate a second picture;

[0092] S3, performing the processing on the first picture as in steps S1 and S2 on the second picture to obtain a third picture.

[0093] The specific implementation of the computer readable storage medium of the present application is substantially the same as that of the above-mentioned image processing method and the specific implementation of the electronic device 10, and will not be repeated here.

[0094] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or other elements inherent to such a process, device, article or method. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, device, article or method including the element.

[0095] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the parts that contribute to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the methods described in various embodiments of the present application.

[0096] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: S1. Obtain a first image, process the first image based on a preset neural network to obtain a first heatmap; perform a closing operation on the first heatmap using a preset program to obtain a first closed operation heatmap; S2. A first matrix is ​​obtained by processing the first closed operation heatmap; a first denoised image is obtained by performing Gaussian blur processing on the first image; and a second image is generated by fusing the first image and the first denoised image based on the first matrix and a preset threshold. S3. Process the second image as in steps S1 and S2 on the first image to obtain the third image; The step of fusing the first image with the first denoised image based on the first matrix and a preset threshold to generate a second image includes: The pixels in the first matrix that have values ​​less than a preset threshold are designated as marked pixels. Based on the marked pixels, the region where the marked pixels are located in the first image is identified and recorded as the first pixel marked region; based on the marked pixels, the region where the marked pixels are located in the first noise-reduced image is identified and recorded as the second pixel marked region; The pixel values ​​in the first pixel marker area are replaced with the corresponding pixel values ​​in the second pixel marker area to obtain the second image.

2. The image processing method as described in claim 1, characterized in that, The process of obtaining the first matrix by processing the first closed-loop heatmap includes: Extract the pixel values ​​of the red channel from the first closed-loop heatmap; The red channel pixel values ​​are normalized using Python's default program. The data obtained from the normalization calculation is processed by Softmax row by row to obtain a new first matrix whose size is unchanged from the first closed operation heatmap.

3. The method for obtaining a first matrix by processing the first closed-loop heatmap as described in claim 2, characterized in that, The normalization calculation of the red channel pixel values ​​using Python's preset program includes: The extracted red channel pixel values ​​are divided by the maximum value of 255 and normalized to the range of [0,1] in decimal form.

4. The method for obtaining a first matrix by processing the first closed-loop heatmap as described in claim 2, characterized in that, The data obtained by normalization calculation is processed row-wise using Softmax to obtain a new first matrix whose size remains unchanged from the first closed-loop heatmap. Where: x is the value of a pixel, exp represents an exponential function with the natural constant e as the base, i is the sequence number of the red channel pixel values ​​in the matrix, and j is the total number of red channel pixel values ​​in the matrix.

5. The image processing method as described in claim 1, characterized in that, The Gaussian blurring process applied to the first image includes: The first image is convolved with the probability density function of a two-dimensional Gaussian distribution. The calculation formula is as follows: Where (x,y) represents the coordinates of a pixel in a two-dimensional image, g(x,y) represents the pixel value after processing, r represents the radius of the convolution kernel, s(x,y) represents the original pixel value, and the function f(u,v) is often called a filtering function.

6. The Gaussian blur processing of the first image as described in claim 5, characterized in that, The filtering function is the probability density function of a two-dimensional Gaussian distribution, and the formula is: dx and dy correspond to the distances from the current horizontal and vertical coordinates to the center of the convolution kernel, respectively. σ is the standard deviation of the Gaussian distribution, and e is the natural constant.

7. An image processing apparatus for performing the image processing method as described in any one of claims 1 to 6, characterized in that, The image processing device includes: The acquisition module is used to acquire a first image, which is a classifying image contained in the test set of the image classification network to be input; The neural network module is used to receive the acquired first image and process the first image based on a preset neural network to obtain a first heatmap; The closing operation module is used to receive the first heat map and use a preset program to perform a closing operation on the first heat map to obtain a first closed operation heat map. The normalization module is used to receive the first closed operation heatmap and extract the pixel values ​​of the red channel from the first closed operation heatmap, and then use a preset program to perform normalization calculation on the pixel values ​​of the red channel. The Softmax module takes over the work of the normalization module and performs Softmax processing on the data obtained from the normalization calculation row by row to obtain a new first matrix whose size is unchanged from the first closed operation heatmap. The Gaussian blur module receives the first image and performs Gaussian blur processing on the first image to obtain the first denoised image; The fusion module fuses the first image with the first denoised image based on the first matrix and a preset threshold to generate a second image.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory stores an image processing program, which, when executed by the processor, implements the image processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes an image processing program, which, when executed by a processor, implements the image processing method as described in any one of claims 1 to 6.

Images