Image processing method, device and storage medium

Abstract

The embodiments of the present application disclose an image processing method, device and storage medium, which belong to the field of image processing. In the embodiments of the present application, by enhancing the detail information in the first infrared image to obtain a detail-enhanced image, the details of the low temperature difference area can be made more prominent. In addition, the reference noise reduction intensity of the target noise reduction method is adjusted according to the ISP gain of the first infrared image so that the noise reduction intensity has better scene adaptability. In this case, the first infrared image is subjected to noise reduction processing using the adjusted noise reduction intensity, so that the processed first infrared image retains more detail information. On this basis, by separating the noise reduction process and the detail enhancement process of the first infrared image and fusing the detail-enhanced image and the noise reduction image, the detail loss caused to the image by the noise reduction process is reduced, and the visualization effect of the infrared image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method, device and storage medium. Background Art

[0002] Infrared thermal imaging devices are one of the main image acquisition devices. They primarily use infrared detectors to receive thermal radiation emitted by the surface of the detected target to generate infrared images containing the detected target. When the detected target is in a low temperature environment, the generated infrared image will have poor visualization quality. Therefore, it is urgent to provide an image processing method to improve the visualization quality of infrared images. Summary of the Invention

[0003] The embodiments of the present application provide an image processing method, device, and storage medium that can improve the problem of poor visualization of infrared images in low-temperature-difference scenarios. The technical solution is as follows:

[0004] In one aspect, an image processing method is provided, the method comprising:

[0005] enhancing detail information of the first infrared image to be processed to obtain a detail-enhanced image;

[0006] performing noise reduction processing on the first infrared image according to an image signal processing (ISP) gain of the first infrared image to obtain a noise-reduced image;

[0007] The detail-enhanced image is fused with the noise-reduced image to obtain a second infrared image.

[0008] Optionally, the enhancing detail information of the first infrared image to be processed to obtain a detail-enhanced image includes:

[0009] Determining a plurality of detail pixels in the first infrared image that meet preset conditions;

[0010] Acquire a third infrared image with highlighted details according to the plurality of detail pixels;

[0011] Detail information in the third infrared image is extracted and enhanced to obtain the detail-enhanced image.

[0012] Optionally, the preset condition is that the gradient value of the pixel point is greater than a first reference threshold, or the grayscale value of the pixel point is greater than a second reference threshold.

[0013] Optionally, obtaining a third infrared image with highlighted details based on the multiple detail pixels includes: increasing the grayscale difference between non-detail pixels adjacent to the multiple detail pixels in the first infrared image and the multiple detail pixels to obtain the third infrared image.

[0014] Optionally, increasing the grayscale difference between non-detail pixels adjacent to the plurality of detail pixels in the first infrared image and the plurality of detail pixels to obtain the third infrared image includes:

[0015] Acquiring a first histogram of the first infrared image;

[0016] According to each detail pixel, the number of pixels corresponding to the grayscale value of the corresponding detail pixel in the first histogram is increased by a specified value to obtain a second histogram;

[0017] In ascending order of grayscale values, the number of pixels corresponding to each grayscale value in the second histogram is updated to the sum of the number of pixels corresponding to the corresponding grayscale value and the number of pixels corresponding to the grayscale value immediately before the corresponding grayscale value, to obtain a third histogram;

[0018] Determining a mapped grayscale value corresponding to each grayscale value in the first histogram according to the third histogram and a reference grayscale number;

[0019] The grayscale value of each pixel in the first infrared image is replaced by a mapped grayscale value corresponding to the corresponding grayscale value in the first histogram to obtain the third infrared image.

[0020] Optionally, the performing noise reduction processing on the first infrared image according to the ISP gain of the first infrared image includes:

[0021] Obtain the reference noise reduction intensity corresponding to the target noise reduction method;

[0022] Taking the product of the ISP gain and the reference noise reduction strength as the target noise reduction strength;

[0023] According to the target noise reduction intensity, the first infrared image is subjected to noise reduction processing using the target noise reduction method.

[0024] Optionally, fusing the detail-enhanced image with the noise-reduced image to obtain a second infrared image includes:

[0025] The grayscale value of each pixel in the detail-enhanced image is fused with the grayscale value of the pixel at the corresponding position in the noise-reduced image to obtain the second infrared image.

[0026] In another aspect, an image processing apparatus is provided, the apparatus comprising:

[0027] A detail enhancement module, configured to enhance detail information of the first infrared image to be processed to obtain a detail-enhanced image;

[0028] A noise reduction module, configured to perform noise reduction processing on the first infrared image according to an image signal processing (ISP) gain of the first infrared image to obtain a noise-reduced image;

[0029] A fusion module is used to fuse the detail-enhanced image with the noise-reduced image to obtain a second infrared image.

[0030] Optionally, the detail enhancement module is mainly used to:

[0031] Determining a plurality of detail pixels in the first infrared image that meet preset conditions;

[0032] Acquire a third infrared image with highlighted details according to the plurality of detail pixels;

[0033] Detail information in the third infrared image is extracted and enhanced to obtain the detail-enhanced image.

[0034] Optionally, the preset condition is that the gradient value of the pixel point is greater than a first reference threshold, or the grayscale value of the pixel point is greater than a second reference threshold.

[0035] Optionally, the detail enhancement module is mainly used to increase the grayscale difference between non-detail pixels adjacent to the multiple detail pixels in the first infrared image and the multiple detail pixels to obtain the third infrared image.

[0036] Optionally, the detail enhancement module is mainly used to:

[0037] Acquiring a first histogram of the first infrared image;

[0038] According to each detail pixel, the number of pixels corresponding to the grayscale value of the corresponding detail pixel in the first histogram is increased by a specified value to obtain a second histogram;

[0039] In ascending order of grayscale values, the number of pixels corresponding to each grayscale value in the second histogram is updated to the sum of the number of pixels corresponding to the corresponding grayscale value and the number of pixels corresponding to the grayscale value immediately before the corresponding grayscale value, to obtain a third histogram;

[0040] Determining a mapped grayscale value corresponding to each grayscale value in the first histogram according to the third histogram and a reference grayscale number;

[0041] The grayscale value of each pixel in the first infrared image is replaced by a mapped grayscale value corresponding to the corresponding grayscale value in the first histogram to obtain the third infrared image.

[0042] Optionally, the noise reduction module is mainly used to:

[0043] Obtain the reference noise reduction intensity corresponding to the target noise reduction method;

[0044] Taking the product of the ISP gain and the reference noise reduction strength as the target noise reduction strength;

[0045] According to the target noise reduction intensity, the first infrared image is subjected to noise reduction processing using the target noise reduction method.

[0046] Optionally, the fusion module is mainly used to:

[0047] The grayscale value of each pixel in the detail-enhanced image is fused with the grayscale value of the pixel at the corresponding position in the noise-reduced image to obtain the second infrared image.

[0048] In another aspect, an image processing apparatus is provided, the apparatus comprising:

[0049] processor;

[0050] a memory for storing processor-executable instructions;

[0051] The processor executes the executable instructions in the memory to perform the above-mentioned image processing method.

[0052] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by a computer, the steps of the above-mentioned image processing method are implemented.

[0053] On the other hand, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to perform the steps of the above-mentioned image processing method.

[0054] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0055] In an embodiment of the present application, noise reduction processing is performed on the first infrared image based on its ISP gain, so that the processed first infrared image retains more detail information. Furthermore, by performing noise reduction and detail enhancement on the first infrared image separately and fusing the detail-enhanced and noise-reduced images, compared to performing detail enhancement after noise reduction or vice versa, detail loss caused by the noise reduction process is reduced, thereby improving the visualization of the infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0057] Figure 1 This is a system architecture diagram involved in an image processing method provided in an embodiment of the present application;

[0058] Figure 2 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0059] Figure 3 This is a flow chart of an embodiment of the present application for enhancing detail information of a first infrared image to be processed to obtain a detail-enhanced image;

[0060] Figure 4 This is a schematic diagram of determining detail pixels using a first infrared image and a gradient map of the first infrared image, provided by an embodiment of the present application;

[0061] Figure 5 is a schematic diagram of a first histogram provided in an embodiment of the present application;

[0062] Figure 6 is a schematic diagram of a second histogram provided in an embodiment of the present application;

[0063] Figure 7 is a schematic diagram of a third histogram provided in an embodiment of the present application;

[0064] Figure 8 is a schematic diagram of a third infrared image provided in an embodiment of the present application;

[0065] Figure 9 This is a flow chart of performing noise reduction processing on a first infrared image according to the ISP gain of the first infrared image to obtain a noise-reduced image, provided by an embodiment of the present application;

[0066] Figure 10 is a structural diagram of an image processing device provided in an embodiment of the present application;

[0067] Figure 11 This is a structural diagram of an image processing server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0069] Before explaining the embodiments of the present application in detail, the application scenarios of the embodiments of the present application are first explained.

[0070] The image processing method provided in the embodiment of the present application can be used to process infrared images collected by infrared thermal imaging equipment in low temperature difference scenarios. For example, in a sea navigation scenario, the safety of the sea area where the ship is sailing at night can be observed at any time by installing infrared thermal imaging equipment on the ship, and the infrared image of the sea area where the ship is sailing can be processed by the method of the embodiment of the present application to improve the visualization effect of the sea area captured in the infrared image, thereby improving the safety of the ship's navigation. For another example, in a non-destructive testing scenario of building quality, an infrared image of a target building can be collected by an infrared thermal imaging device to observe whether the building has quality defects such as cracks and hollows, and the infrared image of the target building obtained can be processed by the method of the embodiment of the present application to improve the visualization effect of the target building in the infrared image, thereby improving the accuracy of the judgment.

[0071] It should be noted that the above are only some exemplary application scenarios given in the embodiments of the present application, and do not constitute a limitation on the application scenarios of the image processing method provided in the embodiments of the present application.

[0072] Figure 1 This is a system architecture diagram of an image processing method provided in an embodiment of the present application. Figure 1 As shown, the system includes an image processing device 101 and an infrared thermal imaging device 102. The infrared thermal imaging device 102 can be connected to the image processing device 101 via a wired network or a wireless network.

[0073] The infrared thermal imaging device 102 collects an infrared image containing the detected target and sends the collected infrared image containing the detected target to the image processing device 101. Correspondingly, the image processing device 101 receives the infrared image of the detected target sent by the infrared thermal imaging device 102 and processes the infrared image of the detected target using the method provided in the embodiment of the present application.

[0074] The infrared thermal imaging device 102 may be an infrared thermal imager or other image acquisition device capable of acquiring infrared images, which is not limited in the present embodiment. The image processing device 101 may be a computer device with data processing capabilities, such as a server or a server cluster, or a cloud platform that provides image processing services. Alternatively, the image processing device 101 may be a terminal device, such as a smartphone, tablet computer, laptop computer, personal computer, etc., which is not limited in the present embodiment.

[0075] Optionally, in some possible implementations, the image processing device 101 may also be integrated into the infrared thermal imaging device 102. That is, the image processing device 101 may be an image processing unit included in the infrared thermal imaging device 102. In the following embodiments, the image processing device 101 is used as a server to illustrate the implementation process of the embodiments of the present application.

[0076] Next, the image processing method provided in the embodiment of the present application is introduced.

[0077] Figure 2 This is an image processing method provided by an embodiment of the present application. This method can be applied to an image processing device. The following description will be made by taking the image processing device as a server as an example. Figure 2 As shown, the method includes the following steps:

[0078] Step 201: enhancing detail information of a first infrared image to be processed to obtain a detail-enhanced image.

[0079] The first infrared image may be an original image captured by an infrared thermal imaging device without compression processing. For example, the first infrared image may be an original grayscale image with a grayscale level of 14 bits captured by the infrared thermal imaging device.

[0080] In an embodiment of the present application, the server may process the first infrared image using histogram equalization to enhance detail information in the first infrared image. The server may then extract and enhance detail information from the processed first infrared image to obtain a detail-enhanced image.

[0081] For example, see Figure 3 The server can process the detail information of the first infrared image to be processed through the following steps 2011-2013 to obtain a detail-enhanced image.

[0082] 2011: Determine a plurality of detail pixels in the first infrared image that meet preset conditions.

[0083] The preset condition may be that the gradient value of the pixel point is greater than a first reference threshold, or that the grayscale value of the pixel point is greater than a second reference threshold.

[0084] When the preset condition is that the gradient value of a pixel point is greater than a first reference threshold, the server first obtains the gradient value of each pixel point in the first infrared image, and then uses the pixel points in the first infrared image whose gradient value is greater than the first reference threshold as detail pixels.

[0085] The gradient value of the pixel point (x, y) in the first infrared image is: the sum of the absolute value of the grayscale difference between the grayscale values ​​of the pixel point (x, y) and the neighboring pixel point (x+1, y) and the absolute value of the grayscale difference between the grayscale values ​​of the pixel point (x, y) and the neighboring pixel point (x, y+1).

[0086] It should be noted that the gradient value of the pixel points in the last row of the first infrared image is the same as the gradient value of the pixel points in the same column in the adjacent previous row; similarly, the gradient value of the pixel points in the last column of the first infrared image is the same as the gradient value of the pixel points in the same row in the adjacent previous column.

[0087] For example, Figure 4 Figure A shows the grayscale values ​​of each pixel in the first infrared image. The gradient value of the pixel with a grayscale value of 1 in the upper left corner of Figure A is calculated as: |1-6|+|1-2|=6. Figure 4 The gradient value of each pixel in the A image is obtained by calculation Figure 4 The gradient value of each pixel point in Figure A is obtained as follows Figure 4 The gradient image shown in Figure B, where the pixel value of each pixel in the gradient image is Figure 4 The gradient value of the pixel at the corresponding position in the A image. Assuming the first reference threshold is 5, then Figure 4 The points with gradient values ​​of 6 and 7 in Figure B are Figure 4 The corresponding pixel point in Figure A is Figure 4 The detail pixels in Figure A are as follows: Figure 4 The pixel points marked with circles in Figure A.

[0088] In another implementation, when the preset condition is that the grayscale value of a pixel point is greater than a second reference threshold, the grayscale value of each pixel point in the first infrared image is compared with the second reference threshold, and the pixel points with grayscale values ​​greater than the second reference threshold are regarded as detail pixels of the first infrared image.

[0089] The second reference threshold can be determined based on the distribution of the grayscale values ​​of the pixels in the first infrared image. For example, the median of the grayscale values ​​of all pixels in the first infrared image can be used as the second reference threshold. Alternatively, the second reference threshold can be the mode of the grayscale values ​​of all pixels in the first infrared image. Of course, the second reference threshold can also be pre-set in other ways, and this embodiment of the present application is not limited thereto.

[0090] 2012: Obtain a third infrared image with highlighted details based on multiple detail pixels.

[0091] After obtaining multiple detail pixels, the server can remap the grayscale values ​​of the pixels in the first infrared image according to the multiple detail pixels and the histogram of the first infrared image, so as to widen the grayscale value range of the pixels in the first infrared image, thereby obtaining a third infrared image with highlighted details.

[0092] Exemplarily, the server can weight the number of detail pixels in the histogram of the first infrared image, and then remap the grayscale values ​​of the pixels in the first infrared image according to the weighted histogram, so as to increase the grayscale difference between the non-detail pixels adjacent to the multiple detail pixels in the first infrared image and the multiple detail pixels, thereby obtaining a third infrared image.

[0093] For example, the server may obtain the third infrared image through the following steps A to E.

[0094] Step A: Acquire a first histogram of a first infrared image.

[0095] In one implementation, the server first obtains the number of pixels of each grayscale value in the first infrared image, and then generates a first histogram of the first infrared image with the grayscale value contained in the first infrared image as the horizontal coordinate and the number of pixels as the vertical coordinate.

[0096] For example, Figure 4 The grayscale values ​​of the pixels in the first infrared image shown in Figure A are 1 to 7. The server then counts the number of pixels of each grayscale value in the first infrared image. For example, the number of pixels with a grayscale value of 1 is 4, and the number of pixels with a grayscale value of 2 is 5. Then, the grayscale values ​​1 to 7 are used as the horizontal coordinates, and the number of pixels is used as the vertical coordinate to obtain the following: Figure 5 The first histogram is shown.

[0097] Step B: According to each detail pixel, the number of pixels corresponding to the grayscale value of the corresponding detail pixel in the first histogram is increased by a specified value to obtain a second histogram.

[0098] The server may start from the first detail pixel among the multiple detail pixels, obtain the grayscale value of the first detail pixel, then search for the number of pixels corresponding to the grayscale value in the first histogram based on the grayscale value of the first detail pixel, then add a specified value to the number of pixels, and update the first histogram based on the number of pixels after the specified value is added to obtain an updated first histogram. Thereafter, the server may obtain the grayscale value of the second detail pixel, then search for the number of pixels corresponding to the grayscale value of the second detail pixel in the updated first histogram based on the grayscale value of the second detail pixel, then add a specified value to the number of pixels, and update the updated first histogram again based on the number of pixels after the specified value is added, and so on, until the first histogram is updated for the last time based on the last detail pixel, and the first histogram after the last update is used as the second histogram.

[0099] Optionally, the server may also obtain the number of detail pixels corresponding to each grayscale value in the first histogram, and update the first histogram according to the number of detail pixels corresponding to each grayscale value and the specified value to obtain a second histogram. For example, taking a certain grayscale value in the first histogram as an example, for the convenience of explanation, it is referred to as the first grayscale value. The server obtains the number of detail pixels corresponding to the first grayscale value, that is, obtains the detail pixels whose grayscale value is the first grayscale value, and then calculates the product of the number of detail pixels corresponding to the first grayscale value and the specified value to obtain the first value. After that, the number of pixels corresponding to the first grayscale value in the first histogram is added to the first value to update the number of pixels corresponding to the first grayscale value. For the number of pixels corresponding to each other grayscale value in the first histogram, they can be updated with reference to the above method to obtain a second histogram.

[0100] For example, for Figure 5 The grayscale value in the first histogram is 1, Figure 4 The number of detail pixels with a grayscale value of 1 in the A image is 2. Assuming the specified value is 3, the first value calculated is 6. On this basis, Figure 5 The number of pixels with a grayscale value of 1 in the first histogram is added to the first value, so that the updated number of pixels corresponding to the grayscale value of 1 is 10. Referring to the above method, according to Figure 4 The detail pixels shown in Figure A are Figure 5 The number of pixels corresponding to each grayscale value in the first histogram shown is updated to obtain the following Figure 6 The second histogram is shown.

[0101] Step C: In order of grayscale values ​​from small to large, update the number of pixels corresponding to each grayscale value in the second histogram to the sum of the number of pixels corresponding to the corresponding grayscale value and the number of pixels corresponding to the previous grayscale value of the corresponding grayscale value, to obtain a third histogram.

[0102] Exemplarily, the server calculates the sum of the number of pixels corresponding to the second grayscale value and the number of pixels corresponding to the first grayscale value based on the grayscale values ​​in the second histogram in ascending order, and then updates the number of pixels corresponding to the second grayscale value to the calculated sum. Thereafter, the server calculates the sum of the number of pixels corresponding to the third grayscale value in the second histogram and the updated number of pixels for the second grayscale value, and updates the number of pixels for the third grayscale value to the sum, and so on, until the number of pixels corresponding to the last grayscale value is updated using the above method, thereby obtaining a third histogram.

[0103] For example, Figure 6 The number of pixels with a grayscale value of 1 is 10, and the number of pixels with a grayscale value of 2 is 8. The sum of the number of pixels with a grayscale value of 2 and the number of pixels with a grayscale value of 1 is calculated to be 18. At this time, the number of pixels with a grayscale value of 2 is updated to 18. Figure 6 The number of pixels with a gray value of 3 is 5. The sum of the number of pixels with a gray value of 3 and the number of pixels with an updated gray value of 2 is calculated to be 23. At this time, the number of pixels with a gray value of 3 is updated to 23. Similarly, the following can be obtained: Figure 7 The third histogram is shown.

[0104] Step D: Determine the mapped grayscale value corresponding to each grayscale value in the first histogram according to the third histogram and the reference grayscale number.

[0105] In one implementation, the server first determines the ratio between the number of pixels corresponding to each grayscale value in the third histogram and the number of pixels corresponding to the maximum grayscale value in the third histogram, obtains the probability of each grayscale value in the third histogram, and then multiplies the obtained probability of each grayscale value by the reference grayscale number to obtain the mapped grayscale value corresponding to each grayscale value.

[0106] The reference grayscale number is determined based on the grayscale level of the second infrared image that the user is interested in. That is, the reference grayscale number is the grayscale number indicated by the grayscale level of the second infrared image that the user desires to obtain. For example, if the user desires an 8-bit second infrared image, the reference grayscale number is 255; if the user desires a 10-bit second infrared image, the reference grayscale number is 1023.

[0107] For example, Figure 7The number of pixels corresponding to the maximum grayscale value is 57, and the number of pixels with a grayscale value of 1 is 10. The probability of grayscale value 1 is 10 / 57=0.175. When the reference grayscale number is 255, Figure 5 The mapped grayscale value corresponding to the grayscale value 1 in the first histogram shown is 0.175*255=44.

[0108] Step E: Replace the grayscale value of each pixel in the first infrared image with the mapped grayscale value corresponding to the corresponding grayscale value in the first histogram to obtain a third infrared image.

[0109] For example, for any grayscale value in the first histogram, such as the first grayscale value, the server may determine pixels in the first infrared image whose grayscale value is the first grayscale value, and then update the grayscale values ​​of these pixels to the mapped grayscale values ​​corresponding to the first grayscale value. For each grayscale value in the first histogram, the grayscale value of the corresponding pixel in the first infrared image may be updated using the aforementioned method, thereby generating a third infrared image.

[0110] Still taking the above example as an example, through the above steps, we can know that if Figure 5 In the first histogram shown, the grayscale value 1 corresponds to a mapping grayscale value of 44. At this time, you can Figure 4 In the A figure, the grayscale value of all pixels with a grayscale value of 1 is replaced by 44. Figure 4 After replacing all grayscale values ​​in the first infrared image with the corresponding mapped grayscale values, the following can be obtained: Figure 8 The third infrared image is shown.

[0111] Optionally, the server can first determine multiple detail pixels in the first infrared image that meet preset conditions. Then, when counting the number of pixels at each grayscale value in the first infrared image to generate a histogram, for any detail pixel, the statistical number of the detail pixel is not 1, but a specified value. That is, when counting the number of pixels corresponding to the grayscale value of the detail pixel, the contribution of a detail pixel to the number of pixels corresponding to that grayscale value is not 1, but a specified value. In this way, the second histogram of the first infrared image described above can be directly obtained. Thereafter, the server can refer to the implementation methods of the aforementioned steps C to E to obtain a third infrared image with highlighted details based on the second histogram.

[0112] 2013: Extract and enhance the detail information in the third infrared image to obtain a detail-enhanced image.

[0113] In one implementation, the server may use an edge detection operator to extract detail-highlighted pixel points in the third infrared image, and increase the grayscale value of each detail-highlighted pixel point to obtain a detail-enhanced image.

[0114] The edge detection operator may be a Roberts operator, a Sobel operator, a Laplacian operator, a Canny operator, or other operators capable of extracting detail information from the third infrared image, which is not limited in this application.

[0115] Taking the Laplace operator as an example, the server first performs a convolution operation on the pixels in the third infrared image using the convolution kernel of the Laplace operator to obtain an amplitude corresponding to each pixel. Pixels whose absolute amplitudes are greater than a third reference threshold are then selected as detail-highlighted pixels in the third infrared image. Next, the grayscale value of each detail-highlighted pixel is multiplied by a preset weight to increase the grayscale value of each detail-highlighted pixel. The increased grayscale value replaces the original grayscale value of the corresponding detail-highlighted pixel. The grayscale values ​​of all pixels in the third infrared image, excluding the detail-highlighted pixels, are then replaced with 0 to obtain a detail-enhanced image. The size of the preset weight can be determined based on the degree of visualization of the detail information in the first infrared image.

[0116] In another implementation method, the server may also use a transform domain extraction method to extract detail highlighting pixels in the third infrared image, and increase the grayscale value of each detail highlighting pixel to obtain a detail enhanced image. Among them, the implementation method of using the transform domain extraction method to extract detail highlighting pixels can refer to relevant technologies, and the embodiments of this application will not be repeated here.

[0117] After obtaining the detail-enhanced image, the server may further perform noise reduction processing on the detail-enhanced image to improve the signal-to-noise ratio of the detail-enhanced image.

[0118] Exemplarily, the server may use an N*N filter window to perform noise reduction on the detail-enhanced image. The server first obtains the grayscale values ​​of the central pixel HF(x,y) at the center of the filter window and all neighboring pixels of the central pixel HF(x,y), calculates the grayscale difference between the grayscale value of the central pixel HF(x,y) and the grayscale value of each neighboring pixel, obtains the grayscale difference of all neighboring pixels of the central pixel HF(x,y), and calculates the variance of the grayscale difference of all neighboring pixels of the central pixel HF(x,y). Furthermore, the server may also calculate the average grayscale value of all neighboring pixels of the central pixel HF(x,y). If the variance of the grayscale difference of all neighboring pixels of the central pixel is less than the noise discreteness discrimination threshold and the average grayscale value of all neighboring pixels of the central pixel is greater than the neighborhood uniformity discrimination threshold, the central pixel is considered to be an isolated pixel. At this point, the grayscale value of the central pixel may be replaced with the average grayscale value of all neighboring pixels of the central pixel. If the variance of the grayscale differences of all neighboring pixels of the central pixel is not less than the noise discreteness judgment threshold or the average grayscale value of all neighboring pixels of the central pixel is not greater than the neighborhood uniformity judgment threshold, then the central pixel is considered not to be an isolated pixel. At this time, the grayscale value of the central pixel is kept unchanged.

[0119] Optionally, after denoising the detail-enhanced image using an N*N filter window, the service may further filter the denoised detail-enhanced image using bilateral filtering to obtain an edge-preserving denoised detail-enhanced image. The implementation of filtering the detail-enhanced image using bilateral filtering can be referenced to related art and will not be further described in detail in this embodiment of the present application.

[0120] Step 202: performing noise reduction processing on the first infrared image according to the ISP gain of the first infrared image to obtain a noise-reduced image.

[0121] refer to Figure 9 In one implementation, the server may perform noise reduction processing on the first infrared image according to the ISP gain of the first infrared image through the following steps 2021 to 2023 to obtain a noise-reduced image.

[0122] 2021: Get the reference noise reduction intensity corresponding to the target noise reduction method.

[0123] In the embodiment of the present application, the server may first obtain a reference noise reduction intensity corresponding to the target noise reduction method according to the noise reduction method to be adopted, that is, the target noise reduction method.

[0124] Among them, the target noise reduction method can be Gaussian filtering, median filtering, mean filtering, low-pass filtering or other filtering methods that can reduce the noise of the first infrared image, and this application does not limit this.

[0125] It should be noted that, since each noise reduction method adopts a different noise reduction principle, each noise reduction method has its own corresponding reference noise reduction intensity according to its own parameters.

[0126] Exemplarily, when the Gaussian filter method is used to perform noise reduction processing on the first infrared image, the reference noise reduction strength of the Gaussian filter is determined by the window size and standard deviation σ of the Gaussian filter.

[0127] 2022: The product of the ISP gain of the first infrared image and the reference noise reduction intensity is used as the target noise reduction intensity.

[0128] 2023: According to the target noise reduction intensity, a target noise reduction method is used to perform noise reduction processing on the first infrared image to obtain a noise-reduced image.

[0129] Still taking Gaussian filtering as an example, the target noise reduction intensity for processing the first infrared image using Gaussian filtering is obtained from step 2022, and the weight coefficient of each point in the Gaussian filtering window is determined according to the target noise reduction intensity. The pixel point in the first infrared image corresponding to the center point of the Gaussian filtering window is used as the target pixel point, and the weight coefficient of each point in the Gaussian filtering window is multiplied by the grayscale value of the corresponding pixel point in the first infrared image covered by the Gaussian filtering window to weight the grayscale value of each pixel point in the first infrared image to obtain the weighted grayscale value of each pixel point, and the weighted grayscale values ​​of all pixels covered by the filtering window are added to obtain the Gaussian blur value of the target pixel point, and the grayscale value of the target pixel point is replaced by the blur value of the target pixel point to complete the filtering process of the target pixel point. The Gaussian blur values ​​of all pixels in the first infrared image are obtained in sequence using this method, and the grayscale values ​​of the corresponding pixels are replaced by the Gaussian blur values ​​to obtain a denoised image.

[0130] The above is an exemplary description using the Gaussian filtering method as an example of the target noise reduction method. If the target noise reduction method is other noise reduction methods, the embodiments of the present application will not be described in detail here.

[0131] In this embodiment of the present application, the server adaptively adjusts the reference noise reduction intensity of the target noise reduction method based on the ISP gain of the first infrared image, thereby making the adjusted noise reduction intensity more adaptable to the scene. In this case, using the adjusted noise reduction intensity to denoise the first infrared image can preserve more detailed information in the noise-reduced image.

[0132] Step 203: Fusing the detail-enhanced image with the noise-reduced image to obtain a second infrared image.

[0133] Exemplarily, the server may fuse the grayscale value of each pixel in the detail-enhanced image with the grayscale value of the pixel at the corresponding position in the noise reduction image, thereby obtaining the second infrared image.

[0134] Optionally, the server adds the grayscale value of each pixel in the detail-enhanced image to the grayscale value of the pixel at the corresponding position in the noise reduction image, thereby obtaining the second infrared image.

[0135] It should be noted that the grayscale value obtained by adding the grayscale value of each pixel in the detail-enhanced image to the grayscale value of the pixel at the corresponding position in the denoised image may exceed the maximum allowable grayscale value of the second infrared image. In this case, the obtained grayscale value can be set to the maximum allowable grayscale value. For example, when the grayscale level of the second infrared image is 8 bits, its maximum allowable grayscale value is 255. In this case, if the grayscale value obtained by adding the grayscale value of a pixel in the detail-enhanced image to the grayscale value of the pixel at the corresponding position in the denoised image is greater than 255, the grayscale value of the pixel in the second infrared image can be set to 255.

[0136] In an embodiment of the present application, the reference noise reduction intensity of the target noise reduction method is adjusted based on the ISP gain of the first infrared image to achieve better scene adaptability. In this case, using the adjusted noise reduction intensity to perform noise reduction on the first infrared image can enable the processed first infrared image to retain more detail information. Furthermore, by performing the noise reduction and detail enhancement processes of the first infrared image separately and fusing the detail-enhanced image with the noise-reduced image, compared to performing detail enhancement after noise reduction or vice versa, the loss of image detail caused by the noise reduction process is reduced, thereby improving the visualization effect of the infrared image.

[0137] Secondly, the first infrared image of the present application can be an uncompressed infrared image with a grayscale level of 14 bits. Compared with the compressed infrared image with a grayscale level of 8 bits, after detail enhancement of the 14-bit infrared image, the obtained detail enhanced image contains more detail information, which can further improve the visualization effect of the detail information in the infrared image.

[0138] Finally, in this embodiment of the present application, by highlighting the detail information in the first infrared image, the details in the low-temperature difference region can be made more prominent. Subsequently, by extracting and enhancing the detail information in the third infrared image obtained through the highlighting process, a detail-enhanced image is obtained. This reduces the difficulty of extracting the detail information. Furthermore, obtaining the final infrared image from the detail-enhanced image improves the visualization quality of the obtained infrared image.

[0139] Next, the image processing device provided in the embodiment of the present application is introduced.

[0140] See also Figure 10 An embodiment of the present application provides an image processing device 1000, which can be implemented in the form of software or hardware as a part of the server in the aforementioned embodiment. The device 1000 includes: a detail enhancement module 1001, a noise reduction module 1002 and a fusion module 1003.

[0141] The detail enhancement module 1001 is used to enhance detail information of the first infrared image to be processed to obtain a detail-enhanced image.

[0142] The first infrared image is an uncompressed original image captured by an infrared thermal imaging device. For example, the grayscale level of the first infrared image is 14 bits.

[0143] The detail enhancement module 1001 is mainly used to determine multiple detail pixels that meet preset conditions in the first infrared image, obtain a third infrared image with highlighted details based on the multiple detail pixels, extract and enhance the detail information in the third infrared image, and obtain a detail-enhanced image.

[0144] In this embodiment of the present application, highlighting the detail information in the first infrared image can enhance the details of the low-temperature difference region. Subsequently, detail information in the third infrared image obtained through the highlighting process is extracted and enhanced to obtain a detail-enhanced image. This reduces the difficulty of extracting detail information. Furthermore, obtaining the final infrared image from the detail-enhanced image improves the visualization quality of the obtained infrared image.

[0145] Exemplarily, the detail enhancement module 1001 may obtain the third infrared image by increasing the grayscale difference between the non-detail pixels adjacent to the multiple detail pixels in the first infrared image and the multiple detail pixels.

[0146] For example, the detail enhancement module 1001 can obtain a first histogram of the first infrared image; according to each detail pixel point, increase the number of pixels corresponding to the grayscale value of the corresponding detail pixel point in the first histogram by a specified value to obtain a second histogram; in order of grayscale values ​​from small to large, update the number of pixels corresponding to each grayscale value in the second histogram to the sum of the number of pixels corresponding to the corresponding grayscale value and the number of pixels corresponding to the previous grayscale value of the corresponding grayscale value to obtain a third histogram; according to the third histogram and the reference grayscale number, determine the mapping grayscale value corresponding to each grayscale value in the first histogram; replace the grayscale value of each pixel point in the first infrared image with the mapping grayscale value corresponding to the corresponding grayscale value in the first histogram to obtain a third infrared image.

[0147] The reference grayscale number can be determined based on the grayscale level of the second infrared image that the user is interested in. That is, the reference grayscale number is the grayscale number indicated by the grayscale level of the second infrared image that the user desires to obtain. For example, if the user desires to obtain an 8-bit second infrared image, the reference grayscale number is 255; if the user desires to obtain a 10-bit second infrared image, the reference grayscale number is 1023.

[0148] The noise reduction module 1002 is configured to perform noise reduction processing on the first infrared image according to the image signal processing (ISP) gain of the first infrared image to obtain a noise-reduced image.

[0149] Exemplarily, the noise reduction module 1002 is mainly used to obtain a reference noise reduction intensity corresponding to a target noise reduction method; take the product of the ISP gain and the reference noise reduction intensity as the target noise reduction intensity; and perform noise reduction processing on the first infrared image using the target noise reduction method according to the target noise reduction intensity.

[0150] The target noise reduction method may be a Gaussian filter, a median filter, a mean filter, a low-pass filter, or other filtering method capable of performing noise reduction processing on the first infrared image, and this application does not limit this. It should be noted that since each noise reduction method adopts a different noise reduction principle, each noise reduction method has its own corresponding reference noise reduction intensity based on its own parameters.

[0151] In this embodiment of the present application, the reference noise reduction intensity of the target noise reduction method is adaptively adjusted using the ISP gain of the first infrared image, thereby making the adjusted noise reduction intensity more adaptable to the scene. In this case, using the adjusted noise reduction intensity to denoise the first infrared image can preserve more detailed information in the denoised image.

[0152] The fusion module 1003 is configured to fuse the detail-enhanced image with the noise-reduced image to obtain a second infrared image.

[0153] Exemplarily, the fusion module 1004 is mainly used to fuse the grayscale value of each pixel in the detail-enhanced image with the grayscale value of the pixel at the corresponding position in the noise reduction image to obtain the second infrared image.

[0154] In summary, in an embodiment of the present application, the reference noise reduction intensity of the target noise reduction method is adjusted based on the ISP gain of the first infrared image to make the noise reduction intensity more adaptable to the scene. In this case, using the adjusted noise reduction intensity to perform noise reduction processing on the first infrared image can enable the processed first infrared image to retain more detail information. On this basis, by separately performing the noise reduction process and the detail enhancement process of the first infrared image and fusing the detail-enhanced image with the noise reduction image, compared to performing detail enhancement after noise reduction or noise reduction after detail enhancement, the loss of detail in the image caused by the noise reduction process is reduced, thereby improving the visualization effect of the infrared image.

[0155] It should be noted that the image processing device provided in the above embodiments is merely illustrated by the division of the above functional modules when performing image processing. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing device provided in the above embodiments and the image processing method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0156] Figure 11 FIG. 1 is a schematic diagram of a server structure according to an exemplary embodiment. The image processing function of the server in the above embodiment can be achieved by Figure 11 This is achieved by using the server shown in . This server can be a server in the background server cluster. Specifically:

[0157] The server 1100 includes a central processing unit (CPU) 1101, a system memory 1104 including a random access memory (RAM) 1102 and a read-only memory (ROM) 1103, and a system bus 1105 connecting the system memory 1104 and the CPU 1101. The server 1100 also includes a basic input / output (I / O) system 1106 that facilitates information transfer between various components within the computer, and a mass storage device 1107 for storing an operating system 1113, application programs 1114, and other program modules 1115.

[0158] The basic input / output system 1106 includes a display 1108 for displaying information and an input device 1109, such as a mouse and keyboard, for user input. Both the display 1108 and the input device 1109 are connected to the central processing unit 1101 via an input / output controller 1110 connected to the system bus 1105. The basic input / output system 1106 may also include an input / output controller 1110 for receiving and processing input from a variety of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1110 also provides output to a display screen, printer, or other types of output devices.

[0159] The mass storage device 1107 is connected to the central processing unit 1101 via a mass storage controller (not shown) connected to the system bus 1105. The mass storage device 1107 and its associated computer-readable media provide non-volatile storage for the server 1100. In other words, the mass storage device 1107 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0160] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage devices, CD-ROM, DVD (Digital Versatile Disc) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above-mentioned ones. The above-mentioned system memory 1104 and mass storage device 1107 can be collectively referred to as memory.

[0161] According to various embodiments of the present application, the server 1100 may also be connected to a remote computer on a network such as the Internet for operation. That is, the server 1100 may be connected to the network 1112 via the network interface unit 1111 connected to the system bus 1105, or the network interface unit 1111 may be used to connect to other types of networks or remote computer systems (not shown).

[0162] The memory further includes one or more programs, which are stored in the memory and configured to be executed by the CPU. The one or more programs include instructions for performing the image processing method provided in the embodiment of the present application.

[0163] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a server, the server can perform the image processing method provided in the above embodiment. For example, the computer-readable storage medium can be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device. It is worth noting that the computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, a non-transient storage medium.

[0164] It should be understood that all or part of the steps for implementing the above embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the steps may be implemented in the form of a computer program product. The computer program product may include one or more computer instructions. The computer instructions may be stored in the computer-readable storage medium.

[0165] That is, in some embodiments, a computer program product including instructions is further provided, which, when executed on a computer, enables the computer to execute the image processing method provided in the above embodiments.

[0166] The above description is not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first infrared image to be processed, where the first infrared image is an original image captured by an infrared thermal imaging device and not compressed; Determining a plurality of detail pixels in the first infrared image that meet a preset condition, wherein the preset condition is that a gradient value of the pixel is greater than a first reference threshold; weighting the number of detail pixels in the histogram of the first infrared image according to the plurality of detail pixels, and then remapping the grayscale values ​​of the pixels in the first infrared image according to the weighted histogram to increase the grayscale difference between non-detail pixels adjacent to the plurality of detail pixels in the first infrared image and the plurality of detail pixels, to obtain a third infrared image; Extracting detail-highlighted pixel points from the third infrared image and increasing the grayscale value of each detail-highlighted pixel point to obtain a detail-enhanced image; Using an N*N filter window to perform noise reduction on the detail-enhanced image; performing noise reduction processing on the first infrared image according to an image signal processing (ISP) gain of the first infrared image to obtain a noise-reduced image; The detail-enhanced image after noise reduction processing is fused with the noise reduction image to obtain a second infrared image.

2. The method according to claim 1, characterized in that The weighting of the number of detail pixels in the histogram of the first infrared image, and then remapping the grayscale values ​​of the pixels in the first infrared image according to the weighted histogram, includes: Acquiring a first histogram of the first infrared image; According to each detail pixel, the number of pixels corresponding to the grayscale value of the corresponding detail pixel in the first histogram is increased by a specified value to obtain a second histogram; In ascending order of grayscale values, the number of pixels corresponding to each grayscale value in the second histogram is updated to the sum of the number of pixels corresponding to the corresponding grayscale value and the number of pixels corresponding to the grayscale value immediately before the corresponding grayscale value, to obtain a third histogram; Determining a mapped grayscale value corresponding to each grayscale value in the first histogram according to the third histogram and a reference grayscale number; The grayscale value of each pixel in the first infrared image is replaced by a mapped grayscale value corresponding to the corresponding grayscale value in the first histogram to obtain the third infrared image.

3. The method according to claim 1, characterized in that The performing noise reduction processing on the first infrared image according to the image signal processing (ISP) gain of the first infrared image includes: Obtain the reference noise reduction intensity corresponding to the target noise reduction method; Taking the product of the ISP gain and the reference noise reduction strength as the target noise reduction strength; According to the target noise reduction intensity, the first infrared image is subjected to noise reduction processing using the target noise reduction method.

4. The method according to any one of claims 1 to 3, characterized in that: The step of fusing the detail-enhanced image after noise reduction processing with the noise reduction image to obtain a second infrared image includes: The grayscale value of each pixel in the detail-enhanced image after noise reduction processing is fused with the grayscale value of the pixel at the corresponding position in the noise reduction image to obtain the second infrared image.

5. An image processing device, characterized in that: The device comprises: a detail enhancement module configured to obtain a first infrared image to be processed, where the first infrared image is an uncompressed original image captured by an infrared thermal imaging device; determine a plurality of detail pixels in the first infrared image that meet a preset condition, where the preset condition is that a gradient value of the pixel is greater than a first reference threshold; weight the number of detail pixels in a histogram of the first infrared image based on the plurality of detail pixels, and then remap the grayscale values ​​of the pixels in the first infrared image based on the weighted histogram to increase the grayscale difference between non-detail pixels adjacent to the plurality of detail pixels in the first infrared image and the plurality of detail pixels, thereby obtaining a third infrared image; extract detail highlight pixels in the third infrared image and increase the grayscale value of each detail highlight pixel to obtain a detail-enhanced image; a noise reduction module, configured to perform noise reduction processing on the detail-enhanced image using an N*N filter window; and perform noise reduction processing on the first infrared image according to an image signal processing (ISP) gain of the first infrared image to obtain a noise-reduced image; A fusion module is used to fuse the detail-enhanced image after noise reduction processing with the noise reduction image to obtain a second infrared image.

6. The device according to claim 5, characterized in that The detail enhancement module is mainly used to: obtain a first histogram of the first infrared image; according to each detail pixel, increase the number of pixels corresponding to the grayscale value of the corresponding detail pixel in the first histogram by a specified value to obtain a second histogram; update the number of pixels corresponding to each grayscale value in the second histogram in ascending order of grayscale value to the sum of the number of pixels corresponding to the corresponding grayscale value and the number of pixels corresponding to the previous grayscale value of the corresponding grayscale value to obtain a third histogram; determine the mapped grayscale value corresponding to each grayscale value in the first histogram according to the third histogram and a reference grayscale number; replace the grayscale value of each pixel in the first infrared image with the mapped grayscale value corresponding to the corresponding grayscale value in the first histogram to obtain the third infrared image; The noise reduction module is mainly used to: obtain a reference noise reduction intensity corresponding to a target noise reduction method; use the product of the ISP gain and the reference noise reduction intensity as the target noise reduction intensity; and perform noise reduction processing on the first infrared image using the target noise reduction method according to the target noise reduction intensity; The fusion module is mainly used to fuse the grayscale value of each pixel in the detail enhanced image after noise reduction processing with the grayscale value of the pixel at the corresponding position in the noise reduction image to obtain the second infrared image.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the image processing method according to any one of claims 1 to 4 is implemented.

Images