A method and system for sharpness processing of infrared images

Abstract

The application provides a sharpness processing method and system for an infrared image, specifically comprising the following steps: reading an infrared image to be processed, and performing Gaussian filtering on the infrared image to be processed to obtain a filtered image; performing convolution operation on the filtered image in several directions respectively to obtain at least two direction weight maps; superimposing the at least two direction weight maps to generate a total position weight map; calculating the image sharpness of the infrared image to be processed according to a weighted gradient map calculated based on a convolution gradient map and the total position weight map; and calculating the sharpness value of the processed image by considering the multi-direction convolution operation on the infrared image to be processed, thereby solving the problem that the image sharpness repair accuracy is affected due to the directionality of noise and the large sharpness calculation of a traditional algorithm.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing, and in particular to a method and system for sharpening infrared images. Background Technology

[0002] Infrared image sharpness evaluation is a crucial indicator of infrared image quality, assessing the clarity and detail of an infrared image. In the field of infrared imaging, the unique characteristics of infrared images, such as low contrast, low resolution, and noise, make sharpness evaluation particularly important. Infrared image sharpness evaluation helps us better understand target information within infrared images, improving their recognition and detection capabilities. For example, with infrared cameras used in access control or door locks, algorithms for face recognition, detection, feature extraction, and anti-spoofing work better only when the captured infrared image has high sharpness (generally considered to be clearer).

[0003] During infrared image imaging, vertical stripe noise typically appears due to the influence of noise from circuits, sensor vibrations, and temperature changes in the infrared imaging system. This noise cannot be removed by traditional denoising methods. Therefore, when calculating the sharpness of these images using traditional methods, this stripe noise is included in the sharpness calculation process, resulting in an overestimation of the calculated sharpness. This ultimately leads to the blurry image being judged as sharp, resulting in inaccurate sharpness restoration of the output image. Summary of the Invention

[0004] The purpose of this invention is to solve the problem that the existing technology cannot process directional noise through traditional methods, resulting in inaccurate image sharpness restoration. The invention proposes a sharpness processing method for infrared images, which includes: reading the infrared image to be processed and performing Gaussian filtering on the infrared image to be processed to obtain a filtered image.

[0005] The filtered image is subjected to convolution operations in several directions to obtain weight maps in at least two directions; wherein, the several directions include at least two directions: width direction, height direction and diagonal direction;

[0006] The at least two directional weight maps are superimposed to generate a total position weight map;

[0007] The image sharpness of the infrared image to be processed is calculated based on the weighted gradient map obtained from the convolution gradient map and the total position weight map; wherein, the convolution gradient map is obtained by performing a Laplace convolution operation on the filtered image.

[0008] The present invention provides an infrared image sharpness processing method that considers weight values ​​in several directions, performs gradient convolution operations on the image simultaneously, and then weights the processed image and the image after gradient convolution. Through convolution gradient operations in multiple directions, the final repair result is obtained through calculation. Not only does this method address directional noise interference, but it also prevents the image sharpness repair from being too sharp, resulting in unclear images, and avoids misjudging blurry images as sharp ones, thus ultimately solving the problem of inaccurate output image sharpness repair.

[0009] Furthermore, the filtered image is subjected to convolution operations in several directions to obtain at least two directional weight maps; wherein, the several directions include at least two directions: width direction, height direction, and diagonal direction, specifically:

[0010] Based on the filtered image, a convolution kernel traverses the entire image and combines several directional weight values. Whenever the convolution kernel traverses a pixel, the completeness of the expanded interval of the pixel is determined, and pixel compensation scheme and directional weight matrix operation are performed to obtain at least two directional weight maps corresponding to the pixel. The expanded interval of the pixel is a 3×3 interval centered on the pixel, and the diagonal directions include: the main diagonal direction and the secondary diagonal direction.

[0011] This invention enables the measurement of the infrared image to be processed from multiple angles when dealing with directional noise, such as vertical stripe noise, through convolution operations in multiple directions, thus laying the foundation for noise elimination from multiple angles.

[0012] Furthermore, whenever the convolution kernel traverses to a pixel, it determines the interval position of the pixel and performs pixel compensation scheme and orientation weight matrix operations, specifically as follows:

[0013] When the pixel is determined to be a 3×3 incomplete interval, a pixel compensation scheme is performed where neighboring data is filled in columns with data and all zeros are filled in columns without data. Then, the convolution operation of the compensated pixel expansion interval matrix and at least two directional weight matrices is performed.

[0014] When the expanded pixel is determined to be a complete 3×3 interval, the convolution operation between the expanded pixel interval matrix and at least two directional weight matrices is directly performed.

[0015] This invention also considers the possibility that when the convolution kernel traverses the entire image and enters the edge pixels, the pixel expansion interval may be incomplete, leading to deviation errors in the result after combining the position weight calculation. This can cause deviations in the sharpness calculation of the edge pixels during subsequent image restoration, resulting in blurred edges after image restoration. Therefore, the pixel compensation scheme can ensure that the convolution operation on the edge pixels is at a reasonable level.

[0016] Furthermore, the step of superimposing the at least two directional weight maps to generate a total position weight map specifically involves:

[0017] Based on statistical methods, a filtering calculation is performed on the weight maps of the at least two directions, using rows and columns as coordinates, to obtain a filtered weight coordinate map.

[0018] Based on the filter weight coordinate map, a normalization calculation is performed with the maximum value of the pixel value in the filter weight coordinate map as the base, to obtain the normalized weight coordinate map.

[0019] Based on the filtered weight coordinate map and the normalized weight coordinate map, the weight coefficients are assigned and superimposed to obtain the total position weight map.

[0020] Furthermore, the convolutional gradient map is obtained by performing a Laplace convolution operation on the filtered image, specifically:

[0021] Based on the filtered image, Gaussian kernels of different scales are acquired, and the original Laplace operator is convolved with the Gaussian kernels of different scales to obtain Laplace operators of different scales.

[0022] Based on the Laplace operators of different scales, the gradient of the image is calculated to obtain Laplace gradient maps of different scales.

[0023] Based on the Laplace gradient maps of different scales, noise is removed by merging to obtain the convolutional gradient map.

[0024] Furthermore, the step of calculating the image sharpness of the infrared image to be processed based on the weighted gradient map obtained from the convolutional gradient map and the total position weight map specifically involves:

[0025] Based on the convolutional gradient map and the total position weight map, perform a matrix point-to-point multiplication operation to obtain a weighted gradient map;

[0026] Based on the weighted gradient map, the average value is calculated to obtain the weighted sharpness average value.

[0027] Based on the weighted average sharpness, variance calculation is performed to obtain the image sharpness of the infrared image to be processed;

[0028] Furthermore, the step of reading the infrared image to be processed and performing Gaussian filtering on the infrared image to obtain a filtered image specifically involves:

[0029] Read the input infrared image to be processed;

[0030] Based on the defined Gaussian kernel size and standard deviation, the Gaussian kernel is applied to each pixel of the infrared image to be processed, and a weighted average is calculated to obtain a new value for each pixel.

[0031] Wait for the Gaussian kernel to traverse the infrared image to be processed to obtain the filtered image.

[0032] This invention also proposes an infrared image sharpness processing system, comprising:

[0033] Filtering module, Laplace convolution module, directional convolution module, overall image generation module, and sharpness calculation module;

[0034] The filtering module is used to read the infrared image to be processed and perform Gaussian filtering on the infrared image to be processed to obtain the filtered image.

[0035] The Laplace convolution processing module is used to perform Laplace convolution operations on the filtered image to obtain a convolution gradient map.

[0036] The directional convolution processing module is used to perform convolution operations in several directions on the filtered image to obtain at least two directional weight maps; wherein, the several directions include at least two directions: width direction, height direction and diagonal direction;

[0037] The overall map generation module is used to overlay the at least two directional weight maps to generate an overall position weight map;

[0038] The sharpness calculation module is used to calculate the image sharpness of the infrared image to be processed based on the weighted gradient map obtained by the convolution gradient map and the total position weight map.

[0039] This invention provides an infrared image sharpness processing system, in which the modules are interconnected and respond quickly to each other. By performing filtering, directional weighting, and convolutional gradient processing on the image, the system repairs the image, eliminates interference from directional noise, reduces the problem of excessive sharpness, and solves the problem of inaccurate image sharpness repair.

[0040] Furthermore, the directional convolution processing module is used to perform convolution operations in several directions on the filtered image to obtain at least two directional weight maps; wherein, the several directions include at least two directions: width direction, height direction, and diagonal direction, specifically:

[0041] Based on the filtered image, several directional weight values ​​are combined by traversing the entire image through a convolution kernel. Whenever the convolution kernel traverses a pixel, the completeness of the expanded interval of the pixel is determined, and pixel compensation scheme and directional weight matrix operation are performed to obtain at least two directional weight maps corresponding to the pixel.

[0042] The expanded interval of the pixel is a 3×3 interval centered on the pixel, and the diagonal direction includes a main diagonal direction and a secondary diagonal direction.

[0043] Furthermore, whenever the convolution kernel traverses to a pixel, it determines the interval position of the pixel and performs pixel compensation scheme and orientation weight matrix operations, specifically as follows:

[0044] When the pixel is determined to be a 3×3 incomplete interval, a pixel compensation scheme is performed where neighboring data is filled in columns with data and all zeros are filled in columns without data. Then, the convolution operation of the compensated pixel expansion interval matrix and at least two directional weight matrices is performed.

[0045] When the expanded pixel is determined to be a complete 3×3 interval, the convolution operation between the expanded pixel interval matrix and at least two directional weight matrices is directly performed.

[0046] Furthermore, the overall map generation module is used to overlay the at least two directional weight maps to generate an overall position weight map, specifically as follows:

[0047] Based on statistical methods, a filtering calculation is performed on the weight maps of the at least two directions, using rows and columns as coordinates, to obtain a filtered weight coordinate map.

[0048] Based on the filter weight coordinate map, a normalization calculation is performed with the maximum value of the pixel value in the filter weight coordinate map as the base, to obtain the normalized weight coordinate map.

[0049] Based on the filtered weight coordinate map and the normalized weight coordinate map, the weight coefficients are assigned and superimposed to obtain the total position weight map.

[0050] Furthermore, the Laplace convolution processing module is used to perform Laplace convolution operations on the filtered image to obtain a convolution gradient map, specifically:

[0051] Based on the filtered image, Gaussian kernels of different scales are acquired, and the original Laplace operator is convolved with the Gaussian kernels of different scales to obtain Laplace operators of different scales.

[0052] Based on the Laplace operators of different scales, the gradient of the image is calculated to obtain Laplace gradient maps of different scales.

[0053] Based on the Laplace gradient maps of different scales, noise is removed by merging to obtain the convolutional gradient map.

[0054] Furthermore, the sharpness calculation module is used to calculate the image sharpness of the infrared image to be processed based on the weighted gradient map obtained from the convolution gradient map and the total position weight map, specifically:

[0055] Based on the convolutional gradient map and the total position weight map, perform a matrix point-to-point multiplication operation to obtain a weighted gradient map;

[0056] Based on the weighted gradient map, the average value is calculated to obtain the weighted sharpness average value.

[0057] Based on the weighted average sharpness, variance calculation is performed to obtain the image sharpness of the infrared image to be processed.

[0058] Furthermore, the filtering module is used to read the infrared image to be processed and perform Gaussian filtering on the infrared image to obtain a filtered image, specifically:

[0059] Read the input infrared image to be processed;

[0060] Based on the defined Gaussian kernel size and standard deviation, the Gaussian kernel is applied to each pixel of the infrared image to be processed, and a weighted average is calculated to obtain a new value for each pixel.

[0061] Wait for the Gaussian kernel to traverse the infrared image to be processed to obtain the filtered image. Attached Figure Description

[0062] Figure 1 A flowchart illustrating the steps of an infrared image sharpness processing method according to a certain embodiment of the present invention;

[0063] Figure 2 A block diagram of an infrared image sharpness processing system provided in a certain embodiment of the present invention;

[0064] Figure 3 A block diagram of a directional convolution processing module provided in a certain embodiment of the present invention;

[0065] Figure 4 A block diagram of a general layout generation module provided in a certain embodiment of the present invention;

[0066] Figure 5 A block diagram of a Laplace convolution processing module provided in a certain embodiment of the present invention;

[0067] Figure 6 A block diagram of a sharpness calculation module provided in a certain embodiment of the present invention;

[0068] Figure 7 This is a block diagram of a filtering processing module provided in a certain embodiment of the present invention;

[0069] Figure 8 This is a schematic diagram of an image after 8×8 filtering according to a certain embodiment of the present invention;

[0070] Figure 9 This is a schematic diagram of a width-direction convolution operation provided in a certain embodiment of the present invention;

[0071] Figure 10 This is a schematic diagram of a height-direction convolution operation provided in a certain embodiment of the present invention;

[0072] Figure 11 This is a schematic diagram of a convolution operation in the main diagonal direction provided in a certain embodiment of the present invention;

[0073] Figure 12 This is a schematic diagram of a convolution operation in the secondary diagonal direction provided in a certain embodiment of the present invention;

[0074] Figure 13 This is a schematic diagram of a pixel compensation scheme provided in a certain embodiment of the present invention;

[0075] Figure 14 This is a schematic diagram of Laplace gradient calculation provided in one embodiment of the present invention. Detailed Implementation

[0076] The technical solution of an infrared image sharpness processing method and system according to the present invention will be clearly and completely described below with reference to specific embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] Example 1

[0078] See Figure 1 This invention proposes a method for sharpening infrared images, as detailed below:

[0079] S101, Read the infrared image to be processed and perform Gaussian filtering on the infrared image to be processed to obtain the filtered image.

[0080] As an example of this embodiment, see Figure 8 Taking an 8×8 image as an example, the process involves reading the input infrared image to be processed; performing Gaussian filtering based on the defined Gaussian kernel size and standard deviation; applying the Gaussian kernel to each pixel of the infrared image to be processed; calculating the weighted average value to obtain the new value of each pixel; and waiting for the Gaussian kernel to traverse the infrared image to be processed to obtain a filtered image with the new values ​​for each pixel.

[0081] S102, perform convolution operations in several directions on the filtered image to obtain at least two directional weight maps; wherein, the several directions include at least two directions: width direction, height direction and diagonal direction;

[0082] As an example of this embodiment, see Figure 9 , Figure 10 , Figure 11 and Figure 12 Based on the filtered image, the entire image is traversed using a convolution kernel, incorporating orientation weights. For each pixel traversed, the pixel is matrix-unfolded and orientation weight matrix operations are performed to obtain at least two orientation weight maps corresponding to that pixel. The width-direction convolution operation is as follows: Figure 9 Convolution operation in the height direction is as follows Figure 10 The pixel is expanded into a matrix of 3×3 intervals centered on the pixel. The diagonal directions also include a main diagonal direction and a secondary diagonal direction. The convolution operation in the main diagonal direction is as follows: Figure 11 Convolution operation in the secondary diagonal direction is as follows Figure 12 .

[0083] This invention enables the measurement of the infrared image to be processed from multiple angles when dealing with directional noise, such as vertical stripe noise, through convolution operations in multiple directions, thus laying the foundation for noise elimination from multiple angles.

[0084] As another example of this embodiment, see Figure 13 Each time the convolution kernel traverses a pixel and performs matrix expansion on that pixel, it performs a check on the completeness of the expanded interval. When traversing as shown... Figure 13 When the edge pixel is shown, the expanded interval of the pixel is an incomplete 3×3 interval. Therefore, a pixel compensation scheme is adopted to compensate the pixel by filling the nearest neighbor data with the data column and filling all zeros with the no data column. After the expanded interval is determined to be a complete 3×3 interval, the direction weight matrix operation is performed.

[0085] This invention also considers the possibility that when the convolution kernel traverses the entire image and enters the edge pixels, the pixel expansion interval may be incomplete, leading to deviation errors in the result after combining the position weight calculation. This can cause deviations in the sharpness calculation of the edge pixels during subsequent image restoration, resulting in blurred edges after image restoration. Therefore, the pixel compensation scheme can ensure that the convolution operation on the edge pixels is at a reasonable level.

[0086] S103, superimpose the at least two directional weight maps to generate a total position weight map;

[0087] As an example of this embodiment, a filtered weight coordinate map is obtained by performing a traversal weight map filtering calculation on the at least two directional weight maps using a statistical method, with rows and columns as coordinates. The specific method is as follows:

[0088]

[0089] Traverse the weight graph using the following formula:

[0090]

[0091] Here, weighted graph filtering is for, for example... Figure 9 , Figure 10 , Figure 11 and Figure 12 The four weighted graphs are processed sequentially, repeating the same steps for each. Taking the width direction (W) weight as an example:

[0092] In the formula, E is the mean, w is the image width, h is the image height, and W0 is the mean. (i,j) W1 represents the weight value at coordinate (i, j) in the width direction (W) weight map, i.e., the standard deviation of the pixel value σ. (i,j) The weight value at coordinate (i, j) after filtering the width-direction (W) weight map is the pixel value.

[0093] Based on the filter weight coordinate map, a normalization calculation is performed using the maximum pixel value in the filter weight coordinate map as the base, to obtain the normalized weight coordinate map, specifically:

[0094] The filtered weighted graph is traversed using the following formula:

[0095]

[0096] Here, weight graph normalization is performed sequentially on the four filtered weight graphs calculated using the statistical method mentioned above, repeating the same steps for each. Taking the weight graph filtered in the width direction (W) as an example:

[0097] In the formula, W1 (i,j)The weight value at coordinates (i, j) after filtering the width-direction (W) weighted map, i.e., the pixel value; W2 (i,j) MAX(W1) represents the weight value at coordinate (i, j) in the normalized weight map along the width direction (W), i.e., the pixel value. (i,j) ) represents the largest weight value in the weight map coordinate graph after filtering in the width direction (W), which is the largest value in the data.

[0098] Based on the filtered weight coordinate map and the normalized weight coordinate map, weight coefficients are assigned and superimposed to obtain the total position weight map, specifically as follows:

[0099] Generate the total position weight map using the following formula:

[0100] W (i,j) =β w ×Ww2 (i,j) +β h ×Wh2 (i,j) +β md ×Wmd2 (i,j) +β sd ×Wsd2 (i,j)

[0101] In the formula:

[0102] W (i,j) β represents the weight value at position (i, j) in the total position weight map. w β is the width-direction weighting coefficient. h β is the weighting coefficient in the height direction. md The weighting coefficient for the main diagonal direction, β sd Let β be the weighting coefficient for the secondary diagonal direction, and: w +β h +β md +β sd =1, the coefficient can be adjusted according to the actual situation; Ww2 (i,j) Wh2 represents the weight value at position (i, j) in the normalized width-weighted map. (i,j) Wmd2 represents the weight value at position (i, j) in the normalized width-weighted map. (i,j) Wsd2 represents the weight value at position (i, j) on the normalized main diagonal weight map. (i,j) This represents the weight value at position (i, j) in the normalized subdiagonal weighted graph.

[0103] S104, calculate the image sharpness of the infrared image to be processed based on the weighted gradient map obtained from the convolution gradient map and the total position weight map; wherein, the convolution gradient map is obtained by performing Laplace convolution operation on the filtered image.

[0104] As an example of this embodiment, see Figure 14 Based on the filtered image, Gaussian kernels of different scales are acquired. The original Laplace operator is convolved with the defined Gaussian kernels of different scales to obtain Laplace operators of different scales. Based on the Laplace operators of different scales, the image gradient is calculated to obtain Laplace gradient maps of different scales. Based on the Laplace gradient maps of different scales, merging and noise reduction are performed to obtain the convolution gradient map. Specifically:

[0105] Step 1: Define Gaussian kernels of different scales: In order to obtain Laplace operators of different scales, it is necessary to define Gaussian kernels of different scales. The size and standard deviation of the Gaussian kernel determine the effect and scale of Gaussian blur.

[0106] Step 2: Convolve the original Laplace operator: Convolve the original Laplace operator with Gaussian kernels of different scales to obtain Laplace operators of different scales. The convolution operation can be implemented using the cv::filter2D function in OpenCV.

[0107] Step 3: Output Laplace operators at different scales: After Gaussian blurring, Laplace operators at different scales can be obtained.

[0108] Step 4: Perform gradient calculation on the image using different Laplace operators: Perform the Laplace gradient map generation calculation on the image using different Laplace operators as described in the original steps above.

[0109] Step 5: Laplace Gradient Map Merging: Merge the different Laplace gradient maps from Step 4. Merging methods can include (mean method, weighted average method, maximum value method, wavelet transform method, etc.). Here, wavelet transform is used for merging (wavelet transform is used to decompose and reconstruct Laplace gradient maps at different scales to obtain a merged gradient map. Wavelet transform can preserve edge information and remove noise). Details are as follows:

[0110] (1) Decompose Laplace gradient maps at different scales using wavelet transform: Decompose Laplace gradient maps at different scales to obtain low-frequency and high-frequency coefficients at different scales.

[0111] (2) Weighted average of low-frequency coefficients: Weighted average of low-frequency coefficients at different scales is obtained. Different weights can be set according to the importance of Laplace gradient maps at different scales to preserve edge information.

[0112] (3) Thresholding of high-frequency coefficients: Thresholding of high-frequency coefficients at different scales is performed to remove noise.

[0113] (4) Reconstruct the weighted average low-frequency coefficients and the threshold-processed high-frequency coefficients: Use wavelet transform to reconstruct the weighted average low-frequency coefficients and the threshold-processed high-frequency coefficients to obtain a merged gradient map.

[0114] As another example of this embodiment, based on the convolutional gradient map and the total position weight map, a matrix-corresponding point multiplication operation is performed to obtain a weighted gradient map, as shown in the following formula:

[0115] G (i,j) =W (i,j) ×L (i,j)

[0116] Based on the weighted gradient map, the average value is calculated to obtain the weighted average sharpness, as shown in the following formula:

[0117]

[0118] Based on the weighted average sharpness, variance is calculated to obtain the image sharpness of the infrared image to be processed, as shown in the following formula:

[0119]

[0120] The present invention provides an infrared image sharpness processing method that considers weight values ​​in several directions, performs gradient convolution operations on the image simultaneously, and then weights the processed image and the image after gradient convolution. Through convolution gradient operations in multiple directions, the final repair result is obtained through calculation. Not only does this method address directional noise interference, but it also prevents the image sharpness repair from being too sharp, resulting in unclear images, and avoids misjudging blurry images as sharp ones, thus ultimately solving the problem of inaccurate output image sharpness repair.

[0121] Example 2

[0122] See Figure 2 The present invention also proposes an infrared image sharpness processing system, comprising:

[0123] Filtering module 205, Laplace convolution module 203, directional convolution module 201, overall image generation module 202, sharpness calculation module 204;

[0124] The filtering module is used to read the infrared image to be processed and perform Gaussian filtering on the infrared image to be processed to obtain the filtered image.

[0125] As an example of this embodiment, see Figure 7 The image acquisition module 701 reads the input infrared image to be processed; according to the Gaussian kernel size and standard deviation defined in the second Gaussian kernel storage module 702, the Gaussian kernel is traversed through each pixel of the infrared image to be processed, and the weighted average calculation module 703 performs weighted average calculation to obtain the new value of each pixel; after waiting for the Gaussian kernel to traverse the infrared image to be processed, the filtered image is obtained from the image output module 704.

[0126] The Laplace convolution processing module is used to perform Laplace convolution operations on the filtered image to obtain a convolution gradient map.

[0127] As an example of this embodiment, see Figure 5 The filtered image is obtained by the second filtered image acquisition module 501, the first Gaussian kernel storage module 502 acquires Gaussian kernels of different scales as defined, and the convolution operation module 503 performs convolution operation between the original Laplace operator and the Gaussian kernels of different scales as defined to obtain Laplace operators of different scales.

[0128] Based on the Laplace operators at different scales, the gradient calculation module 504 performs gradient calculation on the image to obtain Laplace gradient maps at different scales.

[0129] Based on the Laplace gradient maps at different scales, the merging and denoising module 505 performs merging and denoising to obtain the convolutional gradient map.

[0130] The directional convolution processing module is used to perform convolution operations in several directions on the filtered image to obtain at least two directional weight maps; wherein, the several directions include at least two directions: width direction, height direction and diagonal direction;

[0131] As an example of this embodiment, see Figure 3Based on the filtered image acquired by the first filtered image acquisition module 301, a convolution kernel traverses the entire image, combining several directional weight values. Whenever the convolution kernel traverses a pixel, the unfolded interval judgment module 303 judges the completeness of the unfolded interval of the pixel. When the unfolded pixel is determined to be a 3×3 incomplete interval, the pixel compensation module 304 performs a pixel compensation scheme that fills nearby data with data columns and fills all zeros with no data columns. The directional weight matrix operation module 305 then performs a convolution operation between the compensated pixel unfolded interval matrix and at least two directional weight matrices. When the unfolded pixel is determined to be a 3×3 complete interval, the directional weight matrix operation module directly performs a convolution operation between the pixel unfolded interval matrix and at least two directional weight matrices to obtain at least two directional weight maps corresponding to the pixel. Among them, the unfolded interval of the pixel in the convolution kernel unfolding module 302 is a 3×3 interval centered on the pixel, and the diagonal direction also includes: the main diagonal direction and the secondary diagonal direction.

[0132] The overall map generation module is used to overlay the at least two directional weight maps to generate an overall position weight map;

[0133] As an example of this embodiment, see Figure 4 Based on the statistical method, the filtering module 402 performs filtering calculations on the weight maps of the at least two directions collected by the first weight map acquisition module 401, using rows and columns as coordinates, to obtain a filtered weight coordinate map.

[0134] Based on the filter weight coordinate map, the normalization processing module 403 performs normalization calculation with the maximum value of the pixel value in the filter weight coordinate map as the base, and obtains the normalized weight coordinate map.

[0135] Based on the filtered weight coordinate map and the normalized weight coordinate map, the total position weight map generation module 404 assigns and superimposes weight coefficients to obtain the total position weight map.

[0136] The sharpness calculation module is used to calculate the image sharpness of the infrared image to be processed based on the weighted gradient map obtained by the convolution gradient map and the total position weight map.

[0137] As an example of this embodiment, see Figure 6 The total position weight map acquired by the second weight map acquisition module 601 and the convolution gradient map acquired by the gradient map acquisition module 602 are sent to the multiplication weighting module 603 to perform matrix point multiplication to obtain the weighted gradient map.

[0138] Based on the weighted gradient map, the mean calculation module 604 performs mean calculation to obtain the weighted sharpness average.

[0139] Based on the weighted average sharpness, the variance calculation module 605 performs variance calculation to obtain the image sharpness of the infrared image to be processed.

[0140] This invention provides an infrared image sharpness processing system, in which the modules are interconnected and respond quickly to each other. By performing filtering, directional weighting, and convolutional gradient processing on the image, the system repairs the image, eliminates interference from directional noise, reduces the problem of excessive sharpness, and solves the problem of inaccurate image sharpness repair.

[0141] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0142] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0143] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for sharpening infrared images, characterized in that, include: Read the infrared image to be processed and perform Gaussian filtering on the infrared image to be processed to obtain the filtered image; The filtered image is subjected to convolution operations in several directions to obtain at least two directional weight maps. These directions include at least two of the following: width, height, and diagonal. Specifically, based on the filtered image, the convolution kernel traverses the entire image, combining several directional weight values. Whenever the convolution kernel reaches a pixel, the completeness of the pixel's expanded interval is determined, and a pixel compensation scheme and directional weight matrix operations are performed to obtain at least two directional weight maps corresponding to that pixel. The expanded interval of the pixel is a 3×3 interval centered on the pixel. The diagonal directions include: the main diagonal direction and the secondary diagonal direction; whenever the convolution kernel traverses to a pixel, the completeness of the expanded interval of the pixel is determined, and pixel compensation scheme and orientation weight matrix operation are performed. Specifically, when the expanded pixel is determined to be a 3×3 incomplete interval, a pixel compensation scheme is performed where data columns are filled with neighboring data and no data columns are filled with 0, and then the convolution operation of the compensated expanded pixel interval matrix with at least two orientation weight matrices is performed; when the expanded pixel is determined to be a 3×3 complete interval, the convolution operation of the expanded pixel interval matrix with at least two orientation weight matrices is directly performed. The at least two directional weight maps are superimposed to generate a total position weight map; The image sharpness of the infrared image to be processed is calculated based on the weighted gradient map obtained from the convolutional gradient map and the total position weight map. Specifically, the convolutional gradient map is obtained by performing a Laplace convolution operation on the filtered image. This involves: acquiring Gaussian kernels of different scales based on the filtered image; performing a convolution operation between the original Laplace operator and the defined Gaussian kernels of different scales to obtain Laplace operators of different scales; calculating the image gradient based on the Laplace operators of different scales to obtain Laplace gradient maps of different scales; and merging and denoising the Laplace gradient maps of different scales to obtain the convolutional gradient map.

2. The method for sharpening infrared images according to claim 1, characterized in that, The step of superimposing the at least two directional weight maps to generate a total position weight map specifically involves: Based on statistical methods, a filtering calculation is performed on the weight maps of the at least two directions, using rows and columns as coordinates, to obtain a filtered weight coordinate map. Based on the filter weight coordinate map, a normalization calculation is performed with the maximum pixel value in the filter weight coordinate map as the base to obtain the normalized weight coordinate map. Based on the filtered weight coordinate map and the normalized weight coordinate map, the weight coefficients are assigned and superimposed to obtain the total position weight map.

3. The method for sharpening infrared images according to claim 1, characterized in that, The image sharpness of the infrared image to be processed is calculated by using the weighted gradient map obtained from the convolution gradient map and the total position weight map. Specifically: Based on the convolutional gradient map and the total position weight map, perform a matrix point-to-point multiplication operation to obtain a weighted gradient map; Based on the weighted gradient map, the average value is calculated to obtain the weighted sharpness average value. Based on the weighted average sharpness, variance calculation is performed to obtain the image sharpness of the infrared image to be processed.

4. The method for sharpening infrared images according to claim 1, characterized in that, The process of reading the infrared image to be processed and performing Gaussian filtering on the infrared image to obtain the filtered image specifically involves: Read the input infrared image to be processed; Based on the defined Gaussian kernel size and standard deviation, the Gaussian kernel is applied to each pixel of the infrared image to be processed, and a weighted average is calculated to obtain a new value for each pixel. Wait for the Gaussian kernel to complete its traversal of the infrared image to be processed, and obtain the filtered image.

5. A sharpness processing system for infrared images, characterized in that, include: Filtering module, Laplace convolution module, directional convolution module, overall image generation module, and sharpness calculation module; The filtering module is used to read the infrared image to be processed and perform Gaussian filtering on the infrared image to be processed to obtain the filtered image. The Laplace convolution processing module is used to perform Laplace convolution operations on the filtered image to obtain a convolution gradient map. Specifically, it involves: acquiring Gaussian kernels of different scales based on the filtered image; performing convolution operations between the original Laplace operator and the defined Gaussian kernels of different scales to obtain Laplace operators of different scales; calculating the gradient of the image based on the Laplace operators of different scales to obtain Laplace gradient maps of different scales; and merging and denoising the Laplace gradient maps of different scales to obtain the convolution gradient map. The directional convolution processing module is used to perform convolution operations in several directions on the filtered image to obtain at least two directional weight maps. The several directions include at least two of the width, height, and diagonal directions. Specifically, based on the filtered image, the convolution kernel traverses the entire image, combining several directional weight values. Whenever the convolution kernel traverses a pixel, the completeness of the expanded interval of the pixel is determined, and a pixel compensation scheme and directional weight matrix operations are performed to obtain at least two directional weight maps corresponding to the pixel. The expanded interval of the pixel is a 3x3 matrix centered on the pixel. The ×3 interval, the diagonal direction includes: the main diagonal direction and the secondary diagonal direction. Whenever the convolution kernel traverses to a pixel, the completeness of the expanded interval of the pixel is determined, and the pixel compensation scheme and the direction weight matrix operation are performed. Specifically: when the expanded pixel is determined to be a 3×3 incomplete interval, a pixel compensation scheme is performed to fill the nearest neighbor data in the data column and fill all 0s in the no data column, and then the convolution operation of the compensated expanded pixel interval matrix with at least two direction weight matrices is performed; when the expanded pixel is determined to be a 3×3 complete interval, the convolution operation of the expanded pixel interval matrix with at least two direction weight matrices is directly performed. The overall map generation module is used to overlay the at least two directional weight maps to generate an overall position weight map; The sharpness calculation module is used to calculate the image sharpness of the infrared image to be processed based on the weighted gradient map obtained by the convolution gradient map and the total position weight map.

6. The infrared image sharpness processing system according to claim 5, characterized in that, The overall map generation module is used to superimpose the at least two directional weight maps to generate an overall position weight map, specifically: Based on statistical methods, a filtering calculation is performed on the weight maps of the at least two directions, using rows and columns as coordinates, to obtain a filtered weight coordinate map. Based on the filter weight coordinate map, a normalization calculation is performed with the maximum pixel value in the filter weight coordinate map as the base to obtain the normalized weight coordinate map. Based on the filtered weight coordinate map and the normalized weight coordinate map, the weight coefficients are assigned and superimposed to obtain the total position weight map.

7. The infrared image sharpness processing system according to claim 5, characterized in that, The sharpness calculation module is used to calculate the image sharpness of the infrared image to be processed based on the weighted gradient map obtained from the convolution gradient map and the total position weight map. Specifically: Based on the convolutional gradient map and the total position weight map, perform a matrix point-to-point multiplication operation to obtain a weighted gradient map; Based on the weighted gradient map, the average value is calculated to obtain the weighted sharpness average value. Based on the weighted average sharpness, variance calculation is performed to obtain the image sharpness of the infrared image to be processed.

8. The infrared image sharpness processing system according to claim 5, characterized in that, The filtering module is used to read the infrared image to be processed and perform Gaussian filtering on the infrared image to obtain a filtered image, specifically: Read the input infrared image to be processed; Based on the defined Gaussian kernel size and standard deviation, the Gaussian kernel is applied to each pixel of the infrared image to be processed, and a weighted average is calculated to obtain a new value for each pixel. Wait for the Gaussian kernel to complete its traversal of the infrared image to be processed, and obtain the filtered image.

Images