Infrared image super-resolution restoration method

Abstract

The application discloses an infrared image super-resolution restoration method, and belongs to the technical field of image super-resolution restoration. The method first calculates a mask coefficient and a transparency coefficient of an original infrared image. Then, an edge-sharpening image is calculated to reduce the edge blur problem caused in the restoration process. Finally, a corresponding relationship between the super-resolution restored image coordinates and the sharpening image coordinates is constructed, and the restoration calculation is performed pixel by pixel according to the corresponding relationship. The infrared image super-resolution restoration method has the advantages of simple principle, is suitable for application fields such as smart cities, satellite remote sensing, fire rescue and artificial intelligence, and has high popularization value.

Description

Technical Field

[0001] This invention belongs to the field of image super-resolution restoration technology, and specifically relates to an infrared image super-resolution restoration method. Background Technology

[0002] Infrared images are formed by infrared imaging devices capturing the thermal radiation of a target in the infrared band. Unlike traditional visible light images, infrared imaging relies on temperature difference imaging. Since the temperature difference between adjacent areas of a target's surface is generally small, infrared thermal images have low contrast and relatively low resolution, resulting in poor detail resolution. To better facilitate engineering applications, super-resolution reconstruction of infrared images is necessary, which has significant application value in fields such as smart cities, satellite remote sensing, fire rescue, and artificial intelligence.

[0003] Currently, image super-resolution restoration techniques mainly target visible light images, with relatively few techniques available for infrared images. Existing infrared image super-resolution restoration methods primarily employ traditional nearest-neighbor interpolation or bilinear interpolation. However, the nearest-neighbor method sets the pixel value to be interpolated to the pixel value of the nearest point in the original image, resulting in jagged edges and block artifacts in the interpolated image. Linear interpolation methods produce severe blurring of image details and jagged edges during image interpolation. In recent years, example-based super-resolution restoration methods and sparse representation-based methods have emerged. Both of these methods have high computational complexity, especially the learning-based methods, which have some uncontrollable factors, resulting in poor generalization ability of the trained super-resolution restoration model and the occurrence of local image distortion and blurring during image computation.

[0004] The main problem with existing infrared image super-resolution restoration methods is the loss of detail information, which easily leads to edge blurring. Therefore, it is particularly important to develop a method for infrared image super-resolution restoration with clear details, which has significant research and practical value for fields such as smart cities, satellite remote sensing, and artificial intelligence. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method for super-resolution restoration of infrared images. The technical solution adopted by this invention is as follows:

[0006] An infrared image super-resolution restoration method includes the following steps:

[0007] Step S1: Calculate the mask coefficient. Calculate the mask coefficient M(x,y) for each pixel based on the original infrared image F(x,y). The value range of variable x is x = 1, 2, ..., L, and the value range of variable y is y = 1, 2, ..., N. L and N are the number of rows and columns of the infrared image, respectively.

[0008] Step S2: Calculate the transparency coefficient. Calculate the transparency coefficient T(x,y) for each pixel based on F(x,y).

[0009] Step S3: Significant sharpening. Calculate the significantly sharpened image R(x,y) of the original infrared image based on the masking coefficient M(x,y) and the transparency coefficient T(x,y), thereby reducing edge blurring issues in the subsequent super-resolution restoration process.

[0010] Step S4: Coordinate mapping. Based on the scale coefficient A of the infrared image super-resolution restoration, construct the correspondence between the coordinates of the super-resolution restored image S(x′, y′) and the coordinates of the saliency-sharpened image R(x, y). Establish the inverse coordinate mapping calculation formula to determine the pixel points that need to be calculated after super-resolution restoration, where x′=1,2,...,round(L·A), y′=1,2,...,round(N·A), and "round()" is the rounding function.

[0011] Step S5: Image restoration. The specific position of the restored pixel in the original image is determined by the inverse coordinate mapping calculation formula in the previous step, and the pixel value of image S(x′, y′) is calculated pixel by pixel. If Sx′ and Sy′ are both integers, that is, the inverse coordinate mapping result coincides with the corresponding pixel in the saliency-sharpened image R(x, y), then the pixel value of R(x, y) is used. If they do not coincide, the restoration method is used to fit the pixel value of that point.

[0012] Preferably, the formula for calculating the mask coefficient M(x, y) in step S1 is:

[0013] Where "*" represents the convolution operator, and the formula for calculating the binary template H(x, y) is: The parameter ε = 20, and the value of σ is set to 3 in the calculation formulas of M(x,y) and H(x,y).

[0014] Preferably, the formula for calculating the transparency coefficient T(x, y) in step S2 is:

[0015] Where “*” represents the convolution operator, and in this formula, the parameters α = 90 and σ = 2.

[0016] Preferably, the formula for calculating the saliency-sharpened image R(x, y) in step S3 is:

[0017] In this formula, the parameter σ = 3.

[0018] Preferably, the formula for calculating the reverse coordinate mapping in step S4 is:

[0019] Where Sx′ and Sy′ are the x and y coordinates of the inverse coordinate mapping based on the scale factor A, respectively, and x′ and y′ are the x and y coordinates of the super-resolution restored image, respectively. and These are the coordinate mapping adjustment factors for the horizontal and vertical axes, respectively.

[0020] Preferably, the calculation formula for image restoration in step S5 is:

[0021] Where S is a subregion centered at coordinate point (Sx′, Sy′), and parameter δ = 10.

[0022] Compared with existing technologies, the advantages of this invention are: The principle of this invention is simple; by calculating the masking coefficient and transparency coefficient of the original infrared image, a significantly sharpened image is obtained, thereby reducing edge blurring during the restoration process. Furthermore, the super-resolution restoration method in step S5 is simple in principle and suitable for applications such as smart cities, satellite remote sensing, fire rescue, and artificial intelligence, and has high potential for widespread application. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the implementation steps of the present invention. Detailed Implementation

[0024] To facilitate understanding of the technical solutions of the present invention by those skilled in the art, the technical solutions of the present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0025] This invention provides a method for super-resolution restoration of infrared images. The technical solution adopted by this invention is as follows:

[0026] Combination Figure 1 The specific implementation steps of an infrared image super-resolution restoration method are as follows:

[0027] Step S1: Calculate the mask coefficients. Calculate the mask coefficients M(x,y) for each pixel based on the original infrared image F(x,y). The range of values ​​for variable x is x = 1, 2, ..., L, and the range of values ​​for variable y is y = 1, 2, ..., N. L and N are the number of rows and columns of the image, respectively.

[0028] The formula for calculating the masking factor M(x,y) is:

[0029] Where "*" represents the convolution operator, and the formula for calculating the binary template H(x, y) is: The parameter ε = 20, and the value of σ is set to 3 in the calculation formulas of M(x,y) and H(x,y).

[0030] Step S2: Calculate the transparency coefficient. Calculate the transparency coefficient T(x,y) for each pixel based on F(x,y).

[0031] The formula for calculating the transparency coefficient T(x, y) is:

[0032] Where “*” represents the convolution operator, and in this formula, the parameters α = 90 and σ = 2.

[0033] Step S3: Significant sharpening. Calculate the significantly sharpened image R(x,y) of the original infrared image based on the masking coefficient M(x,y) and the transparency coefficient T(x,y), thereby reducing edge blurring issues in the subsequent super-resolution restoration process.

[0034] The formula for calculating the saliency-sharpened image R(x,y) is:

[0035] In this formula, the parameter σ = 3.

[0036] Step S4: Coordinate mapping. Based on the scale coefficient A of the infrared image super-resolution restoration, construct the correspondence between the coordinates of the super-resolution restored image S(x′, y′) and the coordinates of the saliency-sharpened image R(x, y). Establish the inverse coordinate mapping calculation formula to determine the pixel points that need to be calculated after super-resolution restoration, where x′=1, 2, ..., round(L·A), y′=1, 2, ..., round(N·A), and "round()" is the rounding function.

[0037] The formula for calculating the reverse coordinate mapping is:

[0038] Where Sx′ and Sy′ are the x and y coordinates of the inverse coordinate mapping based on the scale factor A, respectively, and x′ and y′ are the x and y coordinates of the super-resolution restored image, respectively. and These are the coordinate mapping adjustment factors for the horizontal and vertical axes, respectively.

[0039] Step S5: Image restoration. The specific position of the restored pixel in the original image is determined by the inverse coordinate mapping calculation formula in the previous step, and the pixel value of image S(x′, y′) is calculated pixel by pixel. If Sx′ and Sy′ are both integers, that is, the inverse coordinate mapping result coincides with the corresponding pixel in the saliency-sharpened image R(x, y), then the pixel value of R(x, y) is used. If they do not coincide, the restoration algorithm is used to fit the pixel value of that point.

[0040] The specific formula for image restoration is as follows:

[0041] Where S is the sub-region centered at coordinate point (Sx′, Sy′), and parameter δ = 10. If the overall brightness of the original infrared image is too high, the value of δ can be 20.

[0042] It is important to note that the coordinates (Sx′, Sy′) are the specific positions of the restored pixels in the original image, determined by the inverse coordinate mapping formula. These coordinates are dummy variables, and their values ​​are usually not integers. Therefore, in the above image restoration formula, the sub-region S is a 5×5 sub-region in the saliency-sharpened image R(x, y) centered on the pixel closest to (Sx′, Sy′), and the pixel coordinates within this sub-region are labeled (i, j). If (Sx′, Sy′) falls on a boundary point of R(x, y), then the 5×5 sub-region S is constructed using symmetrical replication.

[0043] Furthermore, steps S1, S2, and S3 all involve convolution operations. In the specific numerical discretization algorithm of this invention, the function... Choose a 7×7 discretization matrix form.

[0044] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for super-resolution restoration of infrared images, characterized in that: The specific steps are as follows: Step S1: Calculate the masking coefficient based on the original infrared image. Calculate the mask coefficient for each pixel. , where variables x The range of values ​​for is ,variable y The range of values ​​for is , L and N These represent the number of rows and columns in the infrared image, respectively. Step S2: Calculate the transparency factor, based on Calculate the transparency coefficient of each pixel. ; Step S3: Significant sharpening, based on masking coefficient and transparency coefficient Calculate the saliency sharpening of the original infrared image ; Step S4: Coordinate mapping, based on the scale coefficients of the infrared image super-resolution restoration. To construct super-resolution restored images Coordinates and saliency sharpening of the image The correspondence between the coordinates is established, and a reverse coordinate mapping calculation formula is created to determine the pixels that need interpolation after super-resolution restoration. , " " is the rounding function; The formula for calculating the reverse coordinate mapping in step S4 is as follows: ,in and They are based on the scaling factor The x and y coordinates of the reverse coordinate mapping, and These are the x and y coordinates of the super-resolution restored image, respectively. and These are the coordinate mapping adjustment factors for the horizontal and vertical axes, respectively. Step S5: Image restoration. The specific location of the restored pixels in the original image is determined using the inverse coordinate mapping calculation formula from the previous step, and the image is calculated pixel by pixel. The pixel value, if and All are integers, meaning the inverse coordinate mapping result and the saliency-sharpened image. If corresponding pixels overlap, then use If the pixel values ​​do not overlap, a restoration method is used to fit the pixel value of that point. The image restoration calculation formula in step S5 is as follows: ,in S Based on coordinate points The sub-region centered on the pixel coordinates is used. Tags, parameters .

2. The infrared image super-resolution restoration method as described in claim 1, characterized in that: The masking coefficient in step S1 The calculation formula is: ,in" " is the convolution operator, a binary template. The calculation formula is: ,parameter ,exist and In the calculation formula, The values ​​are all set to 3.

3. The infrared image super-resolution restoration method as described in claim 1, characterized in that: Transparency coefficient in step S2 The calculation formula is: ,in" " is the convolution operator. In this formula, the parameter..." , .

4. The infrared image super-resolution restoration method as described in claim 1, characterized in that: The image is significantly sharpened in step S3. The calculation formula is: In this formula, " " is the convolution operator, parameter .

Images