Infrared image deblurring method and system based on coding-decoding

Abstract

The invention relates to an infrared image deblurring method and system based on coding-decoding, and the method comprises the following steps: obtaining original infrared radiation signals of the same calibration object at different sites through infrared imaging equipment which is provided with a coding template in front of an optical lens, and carrying out the nonlinear coding of each original infrared radiation signal; generating a coded infrared image according to the coded infrared radiation signal; obtaining an accurate blurring kernel, and determining a blurring kernel template according to the accurate blurring kernel; determining a fuzzy kernel template for image decoding according to the current distance between the infrared imaging device and the target; and decoding the current infrared image according to the fuzzy kernel template for image decoding. According to the method, the lens is coded, then the fuzzy kernel templates corresponding to different distances can be rapidly and accurately obtained through a reasonable fuzzy kernel calibration program, and a clear infrared decoding result can be accurately obtained in real time according to the fuzzy kernel templates used for decoding.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an infrared image deblurring method and system based on coding and decoding. Background Art

[0002] Since the existing optical structure design cannot produce a perfect focusing system, as well as component assembly errors and other reasons, infrared imaging equipment will still have varying degrees of defocusing when acquiring infrared images, causing the infrared images to be blurred and degraded.

[0003] In order to compensate for the effects of defocus, it is usually necessary to process infrared images through methods such as deblurring algorithms based on variational models and deep learning. Among them, the variational model-based method can effectively remove blur, but it is necessary to iteratively estimate and solve the blur kernel of the image before performing the deblurring calculation. The computational complexity is high and it is difficult to port it to the lower computer for use. It is impossible to achieve real-time deblurring to improve the image quality. The deep learning-based method has good robustness, but its results are heavily dependent on training data and require the lower computer to have strong GPU performance. Simple lower computer equipment is difficult to effectively support the algorithm, which leads to the extremely limited application scope of this method on the lower computer. Summary of the invention

[0004] The purpose of the present invention is to provide an infrared image deblurring method and system based on encoding and decoding, which can quickly and accurately obtain blur kernel templates corresponding to different distances by encoding the lens and then using a reasonable blur kernel calibration procedure, and further obtain clear infrared decoding results in real time and accurately according to the blur kernel template used for decoding, and has strong robustness, practicality and universality.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] On the one hand, a method for deblurring an infrared image based on encoding and decoding is provided, which comprises the following steps:

[0007] At different locations, the original infrared radiation signals of the same calibration object are respectively obtained through an infrared imaging device with a coding template placed in front of the optical lens, and each original infrared radiation signal is nonlinearly encoded to obtain a coded infrared radiation signal;

[0008] generating a coded infrared image according to the coded infrared radiation signal;

[0009] Acquire an accurate blur kernel corresponding to the encoded infrared image, and determine a blur kernel template according to the accurate blur kernel;

[0010] Determine a fuzzy kernel template for image decoding according to the current distance between the infrared imaging device and the target;

[0011] The current infrared image is decoded according to the blur kernel template used for image decoding.

[0012] Preferably, each original infrared radiation signal is nonlinearly encoded according to the following formula:

[0013] U c =f(U)

[0014] Among them, U c is the coded infrared radiation signal obtained after nonlinear coding; U is the original infrared radiation signal of the calibration object obtained by the infrared imaging device without the coding template; f represents the nonlinear response generated when the infrared detector encodes the original infrared radiation signal of the calibration object.

[0015] Preferably, obtaining the precise blur kernel corresponding to the encoded infrared image comprises the following steps:

[0016] Generate the initial blur kernel K of the encoded infrared image i ;

[0017] By using the initial blur kernel K i Get the initial blur kernel K i The corresponding latent clear image U l ;

[0018] Through the potential clear image U l Get the potential clear image U l The corresponding intermediate blur kernel K;

[0019] The intermediate blur kernel K is used as the initial blur kernel K i , and repeat the above steps to perform iterative calculation until the solved intermediate blur kernel K converges, and output the intermediate blur kernel K at this time as the precise blur kernel K0;

[0020] The process of obtaining the precise blur kernel K0 is repeated to obtain the precise blur kernel K0 corresponding to each encoded infrared image.

[0021] Preferably, the potential clear image U is obtained by the following formula: l :

[0022]

[0023] Among them, F(·), F -1 (·) represents Fourier transform and inverse Fourier transform respectively; K T is the initial blur kernel K i Transpose of a matrix; is a first-order gradient operator, is the transpose of the first-order gradient operator; β is the penalty term coefficient greater than 0; μ is the regular term coefficient greater than 0; G is the prior description of sparsity; U c is the coded infrared radiation signal obtained after nonlinear coding.

[0024] Preferably, the intermediate blur kernel K is obtained by the following formula:

[0025]

[0026] in, for is the transpose of , γ is the regularization coefficient greater than 0, U c is the coded infrared radiation signal obtained after nonlinear coding.

[0027] Preferably, the initial blur kernel is generated by the following formula:

[0028]

[0029] Among them, K i is the initial blur kernel, whose size can be KSize*KSize; x and y are the initial blur kernel K i The horizontal and vertical coordinates of the pixel points in ; σ is the standard deviation of the Gaussian kernel function.

[0030] Preferably, determining the blur kernel template according to the precise blur kernel comprises the following steps:

[0031] The precise blur kernel K0 is clipped, and the clipped precise blur kernel K0 is used as a blur kernel template.

[0032] Preferably, determining a blur kernel template for image decoding according to the current distance between the infrared imaging device and the target comprises the following steps:

[0033] If the current distance is ≤ D1, the blur kernel template corresponding to D1 is used as the blur kernel template for image decoding, wherein D1 is the minimum value of the distance between the site and the calibration object;

[0034] If the current distance is ≥ D2, the blur kernel template corresponding to D2 is used as the blur kernel template for image decoding, where D2 is the maximum value of the distance between the site and the calibration object;

[0035] If D1<the current distance<D2, and the current distance is the same as the distance between a certain point and a calibration object, the blur kernel template obtained at the point is used as the blur kernel template for image decoding;

[0036] If D1 < the current distance < D2, and the current distance is different from the distance between any site and the calibration object, bilinear interpolation is performed on the two blur kernel templates corresponding to the two sites adjacent to the current distance to obtain the blur kernel template corresponding to the current distance, and use it as the blur kernel template for image decoding.

[0037] Preferably, decoding the current infrared image according to the blur kernel template for image decoding comprises the following steps:

[0038] Image decoding is performed using the following formula:

[0039]

[0040] Among them, U O is the infrared image obtained after decoding; K p is the blur kernel template used for image decoding; C is a constant greater than 0; F(·) represents the Fourier transform operation, F -1 (·) represents the inverse Fourier transform operation; F(K P )'s conjugate complex number; U c is the coded infrared radiation signal obtained after nonlinear coding.

[0041] On the other hand, an infrared image deblurring system is also provided, comprising:

[0042] Calibration object;

[0043] A coding template, which is used to be placed in front of the optical lens of the infrared imaging device;

[0044] A blur kernel calibration module is used to calibrate blur kernels at different distances to obtain blur kernel templates at different distances;

[0045] A fuzzy kernel template determination module, which is used to determine a fuzzy kernel template for image decoding according to the current distance between the infrared imaging device and the target;

[0046] The image decoding module decodes the current infrared image acquired by the infrared imaging device according to the fuzzy kernel template used for image decoding.

[0047] In summary, the present invention has the following beneficial effects compared with the prior art:

[0048] The present invention encodes the lens so that the acquired image has more high-frequency signals, and then through a reasonable blur kernel calibration procedure, the blur kernel templates corresponding to different distances can be quickly and accurately obtained, and the blur kernel template that matches the current distance between the infrared imaging device and the target and is used for decoding is further determined, so that a clear infrared decoding result can be obtained in real time and accurately, and the method has strong robustness, practicality and universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is an assembly diagram of the optical lens and the coding template in the present invention;

[0050] Figure 2 It is a flowchart of the steps of the infrared image deblurring method based on encoding-decoding in the present invention;

[0051] Figure 3 It is a front view of different types of coding templates in the present invention;

[0052] Figure 4 This is the fuzzy kernel template at different locations within a distance of 8m in the present invention;

[0053] Figure 5 The unprocessed original infrared image and the infrared image processed by the infrared image deblurring method of the present invention;

[0054] Figure 6 It is a structural schematic diagram of the infrared image deblurring system of the present invention. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0056] Example 1

[0057] like Figure 1-2 As shown, this embodiment provides an infrared image deblurring method based on encoding-decoding, which includes the following steps:

[0058] S1. Place a coding template 200 in front of an optical lens F of an infrared imaging device, and make the optical lens F and the coding template 200 coaxially arranged;

[0059] In this embodiment, Figure 3As shown, the coding template 200 is a plate-like structure made of metal or plastic, and is provided with a hollow nonlinear pattern; at the same time, in order to fix the coding template 200, the optical lens F and the coding template 200 can be connected to each other by bonding or the like; the infrared imaging device mainly includes structures such as the optical lens F, a shutter, and an infrared detector, and its structural design and imaging principle are all prior arts and will not be described in detail;

[0060] S2, calibrating the blur kernels at different distances to obtain blur kernel templates at different distances and storing them;

[0061] Specifically, the blur kernels at different distances are calibrated to obtain blur kernel templates at different distances, including the following steps:

[0062] At different locations, the original infrared radiation signals of the same calibration object 100 are respectively obtained by placing an infrared imaging device with a coding template 200, so that each location has an original infrared radiation signal corresponding to it, wherein the calibration object 100 includes various targets (such as checkerboard targets) or structures with other obvious characteristics; the imaging angles of the infrared imaging device when obtaining the original infrared radiation signals at different locations are the same, and the distances between each location and the calibration object 100 are different. Preferably, the distances between two adjacent locations can be the same; for example, Figure 4 As shown, several sites with different distances from the calibration object 100 can be determined within the range of 1008m from the calibration object, and the distance between two adjacent sites is 0.5m.

[0063] The infrared detector of the infrared imaging device performs nonlinear encoding on each original infrared radiation signal according to formula (1), so as to obtain the encoded infrared radiation signal after nonlinear encoding of the original infrared radiation signal of the calibration object 100 at different positions:

[0064] U c =f(U)(1)

[0065] Among them, U c is the coded infrared radiation signal obtained after nonlinear coding; U is the original infrared radiation signal of the calibration object 100 acquired by the infrared imaging device without the coding template 200; f represents the nonlinear response generated when the infrared detector encodes the original infrared radiation signal of the calibration object 100, which can be a nonlinear function;

[0066] Generate a corresponding coded infrared image according to each coded infrared radiation signal; thus, each point corresponds to the original infrared radiation signal of the calibration object 100 acquired at the point, the coded infrared radiation signal obtained after nonlinear decoding of the original infrared radiation signal, and the coded infrared image generated according to the coded infrared radiation signal;

[0067] Generate an initial blur kernel of the encoded infrared image; in this embodiment, the initial blur kernel can be generated based on a Gaussian function, which can be specifically implemented by formula (2):

[0068]

[0069] Among them, K i is the initial blur kernel, and its size may be KSize*KSize. For example, in this embodiment, the value range of KSize may be 10-30; x and y are the initial blur kernel K i The coordinates of the pixel points in the horizontal and vertical directions; σ is the standard deviation of the Gaussian kernel function. In this embodiment, the value range of σ can be 1-10; e is the natural base;

[0070] By using the initial blur kernel K i Get the initial blur kernel K i The corresponding latent clear image U l Specifically, the potential clear image U in this embodiment l It can be obtained by formula (3):

[0071]

[0072] Among them, F(·), F -1 (·) represents Fourier transform and inverse Fourier transform respectively; K T is the initial blur kernel K i Transpose of a matrix; is a first-order gradient operator, is the transpose of the first-order gradient operator; β is the penalty term coefficient greater than 0; μ is the regular term coefficient greater than 0; G is the prior description of sparsity, and:

[0073]

[0074] Through the potential clear image U l Get the potential clear image U l The corresponding intermediate blur kernel K, specifically, the intermediate blur kernel K in this embodiment can be obtained by formula (4):

[0075]

[0076] in, for The transpose of , γ is the regularization coefficient greater than 0;

[0077] The intermediate blur kernel K is used as the initial blur kernel K i , and repeat the above to obtain the potential clear image U l , the step of obtaining an intermediate blur kernel K, so as to perform iterative calculation until the obtained intermediate blur kernel K converges, and output the intermediate blur kernel K at this time as the precise blur kernel K0;

[0078] Repeat the above-mentioned process of obtaining the precise blur kernel K0 to finally obtain the precise blur kernel K0 corresponding to each encoded infrared image, thereby completing the calibration of the blur kernel;

[0079] The precise blur kernel K0 is processed, and the processed precise blur kernel K0 is used as a blur kernel template, and all blur kernel templates are stored; in this embodiment, each precise blur kernel K0 is preferably processed to obtain a blur kernel template corresponding to each encoded infrared image, and the processing includes clipping the precise blur kernel K0 so that the blur kernel template is a square blur kernel, for example, the square blur kernel can be obtained by clipping the pixel points with pixel values ​​of 0-0.5 around the precise blur kernel K0, or it can be obtained by other clipping rules;

[0080] Therefore, each point corresponds to the original infrared radiation signal of the calibration object 100 obtained at the point, the coded infrared radiation signal obtained after nonlinear decoding of the original infrared radiation signal, the coded infrared image generated according to the coded infrared radiation signal, and the blur kernel template corresponding to the coded infrared image;

[0081] S3, obtaining the current distance between the infrared imaging device and the target, and determining a fuzzy kernel template for image decoding according to the current distance;

[0082] Specifically, in this embodiment, the current distance information between the infrared imaging device and the target can be obtained in real time through a Hall sensor, a laser ranging device, etc.;

[0083] If the current distance is ≤ D1, the blur kernel template corresponding to D1 is used as the blur kernel template for image decoding, wherein D1 is the minimum value of the distance between the site and the calibration object 100 (such as 0.5m), and the blur kernel template corresponding to D1 is the blur kernel template obtained at the site with a distance of D1;

[0084] If the current distance is ≥ D2, the blur kernel template corresponding to D2 is used as the blur kernel template for image decoding, wherein D2 is the maximum value of the distance between the site and the calibration object 100 (such as 8m), and the blur kernel template corresponding to D2 is the blur kernel template obtained at the site with a distance of D2;

[0085] If D1 < the current distance < D2, and the current distance is the same as the distance between a certain point and the calibration object 100, the blur kernel template obtained at the point is used as the blur kernel template for image decoding. For example, if the current distance is 6m, and a corresponding blur kernel template is obtained at a point 6m away from the calibration object 100, the blur kernel template is directly used as the blur kernel template for image decoding;

[0086] If D1 < the current distance < D2, and the current distance is different from the distance between any site and the calibration object 100, then the two blur kernel templates corresponding to the two sites adjacent to the current distance are bilinearly interpolated to obtain the blur kernel template corresponding to the current distance, and the blur kernel template is used as the blur kernel template for image decoding; for example, when the current distance is 5.2m, there is no site with a distance of 5.2m from the calibration object 100, and the two sites adjacent to it are sites with distances of 5.0m and 5.5m from the calibration object 100, respectively, and the blur kernel templates corresponding to the two sites are Km and Kn, respectively, then the two blur kernel templates Km and Kn are further bilinearly interpolated to obtain the blur kernel template corresponding to the current distance (i.e., 5.2m);

[0087] And S4, according to the fuzzy kernel template used for image decoding, the current infrared image acquired by the infrared imaging device is decoded; specifically, this embodiment performs image decoding through formula (5):

[0088]

[0089] Among them, U O is the infrared image obtained after decoding; K p is the blur kernel template used for image decoding; C is a constant greater than 0, which is used to represent the signal-to-noise ratio of the damaged image. The larger the value of C is, the better the deblurring effect is; F(·) represents the Fourier transform operation, F -1 (·) represents the inverse Fourier transform operation; F(K P ) is the complex conjugate of .

[0090] like Figure 5 As shown, compared with the unprocessed original infrared image in part (a), after being deblurred by the method in this embodiment, an image with complete contours and clear details can be obtained.

[0091] Therefore, this embodiment can be based on the optical system of the existing infrared imaging equipment, and through simple encoding of the lens, the acquired image has more high-frequency signals, and then through a reasonable blur kernel calibration procedure, the blur kernel template can be quickly and accurately obtained, and the blur kernel template that can be used for decoding can be further determined from the blur kernel template, so that a clear infrared decoding result can be obtained in real time and accurately, solving the defocus problem caused by optical design and assembly, and having strong robustness, practicality, universality and other characteristics.

[0092] Embodiment 2:

[0093] This embodiment provides an infrared image deblurring system for implementing the infrared image deblurring method shown in Embodiment 1, such as Figure 6 As shown, it includes:

[0094] Calibration object 100;

[0095] The coding template 200 is used to be placed in front of the optical lens F of the infrared imaging device and is coaxially arranged with the optical lens F;

[0096] A blur kernel calibration module 300 is used to calibrate blur kernels at different distances to obtain blur kernel templates at different distances, and the process is the same as step S2;

[0097] A blur kernel template determination module 400 is used to determine a blur kernel template for image decoding according to the current distance between the infrared imaging device and the target, and the process is the same as step S3;

[0098] The image decoding module 500 decodes the current infrared image acquired by the infrared imaging device according to the fuzzy kernel template used for image decoding, and the process is the same as step S4.

[0099] In summary, the present invention performs simple encoding on the lens so that the acquired image has more high-frequency signals, and then uses a reasonable blur kernel calibration procedure to quickly and accurately obtain blur kernel templates corresponding to different distances, and further determines the blur kernel template that matches the current distance between the infrared imaging device and the target and is used for decoding, so that a clear infrared decoding result can be obtained in real time and accurately to solve the defocus problem caused by optical design and assembly, and has the characteristics of strong robustness, practicality, universality, etc.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for deblurring infrared images based on encoding and decoding, characterized in that: The steps include: At different locations, the original infrared radiation signals of the same calibration object are respectively obtained through an infrared imaging device with a coding template placed in front of the optical lens, and each original infrared radiation signal is nonlinearly encoded to obtain a coded infrared radiation signal; generating a coded infrared image according to the coded infrared radiation signal; Acquire an accurate blur kernel corresponding to the encoded infrared image, and determine a blur kernel template according to the accurate blur kernel; Determine a fuzzy kernel template for image decoding according to the current distance between the infrared imaging device and the target; The current infrared image is decoded according to the blur kernel template used for image decoding.

2. The infrared image deblurring method according to claim 1, characterized in that: Each original infrared radiation signal is nonlinearly encoded according to the following formula: IN c =f(U) Among them, U c is the coded infrared radiation signal obtained after nonlinear coding; U is the original infrared radiation signal of the calibration object obtained by the infrared imaging device without the coding template; f represents the nonlinear response generated when the infrared detector encodes the original infrared radiation signal of the calibration object.

3. The infrared image deblurring method according to claim 1, characterized in that: Obtaining the precise blur kernel corresponding to the encoded infrared image includes the following steps: Generate the initial blur kernel K of the encoded infrared image i ; By using the initial blur kernel K i Get the initial blur kernel K i The corresponding latent clear image U l ; Through the potential clear image U l Get the potential clear image U l The corresponding intermediate blur kernel K; The intermediate blur kernel K is used as the initial blur kernel K i , and repeat the above steps to perform iterative calculation until the solved intermediate blur kernel K converges, and output the intermediate blur kernel K at this time as the precise blur kernel K0; The process of obtaining the precise blur kernel K0 is repeated to obtain the precise blur kernel K0 corresponding to each encoded infrared image.

4. The infrared image deblurring method according to claim 3, characterized in that: The potential clear image U is obtained by the following formula l : Among them, F(·), F -1 (·) represents Fourier transform and inverse Fourier transform respectively; K T is the initial blur kernel K i Transpose of a matrix; is a first-order gradient operator, is the transpose of the first-order gradient operator; β is the penalty term coefficient greater than 0; μ is the regular term coefficient greater than 0; G is the prior description of sparsity; U c is the coded infrared radiation signal obtained after nonlinear coding.

5. The infrared image deblurring method according to claim 3, characterized in that: The intermediate blur kernel K is obtained by the following formula: in, for is the transpose of , γ is the regularization coefficient greater than 0, U c is the coded infrared radiation signal obtained after nonlinear coding.

6. The infrared image deblurring method according to claim 1, characterized in that: The initial blur kernel is generated by the following formula: Among them, K i is the initial blur kernel, whose size can be KSize*KSize; x and y are the initial blur kernel K i The horizontal and vertical coordinates of the pixel points in ; σ is the standard deviation of the Gaussian kernel function.

7. The infrared image deblurring method according to claim 1, characterized in that: Determining a blur kernel template according to the precise blur kernel comprises the following steps: The precise blur kernel K0 is clipped, and the clipped precise blur kernel K0 is used as a blur kernel template.

8. The infrared image deblurring method according to claim 1, characterized in that: Determining a fuzzy kernel template for image decoding according to the current distance between the infrared imaging device and the target includes the following steps: If the current distance is ≤ D1, the blur kernel template corresponding to D1 is used as the blur kernel template for image decoding, wherein D1 is the minimum value of the distance between the site and the calibration object; If the current distance is ≥ D2, the blur kernel template corresponding to D2 is used as the blur kernel template for image decoding, where D2 is the maximum value of the distance between the site and the calibration object; If D1<the current distance<D2, and the current distance is the same as the distance between a certain point and a calibration object, the blur kernel template obtained at the point is used as the blur kernel template for image decoding; If D1 < the current distance < D2, and the current distance is different from the distance between any site and the calibration object, bilinear interpolation is performed on the two blur kernel templates corresponding to the two sites adjacent to the current distance to obtain the blur kernel template corresponding to the current distance, and use it as the blur kernel template for image decoding.

9. The infrared image deblurring method according to claim 1, characterized in that: According to the fuzzy kernel template used for image decoding, the current infrared image is decoded, including the following steps: Image decoding is performed using the following formula: Among them, U O is the infrared image obtained after decoding; K p is the blur kernel template used for image decoding; C is a constant greater than 0; F(·) represents the Fourier transform operation, F -1 (·) represents the inverse Fourier transform operation; F(K P )'s conjugate complex number; U c is the coded infrared radiation signal obtained after nonlinear coding.

10. An infrared image deblurring system, characterized in that: include: Calibration object; A coding template, which is used to be placed in front of the optical lens of the infrared imaging device; A blur kernel calibration module is used to calibrate blur kernels at different distances to obtain blur kernel templates at different distances; A fuzzy kernel template determination module, which is used to determine a fuzzy kernel template for image decoding according to the current distance between the infrared imaging device and the target; The image decoding module decodes the current infrared image acquired by the infrared imaging device according to the fuzzy kernel template used for image decoding.

Images