Real-time parameter recursive quadric surface fitting correction method, device and equipment based on FPGA

Abstract

Embodiments of the present application provide a real-time parameter recursive quadric surface fitting correction method, device and equipment based on FPGA, and a computer readable storage device. The method comprises: acquiring an infrared image; the infrared image comprises an infrared scene image and a shell radiation image; determining an optimal matching function based on the infrared image; obtaining a restored image based on the optimal matching function; performing quadric surface fitting on the restored image, and performing recursive processing on each parameter of the fitted quadric surface to obtain a corrected image. In this way, the problem of brightening of the four corners of the infrared image caused by the thermal effect of the lens of the uncooled infrared imaging system is solved.

Description

Technical Field

[0001] The embodiments of this application relate to the field of data processing, and more particularly to a method, apparatus, device, and computer-readable storage device for real-time parameter recursive quadratic surface fitting and correction based on FPGA. Background Technology

[0002] When the ambient temperature rises, the lens temperature of an uncooled thermal imager gradually increases, which affects the lens's transfer function. This causes the infrared radiation to be non-uniform across the entire detector's focal plane, resulting in dark spots near the four corners of the focal plane. Over time, these dark spots gradually move towards the center of the detector, causing the corners to darken.

[0003] Therefore, there is an urgent need for a real-time calibrated optical non-uniformity correction method to solve the above problems and improve image quality. Summary of the Invention

[0004] According to an embodiment of this application, a real-time parameter recursive quadratic surface fitting and correction scheme based on FPGA is provided.

[0005] In a first aspect of this application, a real-time parametric recursive quadratic surface fitting and correction method based on FPGA is provided. The method includes:

[0006] Acquire infrared images; the infrared images include infrared scene images and outer shell radiation images;

[0007] Based on the infrared image, determine the optimal matching function;

[0008] Based on the optimal matching function, the restored image is obtained;

[0009] The restored image is fitted with a quadratic surface, and the parameters of the fitted quadratic surface are recursively processed to obtain the corrected image.

[0010] Furthermore, determining the optimal matching function based on the infrared image includes:

[0011] The optimal matching function is determined using the following formula:

[0012]

[0013] in, I ( x , y (This refers to an infrared image;)

[0014] g ( x , y ( ) is a radiation image of the outer shell.

[0015] Furthermore, obtaining the restored image based on the optimal matching function includes:

[0016] The restored image can be obtained using the following formula:

[0017]

[0018] when g ( x ,y)= ( x , y When S ( γ The minimum value is used to obtain the restored image.

[0019] Furthermore, the recursive processing of the parameters of the fitted quadratic surface includes:

[0020] The parameters of the fitted quadratic surface are recursively processed using a second-order infinite impulse response filter.

[0021] In a second aspect of this application, a real-time parametric recursive quadratic surface fitting and correction device based on an FPGA is provided. The device includes:

[0022] The acquisition module is used to acquire infrared images; the infrared images include infrared scene images and outer shell radiation images;

[0023] The determination module is used to determine the optimal matching function based on the infrared image;

[0024] The restoration module is used to obtain the restored image based on the optimal matching function;

[0025] The correction module is used to perform quadratic surface fitting on the restored image and recursively process the parameters of the fitted quadratic surface to obtain the corrected image.

[0026] In a third aspect of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0027] In a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to the first aspect of this application.

[0028] The real-time parameter recursive quadratic surface fitting and correction method based on FPGA provided in this application acquires an infrared image, which includes an infrared scene image and an outer shell radiation image. Based on the infrared image, an optimal matching function is determined. Based on the optimal matching function, a restored image is obtained. The restored image is then fitted with a quadratic surface, and the parameters of the fitted quadratic surface are recursively processed to obtain a corrected image. This method solves the problem of bright corners in infrared images caused by lens thermal effects in uncooled infrared imaging systems.

[0029] It should be understood that the description in the Summary Section is not intended to limit the key or essential features of the embodiments of this application, nor is it intended to restrict the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0030] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0031] Figure 1 A flowchart of an FPGA-based real-time parameter recursive quadratic surface fitting and correction method according to an embodiment of this application is shown;

[0032] Figure 2 A schematic diagram of an infinite impact response implementation scheme according to an embodiment of this application is shown;

[0033] Figure 3 A block diagram of an FPGA-based real-time parameter recursive quadratic surface fitting and correction device according to an embodiment of this application is shown.

[0034] Figure 4 A schematic diagram of a terminal device or server suitable for implementing embodiments of this application is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0036] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0037] Figure 1 A flowchart of a real-time parametric recursive quadratic surface fitting and correction method based on an embodiment of the present disclosure is shown. The method includes:

[0038] S110, acquire infrared image.

[0039] The infrared image includes an infrared scene image and an outer shell radiation image.

[0040] Infrared images are typically acquired using uncooled thermal imagers.

[0041] S120, Based on the infrared image, determine the optimal matching function.

[0042] This disclosure is based on the idea of ​​fitting. It assumes a smooth quadratic surface that minimizes the error between the surface and the sum of squares of all pixels in the image. This surface is obtained from the real-time image and subtracted to obtain the corrected image. Let the image of the infrared scene information be... f ( x , y ), the outer shell radiation image is g ( x , y According to the principle of energy superposition, the infrared image output by the seeker is:

[0043]

[0044] That is, if the hull radiation image in image I is known... g ( x , y Then, the image of the scene can be restored:

[0045]

[0046] Furthermore, based on the principle of optimization, a residual sum of squares function is introduced, and the optimal matching function is determined by minimizing the sum of squares of the errors:

[0047]

[0048] in, I ( x , y (This refers to an infrared image;)

[0049] g (x , y ( ) is a radiation image of the outer shell.

[0050] S130, Based on the optimal matching function, the restored image is obtained.

[0051] The restored image can be obtained using the following formula:

[0052]

[0053] when g ( x , y )= ( x , y When S ( γ The minimum value is used to obtain the restored image.

[0054] S140, the restored image is fitted with a quadratic surface, and the parameters of the fitted quadratic surface are recursively processed to obtain a corrected image.

[0055] In some embodiments, quadratic surface fitting is performed on the entire image, and then... (x, y) is the equation of the surface:

[0056]

[0057] Where M is the matrix of coordinate positions;

[0058] A represents the matrix coefficients;

[0059] Solving using the least squares method, we get:

[0060]

[0061] Substituting the obtained coefficients, we get the restored image:

[0062] ;

[0063] Furthermore, the above-mentioned fitted quadratic surface is calculated in real time, and the fitting result and accuracy are related to the current scene in real time. In order to reduce the accuracy loss caused by temporal noise to the quadratic surface fitting, recursive parameter filtering is further performed on each parameter obtained after the quadratic surface calculation to reduce the temporal fluctuation of the parameters, thereby reducing image jitter.

[0064] In some embodiments, a second-order infinite impulse response filter is used for parameter recursive filtering. For example... Figure 2 As shown, where n≥0, I (n) represents the current parameter input. I(n) is the buffer calculated from the previous frame. and These are the filter weight coefficients, used to adjust the proportion of the current frame. Since the image data of this frame is calculated from the image data of the previous frame, and the image data of the previous frame is calculated from the image data of the frame before that, a recursive processing effect is achieved, reducing the temporal fluctuation of parameters and thus reducing image jitter.

[0065] Specifically, the two-dimensional gradients of the images are calculated separately:

[0066]

[0067]

[0068] Furthermore, solve for the coefficient A:

[0069] Generate coordinate matrix N:

[0070]

[0071] Calculate the restoration coefficient:

[0072]

[0073] In the second-order state, it includes 5 restoration coefficients ( );

[0074] Where N represents the pixel position and I represents the pixel grayscale, the expanded result is:

[0075]

[0076] For each restoration coefficient, calculate separately:

[0077]

[0078] Furthermore, perform recursive time-domain filtering using parameters:

[0079]

[0080] in, and These are the filter weighting coefficients;

[0081] In summary, the corrected image is obtained:

[0082] M = [x^2 y^2 xy xy]

[0083] .

[0084] According to the embodiments of this disclosure, the following technical effects are achieved:

[0085] By adding a real-time quadratic surface recursive correction algorithm to the traditional blackbody calibration, the residual non-uniformity problem introduced to the infrared imaging system by the lens when the ambient temperature changes is solved.

[0086] In practical applications, quadratic surface fitting can be performed according to the real-time scene, and the parameters of the fitted quadratic surface can be recursively processed to reduce the temporal fluctuations of the ground and surface, avoid overall image flicker, and solve the problem of the adaptability of infrared imaging systems in high-heat environments.

[0087] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0088] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.

[0089] Figure 3 A block diagram of an FPGA-based real-time parametric recursive quadratic surface fitting and correction device 300 according to an embodiment of this application is shown below. Figure 3 As shown, the device 300 includes:

[0090] The acquisition module 310 is used to acquire infrared images; the infrared images include infrared scene images and outer shell radiation images;

[0091] The determining module 320 is used to determine the optimal matching function based on the infrared image;

[0092] The restoration module 330 is used to obtain the restored image based on the optimal matching function;

[0093] The correction module 340 is used to perform quadratic surface fitting on the restored image and recursively process the parameters of the fitted quadratic surface to obtain the corrected image.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0095] Figure 4 A schematic diagram of a terminal device or server suitable for implementing embodiments of this application is shown.

[0096] like Figure 4 As shown, the terminal device or server includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 502 or a program loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0097] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0098] Specifically, according to embodiments of this application, the above method flow steps can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined in the system of this application.

[0099] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0101] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor. The names of these units or modules do not, in certain circumstances, constitute a limitation on the unit or module itself.

[0102] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the methods described in this application.

[0103] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions claimed in this application.

Claims

1. A real-time parameter recursive quadric surface fitting correction method based on FPGA, characterized in that, The method comprises the following steps: acquiring an infrared image; the infrared image comprises an infrared scene image and a shell radiation image; determining a best matching function based on the infrared image; Based on the idea of fitting, assuming a smooth quadratic surface, the sum of the square error of all pixels of the image is minimum, in real-time image to obtain the surface and subtract, get the corrected image; infrared scene information image is f ( x , y ), the shell radiation image is g ( x , y ), according to the principle of energy superposition, the output of the infrared image seeker is: That is, if the shell radiance image in the image I is known g (x, y), then the image of the scene can be recovered: further, according to the optimization principle, a residual square sum function is introduced, and the best matching function is determined by minimizing the square sum of errors: wherein I ( x , y ) is an infrared image; g x y ) a radiation image of the outer shell;​​ based on the best matching function, a restored image is obtained; the restored image is obtained through the following formula: When g ( x , y ) = ( , x , y ), S( γ ) is the minimum value, and the restored image is obtained. the restored image is subjected to quadratic surface fitting, and each parameter of the fitted quadratic surface is recursively processed to obtain a corrected image; The whole image is fitted with a quadric surface, and let (x, y) be the quadric surface equation: wherein M is a matrix of coordinate positions; A is a matrix coefficient; according to the least square method, the following equation is obtained: the obtained coefficient is substituted into to obtain the restored image: 。 2. The method of claim 1, wherein, the recursive processing of each parameter of the fitted quadratic surface comprises the following steps: the recursive processing of each parameter of the fitted quadratic surface is performed through a second-order infinite impulse response filter.

3. A real-time FPGA-based parameter recursive quadric surface fitting correction device for implementing the real-time FPGA-based parameter recursive quadric surface fitting correction method according to claim 1, characterized in that, The method comprises the following steps: an acquisition module is configured to acquire an infrared image; the infrared image comprises an infrared scene image and a shell radiation image; a determination module is configured to determine a best matching function based on the infrared image; a restoration module is configured to obtain a restored image based on the best matching function; a correction module is configured to perform quadratic surface fitting on the restored image, and recursively process each parameter of the fitted quadratic surface to obtain a corrected image.

4. The apparatus of claim 3, wherein, the recursive processing of each parameter of the fitted quadratic surface comprises the following steps: the recursive processing of each parameter of the fitted quadratic surface is performed through a second-order infinite impulse response filter.

5. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program, characterized in that, The processor executes the computer program to realize the method of any one of claims 1-2.

6. A computer readable storage device having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the method of any one of claims 1-2.

Images