Self-adaptive ultra-large field-of-view infrared image non-uniformity correction method

Abstract

The invention discloses a self-adaptive ultra-large field-of-view infrared image non-uniformity correction method, and belongs to the field of infrared image quality control. Firstly, an infrared camera laboratory radiometric calibration coefficient is used as correction input, initial parameters of non-uniformity correction are obtained through multi-scene image statistics, and then detector response with an ultra-large view field center and stable response is selected as a standard reference standard. Correcting the correction parameter of each pixel towards the two sides of the detector by adopting a specific sample matching technology according to a standard reference datum to finish the non-uniformity correction of a single detector module, then establishing a high-precision overlapping region correlation detection constraint model for adjacent detectors, and performing feature point matching correction by utilizing the overlapping pixels of the adjacent detectors to finish the non-uniformity correction of the single detector module. Correction parameters between adjacent detector modules are obtained and transmitted in sequence, so that gray level balancing of the multiple detector modules of the infrared camera is achieved, finally, full-field-of-view image self-adaptive dynamic stretching is achieved through histogram statistics, and non-uniformity correction of the whole image is achieved.

Description

Technical Field

[0001] The invention belongs to the field of infrared image quality control, and in particular relates to an adaptive ultra-large field-of-view infrared image non-uniformity correction method. Background Art

[0002] Infrared imaging uses infrared camera detectors to collect infrared radiation characteristics emitted or reflected by objects, convert the thermal radiation of objects into electrical signals, and finally obtain infrared images that present the temperature information of objects. Since infrared imaging does not require auxiliary light sources, it can be imaged all day long. Due to limitations such as device semiconductor materials and manufacturing processes, there are obvious inconsistencies in the responses of each pixel of the infrared detector, resulting in obvious textures in the infrared images output by the camera, that is, there is non-uniform noise between pixel responses, which directly affects the quality of the image. For large-field-of-view infrared cameras, the number of pixels in the infrared detector will be extremely large, so the non-uniform noise between pixel responses will be more obvious. Therefore, effectively correcting the non-uniformity of infrared images is one of the key technologies that need to be solved in infrared imaging technology.

[0003] There are two main types of traditional satellite-borne infrared image non-uniformity correction methods: one is based on blackbody calibration, and the other is based on large sample statistics. For ultra-large field of view infrared camera imaging, in order to take into account both the large field of view and the resolution requirements, the optical aperture of the camera will be very large. Therefore, for the method based on blackbody calibration, setting up a full optical path calibration blackbody device is not only very expensive, but also increases the reliability risk of the camera. The premise for the effective use of traditional correction methods based on large sample statistics is that the background radiation of the camera imaging is stable. Generally, the temperature resolution of infrared imaging can reach the millikelvin level, which is sensitive to temperature changes. At the same time, the ultra-large field of view infrared camera adopts a large-aperture off-axis three-mirror optical system, which cannot guarantee the pupil matching requirements of the traditional infrared system. As the satellite observation attitude changes, the camera background radiation changes rapidly and is more unstable. Therefore, this type of method cannot achieve a good non-uniformity correction effect for infrared images that are sensitive to the background environment, especially ultra-large field of view images. In addition, ultra-large field-of-view infrared cameras are composed of multiple detector modules. Due to the limitations of semiconductor materials and manufacturing processes of detector devices, the responses of each pixel of the infrared detector are obviously inconsistent, which will cause the images obtained when multiple detectors of the camera are imaged to produce different grayscale images. Due to the small number of feature points in the overlapping area between multiple adjacent detector modules, feature matching and registration methods cannot be used. In addition, due to the high authenticity and quantitative requirements of the data, it is difficult to use the grayscale histogram matching method to change the relative correspondence between the original infrared radiation and the code value. Therefore, the above traditional methods are difficult to meet the needs of ultra-large field-of-view infrared image non-uniformity correction. Summary of the invention

[0005] The purpose of this invention is to propose an adaptive ultra-large field of view infrared image non-uniformity correction method, which can effectively solve the problem of ultra-large field of view infrared image non-uniformity correction difficulty.

[0006] (II) Technical solution

[0007] In order to achieve the above-mentioned purpose and solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0008] The infrared camera laboratory radiation calibration coefficient is used as the correction input, and the initial parameters of non-uniformity correction are obtained through multi-view image statistics.

[0009] The detector response at the center of the ultra-large field of view and with stable response is selected as the standard reference benchmark, and the calibration parameters of each pixel are corrected by using a specific sample matching technology on both sides of the detector with the standard reference benchmark to complete the non-uniformity correction of a single detector module;

[0010] Based on the infrared camera optical system, a high-precision overlapping area correlation detection constraint model for adjacent detectors is established, which overcomes the problems of insufficient feature points in the overlapping area and changes in the quantitative relationship of code values. The overlapping pixels of adjacent detectors are used for feature point matching correction, and the correction parameters between adjacent detector modules are obtained and transferred in sequence, thereby realizing the grayscale balancing of multiple detector modules of the infrared camera.

[0011] Finally, based on specific application scenarios, histogram statistics are used to realize adaptive dynamic stretching of the full-field image and achieve non-uniformity correction of the entire image.

[0012] (III) Effective income

[0013] 1. The present invention proposes an adaptive ultra-large field of view infrared image non-uniformity correction method, which can effectively solve the problem of ultra-large field of view infrared image non-uniformity correction;

[0014] 2. The present invention breaks the reliance of the previous infrared camera image correction on the on-board calibrator, and adopts adaptive correction technology to reduce the disadvantage that the infrared detector is sensitive to background temperature changes.

[0015] 3. The correction of infrared images by the present invention is not limited by the camera size, optical field of view and detector linear array length. It reduces the complexity of hardware design and the design risk of satellite-borne cameras, broadens the design concept of satellite-borne infrared cameras, and can achieve the correction effect of ultra-large field of view infrared images at a relatively low cost, effectively improving the image quality.

[0016] 4. Since the non-uniformity of the entire image is largely affected by the differences in actual objects, the present invention fully considers the differences in actual objects and effectively removes the samples affected by objects through the sample screening rules, thereby reducing the influence of object factors in the single-view image. The sample screening rules and correction model fully consider the features in different scene images, making them more adaptable and facilitating the rapid selection of samples in the scene to achieve non-uniformity correction of single-view images.

[0017] 5. Due to the lack of overlapping features in ultra-large field of view images and the high requirements for code value data quantification, it is difficult to use conventional methods to perform grayscale balancing of multiple detector module images. The present invention constrains the overlapping areas of different detectors in the optical system and performs correlation detection of the overlapping areas under the constraints to obtain the optimal overlapping area, thereby avoiding problems such as insufficient feature points and changes in the quantitative relationship of code values. It can effectively balance the grayscale between modules for images of different complex objects between multiple modules and has better adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the method for correcting non-uniformity of ultra-large field of view infrared images of the present invention;

[0019] Figure 2 This is a schematic diagram of grayscale balancing for different detector modules. DETAILED DESCRIPTION

[0020] The present invention will be explained and illustrated in detail below in conjunction with the accompanying drawings.

[0021] The main technical solutions of the present invention are as follows Figure 1 As shown, an adaptive ultra-large field of view infrared image non-uniformity correction method mainly includes the following steps:

[0022] Step 1: Initialize the calibration model acquisition: obtain the initial calibration parameters and calibration model through the radiation calibration data in the low temperature environment of the ground laboratory;

[0023] Step 2: Acquisition of initial correction parameters: Using statistics of the original multi-view images on orbit, new initial correction parameters are obtained according to the initialization correction model in step 1;

[0024] Step 3: Image initialization correction: Use the initial correction parameters and initial correction model in step 2, and bring in the single-view image to be corrected for initialization correction;

[0025] Step 4: Standard reference element selection: by comparing the calibration data with the original statistical data, multiple pixels at the center of the detector linear array with stable responses are selected as standard reference elements;

[0026] Step 5, pixel-by-pixel statistical sample screening: for the image initialized and corrected in step 3, the samples of adjacent pixels are adaptively screened pixel by pixel on both sides of the image based on the standard reference pixel;

[0027] Step 6, generate new correction parameters: perform statistical regression on the original sample data in the image corresponding to the adjacent pixels selected in step 4 to achieve sample matching and obtain new correction coefficients;

[0028] Step 7, generating a single detector module correction image: using the new correction coefficient to perform non-uniformity correction on other images corresponding to the single detector, to complete the non-uniformity correction of the single detector module image.

[0029] Step 8, multi-detector module pixel registration point acquisition: using the initial registration overlap area and registration feature points between multiple detector modules obtained in the laboratory, for the image to be registered, based on the camera optical system constraint relationship, calculate new high-precision registration overlap area and registration feature points.

[0030] Step 9, grayscale balancing between multiple detector modules of ultra-large field of view infrared images: Based on the registration overlap area pixels and registration feature points between modules obtained in step 8, the sample screening algorithm and matching method in steps 5 and 6 are applied to calculate the non-uniformity correction coefficients between multiple detector modules to achieve rapid balancing of ultra-large field of view infrared images. Figure 2 shown.

[0031] Step 10: Adaptive dynamic stretching of the entire image based on the scene: For the balanced infrared image, dynamic stretching is performed based on the histogram statistics of different scenes to achieve a better visual effect.

[0032] Furthermore, in step 1, the calibration data is used to obtain the correction model and correction coefficient using the least square method. The correction model is:

[0033] X′=k0×X+b0 (1)

[0034] Where X is the DN value of a certain pixel in the camera image, X′ is the DN value of the corresponding pixel after non-uniformity correction, k0 is the correction slope coefficient (related to the response rate of the pixel), and b0 is the correction intercept coefficient.

[0035] In step 2, the new initial non-uniformity correction coefficients k1 and b1 in formula 1 are calculated by using a histogram statistical algorithm for the multi-view image.

[0036] For step 3, step 4, and step 5, the initial correction coefficients are mainly used to screen samples in the image that participate in calculating new correction coefficients.

[0037] The effective samples are screened by calculating the adjacent pixel image data in two steps:

[0038] The first step is as follows:

[0039]

[0040]

[0041] In formula 2, x std_i is the gray value of the reference pixel in the i-th frame, std_i represents the reference pixel, and there are n frames in the image; x i is the gray value of the adjacent pixel in the i-th frame; dx i Represents the gray value difference between the reference pixel and the adjacent pixel in the i-th frame; is the standard deviation distribution of the gray value difference between the reference pixel and the adjacent pixels in all imaging frames; Represents the mean gray value of all imaging frames of adjacent pixels; Represents the mean gray value of all imaging frames of the reference pixel; Represents the difference between the mean grayscale values ​​of all imaging frames of the adjacent pixel and the reference pixel. , it can be identified as a correlation area and used as the sample point screened out in the first step.

[0042] The second step is as follows:

[0043]

[0044] Among them, (k 1,i , b 1,i )and The initial correction coefficients correspond to the neighboring i and reference pixel std_i respectively. C is the difference in grayscale values ​​obtained after the initial correction between the neighboring i and reference pixel std_i. Theoretically, for uniform objects, when C≈0, it means that the correction coefficient has a high correction accuracy. Assuming that the object information obtained by adjacent pixels in the image is not very different, based on this assumption, samples with smaller absolute value differences in adjacent pixel C values ​​after initial correction are suitable for effectively calculating new coefficients, while samples with larger absolute value differences in C values ​​need to be eliminated.

[0045] Furthermore, the histogram distribution of the difference C is obtained, and the point with the highest probability of appearing in the histogram is taken as C0, and its probability of appearing is taken as P0. Samples with a value greater than 0.5*P0 are selected in the image as the sample points screened out in the second step.

[0046] The intersection of the sample points screened out in the above two steps is taken as the valid sample points actually screened out.

[0047] In steps 5 and 6, according to the correction model, the selected specific samples are matched by using statistical methods to calculate new correction coefficients (k2, b2). The calculation formulas are shown in Formula 4 and Formula 5, and the non-uniformity correction of the image corresponding to a single detector module is completed.

[0048]

[0049] X′=k2×X+b2 (5)

[0050] Among them, yy i , is the gray value of the i-th valid sample selected from the standard reference element and the mean gray value of all valid samples; i , is the gray value of the i-th valid sample screened out from adjacent pixels and the mean of all valid sample gray values; k2 is the slope coefficient of the new non-uniformity correction, and b2 is the intercept coefficient of the new non-uniformity correction.

[0051] In steps 8 and 9, the initial registration point is selected according to the constraint relationship of the camera optical system, and the optimal correlation area is calculated by the correlation calculation formula at the initial registration point to determine the new registration pixel, such as formula 6 and formula 7. After obtaining the latest registration pixel of the image, the two modules to be registered are equivalent to the single pixel in steps 5 and 6. The same sample screening and statistical regression algorithm are used to perform statistical calculations on the entire module to obtain the non-uniformity correction coefficient between adjacent detector modules. Analogous to the infrared non-uniformity correction method of pixel-by-pixel transfer, the non-uniformity correction coefficient generated by the inter-module balancing is applied to the pixels of other detector modules to be registered, that is, the grayscale balancing between multiple detector modules is completed.

[0052]

[0053] Among them, x n With y n They represent the pixel values ​​of the overlapping area in the module to be registered and the pixel values ​​of the overlapping area in the reference module respectively, N is the total number of pixels in the overlapping area; p0 is the intercept representation value of the image to be registered x after the overlapping area is leveled, p1 is the slope representation value of the image to be registered x after the overlapping area is leveled, and It is the intercept and slope of the two-point correction of the image x to be registered, and R is the similarity of the overlapping area. By fine-tuning the calibration area and finding the area with the highest similarity (R is the largest), the algorithm in steps 5 and 6 can be analogized.

[0054] After non-uniformity correction and inter-module balancing, in order to improve the visual effect of the region of interest, linear stretching is first performed based on Formula 8, at this time γ = 1; then adaptive correction is performed on the actual scene based on Formula 9, and finally the features of the region of interest are enhanced. The dynamic stretching model is as follows:

[0055]

[0056] γ=-0.3×(I AVERAGE ) (9)

[0057] In formulas 8 and 9, I is the pixel value of the original image, I min is the minimum value among all pixels, I max is the maximum value among all pixels, H and L represent the upper and lower grayscale values ​​of the specified stretched code value, respectively. The grayscale value is obtained by statistically analyzing the distribution of grayscale code values ​​in the entire scene and adaptively obtaining it. AVERAGE is the average pixel value of the image after linear stretching, and γ is the correction parameter of adaptive Gamma correction.

[0058] Since the non-uniformity of the entire image is largely affected by the differences in actual objects, the present invention fully considers the differences in actual objects and effectively removes samples affected by objects through sample screening rules, thereby reducing the influence of object factors in single-view images. The sample screening rules and correction model fully consider the features in different scene images, making them more adaptable and facilitating the rapid selection of samples in the scene to achieve non-uniformity correction of single-view images.

[0059] Due to the lack of overlapping features in ultra-large field of view images and the high requirements for code value data quantification, it is difficult to use conventional methods to perform grayscale balancing of multiple detector module images. The present invention constrains the overlapping areas of different detectors in the optical system and performs correlation detection of the overlapping areas under the constraints to obtain the optimal overlapping area, thereby avoiding the problems of insufficient feature points and changes in the quantitative relationship of code values. It can effectively perform grayscale balancing between modules for images of different complex objects between multiple modules, and has better adaptability.

[0060] The present invention proposes an adaptive ultra-large field-of-view infrared image non-uniformity correction method. Aiming at the problem of difficulty in non-uniformity correction of ultra-large field-of-view infrared images that are sensitive to temperature background changes of ground objects, the present invention adopts a specific sample matching technology to correct the correction parameters of each pixel of the detector, and completes the non-uniformity correction of a single detector module. Then, based on the optical system of an infrared camera, a high-precision overlapping area correlation detection constraint model for adjacent detectors is established, and feature point matching correction is performed using overlapping pixels of adjacent detectors. Correction parameters between adjacent detector modules are obtained and transferred in sequence, thereby realizing grayscale balancing of multiple detector modules of the infrared camera. Finally, based on specific application scenarios, histogram statistics are used to realize adaptive dynamic stretching of full-field-of-view images, and non-uniformity correction of the entire image is realized.

[0061] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. An adaptive ultra-large field of view infrared image non-uniformity correction method, characterized in that: The steps include: Step 1: Initialize the calibration model acquisition: obtain the initial calibration parameters and calibration model through the radiation calibration data in the low temperature environment of the ground laboratory; Step 2: Acquisition of initial correction parameters: Using statistics of the original multi-view images on orbit, new initial correction parameters are obtained according to the initialization correction model in step 1; Step 3: Image initialization correction: Use the initial correction parameters and initial correction model in step 2, and bring in the single-view image to be corrected for initialization correction; Step 4: Standard reference element selection: by comparing the calibration data with the original statistical data, multiple pixels at the center of the detector linear array with stable responses are selected as standard reference elements; Step 5, pixel-by-pixel statistical sample screening: for the image initialized and corrected in step 3, the samples of adjacent pixels are adaptively screened pixel by pixel on both sides of the image based on the standard reference pixel; Step 6, generate new correction parameters: perform statistical regression on the original sample data in the image corresponding to the adjacent pixels selected in step 4 to achieve sample matching and obtain new correction coefficients; Step 7, generating a single detector module correction image: using the new correction coefficient to perform non-uniformity correction on other images corresponding to the single detector, and completing the non-uniformity correction of the single detector module image; Step 8, multi-detector module pixel registration point acquisition: using the initial registration overlap area and registration feature points between multiple detector modules obtained in the laboratory, for the image to be registered, based on the camera optical system constraint relationship, calculate new high-precision registration overlap area and registration feature points; Step 9, grayscale balancing between multiple detector modules of ultra-large field of view infrared images: Based on the registration overlap area pixels and registration feature points between modules obtained in step 8, the sample screening algorithm and matching method in steps 5 and 6 are applied to calculate the non-uniformity correction coefficients between multiple detector modules to achieve rapid balancing of ultra-large field of view infrared images; Step 10: Adaptive dynamic stretching of the entire image based on the scene: For the balanced infrared image, dynamic stretching is performed based on the histogram statistics of different scenes to achieve a visual effect.

2. The method for adaptively correcting non-uniformity of ultra-large field of view infrared images according to claim 1, characterized in that: For step 1, the calibration data is used to obtain the correction model and correction coefficient using the least squares method. The correction model is: X′=k0×X+b0 (1) Among them, X is the DN value of a pixel in the camera image, X′ is the DN value of the corresponding pixel after non-uniformity correction; k0 is the correction slope coefficient, which is related to the response rate of the pixel, and b0 is the correction intercept coefficient.

3. The method for adaptively correcting non-uniformity of ultra-large field of view infrared images according to claim 1, characterized in that: In step 2, the new initial non-uniformity correction coefficients k1 and b1 in formula 1 are calculated by using a histogram statistical algorithm for the multi-view image.

4. The method for adaptively correcting non-uniformity of ultra-large field-of-view infrared images according to claim 1, characterized in that: For step 3, step 4, and step 5, the initial correction coefficient is used to screen samples in the image that participate in calculating the new correction coefficient; The effective samples are screened by calculating the adjacent pixel image data in two steps: The first step is as follows: dx i =x std_i -x i In formula 2, x std_i is the gray value of the reference pixel in the i-th frame, std_i represents the reference pixel, and there are n frames in the image; x i is the gray value of the adjacent pixel in the i-th frame; dx i Represents the gray value difference between the reference pixel and the adjacent pixel in the i-th frame; is the standard deviation distribution of the gray value difference between the reference pixel and the adjacent pixels in all imaging frames; Represents the mean gray value of all imaging frames of adjacent pixels; Represents the mean gray value of all imaging frames of the reference pixel; Represents the difference between the mean grayscale values ​​of all imaging frames of the adjacent pixel and the reference pixel. When , it can be identified as a correlation area and used as the sample point screened out in the first step; The second step is as follows: Among them, (k 1,i , b 1,i )and They correspond to the initial correction coefficients of the neighboring i and the reference pixel std_i respectively; C is the difference in grayscale values ​​between the neighboring i and the reference pixel std_i after initial correction; theoretically, for uniform objects, when C≈0, it means that the correction coefficient has a high correction accuracy; assuming that the object information obtained by adjacent pixels in the image is not very different, based on this assumption, samples with smaller absolute value differences in adjacent pixel C values ​​after initial correction are suitable for actually calculating new coefficients, while samples with larger absolute value differences in C values ​​need to be eliminated. Get the histogram distribution of the difference C, take the point with the highest probability of appearing in the histogram as C0, and take its probability of appearing as P0; select samples greater than 0.5*P0 in the image as the sample points screened out in the second step.; The intersection of the sample points screened out in the above two steps is taken as the actual screened out valid sample points.

5. The method for adaptively correcting non-uniformity of ultra-large field of view infrared images according to claim 1, characterized in that: In steps 5 and 6, according to the correction model, a statistical method is used to match the selected specific samples and calculate the new correction coefficient (k2, b2); the calculation formula is shown in formula (4) and formula (5), and the non-uniformity correction of the image corresponding to the single detector module is completed; X′=k2×X+b2 (5) Among them, yy i , The gray value of the i-th valid sample selected by the standard reference element and the mean gray value of all valid samples; xx8, is the gray value of the i-th valid sample screened out from adjacent pixels and the mean of all valid sample gray values; k2 is the slope coefficient of the new non-uniformity correction, and b2 is the intercept coefficient of the new non-uniformity correction.

6. The method for adaptively correcting non-uniformity of ultra-large field of view infrared images according to claim 1, characterized in that: In the steps 8 and 9, the initial registration point is selected according to the constraint relationship of the camera optical system, and the optimal correlation area is calculated by the correlation calculation formula at the initial registration point to determine the new registration pixel. After obtaining the latest registration pixel of the image, the two modules to be registered are equivalent to the single pixel in steps 5 and 6. The same sample screening and statistical regression algorithm are used to perform statistical calculations on the module as a whole to obtain the non-uniformity correction coefficient between adjacent detector modules. By analogy with the infrared non-uniformity correction method of pixel-by-pixel transfer, the non-uniformity correction coefficient generated by the inter-module balancing is applied to the pixels of other detector modules to be registered, that is, the grayscale balancing between multiple detector modules is completed. Among them, x n With y n They represent the pixel values ​​of the overlapping area in the module to be registered and the pixel values ​​of the overlapping area in the reference module respectively, N is the total number of pixels in the overlapping area; p0 is the intercept representation value of the image to be registered x after the overlapping area is leveled, p1 is the slope representation value of the image to be registered x after the overlapping area is leveled, and It is the intercept and slope of the two-point correction of the image x to be registered, and R is the similarity of the overlapping area. By fine-tuning the calibration area and finding the area with the highest similarity, the algorithm in steps 5 and 6 can be analogized.

7. The method for adaptively correcting non-uniformity of ultra-large field of view infrared images according to claim 6, characterized in that: After non-uniformity correction and inter-module balancing, in order to improve the visual effect of the region of interest, linear stretching is performed based on formula (8), at this time γ = 1; then adaptive correction is performed on the actual scene based on formula 9, and finally the characteristics of the region of interest are enhanced; the dynamic stretching model is as follows: γ=-0.3×(I AVERAGE ) (9) In formulas 8 and 9, I is the pixel value of the original image, I min is the minimum value among all pixels, I max is the maximum value among all pixels, H and L represent the upper and lower grayscale values ​​of the specified stretched code value, respectively. The grayscale value is obtained by statistically analyzing the distribution of grayscale code values ​​in the entire scene and adaptively obtaining it. AVERAGE is the average pixel value of the image after linear stretching, and γ is the correction parameter of adaptive Gamma correction.

Images