An on-line real-time information enhancement method for high-resolution uncooled infrared multispectral camera and application thereof

Abstract

The application discloses an on-orbit real-time information enhancement method of a high-resolution uncooled infrared multispectral imaging load and application thereof, and comprises the following processing steps: data preselection, data preprocessing, speed-height ratio matching algorithm, attitude correction algorithm and frame inter-shift superposition; the data preselection selects to-be-calculated signals and removes redundant information through a signal counter at a detector end, so as to reduce data storage pressure of a calculation unit; the data preprocessing is used for non-uniform correction, reference correction and blind element processing of a thermal infrared image; the speed-height ratio matching is used for correcting information dislocation at a pixel level caused by speed and imaging frame frequency mismatch in a system push-broom process; the attitude correction is used for correcting image information mismatch problems caused by system attitude changes; and the frame inter-shift superposition superposes the corrected thermal infrared image according to frame inter-shift. The application can obtain an infrared multispectral image with improved detection sensitivity.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of uncooled infrared image processing, in particular to a high-resolution uncooled infrared multispectral camera on-line real-time information enhancement method and application thereof. BACKGROUND

[0002] Infrared is a kind of electromagnetic wave, also known as infrared light or infrared ray. According to Planck's radiation theorem, any object with an absolute temperature higher than absolute zero (-273℃) can radiate electromagnetic energy, and the intensity of the radiation energy of the object is related to the temperature of the object and the radiation emission capacity of the surface. Based on this theory, infrared detectors have emerged, which detect the infrared radiation intensity of objects through thermal photoelectric and other physical effects. Among them, the most widely used is the photon detector, which is divided into refrigeration type (low temperature) infrared detector and non-refrigeration (room temperature) type infrared detector according to the working temperature.

[0003] Based on the space infrared multispectral imaging load of the refrigeration type infrared detector, a mature technical system has been formed after years of development, which has been well applied in the field of aerospace. In China's Fengyun meteorological satellite series, ocean observation satellite series, resource satellite, environment satellite and high-resolution satellite series, high-resolution infrared multispectral imaging load based on refrigeration type infrared detector is loaded. Due to the limitation of the material characteristics of the detector, the detector often needs to work in a low temperature environment below 100K, and the related load is large in volume and weight, and the development cost is high. The existence of this problem leads to the difficulty of commercial remote sensing aerospace in developing high-resolution infrared multispectral imaging load, and the practical application is less.

[0004] In recent years, uncooled detectors represented by vanadium oxide (VOx) microbolometer have developed rapidly. The detector development process is similar to the traditional CMOS (Complementary Metal Oxide Semiconductor) device process, which can realize large array and does not require refrigeration in the working environment. The largest scale in the laboratory has reached 4Kx4K. With the improvement of process and the performance of thermal sensitive material, the original sensitivity of uncooled infrared detector also develops to better than 50mK level. From the aspects of cost, volume and ease of use, the development of uncooled infrared detector is an important direction of future infrared detection technology, and also an important development direction of future commercial aerospace infrared imaging load.

[0005] The performance of uncooled infrared imaging system directly affects the image quality. The main influencing factor is the dynamic range, which is constrained by many factors, not only related to the charge storage capacity of infrared focal plane array and signal detection method, but also related to the object and its background. Therefore, it is difficult to improve the imaging quality of infrared imaging system by improving the performance of infrared detector alone, and the noise of infrared image is one of the main factors limiting the imaging quality of infrared imaging system. The noise of infrared image has great randomness, which is difficult to evaluate and analyze. Non-cooled infrared detector technology has more application scenarios, so it has become the main research direction of various countries.

[0006] However, due to the imaging mechanism of uncooled infrared imaging system and the reasons of uncooled infrared imaging system itself, the infrared image still has the defects of low image contrast, image blur, low image signal-to-noise ratio, etc. These defects seriously hinder the application of uncooled infrared imaging system in various fields. Therefore, removing the noise in infrared image, improving the signal-to-noise ratio and contrast of infrared image, and enhancing the edge and line of infrared image, etc. have a very important role in improving the image quality of uncooled infrared imaging system. At present, there are two methods to effectively improve the image quality of uncooled infrared imaging system: the first method is to continuously research higher performance uncooled infrared detector, and the second method is to add digital image processing function in uncooled infrared imaging system. In digital image processing technology, image enhancement method is widely concerned because of its simple, effective and universal characteristics. Under the current conditions, infrared image enhancement method is an effective and simple method to improve the quality of infrared image. At present, people are researching to improve the performance of infrared detector, and also exploring real-time infrared image enhancement method, the basic purpose of which is to develop modular infrared image real-time enhancement processing system, so as to effectively improve the quality of infrared image and increase the effective distance of uncooled infrared imaging system.

[0007] The principle of thermal radiation detection of uncooled devices is essentially different from that of refrigeration type photon detectors. The uncooled thermal infrared device converts the infrared radiation signal into the resistance value change of the photosensitive element to evaluate the radiation intensity of the signal, which is essentially a thermistor. The refrigeration type detector is mainly a radiation detector made by using photoelectric effect. The photon type device directly absorbs the energy of the photons in the detector, so that the motion state changes to generate an electric signal. Compared with the photon type infrared detector, the sensitivity of the uncooled detector decreases by about one order of magnitude. Therefore, in order to achieve better imaging effect, in addition to increasing the signal quantity by increasing the integration time and adjusting the integration capacitance in the design of the detector driving circuit, it is necessary to improve the sensitivity of the infrared multispectral image by means of image processing. In the prior art, the infrared image can be enhanced by using image processing methods such as corrosion expansion, opening and closing operation, mean filter, median filter, frequency domain filter and wavelet analysis. However, the corrosion expansion and opening and closing operation method need to perform binaryzation processing on the image, and the gray scale of the image after operation is reduced. Although the mean filter and the median filter can smooth the image to a certain extent and suppress the influence of noise, they cannot completely eliminate the noise, and even increase the area of the noise points. In addition, methods such as multi-frame mean filter can be used to suppress white noise, but this needs to perform multiple gaze imaging on the scene, which is not suitable for push-broom imaging system. SUMMARY

[0008] The first object of the present application is to provide an on-orbit real-time information enhancement method for a high-resolution uncooled infrared multispectral camera, which can enhance the sensitivity of the infrared multispectral image on-orbit.

[0009] To this end, the above object of the present application is achieved by the following technical scheme.

[0010] An on-orbit real-time information enhancement method for a high-resolution uncooled infrared multispectral imaging load, comprising the following processing steps:

[0011] S1, data pre-selection, the signal column counter at the face array detector end counts the row (column) synchronization signal, selects the column signal to be calculated, discards the redundant information, and reduces the data storage pressure of the calculation unit;

[0012] S2, data preprocessing, performing non-uniform correction and reference correction on the infrared image;

[0013] S3, speed-height ratio matching algorithm, in the case that the satellite flight speed and the camera frame frequency are not matched, the noise is reduced by scale change of image resolution without changing the speed and frame frequency through cumulative average, in the process, the field of view positions corresponding to the pixels participating in accumulation should be ensured to be equal, the satellite flight speed and the imaging frame frequency are matched through the image difference method, the speed-height ratio is matched, and the information dislocation problem caused by the mismatch between the push scanning speed and the imaging frame frequency in the system push scanning process is corrected;

[0014] S4, attitude correction algorithm, each pixel point on the image is rotated, the measurement value of the old image pixel point is moved to the new image pixel point to form a new image, the new image is a rotated image, the new image needs to be delimited, the coordinates of the original image pixel point corresponding to each pixel point of the new image after delimitation are found, and the pixel value is obtained through the bilinear interpolation method, the pixel value of each new image point is obtained by corresponding to the new image point, so that the image information mismatch caused by the satellite platform attitude change is corrected.

[0015] S5, frame shift superposition, when a surface detector is used, a large amount of redundant information of multi-column repeated acquisition is generated, the relative displacement relationship between the ground objects and scenes of different frames is calculated by using the redundant information of multi-column repeated acquisition, the pixels corresponding to the same ground object in different frames are superposed and averaged, the white noise in the detection process is reduced through the frame image information matching and accumulation, and the image sensitivity is improved.

[0016] While the above technical solutions are adopted, the application can also adopt or combine the following technical solutions:

[0017] As a preferred technical solution of the application, the data preprocessing comprises non-uniform correction and reference correction, and the order of the two correction parts can be interchanged, the non-uniform correction is essentially a relative radiation calibration method, an infrared uniform radiation source is used as a standard light source in the process, and one-point mapping, two-point linear fitting or multi-point fitting is adopted to correct the response difference of the pixels in the detector to the same radiation; the reference correction is used to correct the low-frequency noise generated in the detection process of the detector, in the detection process, the current frame is selected as a reference frame, the response signal of the detection element corresponding to the dark signal shielding piece in the multi-channel light splitting assembly of the non-cooled infrared focal plane assembly is selected, and the change amount of each frame after the frame is subtracted from the reference frame, the mean value change of part of the pixel points in the detector plane that does not change with the scene is monitored, and the full-width signal is compensated.

[0018] As a preferred technical solution of the application, the non-uniform correction in the data preprocessing can adopt one-point correction, two-point correction or multi-point fitting correction algorithm.

[0019] As a preferred technical scheme of the present application, the infrared uniform radiation source is a black body radiation source.

[0020] As a preferred technical scheme of the present application, the speed-altitude ratio matching algorithm (3) dynamically adjusts the difference value parameter according to the speed information fed back by the imaging load in real time, realizes the matching between the push-broom speed and the frame frequency without changing the imaging load frame frequency, that is, realizes the inter-frame image information matching, and the matching formula is:

[0021] wherein, is the actual flight ground speed (ground speed), is the ideal flight speed calculated through the imaging system parameters, j is the pixel coordinate to be calculated, respectively represent the upward rounding and downward rounding operations, abs() represents the absolute value of data, k is the speed ratio, and the calculation formula is:

[0022]

[0023] The weight of the difference value is calculated through the K value.

[0024] As a preferred technical scheme of the present application, the attitude correction algorithm includes pixel rotation, bilinear interpolation, pixel matching, image cutting, and pixel rotation, which includes first rotating the pixel, in the rectangular coordinate system, rotating the point counterclockwise to the point The transformation formula is:

[0025]

[0026] Then, the measurement value of the old image pixel is moved to the new image pixel by calling the rotation formula for each point on the image, so that the image can be rotated to any position, but the pixel coordinates must be integers, and the pixel coordinates of the rotated image may be non-integers, which need to be processed. Assuming that all pixel points on the graph after counterclockwise rotation by θ are integers, for each pixel point of the rotated graph, the corresponding pixel point coordinates of the graph before rotation are found, and is taken as the inverse rotation angle, and the corresponding relationship between the pixel point after rotation and the pixel point before rotation is:

[0027]

[0028] Here, the pixel value needs to be obtained through difference calculation, and the method is bilinear interpolation. Assuming that the non-integer points between the x and y coordinates are , the coordinates of the four integer points around the non-integer points are wherein is not an integer point, there are 4 integer points that will surround it in the middle, set the longitudinal scale factor , the transverse scale factor , the gray scale function The values at the four integer points are respectively Then The color function value of the point is:

[0029]

[0030] Transformed into a matrix form is:

[0031]

[0032] Suppose that the points of the image are all integer points, the new image is a rotated rectangle, and the new image needs to be delimited so that the new image can be included in a large rectangle with a rotation angle of 0, After the delimitation is completed, only the pixel point coordinates of the original image corresponding to each pixel point of the new image after the delimitation are required, and the pixel values are obtained by means of the bilinear interpolation method and are corresponded to the new image points, so that the pixel values of each new image point can be obtained, after the width pixel number after the rotation is determined, the left edge of the rectangle is determined, and then the position of the rectangle is determined through the right boundary, and finally the final result is obtained by cutting, and the initial infrared multispectral image obtained by the imaging load is rotated and corrected.

[0033] As a preferred technical scheme of the present application: the inter-frame shift superposition method analyzes the multiple frames of data collected continuously, selects the corresponding pixels in the same position for accumulation, uses the redundant information of the multiple column repeated collection, calculates the relative displacement relationship of the ground object scene between frames based on the push-broom speed, and superimposes and averages the pixels corresponding to the same ground object in different frames.

[0034] As a preferred technical scheme of the present application: the inter-frame shift superposition method repeatedly corrects the images generated by different column scanning, can obtain multiple images after matching the speed-height ratio, and obtains the infrared image with information enhancement under the premise of not reducing the spatial resolution by accumulating and averaging the images.

[0035] The second object of the present application is to provide a high-resolution uncooled infrared multispectral imaging load on-orbit real-time information enhancement method in view of the deficiencies in the prior art.

[0036] To this end, the above-mentioned object of the present application is realized by the following technical scheme: ​​​​​​

[0037] The method for on-orbit real-time information enhancement of a high-resolution uncooled infrared multi-spectral imaging payload is run in the information processing circuit of the high-resolution uncooled infrared multi-spectral imaging payload in the form of embedded software.

[0038] The present invention provides a method for enhancing on-orbit real-time information for a high-resolution uncooled infrared multispectral imaging payload and its application. By utilizing the principle of spatiotemporal aliasing imaging, a data processing algorithm is deployed in an image processing module to achieve real-time on-orbit sensitivity enhancement. The method comprises procedures such as data preselection, data preprocessing, speed-to-height ratio matching, attitude correction, and inter-frame shifting and superposition. Data preselection is used to select signals to be calculated using a signal counter at the detector end, discarding redundant information and reducing the data storage pressure of the computing unit. Data preprocessing is used to perform non-uniformity correction, reference correction, and blind pixel processing on thermal infrared images. A speed-to-height ratio matching algorithm is used to correct pixel-level information misalignment caused by mismatches between the speed and imaging frame rate during the system's push-scan process. An attitude correction algorithm is used to correct image information mismatches caused by system attitude changes. Inter-frame shifting and superposition is used to superimpose the corrected thermal infrared images according to inter-frame displacements. After completing the above five steps, an infrared multispectral image with enhanced detection sensitivity can be obtained. The present invention provides a method for real-time on-orbit information enhancement of a high-resolution uncooled infrared multispectral imaging payload and its application. The method utilizes redundant information from data to enhance the detection sensitivity of infrared multispectral images on-orbit for high-resolution uncooled infrared multispectral imaging payloads. The method has broad application prospects in the field of uncooled infrared multispectral imaging image enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Fig. 1 This is a flow chart of a method for on-orbit real-time information enhancement of a high-resolution uncooled infrared multispectral imaging payload according to Example 1 of the present invention;

[0040] Fig. 2 Schematic diagram of the rotation correction algorithm;

[0041] In the accompanying drawings, data pre-selection 1, data pre-processing 2, speed-to-height ratio matching algorithm 3, posture correction algorithm 4, and inter-frame shift superposition 5 are performed. DETAILED DESCRIPTION

[0042] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0043] The on-orbit real-time information enhancement method of the high-resolution non-cooled infrared multi-spectral imaging load of the application first converts the infrared radiation signal of the scene to be detected into a digital electrical signal by using a non-cooled infrared focal plane assembly, the non-cooled infrared focal plane assembly is a surface array detector, a large amount of redundant information is generated in the process of system push scanning imaging, the on-orbit real-time information enhancement method of the high-resolution non-cooled infrared multi-spectral imaging load of the application mainly reduces the white noise level in the detection process and improves the sensitivity of the infrared image through the inter-frame image information matching accumulation method. The method runs in the form of embedded software in the information processing circuit of the high-resolution non-cooled infrared multi-spectral imaging load, and the method includes five processing steps: data pre-selection 1, data preprocessing 2, speed-height ratio matching algorithm 3, attitude correction algorithm 4, and inter-frame shift superposition 5.

[0044] In the application, first, the infrared radiation signal of the scene to be detected is converted into a digital electrical signal by using a non-cooled infrared focal plane assembly, the non-cooled infrared focal plane assembly is a surface array detector, a large amount of redundant information is generated in the process of system push scanning imaging, the method mainly reduces the white noise level in the detection process and improves the sensitivity of the infrared image through the inter-frame image information matching accumulation method. The method includes the following steps: first, data pre-selection 1 is realized, the data pre-selection 1 is to select the column signal to be calculated by the signal column counter at the detector end, discard the redundant information, and reduce the data storage pressure of the calculation unit. Data preprocessing 2 is mainly used for non-uniform correction and reference correction of the infrared multi-spectral image. The speed-height ratio matching algorithm 3 is used for correcting the information misplacement problem caused by the mismatch between the push scanning speed and the imaging frame frequency in the system push scanning process, and the attitude correction algorithm 4 is used for correcting the image information mismatch caused by the satellite platform attitude change, both of which are to avoid reducing the spatial resolution in the inter-frame shift superposition 5 process after mismatch.

[0045] Further, the data pre-selection 1 specifically refers to a method of collecting the signal to be measured through the column synchronization signal. The data readout format of the surface array detector generally has frame synchronization and row and column synchronization signals, the frame synchronization signal is used for identifying the starting position of the frame image, and the row and column synchronization signals are used for identifying the starting position of the row and column. The data pre-selection specifically refers to a data selection method of extracting useful signals by counting the row and column synchronization signals, which avoids the data storage pressure and calculation pressure of the hardware system.

[0046] Furthermore, the data preprocessing 2 mainly includes two parts: non-uniformity correction and reference correction, and the correction order of the two parts can be interchanged. The non-uniformity correction is essentially a relative radiation calibration method. During the process, an infrared uniform radiation source, generally a black body, is used as a standard light source. One-point mapping, two-point linear fitting, or multi-point fitting are used to correct the response differences of the pixels in the detector to the same radiation amount. Reference correction is used to correct the low-frequency noise generated by the detector during the detection process. During the detection process, the signal stability of the uncooled detector is affected by the external environment and the noise of the detector itself, and there will be large signal fluctuations on the detector plane. This noise is characterized by a uniform change of the entire signal. Therefore, the full-scale signal can be compensated by monitoring the mean change of some pixels in the detector plane that do not change with the scene. The reference correction selects the current frame as the reference frame, selects the response signal of the detection element corresponding to the dark signal shield in the payload multi-channel spectrometer component in the uncooled infrared focal plane component, and subtracts the change from the reference frame in each frame after this frame.

[0047] Furthermore, the speed-to-height ratio matching algorithm 3 is an image processing method that uses image interpolation to match the satellite's flight speed with the imaging frame rate. Onboard image processing is essentially a temporal and spatial misalignment accumulation algorithm that reduces noise through cumulative averaging. During this process, the pixels participating in the accumulation must be aligned in the field of view. The purpose of speed-to-height ratio matching is to achieve speed-to-height ratio matching without changing the speed or frame rate by scaling the image resolution when the satellite's flight speed and camera frame rate do not match.

[0048] The specific matching formula is:

[0049]

[0050] in, is the actual flight ground speed (ground speed), is the ideal flight speed calculated by the imaging system parameters, j is the pixel coordinate to be calculated, They represent rounding up and rounding down respectively, abs() represents taking the absolute value of the data, k is the speed ratio, and the calculation formula is

[0051]

[0052] The weight of the difference is mainly calculated through the K value.

[0053] Furthermore, the posture correction algorithm 4 is mainly divided into 4 parts: pixel rotation, bilinear interpolation, pixel matching, and image capture. To rotate the image to a certain angle, the pixel must be rotated first. In the rectangular coordinate system, the pixel Counterclockwise rotation Arrive The transformation formula is:

[0054]

[0055] Then, by calling this rotation formula for each point on the image and moving the measured values ​​of the old image pixels to the new image pixels, the image can be rotated to any position. However, the pixel coordinates must be integers. The pixel coordinates of the rotated image may be non-integer, which needs to be processed. Assuming that all pixels on the image after counterclockwise rotation of θ are integers, for each pixel of the rotated image , find the coordinates of the pixel points corresponding to the image before rotation ,Pick is the reverse rotation angle, then the corresponding relationship between the pixel points after rotation and the pixel points before rotation is:

[0056]

[0057] Here The pixel value needs to be obtained by difference calculation, and the method used is bilinear interpolation. Assume that the non-integer point between x and y coordinates The coordinates of the four surrounding points are , , , ( is a non-integer point), there are 4 integer points Surrounded in the middle, set the vertical scale coefficient , lateral scale factor , grayscale function The values ​​at the four integral points are , , , ,but The color function value of the point is:

[0058]

[0059] Converted into matrix form:

[0060]

[0061] Assuming that the points of the image are all integers, the new image is a rotated rectangle. It is necessary to delimit the new image so that it can be included in a large rectangle with a rotation angle of 0. After the delimitation is completed, only the original image pixel point coordinates corresponding to each pixel point of the new image after the delimitation are required to be output, and pixel values are obtained according to a bilinear interpolation method and correspond to new image points, so that the pixel value of each new image point can be obtained. After the number of pixels in the width after rotation is determined, the left edge of the rectangle is determined, and then the position of the rectangle is determined through the right boundary, so that the final result is obtained by cutting.

[0062] The attitude correction algorithm is an image rotation correction based on bilinear interpolation. Since there is a certain inclination between the shooting attitude of the imaging load to the scene to be imaged and the orbit running during shooting, the obtained infrared multispectral image also has a certain inclination. In order to facilitate the realization of space-time aliasing accumulation processing, the initial infrared multispectral image obtained by the imaging load needs to be rotated and corrected. First, the initial image is rotated and placed in an empty matrix with a rotation angle of 0°, and then the required part is cut off to obtain the unit required for the subsequent accumulation step.

[0063] Further, the inter-frame shift superposition 5 is a space-matched image superposition method. During satellite flight, only one column perpendicular to the scanning direction is generally required to collect the ground information scanned by the flight process. When a surface array detector is used, a large amount of redundant information (multiple column repeated collection) is actually generated. The method uses the redundant information of the multiple column repeated collection to calculate the relative displacement relationship between the frames and the frames of the ground scene, and superimposes and averages the pixels corresponding to the same ground object in different frames of images. Through the foregoing method introduction, the image data used for the inter-frame shift superposition has been subjected to data correction, which can ensure that the corresponding field of view contents of the pixel superposition are the same, and the sensitivity can be improved without affecting the spatial imaging resolution.

[0064] The inter-frame shift superposition 5 is a space-matched image superposition method, which analyzes the multiple frames of data collected in succession, selects the corresponding pixels at the same position for accumulation, and because the corresponding field of view contents of the multiple superpositions are the same, the purpose of infrared image information enhancement can be achieved without affecting the spatial imaging resolution.

[0065] When a surface array detector is used to realize multispectral push-broom imaging, only dozens of column signals within the waveband are effective data, and a large amount of signals are redundant. These redundant signals image a same target at different times. Based on this premise, the application provides a non-cooled infrared multispectral inter-frame shift superposition imaging method based on space-time accumulation, and discusses the application in remote sensing imaging. Compared with the prior art, the application has the following remarkable advantages:

[0066] (1) Different from the infrared image enhancement technology of increasing signal quantity by increasing integration time or adjusting integration capacitor, the present application mainly reduces the white noise in the detection process and improves the image sensitivity by the way of inter-frame image information matching accumulation.

[0067] (2) The speed-height ratio matching algorithm is a real-time image difference algorithm, which dynamically adjusts the difference parameters according to the real-time feedback speed information of the imaging load, so as to realize the matching between the push scanning speed and the frame frequency, that is, the inter-frame image information matching, without changing the frame frequency of the imaging load.

[0068] (3) The speed-height ratio matching algorithm is used to correct the information dislocation caused by the mismatching between the push scanning speed and the imaging frame frequency in the system push scanning process, and the attitude correction algorithm is used to correct the image information mismatch caused by the satellite platform attitude change, both of which are to avoid the reduction of spatial resolution in the inter-frame displacement and superposition process of the mismatched image information, so as to obtain the infrared multispectral image with improved detection sensitivity.

[0069] Embodiment 1

[0070] In combination with Figs. 1-2 , the present application selects a camera face array with a size of 1280*1024, and the data interface is LVCMOS. The LVCMOS digital video includes 1 clock signal line, 1 frame synchronization (field synchronization) signal line, 1 row synchronization signal line and 14 parallel data signal lines. When a frame of data arrives, the frame synchronization signal is set to high level, indicating that the following data is the same frame data. After the end of this frame of data, the frame synchronization signal is set to low level, indicating the end of the frame data. Similarly, when a row of data arrives, the column synchronization signal is set to high level, and when the column of data ends, the column synchronization signal is set to low level. There are 1024 column synchronization signals per frame of data, and through calculation, the selected columns can be divided into four groups, each group being divided into (231~280), (359~408), (615~664) and (871~920) respectively.

[0071] The non-uniform correction adopts a two-point non-uniform correction algorithm (two-point NUC) for correction. The principle of the two-point NUC algorithm is as follows: assuming that the detector response changes linearly with temperature, the linear relationship between the incident light intensity of each pixel and the output signal of the detector is calculated according to the blackbody data of two temperatures. Each pixel responds differently, and the actual collected pixel signal is corrected to the fitted linear relationship through gain and offset. The relationship between the response of each pixel before correction and the response after correction is , and each pixel ( ) corresponds to a group of .

[0072] ​​The reference correction is to take the structure occlusion part of the image as the noise reference value, and subtract the noise reference value from the whole image, which can reduce the time domain noise to a certain extent.

[0073] The lens focal length of the imaging system is 50mm, the detector pixel size is 12um, and the detector frame frequency is fixed at 30Hz. The satellite flight height is 500Km, the instantaneous field of view is calculated as 120m, and if a 1:2 matching high-speed ratio model is scanned for imaging, the theoretical satellite speed is calculated as: 120*2*30=7.2Km / s. Here, it is assumed that the actual satellite ground speed is 7.6Km / s, and the speed-height ratio matching is performed through the image processing algorithm introduced above. According to the related formula of the speed-height ratio matching algorithm, under the premise that the satellite flight speed is fixed, the imaging speed is slow (7.6 / 7.2-1≈0.055555). As an example, this means that when 18 columns of actual column 19 are collected (the essence is to take 0.0555 as the offset of interpolation), the actual column pixel value (19 column data) can be calculated by bringing the collected value (18 column data) into formula 1.

[0074] The attitude correction algorithm is essentially an image rotation correction method. In this embodiment, it is assumed that the flight attitude rotation angle is 10°, and rotation correction is performed. A rectangular coordinate system is established with the lower left vertex O of the picture as the coordinate origin, the left side as the y-axis, and the bottom side as the x-axis, and each pixel in the image is rotated counterclockwise by 10° around the coordinate origin to obtain Fig. 2 a, which is expressed in matrix form as

[0075]

[0076] The coordinates of the rotated B', C', and D' of the boundary points B, C, and D are

[0077]

[0078] According to the four boundary points (including point O), the rotated image can be placed in a rectangular coordinate system with a rotation angle of 0°, and the following is obtained Fig. 2 b.

[0079] Then, the bilinear interpolation is used to handle the non-integer situation of the rotated coordinates. Taking the center point A of the character "A" as an example, it is assumed that the coordinates of the rotated A' are integers, and set Then the coordinates of A before rotation are

[0080]

[0081] Point A can be surrounded by the four nearest integer points, as shown in Fig. 2 ​​c, where x = 839.94, y = 156.52, x1 = 839, x2 = 840, y1 = 156, y2 = 157. Let the longitudinal scale factor , the transverse scale factor , the gray value function The values at the four integer points are respectively , , , Then The gray value function value of the point is:

[0082]

[0083] Similarly, the rotated pixel integer points can all be obtained by bilinear interpolation from the points before rotation, and the rotation part is completed. Then is the image cutting part, under the condition of a given cutting width pixel number, a unique cutting rectangular region can be determined, and then the gray value corresponding to the pixel points in the cutting rectangular region is extracted to obtain the final rotated and corrected result.

[0084] After the above column data pre-selection, image correction, speed-height ratio matching algorithm and attitude correction algorithm, the biggest difference between the obtained image and the image before processing is that the corresponding scene information between the frames after processing can be matched by simple column displacement.

[0085] The frame displacement superposition method can accumulate and reduce time and space noise at the same time. This method uses a planar array device to push-scan the target, and then performs matching accumulation on the non-uniformly corrected digital domain image to weaken the time noise and space noise. This method does not limit the number of superpositions, and is performed on the digital image after image acquisition in the back end, without hardware restrictions. The only thing to do is to synchronously collect the redundant information of the spatial dimension of the planar array camera to meet the signal collection of different pixels to the same ground target. The pixel matching between different adjacent frames is based on the push-scan speed, and the best matching value is calculated by the speed-height ratio matching algorithm to obtain the best superposition image. Through the frame displacement superposition algorithm described in the specification, the above correction is performed on the images generated by different column scans in a loop to obtain multiple images after speed-height ratio matching. The images are accumulated and averaged to obtain an infrared image with enhanced information without reducing the spatial resolution.

[0086] The above specific embodiments are used to explain and illustrate the present application, and are only preferred embodiments of the present application, but not limit the present application. Any modification, equivalent replacement, improvement, etc. made to the present application within the spirit of the present application and the protection scope of the claims, all fall within the protection scope of the present application.

Claims

1. A high-resolution uncooled infrared multispectral imaging payload on-orbit real-time information enhancement method, comprising the following processing steps: S1, data pre-selection, by counting the column synchronization signal through the signal column counter at the end of the area array detector, selecting the column signal to be calculated, discarding redundant information, and reducing the data storage pressure of the calculation unit; S2, data preprocessing, performing non-uniform correction and reference correction on the infrared image; S3, speed-height ratio matching algorithm, according to the speed information fed back by the imaging payload in real time, dynamically adjusting the difference value, so that the matching between the push-broom speed and the frame frequency is realized without changing the frame frequency of the imaging payload, to correct the information misplacement problem caused by the mismatch between the push-broom speed and the imaging frame frequency in the system push-broom process; S4, attitude correction algorithm, rotating each pixel point on the image, moving the measurement value of the old image pixel point to the new image pixel point to form a new image, the new image is a rotated image, the new image needs to be delimited, and the coordinates of the original image pixel point corresponding to each pixel point of the new image after delimiting are calculated, and the pixel value is obtained according to the bilinear interpolation method, and the pixel value of each new image point is obtained, to correct the image information mismatch caused by the satellite platform attitude change; S5, frame shift superposition, when the area array detector is used, a large amount of redundant information of multi-column repeated acquisition is generated, the relative displacement relationship between the ground objects in different frames is calculated by using the redundant information of multi-column repeated acquisition, the pixels corresponding to the same ground object in different frames are superimposed and averaged, and the white noise in the detection process is reduced through the frame image information matching accumulation, and the image sensitivity is improved.

2. The on-orbit real-time information enhancement method for high-resolution uncooled infrared multispectral imaging payload according to claim 1, characterized in that: The data preprocessing includes non-uniform correction and reference correction, and the order of the two parts of correction can be interchanged, the non-uniform correction is essentially a relative radiation calibration method, an infrared uniform radiation source is used as a standard light source in the process, and a one-point mapping, two-point linear fitting or multi-point fitting method is adopted to correct the response difference of the pixels in the detector to the same radiation; the reference correction is used to correct the low-frequency noise generated in the detection process of the detector, in the detection process, the current frame is selected as the reference frame, the response signal of the detection element corresponding to the dark signal shield piece in the multi-channel light splitting assembly of the uncooled infrared focal plane assembly is selected, and the change amount from the reference frame is subtracted from each frame after the frame, and the full-width signal is compensated by monitoring the average change of the pixel points in the detector plane that do not change with the scene.

3. The on-orbit real-time information enhancement method for high-resolution uncooled infrared multispectral imaging payload according to claim 1, characterized in that: The non-uniform correction in the data preprocessing can adopt one-point correction, two-point correction or multi-point fitting correction algorithm.

4. The on-orbit real-time information enhancement method for high-resolution uncooled infrared multispectral imaging payload of claim 3, wherein: The infrared uniform radiation source selects a blackbody radiation source.

5. The on-orbit real-time information enhancement method for high-resolution uncooled infrared multispectral imaging payload of claim 3, wherein: In the speed-height ratio matching algorithm, the matching formula for realizing the frame image information matching is: wherein, is the actual ground speed of flight, is the ideal flight speed calculated by the imaging system parameters, j is the pixel coordinate to be calculated, respectively represent the upward rounding and downward rounding operations, abs() represents the absolute value of the data, k is the speed ratio, and the calculation formula is: The weight of the difference value is calculated by k value.

6. The on-orbit real-time information augmentation method for high resolution uncooled infrared multispectral imaging payloads of claim 1, wherein: The posture correction algorithm comprises pixel rotation, bilinear interpolation, pixel matching, image interception, and pixel rotation, which comprises rotating the pixel points first, transforming the pixel points in the rectangular coordinate system to the pixel points counterclockwise rotation to the pixel points The transformation formula is: So call this rotation formula on each point on the image, the old image pixel point measurement to move to the new image pixel point, you can rotate the image to any position, but the pixel point coordinates must be integer, the pixel point coordinates of the image after rotation will have non-integer cases, need to be handled, assume that all pixel points on the graph after counterclockwise rotation θ are integer points, for each pixel point of the rotated graph , find the pixel point coordinates of the graph before rotation , take as the inverse rotation angle, then the corresponding relationship between the pixel points after rotation and the pixel points before rotation is: Here The pixel value needs to be obtained by difference calculation, and the method is to choose bilinear interpolation. It is assumed that the non-integer point between x and y coordinates The four integer point coordinates around the non-integer point are respectively , , , , wherein is a non-integer point, and the four integer points will be surrounded in the middle, the longitudinal proportion coefficient is set as , the transverse proportion coefficient is , and the gray scale function is The values of the four integer points are respectively , , , , , The color function value of the point is: The matrix form is: Assuming that the points of the image are all integral points, the new image is a rotated rectangle, and the new image needs to be delimited so that the new image can be included in a large rectangle with a rotation angle of 0, After the delimitation is completed, only the pixel point coordinates of the original image corresponding to each pixel point of the new image after the delimitation are required, and the pixel values are obtained by the method of bilinear interpolation, and the pixel values are corresponded to the new image points, so that the pixel values of each new image point can be obtained. After the width of the rotated rectangle is determined, the left edge of the rectangle is determined, and then the position of the rectangle is determined through the right boundary, and the final result is obtained by cutting, so that the initial infrared multispectral image obtained by the imaging load is rotated and corrected.

7. The on-orbit real-time information enhancement method for high-resolution uncooled infrared multispectral imaging payload of claim 6, wherein: In the frame shift superposition method, the data of multiple frames after continuous acquisition are analyzed, the corresponding pixels at the same position are selected for accumulation, the relative displacement relationship between the ground object scenes in different frames is calculated by using the redundant information of multi-column repeated acquisition, the pixels corresponding to the same ground object in different frames are superimposed and averaged.

8. The on-orbit real-time information enhancement method for high-resolution uncooled infrared multispectral imaging payload of claim 7, wherein: The inter-frame shift superposition method cyclically corrects images generated by different column scans, can obtain multiple images after matching the speed and the ratio, and obtains an infrared image with enhanced information under the premise of not reducing the spatial resolution by accumulating and averaging the images.

9. Use of the high-resolution non-cooled infrared multispectral imaging load on-orbit real-time information enhancement method according to any one of claims 1-8, wherein the method is run in the form of embedded software in an information processing circuit of the high-resolution non-cooled infrared multispectral imaging load.

Images