A long-time stable infrared focal plane temperature drift self-adaptive correction method

Abstract

The application provides a long-time stable infrared focal plane temperature drift self-adaptive correction method, which comprises the following steps: firstly, extracting feature points of output images of a fixed scene and a fixed temperature by an IRFPA camera, and recording time stamps of each output image; then, using the relationship that the working time is proportional to the temperature drift influence, constructing a feature point affected by temperature drift and time function by replacing temperature with time; recording feature points and time stamps of output images by the IRFPA camera through a large number of different scenes and different working temperatures; since the scene and the temperature are the same, the change of the feature points can only be caused by the temperature drift, so that a drift correction model can be constructed; finally, in the actual work of the IRFPA camera, the image feature points of adjacent frames are matched, the image is generated into a new image through the drift correction model by the feature point affected by the temperature drift and the time function, and thus the temperature drift correction is completed.

Description

Technical Field

[0001] This invention belongs to the field of infrared image processing technology, specifically relating to a long-term stable infrared focal plane temperature drift adaptive correction method. Background Technology

[0002] In infrared imaging systems, widely used operable infrared focal plane array (IRFPA) detectors can be divided into cooled and uncooled types. Cooled infrared thermal imagers significantly reduce thermal noise by using cryogenic coolants, thus achieving a high signal-to-noise ratio. However, the cooling system increases the overall weight of the infrared imager and can be very energy-intensive. In contrast, uncooled infrared focal plane array detectors, with their small size, light weight, low power consumption, and high cost-effectiveness, greatly enhance the applicability and popularity of infrared detectors, making them more popular.

[0003] Due to issues with materials and manufacturing processes, temperature variations during prolonged operation of infrared focal plane arrays can cause focal plane drift in the imaging system, a phenomenon known as temperature drift, which blurs the image. Although most infrared imagers are equipped with non-uniformity correction modules to mitigate drift, these measures are sometimes insufficient because environmental conditions may change faster than the modules can handle, and typical non-uniformity correction methods are not effective in eliminating the effects of drift.

[0004] Therefore, assessing and eliminating drift is very challenging. Summary of the Invention

[0005] To address the impact of temperature drift on IRFPA, a long-term stable adaptive correction method for infrared focal plane array temperature drift will be provided without altering the original infrared focal plane array detector.

[0006] This invention provides a long-term stable adaptive correction method for infrared focal plane temperature drift, comprising the following steps:

[0007] S1. Use an IRFPA camera to capture images of a fixed scene at a fixed temperature and record the timestamp of each image.

[0008] S2. Extract feature points and timestamps from the obtained image;

[0009] S3. After repeating step 2 H*N times, obtain the image feature point results and timestamps, and construct feature points that include the effects of temperature drift. Functions at different times ;

[0010] in, This refers to the number of image feature points when the IRFPA camera first starts working (i.e., the number of feature points that are not affected by temperature drift or are only slightly affected by temperature drift). In time The number of feature points affected by temperature drift;

[0011] S4. After obtaining the feature points and time function affected by temperature drift in different scenarios under different operating temperatures through step S3; construct a drift correction model based on the images corresponding to different timestamps and the images not affected by temperature drift:

[0012] (8);

[0013] in, For temperature Images containing the effects of temperature drift; Images unaffected by temperature drift; The function type can be exponential function, exponential function, linear function, and second, third and fourth order polynomial functions;

[0014] S5. When the camera is in operation, after the calibration steps provide the feature points affected by temperature drift, the time function, and the drift correction model, for the camera operating independently, perform feature matching on images from adjacent frames, and provide the number of feature points. ;

[0015] S6. The number of feature points given in step S5 Substitute the corresponding feature points affected by temperature drift with the time function. Find ,according to Determine the drift correction model This generates a new image.

[0016] Furthermore, images are acquired at fixed scenes and temperatures, and the timestamp of each image is recorded. To ensure the performance of infrared image temperature drift correction, H is at least 50, and the time step can be 5 min or 10 min.

[0017] Furthermore, in step S2, the timestamps of the obtained images are extracted, which are the time from the power-on operation time to the shooting time and the specific time of each frame.

[0018] Furthermore, in step S2, the image features of the obtained image are extracted primarily using the ORB method, with the specific steps as follows:

[0019] Step 21. FAST corner detection;

[0020] First, select a point P to be detected in the image, and use I... pLet I represent the gray value of point P; and compare it with the gray values ​​of 16 pixels on the edge of its circular neighborhood with a radius of three. If the gray values ​​of 9 pixels are greater than I, then... p Whether it is large or small, then point Р is identified as a feature point;

[0021] Step 22. Corner Response;

[0022] Let I(x, y) be the gray value of a point in the image; and suppose that after the rectangular window w(x, y) centered at this point is translated u units horizontally and v units vertically, the gray value change E(u, v) produced in the rectangular window w(x, y) is as shown in equation (1):

[0023] (1)

[0024] In equation (1) After performing a first-order two-dimensional Taylor expansion and substituting it into the original expression, we obtain equation (2):

[0025] (2)

[0026] (3)

[0027] In the formula It is a 5×5 Gaussian sliding window, where x and y are the pixel coordinates corresponding to window W. x It is the gradient value of the pixel within the rectangular window in the x-direction, and similarly, I y It is the gradient value in the y-direction;

[0028] Since the image is classified based on the magnitude of the two feature values ​​of M, the variable R that measures the Harris corner response is shown in equation (4):

[0029] (4)

[0030] (5)

[0031] (6)

[0032] k is an empirical value, and λ1 and λ2 are the eigenvalues ​​of M;

[0033] The variable F, which measures the Shi-Tomasi corner response, is defined as shown in equation (7):

[0034] (7).

[0035] Furthermore, in step S5, when the camera is in operation, after the calibration step provides the feature points affected by temperature drift, the time function, and the drift correction model, for the camera operating independently, feature matching is performed on images of adjacent frames, and the number of feature points is given. The specific process is as follows:

[0036] S51. Extract feature points;

[0037] S52, Feature point matching;

[0038] First, select n point pairs from the neighborhood interval of the feature point. Then, compare these n point pairs according to the point pair comparison criterion τ. Perform the above binary assignment on each point pair, as shown in equation (9):

[0039] (9)

[0040] In the formula, I(p;x) is the gray value of point x in the region, and I(p;y) is the gray value of point y in the region;

[0041] Finally, the results of comparing these point pairs are combined to form the descriptor of the feature point p. As shown in equation (10):

[0042] (10)

[0043] in It is a point logarithm;

[0044] In this algorithm, these n point pairs (x, y) are combined to form a 2×n matrix S as shown in equation (11):

[0045] (11)

[0046] Using formula (11) neighborhood direction With the corresponding rotation matrix As shown in equation (12):

[0047] (12)

[0048] Build a corrected version of S As shown in equation (13):

[0049] (13)

[0050] Finally, the Steer BRIEF descriptor was obtained. As shown in equation (14):

[0051] (14)

[0052] Feature point matching is performed using Hamming Distance (D), as shown in Equation (15):

[0053] (15)

[0054] Here, b1 and b2 are descriptors for the matching point pairs. If D meets the requirements, the match is successful. After matching, there are 20 matching point pairs.

[0055] Based on the transformation matrix and image size, calculate the position and size of the overlapping region; within the overlapping region, count the number of feature points at the overlapping position.

[0056] The advantages of this invention are: This invention provides a long-term stable infrared focal plane temperature drift adaptive correction method by calibrating the given feature points affected by temperature drift and time function and drift correction model before the camera is working, so that the camera can solve the influence caused by temperature drift by using image feature points of adjacent frames during long-term operation.

[0057] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of feature point extraction using FAST in this invention.

[0059] Figure 2 This is a flowchart of the feature extraction algorithm of the present invention.

[0060] Figure 3 This is the feature point matching diagram of the present invention.

[0061] Figure 4 This is a schematic diagram of the process of the present invention.

[0062] Figure 5 This is a statistical diagram illustrating the detection of feature points between two adjacent frames using FAST corner detection.

[0063] Figure 6 This is a schematic diagram showing the comparison before and after correction of an image affected by temperature drift. Detailed Implementation

[0064] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the specific implementation methods, structural features and effects of the present invention are described in detail below with reference to the accompanying drawings and embodiments.

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] All features disclosed in this specification, or all steps in all disclosed methods or processes, may be combined in any way, except for mutually exclusive features and / or steps.

[0067] Any feature disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by other equivalent or similar features for a similar purpose, unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features.

[0068] Example 1

[0069] To address the impact of temperature drift on IRFPA, a long-term stable infrared focal plane array (IRFA) temperature drift adaptive correction method will be provided without altering the original IRFA detector. First, feature points of the output images under fixed scenes and temperatures are extracted using the IRFPA camera, and the timestamp of each output image is recorded. Then, by using a method where operating time is proportional to the temperature drift effect, time is used to replace temperature, constructing a feature point and time function affected by temperature drift. The IRFPA camera is used to record feature points and timestamps of the output images under numerous different scenes and operating temperatures. Since the scene and temperature are the same, the changes affecting the feature points can only be due to temperature drift, thus constructing a drift correction model. Finally, in actual operation of the IRFPA camera, feature point matching between adjacent frames is used. Since environmental changes within such a short time are negligible, the main cause of instrument instability is drift, resulting in inaccurate temperature values. Therefore, by using the feature points and time function containing the temperature drift effect, a new image is generated through the drift correction model, thus completing the temperature drift correction.

[0070] The long-term stable infrared focal plane temperature drift adaptive correction method provided in this embodiment includes the following steps:

[0071] S1. Use an IRFPA camera to capture images of a fixed scene at a fixed temperature and record the timestamp of each image.

[0072] S2. Extract feature points and timestamps from the obtained image;

[0073] S3. After repeating step 2 H*N times, obtain the image feature point results and timestamps, and construct feature points that include the effects of temperature drift. Functions at different times ;

[0074] Where N is the number of images captured in a single acquisition, and H is the number of times images are captured. This refers to the number of image feature points when the IRFPA camera first starts working (i.e., the number of feature points that are not affected by temperature drift or are only slightly affected by temperature drift). In time The number of feature points affected by temperature drift;

[0075] S4. After obtaining the feature points and time function of the fixed scene under a fixed operating temperature through step S3, a drift correction model is constructed based on the images corresponding to different timestamps and the images not affected by temperature drift:

[0076] (8);

[0077] in, For temperature Images containing the effects of temperature drift; Images unaffected by temperature drift; The function type can be exponential function, exponential function, linear function, and second, third and fourth order polynomial functions;

[0078] S5. When the camera is in operation, after the calibration steps provide the feature points affected by temperature drift, the time function, and the drift correction model, for the camera operating independently, perform feature matching on images from adjacent frames, and provide the number of feature points. ;

[0079] S6. The number of feature points given in step S5 Substitute the corresponding feature points affected by temperature drift with the time function. Find ,according to Determine the drift correction model This generates a new image.

[0080] Furthermore, images are acquired at fixed scenes and temperatures, and the timestamp of each image is recorded. To ensure the performance of infrared image temperature drift correction, H is at least 50, and the time step can be 5 min or 10 min.

[0081] Furthermore, in step S2, the timestamps of the obtained images are extracted, which are the time from the power-on operation time to the shooting time and the specific time of each frame.

[0082] Furthermore, in step S2, the image features of the obtained image are extracted primarily using the ORB method, with the specific steps as follows:

[0083] Step 21. FAST corner detection;

[0084] First, select a point P to be detected in the image, and use I... p This represents the grayscale value of point P; it is compared with the grayscale values ​​of 16 pixels on the edge of its circular neighborhood with a radius of three, such as... Figure 1 As shown, if the grayscale values ​​of 9 pixels are all greater than I... p Whether it is large or small, then point Р is identified as a feature point;

[0085] Step 22. Corner Response;

[0086] Let I(x, y) be the gray value of a point in the image; and suppose that after the rectangular window w(x, y) centered at this point is translated u units horizontally and v units vertically, the gray value change E(u, v) produced in the rectangular window w(x, y) is as shown in equation (1):

[0087] (1)

[0088] In equation (1) After performing a first-order two-dimensional Taylor expansion and substituting it into the original expression, we obtain equation (2):

[0089] (2)

[0090] (3)

[0091] In the formula It is a 5×5 Gaussian sliding window, where x and y are the pixel coordinates corresponding to window W. x It is the gradient value of the pixel within the rectangular window in the x-direction, and similarly, I y It is the gradient value in the y-direction;

[0092] Since the image is classified based on the magnitude of the two feature values ​​of M, the variable R that measures the Harris corner response is shown in equation (4):

[0093] (4)

[0094] (5)

[0095] (6)

[0096] k is an empirical value, and λ1 and λ2 are the eigenvalues ​​of M;

[0097] The variable F, which measures the Shi-Tomasi corner response, is defined as shown in equation (7):

[0098] (7).

[0099] Furthermore, in step S5, when the camera is in operation, after the calibration step provides the feature points affected by temperature drift, the time function, and the drift correction model, for the camera operating independently, feature matching is performed on images of adjacent frames, and the number of feature points is given. The specific process is as follows:

[0100] S51. Extract feature points; This step is the feature point extraction part, and the method is the same as steps 21 and 22.

[0101] S52, Feature point matching;

[0102] First, select n point pairs from the neighborhood interval of the feature point. Then, compare these n point pairs according to the point pair comparison criterion τ. Perform the above binary assignment on each point pair, as shown in equation (9):

[0103] (9)

[0104] In the formula, I(p;x) is the gray value of point x in the region, and I(p;y) is the gray value of point y in the region;

[0105] Finally, the results of comparing these point pairs are combined to form the descriptor of the feature point p. As shown in equation (10):

[0106] (10)

[0107] in It is a point logarithm;

[0108] In this algorithm, these n point pairs (x, y) are combined to form a 2×n matrix S as shown in equation (11):

[0109] (11)

[0110] Using formula (11) neighborhood direction With the corresponding rotation matrix As shown in equation (12):

[0111] (12)

[0112] Build a corrected version of S As shown in equation (13):

[0113] (13)

[0114] Finally, the Steer BRIEF descriptor was obtained. As shown in equation (14):

[0115] (14)

[0116] Feature point matching is performed using Hamming Distance (D), as shown in Equation (15):

[0117] (15)

[0118] Here, b1 and b2 are descriptors for the matching point pairs. If D meets the requirements, the match is successful. After matching, there are 20 matching point pairs.

[0119] Based on the transformation matrix and image size, calculate the position and size of the overlapping region; within the overlapping region, count the number of feature points at the overlapping position.

[0120] In summary, this long-term stable infrared focal plane temperature drift adaptive correction method involves collecting data under fixed scenes and temperatures before the camera operates, constructing a temperature drift model, obtaining the feature points affected by temperature drift and their time function, and then using the feature points affected by temperature drift, the time function, and the temperature drift model to correct for temperature drift variations under different scenes and temperatures. After calibration to obtain the given feature points affected by temperature drift, the time function, and the drift correction model, the camera can mitigate the effects of temperature drift during long-term operation by using image feature points from adjacent frames.

[0121] Example 2

[0122] The image is corrected using a long-term stable infrared focal plane temperature drift adaptive correction method. Feature points in adjacent frames are statistically analyzed using FAST corner detection. Figure 5 As shown; through feature point matching, based on the feature points affected by temperature drift and the function at different times, the temperature drift-affected image of the next frame is corrected using the feature points affected by temperature drift, the time function, and the drift correction model. Comparisons before and after correction are shown below. Figure 6 As shown in the results, the correction effect of the present invention is significant.

[0123] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A long-term stable adaptive correction method for infrared focal plane temperature drift, characterized in that, Includes the following steps: S1. Use an IRFPA camera to capture images of a fixed scene at a fixed temperature and record the timestamp of each image. S2. Extract feature points and timestamps from the obtained image; S3. After repeating step 2 H*N times, obtain the image feature point results and timestamps, and construct feature points that include the effects of temperature drift. Functions at different times ; in, This refers to the number of image feature points when the IRFPA camera first starts working, i.e., the number of feature points that are not affected by temperature drift or are only slightly affected by temperature drift. In time The number of feature points affected by temperature drift; S4. After obtaining the feature points and time function affected by temperature drift in different scenarios under different operating temperatures through step S3; construct a drift correction model based on the images corresponding to different timestamps and the images not affected by temperature drift: （8）； in, For temperature Images containing the effects of temperature drift; Images unaffected by temperature drift; The function type can be exponential function, exponential function, linear function, and second, third and fourth order polynomial functions; S5. When the camera is in operation, after the calibration steps provide the feature points affected by temperature drift, the time function, and the drift correction model, for the camera operating independently, perform feature matching on images from adjacent frames, and provide the number of feature points. ; The specific process is as follows: S51. Extract feature points; S52, Feature point matching; First, select n point pairs from the neighborhood interval of the feature point. Then, compare these n point pairs according to the point pair comparison criterion τ. Perform the above binary assignment on each point pair, as shown in equation (9): （9） In the formula, I(p;x) is the gray value of point x in the region, and I(p;y) is the gray value of point y in the region; Finally, the results of comparing these point pairs are combined to form the descriptor of the feature point p. As shown in equation (10): （10） in It is a point logarithm; In this algorithm, these n point pairs (x, y) are combined to form a 2×n matrix S as shown in equation (11): （11） Using formula (11) neighborhood direction With the corresponding rotation matrix As shown in equation (12): （12） Build a corrected version of S As shown in equation (13): （13） Finally, the Steer BRIEF descriptor is obtained. As shown in equation (14): （14） Feature point matching is performed using Hamming Distance (D), as shown in Equation (15): （15） Where b1 and b2 are descriptors for the matching point pairs; if D meets the requirements, the matching is successful; after matching, there are 20 matching point pairs. Based on the transformation matrix and image size, calculate the position and size of the overlapping region; within the overlapping region, count the number of feature points at the overlapping position; S6. The number of feature points given in step S5 Substitute the corresponding feature points affected by temperature drift with the time function. Find ,according to Determine the drift correction model This generates a new image.

2. The long-term stable infrared focal plane temperature drift adaptive correction method as described in claim 1, characterized in that: Images are acquired at a fixed scene and temperature, and the timestamp of each image is recorded. To ensure the performance of infrared image temperature drift correction, H should be at least 50, and the time step can be 5 min or 10 min.

3. The long-term stable infrared focal plane temperature drift adaptive correction method as described in claim 1, characterized in that: In step S2, image features are extracted from the obtained image using the ORB method. The specific steps are as follows: Step 21. FAST corner detection; First, select a point P to be detected in the image, and use I... p Let I represent the gray value of point P; and compare it with the gray values ​​of 16 pixels on the edge of its circular neighborhood with a radius of three. If the gray values ​​of 9 pixels are greater than I, then... p Whether it is large or small, then point Р is identified as a feature point; Step 22. Corner Response; Let I(x, y) be the gray value of a point in the image; and suppose that after the rectangular window w(x, y) centered at this point is translated u units horizontally and v units vertically, the gray value change E(u, v) produced in the rectangular window w(x, y) is as shown in equation (1): （1） In equation (1) After performing a first-order two-dimensional Taylor expansion and substituting it into the original expression, we obtain equation (2): （2） （3） In the formula It is a 5×5 Gaussian sliding window, where x and y are the pixel coordinates corresponding to window W. x It is the gradient value of the pixel within the rectangular window in the x-direction, and similarly, I y It is the gradient value in the y-direction; Since the image is classified based on the magnitude of the two feature values ​​of M, the variable R that measures the Harris corner response is shown in equation (4): （4） （5） （6） k is an empirical value, and λ1 and λ2 are the eigenvalues ​​of M; The variable F, which measures the Shi-Tomasi corner response, is defined as shown in equation (7): （7）。

Images