A method for stitching infrared images

Abstract

The present application relates to a kind of splicing methods of infrared image;Solve the technical problems that the same target in the splicing image exists the inconsistent gray value, affect the gray consistency of image, lead to the existence of obvious splicing seam area of splicing image using the three commonly used image gray consistency correction methods;Specifically including the following steps: step 1: arrange first camera, second camera, control first camera and second camera simultaneously shoot the scene to be spliced and output infrared image U A With U B ;Step 2: set parallel light pipe, and obtain the same point set PA and PB, obtain the gray mapping function relationship f (g) using D formula fitting method, step 3: each pixel gray in U B According to the gray mapping function relationship f (g) mapping, obtain the gray correction image U BF ;Step 4: obtain mapping relationship S, and U A And U BF According to the splicing position mapping relationship S, respectively mapping into mapping image U AS And mapping image U BFS ;Step 5: mapping image U AS And mapping image U BFS Synthesize into an image U ABFS .

Description

Technical Field

[0001] This invention relates to an optical image stitching method, specifically to an infrared image stitching method. Background Technology

[0002] With the continuous development of industrial and remote sensing image processing and analysis technologies, image grayscale consistency correction technology, as a key step in the infrared stitching process, is also constantly evolving and improving. Image grayscale consistency correction refers to adjusting the grayscale distribution of two or more infrared digital images to be stitched, so that the stitched large-field-of-view infrared image has a consistent grayscale distribution.

[0003] Currently, commonly used image grayscale consistency correction methods include grayscale histogram correction, grayscale linear stretching correction, and average grayscale adjustment. Among these, the average grayscale adjustment method first calculates the average grayscale value of the input images to be stitched, obtaining the average grayscale level of each image. Then, using the average grayscale of one image as a benchmark, the average grayscale of the other images is adjusted to match the benchmark image's average grayscale. Finally, the images with average grayscale adjustments are stitched together according to spatial mapping relationships to obtain a large field-of-view infrared image. This method is computationally simple and fast. However, although the stitched infrared image has a consistent average grayscale, grayscale differences still exist in the high and low brightness regions of the image content. This results in inconsistent grayscale values ​​for the same target in the large field-of-view image, affecting image grayscale consistency and leading to noticeable stitching seams.

[0004] The gray-level histogram correction method involves statistically analyzing the gray-level histogram of the images to be stitched, then uniformly expanding the gray-level distribution of the histogram to obtain a new histogram. The images are then output according to the gray-level mapping of the new histogram. Finally, the histogram-corrected images are stitched together according to the spatial mapping relationship to obtain a large field-of-view infrared image. This method adjusts the gray-level levels of the images to be stitched to make their distribution uniform. However, the gray-level values ​​of the same target in the stitched image are mapped to different gray-level values, affecting the gray-level consistency of the image and producing obvious seam areas. The computational load is also relatively large.

[0005] The grayscale linear stretching correction method directly uses linear transformation to map the grayscale of the images to be stitched to a uniform grayscale range, and then stitches the grayscale linearly stretched and corrected images together according to the spatial mapping relationship to obtain a large field-of-view infrared image. Although this method requires less computation, the grayscale values ​​of the same target in the stitched image still have some differences, affecting the grayscale consistency of the image and still producing obvious stitching seam areas;

[0006] In summary, the stitched images obtained using the three commonly used correction methods exhibit inconsistent grayscale values ​​for the same target within the stitched images, affecting the grayscale consistency of the images and resulting in obvious stitching seam areas. Summary of the Invention

[0007] The purpose of this invention is to solve the technical problem that when stitched images are obtained using three existing common image grayscale consistency correction methods, the grayscale values ​​of the same target in the stitched image are inconsistent, which affects the grayscale consistency of the image and results in obvious stitching seam areas in the stitched image. Therefore, this invention provides an infrared image stitching method.

[0008] The technical solution adopted in this invention is as follows:

[0009] A method for stitching infrared images, characterized by the following steps:

[0010] Step 1: Deploy the first camera and the second camera so that there is an overlapping field of view between them. Control the first camera and the second camera to simultaneously capture the scene to be stitched together, and output infrared images U respectively. A with U B Set U A As a reference image;

[0011] Step 2: Set up a collimator so that the star point target in the collimator is aligned with the center of the overlapping area of ​​the field of view. Based on the position of the collimator and the first camera and the second camera, obtain the corresponding point set PA of the first camera and the corresponding point set PB of the second camera. Use the D-term fitting method to obtain the gray-scale mapping function relationship f(g), where 1≤D≤3.

[0012] Step 3: Put U B Each pixel's gray level is mapped according to the gray-level mapping function f(g) to obtain the gray-level corrected image U. BF ;

[0013] Step 4: Obtain the mapping relationship S based on the positions of the collimator, the first camera, and the second camera, and then... A and U BF According to the splicing position mapping relationship, S is mapped to the mapped image U. AS and mapped image U BFS ;

[0014] Step 5: Map the image U AS and mapped image U BFS Combined into one image U ABFS and synthesize the image U ABFS As a stitched wide field-of-view infrared image.

[0015] Furthermore, in step 2, the method for obtaining the corresponding point set PA of the first camera and the corresponding point set PB of the second camera is as follows:

[0016] The target star is controlled to scan upwards and downwards from the center point along the vertical direction of the overlapping area of ​​the field of view. The scan stops after each angle δ, and the first and second cameras acquire images, obtaining image I. Asi with I Bsi I AXj with I BXj Place the star-shaped target in each image I Asi with I AXj The coordinate value QI in ATsi With QI ATXj Let PA be the set of points with the same name, and let the star point target be in each image I. Bsi with I BXj The coordinate value QI in BTsi With QI BTXj Let PB be the set of points with the same name, where I Asi For the image acquired by the first camera after moving upwards by i δ angles, I Bsi For the image acquired by the second camera after moving upwards by i δ angles, I AXj For the image acquired by the first camera after it has traveled downwards by j δ angles, I BXj Let be the image captured by the second camera after it has traveled downwards by j δ angles, where j = i and i ≥ 2.

[0017] Furthermore, the array length of PA and PB is the length of the long side of the overlapping region of the field of view.

[0018] Furthermore, in step 2, D = 2.

[0019] Furthermore, in step 2, the step of obtaining the gray-level mapping function relationship f(g) using the D-term fitting method is as follows:

[0020] A. Obtain the grayscale set w of the point set PB with the same name. j And the grayscale set w of the point set PA with the same name i ;

[0021] B. Let the binomial be f(g) = a + bg + cg. 2 Where a is the constant coefficient, b is the linear coefficient, c is the quadratic coefficient, g is the binomial independent variable, and f(g) is the binomial dependent variable; the grayscale set w j Substitute each element into g in the binomial, and simultaneously add the gray set w. i Substitute the corresponding elements into f(g) in the binomial, solve for a, b, and c of the quadratic polynomial, and obtain the gray-scale mapping function relationship f(g).

[0022] Furthermore, step 3 specifically includes:

[0023] Extract U B The gray level of each pixel is calculated and substituted as an independent variable into the gray-level mapping function relationship f(g) obtained in step 2. The calculated f(g) is the corrected gray level, and the corrected gray level is used to replace U. B Pixels at the same position in the image are used to obtain a grayscale corrected image U. BF ;

[0024] Specifically, when the corrected gray level is less than 0, the corrected gray level is set to 0; when the corrected gray level is greater than the maximum gray level LN, the corrected gray level is set to the maximum resolution of the second camera.

[0025] Furthermore, in step 4, the method for obtaining the mapping relationship S based on the positions of the collimator and the first and second cameras is as follows:

[0026] V1. By placing a checkerboard calibration board in the overlapping area of ​​the field of view, the matching feature point group of the first camera and the second camera is obtained;

[0027] V2. Calculate the transformation matrix M of the first camera relative to the second camera using the matching feature point sets of the first and second cameras. BA and the transformation matrix M BA The distance error d after projection of each pair of matched feature points o And based on the matching error d o Obtain the transformation matrix M BA The number of interior points, where the transformation matrix M BA Based on the second camera;

[0028] V3. Repeat step V2 multiple times, and transform the matrix M with the largest number of interior points. BA Set as homography matrix H BA ;

[0029] V4. Based on the homography matrix H BA Calculate the stitching position mapping relationship S between the first camera 1 and the second camera 2: S = K B -1 ·H BA ·K A .

[0030] Further, step V1 specifically includes:

[0031] Twenty pairs of chessboard calibration board images with different poses at the same time were acquired using a first camera and a second camera. Then, corner detection was performed on the chessboard calibration board images acquired by the first camera and the second camera, respectively. The corner coordinates of the chessboard images with successfully detected corners were recorded in the order of acquisition time. The corner coordinates of the chessboard images recorded by the first camera and the second camera were matched one by one according to the chessboard corner distribution rules to obtain the matching feature point groups of the first camera and the second camera.

[0032] Furthermore, in step V2, based on the matching error d o Obtain the transformation matrix M BA The method for determining the number of interior points is as follows:

[0033] Set a distance threshold dT, where dT > 0. Specifically, draw a circle with radius dT. If dT > 0... o <dT, then it means d o Let d be an interior point of the circle. o If the value is ≥dT, then no counting is performed. After the statistics are completed, the transformation matrix M is recorded. BA The number of interior points in the array.

[0034] The beneficial effects of this invention are:

[0035] 1. The present invention proposes an infrared image stitching method that pre-calibrates corresponding points in the overlapping area of ​​the external field-of-view stitching cameras. Then, during image stitching, the pixel grayscale of the image to be stitched is extracted using the results of the corresponding point marking. Based on the grayscale mapping relationship of the corresponding points, grayscale mapping is performed on the image to be stitched to make the grayscale levels of the reference image and the image to be stitched consistent. Finally, a large field-of-view image is calculated based on the image stitching space mapping relationship. The stitched large field-of-view image has no stitching seam area, effectively solving the problem of unsatisfactory stitching display effect of traditional correction methods.

[0036] 2. The infrared image stitching method proposed in this invention is mainly based on grayscale mapping of corresponding points to perform grayscale consistency correction, which can effectively solve the problem of unsatisfactory stitching display effect of traditional grayscale correction methods.

[0037] 3. The infrared image stitching method proposed in this invention, compared with the commonly used average gray level compensation method, can reduce the gray level difference between the same background and target content in different fields of view images to be stitched, ensure the gray level consistency of large field of view stitched images, improve the display and viewing effect, and at the same time, the algorithm has a low computational load and is suitable for real-time infrared image stitching processing.

[0038] Compared with grayscale histogram correction methods, it can ensure the consistency of grayscale levels in large field-of-view stitched images, improve the display effect, and at the same time, the algorithm has a low computational load, making it suitable for real-time infrared image stitching processing.

[0039] Compared with the grayscale linear stretching correction method, it can reduce the difference in grayscale values ​​of the same target in different fields of view images to be stitched, ensure the consistency of grayscale levels in large field of view stitched images, and improve the display and viewing effect.

[0040] 4. The infrared image stitching method proposed in this invention reduces the grayscale difference between the same background and target content in different field-of-view images to be stitched, ensures the consistency of grayscale levels in large field-of-view stitched images, and improves the display and viewing effect.

[0041] 5. The purpose of setting the star point target in the collimator to be aligned with the center of the overlapping area of ​​the camera's field of view in the infrared image stitching method proposed in this invention is to ensure the uniformity of data acquisition, thereby ensuring the accuracy of the correction results.

[0042] 6. The purpose of using a quadratic polynomial in the infrared image stitching method proposed in this invention is that the quadratic polynomial can better fit the gray-level mapping relationship of adjacent field-of-view images, while avoiding the stitching seam area caused by the excessive fitting residual of the first-order polynomial, as well as the gray-level inversion effect caused by the overfitting phenomenon of the third-order and higher-order polynomials.

[0043] In summary, this invention proposes an infrared image stitching method. This method mainly uses gray-level mapping of corresponding points to correct gray-level consistency. It can effectively solve the problem that the gray-level values ​​of the same target in the stitched image are inconsistent when using the three commonly used correction methods, which affects the gray-level consistency of the image and leads to obvious stitching seam areas in the stitched image. Attached Figure Description

[0044] Figure 1 This is a flowchart of the present invention;

[0045] Figure 2 This is a schematic diagram of the optical device used in the embodiments of the present invention;

[0046] Figure 3 This is the pre-stitching effect obtained by directly displaying the output images of six cameras according to their spatial positions in an embodiment of the present invention;

[0047] Figure 4 This is the effect of stitching together the output images of six cameras using the method described in this invention, as shown in this embodiment of the invention.

[0048] Figure 5 This is the result of stitching two images of the same size together horizontally using the average grayscale compensation method.

[0049] Figure 6This is the result of stitching two images of the same size together using a grayscale histogram correction method.

[0050] Figure 7 This is the result of stitching two images of the same size together using the grayscale linear stretching correction method.

[0051] Figure 8 This is the effect of stitching two images of the same size together horizontally using the present invention.

[0052] Figure 2 In the diagram, 1. First camera; 2. Second camera; 3. Collimator. Detailed Implementation

[0053] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0054] Specifically, this invention is described in detail using two infrared cameras that can generate W (width) * H (height) pixels and whose grayscale resolution values ​​for each pixel are in the range of 0-(LN-1) as image sources to be stitched together. Here, LN is the grayscale resolution of the infrared camera. In this embodiment, LN = 16384.

[0055] like Figure 1 As shown, this invention proposes an infrared image stitching method to enable users to perform infrared large field-of-view image stitching processing, specifically including the following steps:

[0056] Step 1: As Figure 2 As shown, a first camera 1 and a second camera 2 are arranged such that there is an overlapping field of view between the first camera 1 and the second camera 2. The first camera 1 and the second camera 2 are controlled to simultaneously capture the scene to be stitched together, and output infrared images U respectively. A with U B Set U A As the reference image, U A with U B All are LN=16384 grayscale infrared digital images;

[0057] Specifically, the first camera 1 and the second camera 2 have the same specifications, that is, the horizontal field of view of the first camera 1 and the second camera 2 are both α; the horizontal angle between the optical axis extension line of the first camera 1 and the optical axis extension line of the second camera 2 is β, and there is no relative pitch angle. The purpose of having no relative pitch angle is to ensure that the height field of view of the first camera 1 and the second camera 2 are consistent.

[0058] Step 2: Set up collimator 3 so that the star point target in collimator 3 is aligned with the center of the overlapping area of ​​the field of view. Based on the position of collimator 3 and the first camera 1 and the second camera 2, obtain the corresponding point set PA of the first camera 1 and the corresponding point set PB of the second camera 2. Use the D-term fitting method to obtain the gray-scale mapping function relationship f(g), where 1≤D≤3.

[0059] 2.1 As Figure 2 As shown, the collimator 3 is mounted on a two-dimensional turntable. The position of the turntable is adjusted so that the star target in the collimator 3 is located on the vertical symmetrical center line of the overlapping field of view. The pitch angle of the two-dimensional turntable is controlled so that the star target in the collimator 3 is aligned with the center point of the vertical symmetrical center line, that is, the star target in the collimator 3 is aligned with the center of the overlapping field of view of the first camera 1 and the second camera 2. The first camera 1 is set to output image I. A The second camera 2 outputs image I B Then the star target is in I A The image above is I AT The star-shaped target is in I B The image above is I BT ;

[0060] 2.2 Obtain the point set PA of the first camera 1 and the point set PB of the second camera 2;

[0061] The target star is controlled to scan upwards from its center point along the vertical direction of the overlapping area of ​​the field of view, stopping once after passing an angle δ. The first camera 1 and the second camera 2 simultaneously acquire images, obtaining image I. Asi with I Bsi Record the star-shaped target in each image I Asi The coordinate value QI in ATsi In each image I Bsi The coordinate value QI in BTsi After the target star returns to the center of the overlapping area of ​​the field of view, it scans downwards from the center point, stopping once every time it moves an angle δ. The first camera 1 and the second camera 2 simultaneously acquire images, obtaining image I. AXj with I BXj Record the star-shaped target in each image I AXj The coordinate value QI in ATXj In each image I BXj The coordinate value QI in BTXj QI ATsi With QI ATXj Let PA be the set of points with the same name, and let QI be the set of points with the same name. BTsi With QI BTXj Let PB be the set of points with the same name, where I Asi For the image captured by the first camera 1 after moving upwards by i δ angles, I BsiFor the image acquired by the second camera 2 after moving upwards by i δ angles, I AXj The image acquired by the first camera 1 after moving downwards by j δ angles, I BXj The image is captured by the second camera 2 after moving downwards by j δ angles, where j = i and i ≥ 2. PA and PB are two two-dimensional arrays of the same length. The optimal array is the length of the long side of the overlapping area of ​​the image, which results in a more uniform data distribution and better correction effect.

[0062] 2.3 The gray-level mapping function relationship f(g) is calculated using the polynomial fitting method;

[0063] 2.3.1 Based on each coordinate value in the set of corresponding points PB, extract the gray value w of the corresponding pixel in the image. Bsi and w BXj and all extracted grayscale values ​​w si and w Xj Let w be the gray set j Based on each coordinate value in the set of points PA with the same name, extract the gray value w of the corresponding pixel in the image. Asi and w AXi and all extracted grayscale values ​​w Asi and w AXi The record is denoted as grayscale set w. i ;

[0064] 2.3.2 In this embodiment, taking a quadratic polynomial as an example, the grayscale mapping function relationship is set as f(g)=a+bg+cg. 2 Where a is the constant coefficient, b is the coefficient of the linear term, c is the coefficient of the quadratic term, g is the binomial independent variable, and f(g) is the binomial dependent variable; w j Substitute each grayscale value in the above equation as the independent variable g, and then substitute w. i In and w j Substitute the gray value corresponding to the same position of each gray value in the image into the dependent variable f(g), and use the least squares method to determine the formula f(g) = a + bg + cg. 2 The optimal values ​​of parameters a, b, and c in the formula are substituted into the above formula to obtain the gray-scale mapping function f(g).

[0065] Step 3: Put U B Each pixel's gray level is mapped according to the gray-level mapping function f(g) to obtain the gray-level corrected image U. BF ;

[0066] Extract U B The gray level of each pixel is calculated and substituted as an independent variable into g in the gray-level mapping function relationship f(g) obtained in step 2.3.2. The calculated f(g) is the corrected gray level, and U is replaced with the corrected gray level.B Pixels at the same position in the array, traversing the entire U B Obtain the grayscale corrected image U BF ;

[0067] Specifically, when the corrected gray level is less than 0, the corrected gray level is set to 0; when the corrected gray level is greater than the maximum gray level LN, the corrected gray level is set to the maximum gray level resolution of the second camera 2.

[0068] Step 4: Obtain the mapping relationship S based on the positions of the collimator 3, the first camera 1, and the second camera 2, and then... A and U BF According to the splicing position mapping relationship, S is mapped to the mapped image U. AS and mapped image U BFS ;

[0069] 4.1 Obtain the mapping relationship S between the stitching positions of the first camera 1 and the second camera 2;

[0070] 4.1.1 Place a checkerboard calibration board in the overlapping area of ​​the field of view. Use the first camera 1 and the second camera 2 to acquire 20 pairs of checkerboard calibration board images with different poses at the same time. Then, perform corner detection on the checkerboard calibration board images acquired by the first camera 1 and the second camera 2 respectively, and record the corner coordinates of the checkerboard images with successful corner detection in the order of acquisition time. Match the corner coordinates of the checkerboard images recorded by the first camera 1 and the second camera 2 one by one according to the checkerboard corner distribution rules to obtain the matching feature point group of the first camera 1 and the second camera 2. The matching feature point group includes multiple pairs of matching feature points.

[0071] 4.1.2 Calculate the transformation matrix M of the first camera 1 relative to the second camera 2 using the matching feature point sets of the first camera 1 and the second camera 2. BA and the transformation matrix M BA The distance error d after projection of each pair of matched feature points o And based on the matching error d o Obtain the transformation matrix M BA The number of interior points, where the transformation matrix M BA Based on the second camera 2;

[0072] The RANSAC (Random Sample Consensus) algorithm is used to process the matching feature point groups of the first camera 1 and the second camera 2. Four pairs of matching feature points are selected respectively, and the transformation matrix M is calculated using the linear transformation method. BA ,in, m1, m2, m3, m4, m5, m6, m7, m8 represent the calculated M BA The matrix parameter M BAThe reason for using a two-dimensional matrix with eight parameters is that a two-dimensional to two-dimensional image homography transformation requires eight degrees of freedom. Based on these eight degrees of freedom, M... BA It is a two-dimensional matrix with 8 parameters;

[0073] Based on M BA Calculate the distance error d after projection for each pair of matched feature points. o , where d o This represents the distance error after projection of the o-th pair of matched feature points, 1≤o≤4;

[0074] Set a distance threshold dT, where dT > 0. Specifically, draw a circle with radius dT. If dT > 0... o <dT, then it means d o Let d be an interior point of the circle. o If the value is ≥dT, then no counting is performed. After the statistics are completed, the transformation matrix M is recorded. BA The number of interior points in the array;

[0075] Distance error d o The calculation method is as follows:

[0076] Setting (x) o A y o A )and These are the coordinates of any pair of matching feature points from four pairs of matching feature points in the first camera 1 and the second camera 2, and their projected distance error d. o The calculation is shown in the following formula:

[0077]

[0078] B3. Repeat step B2 multiple times to obtain the transformation matrix M with the largest number of interior points. BA Set as homography matrix H BA In this embodiment, the number of repetitions is 400.

[0079] B4. According to the homography matrix H BA Calculate the stitching position mapping relationship S between the first camera 1 and the second camera 2;

[0080] Let the coordinates of a 3D target point in the scene be P = (X, Y, Z), and its coordinates in the images captured by the first camera 1 and the second camera 2 be F = (x, y, 1). Then P and F satisfy the relationship: P = R -1 K -1 F, where K is the camera's intrinsic parameter matrix, which can be obtained by capturing at least two chessboard images in different poses and solving for corner detection and matching.

[0081] Due to the homography matrix H between the first camera 1 and the second camera 2 BA Also known as the perspective transformation matrix H BA The definition is as follows:

[0082] H BA =K B ·R B ·R A -1 ·K A -1

[0083] Among them, K B R is the intrinsic parameter matrix of the second camera 2, which can be obtained by acquiring at least two chessboard images with different poses using the second camera 2 and performing corner detection and matching. B K is the rotation matrix for the second camera 2; A R is the intrinsic parameter matrix of the first camera 1, which can be obtained by corner detection and matching of at least two chessboard images with different poses acquired by the first camera 1. A Let be the rotation matrix of the first camera 1; the rotation matrix of the first camera 1 relative to the second camera 2 can be derived from this. but,

[0084] Step 5: Put U A and U BF According to the splicing position mapping relationship, S is mapped to the mapped image U. AS and mapped image U BFS Mapped image U AS The calculation formula is: U AS =U A ·S, mapped image U BFS The calculation formula is: U BFS =U BF ·S;

[0085] Step 6: Map the image U AS and mapped image U BFS Combined into one image U ABFS and synthesize the image U ABFS As a stitched wide field-of-view infrared image.

[0086] In another embodiment provided by the present invention, U is set B This is the baseline image.

[0087] In another embodiment of the present invention, the number of cameras is six.

[0088] To further demonstrate the effectiveness of this invention, the following verifications were performed:

[0089] 1. Comparison of effects before and after splicing using the scheme in this invention

[0090] Figure 3 The image is the result of directly displaying the output images of six infrared cameras according to their spatial positions before stitching. Since the infrared images are directly displayed without spatial mapping processing and grayscale consistency correction, it can be seen that there are obvious object misalignment and grayscale inconsistency in the junction area of ​​adjacent images.

[0091] Figure 4 The output images from six infrared cameras are first spatially mapped, then grayscale consistency is corrected, and finally the output is displayed. It can be seen that there is no misalignment of objects in the junction area of ​​adjacent images, and the grayscale consistency is also significantly improved.

[0092] 2. Comparison of simulation effects of four grayscale consistency correction methods

[0093] The simulation results show that two images of the same size are stitched together horizontally, and four grayscale consistency correction methods are used to correct the grayscale of the images to be stitched and then display them.

[0094] like Figure 5 As shown, the average gray level compensation method is used. It can be seen that there is a clear vertical line where gray levels meet in the middle of the image, indicating that the gray levels of the left and right images are inconsistent.

[0095] like Figure 6 As shown, the grayscale histogram correction method is used. It can be seen that there are still vertical lines where grayscale levels intersect in the middle of the image, indicating that the grayscale inconsistency between the left and right images has been reduced but still exists.

[0096] like Figure 7 As shown, the grayscale linear stretching correction method was used, and it can be seen that the grayscale inconsistency between the left and right splicing areas is still very obvious.

[0097] like Figure 8 As shown, the grayscale consistency correction method described in this invention is applied. It can be seen that the grayscale transition area in the center of the image is natural and there is no grayscale transition area. The grayscale consistency of the left and right spliced ​​images is very good, and the image effect is the best.

Claims

1. A method of stitching infrared images, characterized by, It comprises the following steps: Step 1: arrange the first camera (1) and the second camera (2) so that there is a field of view overlap area between the first camera (1) and the second camera (2), control the first camera (1) and the second camera (2) to shoot the scene to be spliced at the same time, and output infrared images U A and U B , set U A as the reference image; Step 2: set the collimator (3), the star point target in the collimator (3) is aligned with the center of the field of view overlap area, according to the position of the collimator (3) and the first camera (1), the second camera (2), the homonymic point set PA of the first camera (1) and the homonymic point set PB of the second camera (2) are obtained, and the gray mapping function relationship f(g) is obtained by using D polynomial fitting method, wherein 1≤D≤3; Step 3: mapping each pixel gray value of U according to a gray mapping function f(g) to obtain a gray corrected image U B Step 3: mapping each pixel gray value of U according to a gray mapping function f(g) to obtain a gray corrected image U BF ; Step 4: Obtain the mapping relationship S based on the positions of the collimator (3) and the first camera (1) and the second camera (2), and then map U... A and U BF According to the splicing position mapping relationship, S is mapped to the mapped image U. AS and mapped image U BFS ; Step 5: Map the image U AS and the map image U BFS into one image U ABFS , and the synthesized image U ABFS as the large field of view infrared image after stitching; In step 4, according to the position of the collimator (3) and the first camera (1), the second camera (2), the method of obtaining the mapping relationship S is as follows: V1, by placing a checkerboard calibration board in the field of view overlap area, the matching feature point group of the first camera (1) and the second camera (2) is obtained; V2. Calculate the transformation matrix M of the first camera (1) relative to the second camera (2) through the matched feature point sets of the first camera (1) and the second camera (2) BA and the transformation matrix M BA The distance error d of the projection of each pair of matched feature points o and based on the matching error d o Obtain the transformation matrix M BA The number of inliers of the transformation matrix M BA with the second camera (2) as the reference; V3, repeat step V2 multiple times, the transformation matrix M with the largest number of inner points BA Set to homography H BA ; V4. According to the homography matrix H BA The stitching position mapping relationship S of the first camera 1 relative to the second camera 2 is calculated: S = K B -1 · H BA · K A ; In step V2, based on the matching error d o obtaining the transformation matrix M BA The method for obtaining the number of inliers of the transformation matrix M is: Set a distance threshold dT, dT > 0, specifically draw a circle with radius dT, if d o < dT, it means d o is an inner point, if d o ≥ dT, do not count, after counting, record the number of inner points in the transformation matrix M BA .

2. The infrared image splicing method according to claim 1, characterized in that: In step 2, the method for obtaining the homonymic point set PA of the first camera (1) and the homonymic point set PB of the second camera (2) is as follows: The star point target is scanned upward and downward from the center point along the vertical direction of the coincident area of the field of view, stopping once every angle δ, and the first camera (1) and the second camera (2) perform image acquisition to obtain images I Asi With I Bsi , I AXj With I BXj , the coordinate values QI Asi With QI AXj of the star point target in each image I ATsi With QI ATXj are recorded as the same point set PA, and the coordinate values QI Bsi With QI BXj of the star point target in each image I BTsi With QI BTXj are recorded as the same point set PB, wherein I Asi is the image acquired by the first camera (1) after scanning upward by i angles δ, I Bsi is the image acquired by the second camera (2) after scanning upward by i angles δ, I AXj is the image acquired by the first camera (1) after scanning downward by j angles δ, and I BXj is the image acquired by the second camera (2) after scanning downward by j angles δ, j = i and i ≥ 2.

3. The infrared image splicing method according to claim 2, characterized in that: The array length of PA and PB is the length of the long side of the field of view overlap area.

4. The infrared image splicing method according to claim 3, characterized in that: In step 2, D=2.

5. The infrared image splicing method according to claim 4, characterized in that: In step 2, the steps of obtaining the gray mapping function relationship f(g) by using D polynomial fitting method are as follows: A, obtain the gray set w of the homonymy point set PB j and the gray set w of the homonymy point set PA i ; B, set the binomial as f(g) = a + bg + cg 2 , where a is a constant coefficient, b is a linear term coefficient, c is a quadratic term coefficient, g is a binomial independent variable, f(g) is a binomial dependent variable; replace each element in the gray set w j with g in the binomial, and replace the element at the corresponding position in the gray set w i with f(g) in the binomial, solve a, b, and c of the quadratic polynomial, and obtain the gray mapping function relationship f(g).

6. The infrared image splicing method according to claim 5, characterized in that: The step 3 is specifically: extract U B the gray scale of each pixel as the independent variable into the gray scale mapping function relationship f(g) obtained in step 2, and calculate f(g) to obtain the corrected gray scale, and replace U with the corrected gray scale B pixels at the same position to obtain a gray scale correction image U BF ; Wherein, when the corrected gray is less than 0, the corrected gray is set to 0, and when the corrected gray is greater than the maximum gray level LN, the corrected gray is set to the maximum resolution of the second camera (2).

7. The infrared image splicing method according to claim 6, characterized in that: The step V1 is specifically: 20 pairs of checkerboard calibration board pictures of the same time and different poses are collected by using the first camera (1) and the second camera (2), then the corner point detection is carried out on the checkerboard calibration board pictures collected by the first camera (1) and the second camera (2) respectively, and the corner point coordinates of the checkerboard pictures with successful corner point detection are recorded in time sequence, the checkerboard picture corner point coordinates recorded by the first camera (1) and the second camera (2) are matched one by one according to the distribution rule of the checkerboard corner points, and the matching feature point group of the first camera (1) and the second camera (2) is obtained.

Images