A method and system for registering visible light and infrared thermal imaging images

Abstract

The present invention discloses a method and system for registering visible light and infrared thermal images. The method includes the following steps: fitting an affine transformation curve; acquiring an image and distance information of a target object; identifying and locating the target using a single-stage target detection model and outputting the target pixel area; initially registering the visible light image with the infrared thermal image; accurately registering the visible light image with the infrared thermal image to obtain feature point pairs; and calculating the confidence level of the feature point pairs. If the confidence level is greater than a preset confidence threshold, the registration accuracy of the visible light image and the infrared thermal image is determined to meet the required accuracy. The present invention reduces the detection range of the image area, analyzes the feature information of the target object area, and uses an improved ORB algorithm with advantages in efficiency for feature extraction and matching, thus solving problems such as slow image registration speed, poor real-time performance, and insufficient stability.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and system for registering visible light and infrared thermal imaging images. Background Art

[0002] Infrared thermal imaging detects thermal radiation from objects, providing precise surface temperature information unaffected by light. It has widespread application in military, industrial, and medical fields. However, identifying and locating targets using infrared thermal imaging currently faces numerous limitations. Compared to visible light images, infrared thermal imaging has lower resolution, a lack of detail, and limited texture information, all of which can affect the accuracy of target identification and location.

[0003] Visible light images, with their high resolution and rich texture information, are widely used for target identification and positioning. However, they are easily affected by lighting. Therefore, infrared thermal imaging and visible light images are highly complementary. Combining the strengths of these two, establishing a corresponding relationship between them, and achieving dual-spectral matching and fusion of visible light and infrared thermal imaging information can enhance the precise perception of target objects.

[0004] Establishing a correspondence between visible light images and infrared thermal images, that is, establishing a geometric transformation relationship between the coordinates of visible light and infrared thermal images. Currently, mainstream visible light and infrared thermal image registration methods, whether based on the spatial domain, frequency domain (such as maximum mutual information, wavelet transform), or feature-based registration (such as point features, line features, and surface features), require traversing the entire image area of ​​visible light and infrared thermal images to analyze, extract, and align their grayscale information or image features. This results in slow image registration, poor real-time performance, and low efficiency on embedded systems. Summary of the Invention

[0005] Technical purpose: In response to the above technical problems, the present invention proposes a visible light and infrared thermal imaging image registration method and system, which only analyzes the feature information of the target object area and uses the improved ORB algorithm with efficiency advantages for feature extraction and matching. It can quickly realize the registration of visible light and infrared thermal imaging images, and has the advantages of good real-time performance and excellent stability.

[0006] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] A method for registering visible light and infrared thermal imaging images, characterized by comprising the steps of:

[0008] S1. Fitting the affine transformation curve: Perform optical calibration on the image acquisition device in advance, collect visible light images and infrared thermal imaging images of the calibration plate at multiple points at different distances within the working range, calculate the image pixel geometric transformation relationship between the visible light image and the infrared thermal imaging image, and fit the affine transformation curve;

[0009] S2. Acquiring images and distance information of the target object: Acquiring visible light images, infrared thermal imaging images, and target distance information of the target object;

[0010] S3. Use a single-stage target detection model to identify and locate the target: The visible light image is used as the input source of the single-stage target detection model to identify and locate the target and output the target pixel area;

[0011] S4. Initial registration of the visible light image and the infrared thermal imaging image: Based on the affine transformation curve of step S1, the target distance information of step S2 is input and the corresponding affine transformation matrix is ​​calculated; the target pixel area outputted in step S3 is subjected to an affine transformation using the affine transformation matrix calculated in this step;

[0012] The target pixel area of ​​the visible light image is used as the image to be registered after affine transformation, and the infrared thermal imaging image after the initial registration is used as the reference image;

[0013] S5. Accurate registration of visible light images and infrared thermal images: Extract and match feature points between the image to be registered and the reference image to obtain feature point pairs;

[0014] S6. Calculate the confidence of the feature point pair: Calculate the confidence of the feature point pair obtained in step S5. If the confidence is greater than a preset confidence threshold, it is determined that the registration accuracy of the visible light image and the infrared thermal imaging image meets the requirements.

[0015] Preferably, in step S5, feature point extraction and feature point matching are performed on the image to be registered and the reference image using an improved ORB algorithm, including the following steps:

[0016] S5.1. Extract key points from the target object's image to be registered and the reference image using the FAST algorithm.

[0017] S5.2. Calculate the principal direction of the key point. Based on the similarity between visible light and infrared thermal imaging target contours, select the tangent direction of the edge region where the key point is located as the principal direction of the key point.

[0018] S5.3. Use BRIEF binary feature descriptor to describe the detected key points;

[0019] S5.4. Use Hamming distance to match feature points. When the similarity between the descriptors of the visible light image and the infrared thermal imaging image is greater than a preset similarity threshold, the feature points are determined to be matched.

[0020] Preferably, the step of extracting key points using the FAST algorithm includes:

[0021] Step S5.1.1: In the image, select a first number of pixels on a circle of a preset radius centered at pixel p, and calculate the grayscale difference between the first number of pixels and pixel p. If the number of pixels whose grayscale difference exceeds a pixel threshold exceeds a first preset value, pixel p is selected as a candidate keypoint and the next step is performed. Otherwise, pixel p is not selected as a keypoint, and a new pixel is selected as the center.

[0022] Step S5.1.2: If p is a candidate key point, calculate the grayscale difference between a second number of pixels on the circle and the center p. If the number of pixels whose grayscale difference exceeds the threshold t is greater than a second preset value, the center p is a key point, and proceed to the next step. Otherwise, the center p is not a key point, and the process returns to step S5.1.1 to reselect a pixel as the center. The second number is greater than the first number, and the second preset value is greater than the first preset value.

[0023] Step S5.1.3, performing non-maximum suppression on the image: calculating a score value of the key point p, where the score value is the sum of the absolute values ​​of the grayscale differences between a second preset number of pixels on the circle and the key point p;

[0024] If there are multiple key points in the preset neighborhood centered on the key point p, the score value of each key point is determined. If the score value of the key point p is the largest among all the key points in the preset neighborhood, the key point p is retained; otherwise, the key point p is suppressed; if there is only one key point in the preset neighborhood, the key point p is retained.

[0025] Preferably, in step S1, the calibration plate is made of transparent, heat-conductive optical glass, a regular pattern with clear feature points is used on the optical glass, a heat-insulating material is applied in a preset area, and no heat-insulating material is applied in other areas, and the calibration plate can obtain a contour image with a clarity that meets the requirements in both the visible light band and the infrared band;

[0026] The infrared thermal image of the calibration plate is used as the reference image, and the visible light image is used as the image to be registered. For each pair of visible light image and infrared thermal image, three stable diagonal points are selected as registration points, and the corresponding affine transformation matrix is ​​calculated. For multiple groups of affine transformation matrices at different distances, the affine transformation curve is fitted using the least squares polynomial curve.

[0027] Preferably, in step S1, the affine transformation matrix obtained is (m 00 , m 01 , m02 , m 10 , m 11 , m 12 ), and the distance parameter d i (i=1,2,3,...,n) form the sample data, and the sample data is (m 00 , d i )、(m 01 d i )、(m 02 , d i )、(m 10 , d i )、(m 11 , d i )、(m 12 , d i ) sample data, using the least squares polynomial curve to fit the affine transformation parameter m 00 、m 01 、m 02 、m 10 、m 11 、m 12 curve.

[0028] Preferably, in step S4, the affine transformation matrix is ​​used to perform affine transformation on the target pixel area of ​​the visible light image to obtain the image to be registered, and the geometric transformation calculation is shown in formula (2):

[0029]

[0030] (x, y) is the pixel point of the visible light image of the target object, and (x′, y′) is the pixel point registered to the infrared thermal imaging after affine transformation.

[0031] Preferably, the confidence level in step S6 is calculated as follows: the Euclidean distance of each pair of feature points is calculated, and the ratio of the number of feature point pairs whose Euclidean distance is less than a preset Euclidean distance threshold to the total number of feature point pairs is the confidence level.

[0032] A system for registering visible light and infrared thermal imaging images, characterized in that:

[0033] The affine transformation curve fitting module is used to pre-calibrate the image acquisition device optically, collect visible light images and infrared thermal imaging images of the calibration plate at multiple points at different distances within the working range, and fit the affine transformation curve;

[0034] Target object information acquisition module, used to acquire visible light images, infrared thermal imaging images and target distance information of the target object;

[0035] The image recognition and positioning module is used to identify and locate targets in visible light images using a single-stage target detection model and output the target pixel area;

[0036] The primary registration module is used to perform the primary registration of the visible light image and the infrared thermal image. The target pixel area of ​​the visible light image is subjected to affine transformation and used as the image to be registered. The infrared thermal image after the primary registration is used as the reference image.

[0037] The precise registration module is used to extract and match feature points between the image to be registered and the reference image, and obtain feature point pairs through matching;

[0038] The discrimination module is used to calculate the confidence of the feature point pair. If the confidence is greater than a preset confidence threshold, it is determined that the registration accuracy of the visible light image and the infrared thermal imaging image meets the requirements.

[0039] An electronic device comprises a memory and a processor, wherein a computer program is stored in the memory and the processor is configured to run the computer program to execute the method.

[0040] A computer-readable storage medium is characterized in that the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method.

[0041] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0042] The present invention discloses a method and system for registering visible light and infrared thermal imaging images. Compared with image registration technologies that only use spatial domain / frequency domain / features, the present invention reduces the detection range of the image area, analyzes only the feature information of the target object area, and uses an improved ORB algorithm with efficiency advantages for feature extraction and matching. This solves the problems of slow speed, poor real-time performance, and insufficient stability in visible light and infrared thermal imaging image registration on embedded systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a structural schematic diagram of a visible light and infrared thermal imaging image registration system according to the present invention;

[0044] Figure 2 It is a schematic diagram of the three-dimensional structure of a visible light and infrared thermal imaging image registration system according to the present invention;

[0045] Figure 3 This is a flow chart of a method for registering visible light and infrared thermal imaging images according to the present invention;

[0046] Figure 4 It is a schematic diagram of key points detected by the FAST algorithm of a visible light and infrared thermal imaging image registration method described in the present invention. DETAILED DESCRIPTION

[0047] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0048] Example 1

[0049] This paper proposes a method for image registration between visible light and infrared thermal imaging. This method pre-calibrates the geometric transformation relationship between visible light and infrared thermal images at multiple different working distances and then uses the least squares method to fit a curve of affine transformation parameters. The method then rapidly performs an initial pairing of the visible light and infrared thermal images based on object distance information. Using a renowned single-stage target detection model, the method identifies and locates the target object from the visible light image, effectively reducing the number of feature point analysis required for image registration and improving image registration efficiency. Furthermore, the ORB algorithm, which is highly efficient due to the different imaging principles of visible light and infrared thermal imaging, is modified to extract and match features between the registered image and the reference image. The improved ORB algorithm not only boasts high efficiency but also exhibits rotational invariance, taking into account the characteristics of visible light and infrared thermal image contour matching. Finally, by comparing the spatial relationships of matched feature points, mismatches of feature points are eliminated, ensuring that the image registration accuracy of visible light and infrared thermal imaging meets the required standards.

[0050] like Figure 1 、 Figure 2 The visible light and infrared thermal imaging image registration system shown includes a laser rangefinder 1 , an infrared thermal imaging device 2 and a visible light device 3 .

[0051] like Figure 3 As shown in FIG, the visible light and infrared thermal imaging image registration method based on the target object includes several steps:

[0052] Step S1, fitting the affine transformation curve: The image acquisition device is optically calibrated in advance. Visible light images and infrared thermal images of a calibration plate are collected at multiple distances within the operating range. The infrared thermal image serves as the reference image, and the visible light image serves as the image to be registered. For each pair of visible light and infrared thermal images, three stable diagonal points are selected as registration points, and the corresponding affine transformation matrix is ​​calculated. The affine transformation curve is then generated by fitting the affine transformation matrices of multiple sets of points at different distances using a least-squares polynomial curve fit.

[0053] That is, optical calibration is performed on the visible light and infrared thermal imaging equipment in advance, and the pixel coordinate geometric transformation curves of visible light and infrared thermal imaging at the working distance points are fitted.

[0054] The calibration plate of this embodiment uses a black and white checkerboard made of transparent, heat-conducting optical glass. The black checkerboard is coated with heat-insulating material, while the white checkerboard is not. This checkerboard can produce clear contour images in imaging equipment for both visible light and infrared light bands.

[0055] The image of the calibration plate is collected, and the pixel point (x, y) of the visible light image is registered to the pixel point (x′, y′) of the infrared thermal imaging through affine transformation. The matrix operation is shown in formula (1):

[0056]

[0057] Among them, (m 00 , m 01 , m 02 , m 10 , m 11 , m 12 ) is the affine transformation matrix, (m 02 , m 12 ) represents the image translation, (m 00 , m 01 , m 10 , m 11 ) represents image scale change and image rotation.

[0058] The affine transformation matrix solution process uses the three diagonal points of the chessboard as pairing points and uses the least squares method to solve the affine transformation matrix. The least squares method for solving the affine transformation matrix is ​​a known prior art and will not be described in detail in this embodiment.

[0059] The obtained affine transformation matrix of different distance points within the working range is combined with the distance parameter to form sample data, and the (m 00 , d i )、(m 01 d i )、(m 02 , d i )、(m 10 , d i )、(m 11 , d i )、(m 12 , d i ) Sample data, d i (i=1, 2, 3, ..., n) is the distance parameter. In order to take into account both real-time performance and accuracy, a least squares polynomial curve with an order not greater than 6 is used to fit the affine transformation parameter m 00 、m 01 、m 02 、m 10 、m 11 、m 12 curve.

[0060] Step S2, use Figure 1 、 Figure 2 The system shown, which includes a visible light device, an infrared thermal imaging device, and a laser rangefinder, captures visible light images, infrared thermal images, and target distance information of a target object within its operating range. When capturing the visible light image, infrared thermal image, and target distance information, the image capture device and the laser rangefinder maintain the same position, shooting direction, and shooting angle.

[0061] In step S3, the YOLO4 single-stage target detection model is used to identify and locate the target in the visible light image to obtain the target pixel area.

[0062] Step S4: The target distance information obtained in step S2 is input into the affine transformation curve obtained in step S1 to calculate the affine transformation matrix parameter m of the distance point. 00 、m 01 、m 02 、m 10 、m 11 、m 12 The affine transformation matrix is ​​used to perform affine transformation on the target pixel area of ​​the visible light image to obtain the image to be registered. The geometric transformation calculation is shown in formula (2):

[0063]

[0064] In step S5, feature point analysis, extraction, and matching are performed on the image to be registered (visible light image) and the reference image (infrared thermal image) using the improved ORB algorithm. Both the image to be registered and the reference image are in the target area. Since feature point analysis and matching do not need to be performed on the entire image, the image analysis and matching computational effort can be reduced, improving matching efficiency.

[0065] Feature point extraction and feature point pairing based on the improved ORB algorithm include the following steps:

[0066] Step S5.1: Use the FAST algorithm to extract key points from the reference image and the image to be registered. The steps for extracting key points using the FAST algorithm are as follows:

[0067] Step S5.1.1, as Figure 4 As shown in the figure, there are 16 pixels (p1, p2, ..., p16) on a circle with a radius of 3 and centered on pixel p. Set the grayscale threshold to t. Calculate the grayscale difference between p1, p9, p5, p13 and the center p. If there are more than 3 pixels with a grayscale difference exceeding the threshold t, the center p is considered a candidate keypoint; otherwise, the center p is not a keypoint.

[0068] Step S5.1.2: If p is a candidate key point, calculate the grayscale difference between the 16 pixels p1 to p16 and the center p. If there are more than 8 pixels with a grayscale difference exceeding the threshold t, the center p is a key point; otherwise, the center p is not a key point.

[0069] Step S5.1.3, perform non-maximum suppression on the image: first calculate the score value of the key point, such as Figure 4 As shown in the figure, the absolute value of the grayscale difference between keypoint p and 16 points on a circle with a radius of 3, centered on keypoint p, is summed. Non-maximum suppression is then performed. The rule is that if there are multiple keypoints within a 3x3 neighborhood centered on keypoint p, the score of each keypoint is determined. If the score of keypoint p is the largest among all keypoints in the 3x3 neighborhood, it is retained; otherwise, it is suppressed. If there is only one keypoint within the 3x3 neighborhood, it is retained.

[0070] Step S5.2, calculate the main direction of the key point. According to the similarity between the target contours of visible light and infrared thermal imaging, the tangent direction of the edge area where the key point is located is selected as the main direction of the key point, so that the improved ORB algorithm has rotation invariance. The calculation rule of the main direction of the key point is to assume that the position coordinates of any key point p(i) are (x i ,y i ), select two feature points p(ik) and p(i+k) with a distance k from the key point p(i) in the adjacent area, and the position coordinates of these two key points are (x i-k y i-k ) and (x i+k ,y i+k ), the main direction calculation formula of the key point p(i) is:

[0071]

[0072] The purpose of calculating the main direction of the key points is to solve the problem that the images taken may have inconsistent relative angles due to the installation deviation angles of the visible light device and the infrared thermal imaging device. This requires the visible light and infrared thermal imaging registration algorithm to be rotationally invariant.

[0073] In step S5.3, the detected key points are described using the BRIEF binary feature descriptor. Because the visible light image has been affine transformed in step S4, the image to be registered and the reference image have the same resolution, so Gaussian filtering is not used to process the image. The specific steps for generating feature descriptors using the BRIEF algorithm are as follows:

[0074] In step S5.3.1, a 31x31 window is generated with the key point obtained in step S5.1 as the center, and 256 pixel pairs are randomly selected in this window to construct a 256-dimensional bit vector.

[0075] Step S5.3.2: Rotate the 256 pixel pairs according to the main direction of the key point obtained in step S5.2 so that the direction of the 256 pixel pairs is consistent with the key point.

[0076] In step S5.3.3, the grayscale values ​​of these 256 pixel pairs are compared, 1 and 0 are assigned accordingly, and the corresponding feature vectors are created.

[0077] In step S5.4, the Hamming distances of the feature points are compared to match the feature points. When the similarity between the descriptors of the visible light and infrared thermal images exceeds a preset similarity threshold, the pair of feature points is considered a match. A descriptor is also known as a feature descriptor, and is generally used to refer to an algorithm.

[0078] Step S6: Calculate the confidence level of the feature point pairs. To further improve the stability of feature point matching, the confidence level of the feature point registration between visible light and infrared thermal imaging is calculated for the feature point pairs obtained using the improved ORB algorithm. The confidence level is calculated by calculating the Euclidean distance between each pair of matching points, i.e., the feature point pair (x1, x2) (x2, y2). The calculation formula is as follows:

[0079]

[0080] The percentage of feature point pairs whose Euclidean distance is less than the preset distance threshold represents the confidence level of the visible light and infrared thermal image registration. If the confidence level exceeds the preset confidence threshold, the registration accuracy of the visible light and infrared thermal images meets the requirements. A higher confidence level indicates a higher accuracy rate for the visible light and infrared thermal image registration.

[0081] The contents described in this embodiment are preferred specific implementation forms of the present invention. The scope of protection of the present invention should not be considered as limited to the specific forms described in the embodiment. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for registering visible light and infrared thermal imaging images, characterized in that: Including steps: S1. Fitting the affine transformation curve: Perform optical calibration on the image acquisition device in advance, collect visible light images and infrared thermal imaging images of the calibration plate at multiple points at different distances within the working range, calculate the image pixel geometric transformation relationship between the visible light image and the infrared thermal imaging image, and fit the affine transformation curve; S2. Acquiring images and distance information of the target object: Acquiring visible light images, infrared thermal imaging images, and target distance information of the target object; S3. Use a single-stage target detection model to identify and locate the target: The visible light image is used as the input source of the single-stage target detection model to identify and locate the target and output the target pixel area; S4. Initial registration of the visible light image and the infrared thermal imaging image: Based on the affine transformation curve of step S1, the target distance information of step S2 is input and the corresponding affine transformation matrix is ​​calculated; the target pixel area outputted in step S3 is subjected to an affine transformation using the affine transformation matrix calculated in this step; The target pixel area of ​​the visible light image is used as the image to be registered after affine transformation, and the infrared thermal imaging image after the initial registration is used as the reference image; S5. Accurate registration of visible light images and infrared thermal images: Extract and match feature points between the image to be registered and the reference image to obtain feature point pairs; S6. Calculating the feature point registration confidence of the feature point pair: For the feature point pair obtained in step S5, calculate the feature point registration confidence of the feature point pair. If the feature point registration confidence is greater than a preset feature point registration confidence threshold, it is determined that the registration accuracy of the visible light image and the infrared thermal imaging image meets the requirements. The calculation rule of the feature point registration confidence is to calculate the Euclidean distance of each pair of feature points. The proportion of the number of feature point pairs whose Euclidean distance is less than the preset Euclidean distance threshold to all feature point pairs is the feature point registration confidence. The step S5 includes the following steps: S5.

1. Extract key points from the target object's image to be registered and the reference image using the FAST algorithm; S5.

2. Calculate the main directions of key points; The calculation rule of the main direction of the key point is that the position coordinates of any key point p(i) are ( , ), select two feature points p(ik) and p(i+k) with a distance k from the key point p(i) in the adjacent area, and the position coordinates of these two key points are ( , ) and( , ), the main direction calculation formula of the key point p(i) is: （3）； S5.

3. Use BRIEF binary feature descriptor to describe the detected key points; S5.

4. Use Hamming distance to match feature points. When the similarity between the descriptors of the visible light image and the infrared thermal imaging image is greater than a preset similarity threshold, the feature points are determined to be matched.

2. The method for registering visible light and infrared thermal imaging images according to claim 1, characterized in that: The steps of extracting key points using the FAST algorithm include: Step S5.1.1: In the image, select a first number of pixels on a circle with a preset radius centered on pixel p, and calculate the grayscale difference between the first number of pixels and pixel p. If the number of pixels with a grayscale difference exceeding a pixel threshold exceeds a first preset value, pixel p is selected as a candidate keypoint and the next step is performed. Otherwise, pixel p is not selected as a keypoint, and a new pixel is selected as the center. Step S5.1.2: If p is a candidate key point, calculate the grayscale difference between a second number of pixels on the circle and the center p. If the number of pixels with a grayscale difference exceeding a threshold t is greater than a second preset value, the center p is a key point, and proceed to the next step. Otherwise, the center point p is not a key point, and the process returns to step S5.1.1 to reselect a pixel point as the center; the second number is greater than the first number, and the second preset value is greater than the first preset value; Step S5.1.3, performing non-maximum suppression on the image: calculating a score value of the key point p, where the score value is the sum of the absolute values ​​of the grayscale differences between a second preset number of pixels on the circle and the key point p; If there are multiple key points in the preset neighborhood centered on the key point p, the score value of each key point is determined. If the score value of the key point p is the largest among all the key points in the preset neighborhood, the key point p is retained; otherwise, the key point p is suppressed; if there is only one key point in the preset neighborhood, the key point p is retained.

3. The method for registering visible light and infrared thermal imaging images according to claim 1, characterized in that: In step S1, the calibration plate is made of transparent, heat-conductive optical glass with a regular pattern of clear feature points. A thermal insulation material is applied to a predetermined area, while other areas are left uncoated. The calibration plate can obtain contour images with the required clarity in both the visible and infrared bands. The infrared thermal image of the calibration plate is used as the reference image, and the visible light image is used as the image to be registered. For each pair of visible light image and infrared thermal image, three stable diagonal points are selected as registration points, and the corresponding affine transformation matrix is ​​calculated. For multiple groups of affine transformation matrices at different distances, the affine transformation curve is fitted using the least squares polynomial curve.

4. The method for registering visible light and infrared thermal imaging images according to claim 3, characterized in that: In step S1, the affine transformation matrix obtained is ( , , , , , ), and the distance parameter (i=1,2,3,…, ) constitutes the sample data, and the sample data is ( , )、( )、( , )、( , )、( , )、( , ) sample data, using the least squares polynomial curve to fit the affine transformation parameters ( , )、( )、( , )、( , )、( , )、( , ) curve.

5. The method for registering visible light and infrared thermal imaging images according to claim 4, characterized in that: In step S4, the affine transformation matrix is ​​used to perform affine transformation on the target pixel area of ​​the visible light image to obtain the image to be registered. The geometric transformation calculation is shown in formula (2): (2) ( , ) is the pixel point of the visible light image of the target object, ( , ) is the pixel point registered to the infrared thermal imaging through affine transformation.

6. A system for registering visible light and infrared thermal imaging images, executing the method according to any one of claims 1 to 5, characterized in that: The affine transformation curve fitting module is used to pre-calibrate the image acquisition device optically, collect visible light images and infrared thermal imaging images of the calibration plate at multiple points at different distances within the working range, and fit the affine transformation curve; Target object information acquisition module, used to acquire visible light images, infrared thermal imaging images and target distance information of the target object; The image recognition and positioning module is used to identify and locate targets in visible light images using a single-stage target detection model and output the target pixel area; The primary registration module is used to perform the primary registration of the visible light image and the infrared thermal image. The target pixel area of ​​the visible light image is subjected to affine transformation and used as the image to be registered. The infrared thermal image after the primary registration is used as the reference image. The precise registration module is used to extract and match feature points between the image to be registered and the reference image, and obtain feature point pairs through matching; The discrimination module is used to calculate the feature point registration confidence of the feature point pair. If the feature point registration confidence is greater than a preset feature point registration confidence threshold, it is determined that the registration accuracy of the visible light image and the infrared thermal imaging image meets the requirements.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 5.

Images