Image registration method, electronic device and storage medium

Abstract

The present application discloses an image registration method, electronic device, and computer-readable storage medium. The method includes: acquiring a first visible light image and a first infrared image, wherein the first visible light image and the first infrared image are obtained by capturing a target feature by a visible light camera and an infrared camera, respectively, of an imaging device in a first position; determining a first distance from the target feature to the imaging device in the first position; and determining a second mapping relationship between the first visible light image and the first infrared image based on the first distance and the second distance, as well as a first mapping relationship between a second visible light image and a second infrared image, wherein the second visible light image and the second infrared image are obtained by capturing a reference feature by a visible light camera and an infrared camera, respectively, of the imaging device in a second position, wherein the second distance is the distance from the reference feature to the imaging device in the second position. This method can reduce the processing overhead required for image registration.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image registration method, an electronic device, and a computer-readable storage medium. Background Art

[0002] The imaging principles of infrared images and visible light images are different. Visible light images are formed based on the different reflectivities of feature objects, while infrared images are formed based on the different temperatures or emissivities of feature objects. Therefore, the infrared images and visible light images taken of the same feature object can express the feature object from different dimensions. For example, the infrared image expresses the temperature of the feature object, while the visible light image expresses the texture, color and other information of the feature object.

[0003] Therefore, combining infrared and visible light images enables better feature analysis, with applications in remote sensing, feature detection, model reconstruction, motion estimation, feature recognition, and image fusion. However, since parallax often exists between the infrared camera used to capture infrared images and the visible light camera used to capture visible light images, the prerequisite for combining infrared and visible light images for analysis is to achieve image registration. This registration aims to obtain a mapping from the visible light image to the infrared image. This mapping allows for a one-to-one correspondence between pixels in the visible and infrared images that correspond to the same spatial point.

[0004] The existing registration method process can be roughly described as extracting feature points of infrared images and visible light images, matching the infrared images and visible light images based on the feature points, and finding the mapping relationship between the visible light image and the infrared image based on the matching results.

[0005] However, existing methods have high computational complexity and demanding hardware, thus requiring excessive processing overhead. Summary of the Invention

[0006] The present application provides an image registration method, an electronic device, and a computer-readable storage medium, which can solve the problem that existing image registration methods require excessive processing overhead.

[0007] To solve the above-mentioned technical problems, the present application adopts a technical solution: providing an image registration method. The method includes: acquiring a first visible light image and a first infrared image, wherein the first visible light image and the first infrared image are obtained by respectively photographing a target feature by a visible light camera and an infrared camera of an imaging device in a first posture; determining a first distance from the target feature to the imaging device in the first posture; and determining a second mapping relationship between the first visible light image and the first infrared image based on the first distance and the second distance, as well as a first mapping relationship between a second visible light image and a second infrared image, wherein the second visible light image and the second infrared image are obtained by respectively photographing a reference feature by a visible light camera and an infrared camera of the imaging device in a second posture, wherein the second distance is the distance from the reference feature to the imaging device in the second posture.

[0008] To solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, which includes a processor and a memory coupled to each other, wherein the memory stores program instructions; the processor is used to execute the program instructions stored in the memory to implement the above method.

[0009] In order to solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, storing program instructions that can be executed, and the program instructions can implement the above method when executed.

[0010] Through the above approach, in this application, a first mapping relationship between the second visible light image and the second infrared image, as well as a second distance from the reference feature to the imaging device in a second position, is pre-given. Based on the second distance, the first distance, and the first mapping relationship, a second mapping relationship can be calculated. This eliminates the need for feature point extraction and matching during each registration process, reducing the amount of computation and computational complexity, and lowering the processing overhead required for image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flow chart of an embodiment of the image registration method of the present application;

[0012] Figure 2 This is a flow chart of an embodiment of the image registration method of the present application;

[0013] Figure 3 It is a mapping relationship diagram under the visible to infrared registration method;

[0014] Figure 4 It is a mapping relationship diagram under the infrared to visible registration method;

[0015] Figure 5 It is a schematic diagram of two images taken by the same camera at different poses;

[0016] Figure 6 This is a flowchart of another embodiment of the image registration method of the present application;

[0017] Figure 7 yes Figure 6 A specific flow diagram of S31;

[0018] Figure 8 yes Figure 6 Another specific flow diagram of S31;

[0019] Figure 9 yes Figure 6 A specific flow diagram of S32;

[0020] Figure 10 yes Figure 6 Another specific flow diagram of S32;

[0021] Figure 11 This is a structural diagram of an embodiment of an electronic device of the present application;

[0022] Figure 12 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0024] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically specified.

[0025] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments unless there is a conflict.

[0026] The image registration method provided in this application can be used to register visible light images to infrared images, or to register infrared images to visible light images, depending on the application scenario. For ease of understanding, before introducing the image registration method provided in this application, its specific application scenario is first illustrated with examples:

[0027] Application Scenario 1: Measuring the temperature of a belt conveyor roller in a substation. A pre-planned route is prepared for the inspection robot, including several inspection points. The camera position at each inspection point is known. The inspection robot, equipped with a camera, moves along the planned route. At each inspection point, the camera captures images at the corresponding position, producing infrared and visible light images. The visible light image is inspected to determine the roller position in the visible light image. The visible light image is then registered with the infrared image. Based on the roller position in the visible light image, the roller position is determined from the registered infrared image, thereby obtaining the roller temperature.

[0028] Application Scenario 2: Image fusion and feature detection in a scene. A visible light image is captured of the scene to obtain a visible light image and an infrared image. The infrared image is then registered with the visible light image, and the registered infrared image is fused with the visible light image. This allows the corresponding pixels in the visible light image to be assigned information representing the emissivity, and feature detection is performed on the fused visible light image. Because the emissivity of the same feature falls within a certain range, it can, to a certain extent, characterize the target's location. Therefore, feature detection based on the fused visible light image can yield more accurate detection results.

[0029] The image registration method provided in this application is introduced as follows:

[0030] Figure 1 It is a flow chart of an embodiment of the image registration method of the present application. It should be noted that if there is substantially the same result, this embodiment does not use Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment may include:

[0031] S11: Acquire a first visible light image and a first infrared image.

[0032] The first visible light image and the first infrared image are obtained by respectively photographing the target feature object by the visible light camera and the infrared camera of the imaging device in the first posture.

[0033] The capture time of the first visible light image and the first infrared image may be the same or different, depending on the application scenario. For example, in an image fusion scenario, in order to avoid errors caused by time differences (such as errors caused by different lighting at different times), the capture time of the visible light image and the infrared image is consistent. The position and orientation of the imaging device mentioned in this application include the position of the imaging device and the orientation of the mirror.

[0034] The first visible light image and the first infrared image are respectively the visible light image and the infrared image to be registered. The first visible light image and the first infrared image can be obtained by real-time shooting or non-real-time shooting, which depends on the requirements of the application scenario.

[0035] S12: Determine a first distance from the target feature object to the imaging device in the first pose.

[0036] In some embodiments, the first distance can be obtained by laser ranging. Specifically, the laser can emit a laser beam toward a target feature, and the first distance can be calculated based on the reflection time of the laser beam. If the position of the laser rangefinder and the imaging device are inconsistent, a conversion relationship can be set between the position of the imaging device and the position of the laser, and the distance measured by the laser can be converted based on the conversion relationship to obtain the first distance.

[0037] In some embodiments, the first distance may also be calculated based on the detection result of the target feature in the first visible light image.

[0038] Specifically, the target feature and the reference feature can be regarded as particles, and the first visible light image and the second visible light image can be detected respectively to obtain the position of the target feature in the first visible light image and the position of the reference feature in the second visible light image; the ratio between the position of the target feature in the first visible light image and the position of the reference feature in the second visible light image is calculated; the ratio is multiplied by the second distance to obtain the first distance.

[0039] Alternatively, the first and second visible light images can be detected separately to obtain the location of the target feature in the first visible light image and the location of the reference feature in the second visible light image; the size ratio of the location of the target feature in the first visible light image to the location of the reference feature in the second visible light image can be calculated; and the size ratio can be multiplied by the second distance to obtain the first distance. The location of the target feature / reference feature can be a connected domain or a circumscribed rectangular box of the connected domain. The size ratio can be the length ratio or width ratio of the circumscribed rectangular box; the length ratio or width ratio of the connected domain, etc.

[0040] S13: Determine a second mapping relationship between the first visible light image and the first infrared image based on the first distance and the second distance, and the first mapping relationship between the second visible light image and the second infrared image, so as to achieve registration of the first visible light image and the first infrared image.

[0041] The second visible light image and the second infrared image are obtained by respectively photographing the reference feature object by the visible light camera and the infrared camera of the imaging device in the second posture, and the second distance is the distance from the reference feature object to the imaging device in the second posture.

[0042] The mapping relationship between two images includes the mapping relationship between the pixels in the two images that correspond to the same spatial point, indicating which pixels in the two images belong to the same spatial point. The mapping relationship can be in the form of, but is not limited to, a homography matrix. Taking the homography matrix as an example, the mapping relationship between the pixels in the two images can be expressed as:

[0043] x2=Hx1;

[0044] Where x1 represents a pixel in one image, x2 represents a pixel in the other image, and H represents the homography matrix.

[0045] It is understood that the registration of the first visible light image and the first infrared image can be the registration of the first visible light image to the first infrared image (hereinafter referred to as visible-to-infrared registration) or the registration of the first infrared image to the first visible light image (hereinafter referred to as infrared-to-visible registration). In the case of visible-to-infrared registration, the first mapping relationship is the first mapping relationship between the second visible light image and the second infrared image, and the second mapping relationship is the second mapping relationship between the first visible light image and the first infrared image. In the case of infrared-to-visible registration, the first mapping relationship is the first mapping relationship between the second infrared image and the second visible light image, and the second mapping relationship is the second mapping relationship between the first infrared image and the first visible light image.

[0046] The first mapping relationship can be obtained through calibration. The calibration method can be manual calibration or automatic calibration. For example, in the automatic calibration method, the reference feature object can be a feature object with significant feature points (such as a checkerboard), based on which the feature points of the second visible light image and the second infrared image can be extracted and matched to obtain matching feature point pairs in the second visible light image and the second infrared image; based on the coordinates of the corresponding pixel points of the matched feature point pairs in the second visible light image and the second infrared image, the first mapping relationship between the second visible light image and the second infrared image is fitted. The algorithm based on which the feature point extraction is based can be SIFT, FAST, ORB, RANSAC, Sobel algorithm, etc. The first distance can be obtained through calibration.

[0047] In some embodiments, the first mapping relationship can be adjusted based on the difference between the first distance and the second distance to obtain a second mapping relationship. In other words, the first visible light image can be converted based on the first mapping relationship to obtain a preliminarily registered infrared image; and the preliminarily registered infrared image can be compensated based on the difference between the first distance and the second distance to obtain a second mapping relationship between the first visible light image and the first infrared image.

[0048] In some embodiments, a third mapping relationship between the first visible light image and the second visible light image can be obtained based on the difference between the first distance and the second distance, and a fourth mapping relationship between the second infrared image and the first infrared image can be obtained. The second mapping relationship can be obtained based on the first mapping relationship, the third mapping relationship, and the fourth mapping relationship. This approach will be described in detail in later embodiments.

[0049] It's understandable that in related technologies, each registration of a visible light image with an infrared image requires feature point extraction and matching. Matching feature point pairs are then determined based on the matching results, and the mapping relationship between the visible light image and the infrared image is derived based on the matched feature point pairs. However, this approach requires a large and complex computational effort, resulting in excessive computational overhead, as feature point extraction and matching must be performed each time.

[0050] To this end, in this application, a first mapping relationship between the second visible light image and the second infrared image, as well as a second distance from the reference feature to the imaging device in a second position, is pre-given. Based on the second distance, the first distance, and the first mapping relationship, a second mapping relationship can be calculated. This eliminates the need for feature point extraction and matching during each registration process, reducing the amount of computation and computational complexity, and lowering the processing overhead required for image registration.

[0051] Figure 2 It is a flow chart of another embodiment of the image registration method of the present application. It should be noted that if there is substantially the same result, this embodiment does not use Figure 2 The process sequence shown is limited. This embodiment is a further extension of S13. Figure 2 As shown, this embodiment may include:

[0052] S21: Determine a third mapping relationship between the first visible light image and the second visible light image, and a fourth mapping relationship between the second infrared image and the first visible light image based on the first distance and the second distance.

[0053] In the visible-to-infrared registration method, the third mapping relationship is the third mapping relationship between the first visible light image and the second visible light image, and the fourth mapping relationship is the fourth mapping relationship between the second infrared image and the first infrared image. In the infrared-to-visible registration method, the third mapping relationship is the third mapping relationship between the second visible light image and the first visible light image, and the fourth mapping relationship is the fourth mapping relationship between the first infrared image and the second infrared image.

[0054] In other words, in the visible-to-infrared registration method, the mapping relationship H2 from the first visible light image to the first infrared image can be calculated based on the third mapping relationship U from the first visible light image to the second visible light image, the first mapping relationship H1 from the second visible light image to the second infrared image, and the fourth mapping relationship V from the second infrared image to the first infrared image. For specific mapping relationship diagrams, please refer to Figure 3 In the infrared to visible registration method, based on the fourth mapping relationship V' between the first infrared image and the second infrared image, the first mapping relationship H1' between the second infrared image and the second visible light image, and the third mapping relationship U' between the second visible light image and the first visible light image, the mapping relationship H2' between the first infrared image and the first visible light image is calculated. For the specific mapping relationship diagram, please refer to Figure 4 .

[0055] S22: Obtain a second mapping relationship based on the first mapping relationship, the third mapping relationship, and the fourth mapping relationship.

[0056] In the visible-to-infrared registration method, U, H1, and V can be multiplied in sequence to obtain H2, that is, H2 = U * H1 * V. In the infrared-to-visible registration method, V', H1', ​​and U' can be multiplied to obtain H2', that is, H2' = V' * H1' * U'.

[0057] Further, combined with Figure 5 The mapping relationship between two images taken by the same camera at different positions is explained below:

[0058] The camera captures plane π in the left and right poses, respectively, to obtain images m and m'. The unit normal vector of plane π in the camera coordinate system of the left pose is n, and the first distance from plane π to the camera of the left pose is d. The mapping relationship from m to m' is:

[0059] H=K(R+t·n T / d)K -1 ;

[0060] Among them, K represents the intrinsic parameter of the camera, R represents the camera rotation parameter from m to m', and t represents the camera translation parameter from m to m'.

[0061] Based on this, S21 can be further expanded:

[0062] Figure 6 It is a flow chart of another embodiment of the image registration method of the present application. It should be noted that if there is substantially the same result, this embodiment does not use Figure 6 The process sequence shown is limited. Figure 6 As shown, this embodiment may include:

[0063] S31: Based on the first distance and the second distance, and the rotation parameters of the visible light camera, estimate the visible translation vector between the first visible light image and the second visible light image and the first unit normal vector of the plane where the target feature object is located; based on the first distance and the second distance, and the rotation parameters of the infrared camera, estimate the infrared translation vector between the second infrared image and the first infrared image and the second unit normal vector of the plane where the target feature object is located.

[0064] The visible light camera rotation parameter is the angle between the optical axis of the visible light camera and the normal direction of the plane where the target feature object is located; the infrared camera rotation parameter is the angle between the optical axis of the infrared camera and the normal direction of the plane where the target feature object is located.

[0065] The visible light camera rotation parameters and the infrared camera rotation parameters can be obtained through calibration. When the plane on which the imaging device carrier is located is perpendicular or nearly perpendicular to the plane on which the target feature is located, the visible light camera rotation parameters and the infrared camera rotation parameters can be calibrated to (0, 0). The visible light translation vector and the first unit normal vector can be calculated based on the first angle between the optical axis direction of the visible light camera and the plane on which the imaging device carrier is located, as well as the first distance and the second distance. The first angle can be obtained through pre-calibration. The imaging device carrier can be a robot, a fixed support, etc. In the visible-to-infrared alignment method, the first unit normal vector is relative to the visible light camera coordinate system in the first pose, that is, the first unit normal vector is the unit normal vector of the plane on which the target feature is located in the visible light camera coordinate system in the first pose. In the infrared-to-visible alignment method, the first unit normal vector is relative to the infrared camera coordinate system in the second pose, that is, the first unit normal vector is the unit normal vector of the plane on which the target feature is located in the infrared camera coordinate system in the second pose.

[0066] Specifically, refer to Figure 7 , it can be seen that the steps of obtaining the translation vector and the first unit normal vector may include:

[0067] S311: Calculate a first difference between the first distance and the second distance.

[0068] S312: Calculate a first unit normal vector based on the first angle.

[0069] S313: Calculate the product of the first difference and the first unit normal vector as the visible translation vector. The above S311 to S313 can be expressed as the following formula:

[0070] t1=(d1-d2)·n1;

[0071]

[0072] in, represents the first angle, n1 represents the first unit normal vector, d1 represents the first distance, d2 represents the second distance, and t1 represents the visible translation coefficient.

[0073] In addition, considering that the first angle is (0, 0) or close to (0, 0) (the optical axis direction of the visible light camera is parallel to the plane where the carrier of the imaging device is located), the first difference can be directly used as the visible light translation vector, and (0, 0, 1) T As the first unit normal vector. Therefore, before estimating the visible translation vector and the first unit normal vector in the above method, it is possible to determine whether the first angle meets the angle requirement; if it does not meet the angle requirement, the visible translation vector is estimated in the above method; otherwise, the first difference between the first distance and the second distance is calculated as the visible light camera translation parameter, and (0, 0, 1) is converted to T As the first unit normal vector, the amount of calculation can be reduced. The angle requirement can be (0, 0) or close to (0, 0).

[0074] In the visible-to-infrared alignment method, the infrared translation vector and the second unit normal vector can be calculated based on the second angle between the infrared camera's optical axis and the plane on which the imaging device's carrier is located, as well as the first distance and the second distance. The second angle can be obtained by pre-calibration. In the visible-to-infrared alignment method, the second unit normal vector is relative to the infrared camera coordinate system in the first pose, that is, the second unit normal vector is the unit normal vector of the plane where the target feature is located in the infrared camera coordinate system in the first pose. In the infrared-to-visible alignment method, the second unit normal vector is relative to the visible light camera coordinate system in the second pose, that is, the second unit normal vector is the unit normal vector of the plane where the target feature is located in the visible light camera coordinate system in the second pose.

[0075] Specifically, refer to Figure 8 , the steps of obtaining the infrared translation vector and the second unit normal vector may include:

[0076] S314: Calculate a second difference between the second distance and the first distance.

[0077] S315: Calculate a second unit normal vector based on the second angle.

[0078] S316: Calculate the product of the second difference and the second unit normal vector as the infrared translation vector. S314 to S316 can be expressed as the following formula:

[0079] t2=(d2-d1)·n2;

[0080]

[0081] in, represents the second angle, n2 represents the second unit normal vector, and t2 represents the infrared translation coefficient.

[0082] In addition, considering that the second angle is (0, 0) or close to (0, 0) (the optical axis direction of the infrared camera is parallel to the plane where the carrier of the imaging device is located), the second difference can be directly used as the infrared translation vector, and (0, 0, 1) T As the second unit normal vector. Therefore, before estimating the infrared translation vector in the above method, it is possible to determine whether the second angle meets the angle requirement; if it does not meet the angle requirement, the infrared translation vector is estimated in the above method; otherwise, the second difference between the second distance and the first distance is calculated as the infrared translation vector, and (0, 0, 1) is replaced by T As the second unit normal vector, this can reduce the amount of calculation.

[0083] S32: Acquire a third mapping relationship based on the first distance, the visible translation vector, and the first unit normal vector; acquire a fourth mapping relationship based on the second distance, the infrared translation vector, and the second unit normal vector.

[0084] See also Figure 9 The step of obtaining the third mapping relationship in S32 may include:

[0085] S321: Calculate the first product of the visible translation vector and the first unit normal vector.

[0086] S322: Calculate a first quotient of the first product and the second distance.

[0087] S323: Calculate the first quotient and the first sum of the identity matrix.

[0088] S324: Obtain a third mapping relationship based on the first sum and the internal parameter of the visible light camera.

[0089] In the visible-to-infrared registration method, the formulas for the above S321 to S324 can be as follows:

[0090]

[0091] Among them, U represents the intrinsic parameter of the visible light camera, k1 represents the third mapping relationship, R1 represents the rotation parameter of the visible light camera, I represents the unit matrix, and t1 represents the visible translation vector.

[0092] See also Figure 10 The step of obtaining the fourth mapping relationship in S32 may include:

[0093] S325: Calculate the second product of the infrared translation vector and the second unit normal vector.

[0094] S326: Calculate a second quotient of the second product and the first distance.

[0095] S327: Calculate the second sum of the second quotient and the identity matrix.

[0096] S328: Obtain a fourth mapping relationship based on the second sum and the internal parameter of the infrared camera.

[0097] In the visible-to-infrared registration method, the formulas for the above S325 to S328 can be as follows:

[0098]

[0099] Among them, V represents the intrinsic parameter of the infrared camera, k2 represents the fourth mapping relationship, R2 represents the rotation parameter of the infrared camera, and t2 represents the infrared translation vector.

[0100] Figure 11 This is a schematic diagram of the structure of an embodiment of the electronic device of the present application. Figure 11 As shown, the speech recognition device includes a processor 21 and a memory 22 coupled to each other.

[0101] The memory 22 stores program instructions for implementing the method of any of the above embodiments; the processor 21 is used to execute the program instructions stored in the memory 22 to implement the steps of the above method embodiments. The processor 21 can also be called a CPU (Central Processing Unit). The processor 21 may be an integrated circuit chip with signal processing capabilities. The processor 21 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0102] In some embodiments, the electronic device may further include an imaging device (not shown), which includes a visible light camera and an infrared camera. The visible light camera can be used to capture visible light images, and the infrared camera can capture infrared images. The captured visible light images and infrared images can be used for image registration.

[0103] In some embodiments, the electronic device may be connected to an external imaging device to receive visible light images and infrared images from the imaging device to achieve image registration.

[0104] For the description of image registration, please refer to the previous embodiment and will not be repeated here.

[0105] Figure 12 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of the present application. Figure 12 As shown, the computer-readable storage medium 30 of the embodiment of the present application stores program instructions 31, and when the program instructions 31 are executed, the method provided in the above embodiment of the present application is implemented. Among them, the program instructions 31 can form a program file and be stored in the above-mentioned computer-readable storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) executes all or part of the steps of the various embodiments of the present application. The aforementioned computer-readable storage medium 30 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0106] Among them, and / or, in addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of this application.

Claims

1. An image registration method, characterized in that: include: Acquire a first visible light image and a first infrared image, where the first visible light image and the first infrared image are obtained by respectively photographing a target feature object by a visible light camera and an infrared camera of an imaging device in a first posture; Determining a first distance from the target feature to the imaging device in the first posture; Based on the first distance and the second distance, and the first mapping relationship between the second visible light image and the second infrared image, the second mapping relationship between the first visible light image and the first infrared image is determined, the second visible light image and the second infrared image are obtained by respectively photographing the reference feature object by the visible light camera and the infrared camera of the imaging device in the second posture, the second distance is the distance from the reference feature object to the imaging device in the second posture, the second distance and the first mapping relationship are given in advance, and the first mapping relationship is obtained by extracting and matching feature points of the second visible light image and the second infrared image.

2. The method according to claim 1, characterized in that The determining, based on the first distance and the second distance, and a first mapping relationship between the second visible light image and the second infrared image, a second mapping relationship between the first visible light image and the first infrared image includes: Determining a third mapping relationship between the first visible light image and the second visible light image, and a fourth mapping relationship between the second infrared image and the first visible light image based on the first distance and the second distance; The second mapping relationship is obtained based on the first mapping relationship, the third mapping relationship, and the fourth mapping relationship.

3. The method according to claim 2, characterized in that The determining, based on the first distance and the second distance, a third mapping relationship between the first visible light image and the second visible light image, and a fourth mapping relationship between the second infrared image and the first visible light image includes: Based on the first distance and the second distance, and the visible light camera rotation parameter, estimate the visible translation vector between the first visible light image and the second visible light image and the first unit normal vector of the plane where the target feature is located; based on the first distance and the second distance, and the infrared camera rotation parameter, estimate the infrared translation vector between the second infrared image and the first infrared image and the second unit normal vector of the plane where the target feature is located; wherein the visible light camera rotation parameter is the angle between the optical axis direction of the visible light camera and the normal direction of the plane where the target feature is located, and the infrared camera rotation parameter is the angle between the optical axis direction of the infrared camera and the normal direction of the plane where the target feature is located; The third mapping relationship is acquired based on the first distance, the visible translation vector, and the first unit normal vector; and the fourth mapping relationship is acquired based on the second distance, the infrared translation vector, and the second unit normal vector.

4. The method according to claim 3, characterized in that The acquiring the third mapping relationship based on the first distance, the visible translation vector, and the first unit normal vector includes: Calculating a first product of the visible translation vector and the first unit normal vector; calculating a first quotient of the first product and the second distance; calculating a first sum of the first quotient and the identity matrix; Obtaining the third mapping relationship based on the first sum and an internal parameter of the visible light camera; The acquiring the fourth mapping relationship based on the second distance, the infrared translation vector, and the second unit normal vector includes: Calculating a second product of the infrared translation vector and the second unit normal vector; calculating a second quotient of the second product and the first distance; calculating a second sum of the second quotient and the identity matrix; The fourth mapping relationship is obtained based on the second sum and the internal parameter of the infrared camera.

5. The method according to claim 3, characterized in that The step of estimating the visible translation vector and the first unit normal vector comprises: Calculate the visible translation vector and the first unit normal vector based on a first angle between the optical axis of the visible light camera and the plane where the carrier of the imaging device is located, as well as the first distance and the second distance; The step of estimating the infrared translation vector and the second unit normal vector comprises: The infrared translation vector and the second unit normal vector are calculated based on a second angle between the optical axis direction of the infrared camera and the plane where the carrier of the imaging device is located, as well as the first distance and the second distance.

6. The method according to claim 5, characterized in that Before calculating the visible translation vector based on a first angle between the optical axis of the visible light camera and the plane on which the carrier of the imaging device is located, and the first distance and the second distance, the method further includes: Determining whether the first angle meets the angle requirement; If the angle requirement is not met, performing the step of calculating the visible translation vector and the first unit normal vector based on the first angle between the optical axis direction of the visible light camera and the plane where the carrier of the imaging device is located, as well as the first distance and the second distance; Before calculating the infrared translation vector based on the second angle between the optical axis direction of the infrared camera and the plane where the carrier of the imaging device is located, and the first distance and the second distance, the method further includes: Determining whether the second angle meets the angle requirement; If the angle requirement is not met, the step of calculating the infrared translation vector and the second unit normal vector based on the second angle between the optical axis direction of the infrared camera and the plane where the carrier of the imaging device is located, as well as the first distance and the second distance is executed.

7. The method according to claim 5, characterized in that The calculating the visible translation vector and the first unit normal vector based on a first angle between the optical axis direction of the visible light camera and the plane where the carrier of the imaging device is located, as well as the first distance and the second distance, includes: calculating a first difference between the first distance and the second distance; Calculating the first unit normal vector based on the first angle; Calculating a product of the first difference and the first unit normal vector as the visible translation vector; The step of calculating the infrared translation vector and the second unit normal vector based on a second angle between the optical axis of the infrared camera and the plane where the carrier of the imaging device is located, as well as the first distance and the second distance, includes: calculating a second difference between the second distance and the first distance; Calculate the second unit normal vector based on the second angle; The product of the second difference and the second unit normal vector is calculated as the infrared translation vector.

8. The method according to claim 1, characterized in that Determining a first distance from the target feature to the imaging device in the first posture includes: detecting the first visible light image and the second visible light image respectively to obtain a position of the target feature in the first visible light image and a position of the reference feature in the second visible light image; Calculating a size ratio between a position of a target feature in the first visible light image and a position of the reference feature in the second visible light image; The first distance is obtained by multiplying the size ratio and the second distance.

9. An electronic device, characterized in that: comprising a processor and a memory coupled to each other, wherein: The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable program instructions, and when the program instructions are executed, the method according to any one of claims 1 to 8 is implemented.

Images