Image registration method, device and computer readable storage medium

Abstract

The embodiment of the present application provides a kind of image registration method, device and computer readable storage medium, the method comprises: parameter initialization is carried out to system;Extract the element set containing at least 2 matching elements, and judge whether the element set satisfies the minimum requirement for estimating rigid transformation matrix based on 2D plane;For the element set satisfying the minimum requirement, the rotation matrix R and displacement t of each matching element in element set are obtained;Local rigid transformation matrix T 局 Is calculated, the minimum value of the distance of all elements is evaluated and recorded, and the iteration number is also recorded, and the local iteration optimization operation is ended when the iteration number reaches the iteration number threshold preset in parameter initialization or the minimum value of distance is not greater than distance convergence threshold;Global rigid transformation matrix T 全 Is solved for all inliers;Distance evaluation is carried out to all extracted matching elements, and image fusion registration is carried out based on new inlier.The embodiment of the present application improves image registration accuracy, and robustness is strong.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image registration method, apparatus and computer-readable storage medium. Background Technology

[0002] In computer vision and image processing, rigid transformation matrices are commonly used to describe the rigid transformations of objects in space. They are crucial for tasks such as image alignment, registration, and target tracking. Rigid transformation matrices generally only involve two basic operations: rotation and translation. Most of them are based on the ICP (Iterative Closest Point) algorithm, which is based on point cloud registration. Its purpose is to find the optimal rotation matrix R and translation vector t from the source point cloud to the target point cloud.

[0003] Traditional ICP methods mainly rely on point matching to estimate the transformation between two datasets. In 2D planar space, when there are few or no point clouds, it is easy to lack sufficient feature information to directly solve the rigid transformation matrix, resulting in poor image registration accuracy. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide an image registration method that improves image registration accuracy and has strong robustness.

[0005] A further technical problem to be solved by the embodiments of the present invention is to provide an image registration device that improves image registration accuracy and has strong robustness.

[0006] A further technical problem to be solved by the embodiments of the present invention is to provide a computer-readable storage medium that improves image registration accuracy and has strong robustness.

[0007] To address the aforementioned technical problems, this invention first provides the following technical solution: an image registration method, the method comprising the following steps: The system parameters are initialized, including at least initializing the rigid transformation matrix T and setting the iteration number threshold and the distance convergence threshold; Extract an element set containing at least two matching elements from video images captured by an in-vehicle camera, and determine whether the element set meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane. The rotation matrix R and displacement t of each matching element in the set of elements are obtained by performing local iterative optimization operation on the set of elements that meet the minimum requirements for rigid transformation matrix estimation based on 2D plane. During each local iterative optimization operation, the local rigid transformation matrix T is calculated based on the obtained rotation matrix R and displacement t.局 And based on T 局 The minimum distance of all current elements is evaluated and recorded, and the number of iterations is also recorded. The local iterative optimization operation ends when the number of iterations reaches the preset iteration threshold during parameter initialization or when the minimum distance is not greater than the distance convergence threshold. Extract the optimal local rigid transformation matrix T from all local iterative optimization operations. 局 Solve for all interior points the global rigid transformation matrix T. 全 ;as well as Based on the global rigid transformation matrix T 全 Distance evaluation is performed on all extracted matching elements to obtain new inliers, and image fusion registration is performed based on the new inliers.

[0008] Furthermore, the parameter initialization also includes setting the matching line parallel angle threshold, the interior point attribution threshold, the confidence level, and the execution termination state quantity.

[0009] Furthermore, the at least two matching elements include two sets of non-coincident matching points. During the local iterative optimization operation, the two lines are estimated using the two sets of non-coincident matching points, and the rotation matrix R and displacement t are calculated according to the following formula:

[0010] Where θ represents the angle θ formed between the two lines at the two sets of matching points. and Let t1 and t2 represent the average values ​​of the translation vectors t1 and t2 at the two sets of matching points, respectively.

[0011] Furthermore, the global rigid transformation matrix T is solved using the singular value decomposition algorithm for all interior points. 全 .

[0012] Furthermore, the at least two matching elements include a set of matching points and a set of matching lines. During the local iterative optimization operation, the straight line is estimated using the set of matching points, and the rotation matrix R and displacement t are calculated according to the following formula:

[0013] Where θ represents the angle θ between the line estimated by a set of matching points and the matching line. and Let t1 and t2 represent the translation vectors t1 and t2 at a set of matching points, respectively.

[0014] Furthermore, the global rigid transformation matrix T is solved for all interior points. 全For multiple sets of matching points, the singular value decomposition algorithm is used to solve for the angle θ1 formed between the two lines on the two sets of matching points; for multiple sets of matching lines, multiple angles θ are calculated, and the median filtering is applied to the multiple angles θ to obtain the average angle θ2; for multiple sets of matching lines, the intersection point between each pair of matching lines is determined, and the displacement t of each set of matching lines is calculated based on the intersection point. i (i=1,2,…N), and t is the translation of each matching line. i1 t i2 Equal to the average value of the included angles θ1 and θ2. Then, the rotation matrix R and displacement t are calculated according to the following formulas: .

[0015] Furthermore, the at least two matching elements include two sets of non-parallel matching lines. During the local iterative optimization operation, the intersection points t1 and t2 are obtained from the two sets of non-parallel matching lines, and the included angles θ1 and θ2 are calculated from the two sets of non-parallel matching lines. The average angle of included angles θ1 and θ2 is then calculated. Then, the rotation matrix R and displacement t are calculated according to the following formulas: .

[0016] Furthermore, the global rigid transformation matrix T is solved for all interior points. 全 For multiple sets of matching lines, multiple included angles θ are calculated, and median filtering is applied to these multiple included angles θ to obtain the average value of the included angles θ. For multiple sets of matching lines, determine the intersection point between each pair of matching lines, and calculate the displacement t of each set of matching lines based on the intersection point. i (i=1,2,…N), and t is the translation of each matching line. i1 t i2 Then, calculate the rotation matrix R and displacement t according to the following formulas: .

[0017] On the other hand, in order to solve the above-mentioned further technical problems, the present invention provides the following technical solution: an image registration device connected to a vehicle camera, characterized in that the image registration device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the image registration method as described in any one of claims 1 to 8.

[0018] Furthermore, in order to solve the aforementioned technical problems, the present invention provides the following technical solution: a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the image registration method as described in any one of the above.

[0019] By adopting the above technical solution, the embodiments of the present invention have at least the following beneficial effects: The embodiments of the present invention first initialize the system parameters, including at least initializing the rigid transformation matrix T and setting an iteration number threshold and a distance convergence threshold; then, extracting an element set containing at least two matching elements from the video images provided by the vehicle-mounted camera, and determining whether the element set meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane; then, performing local iterative optimization operations on the element set that meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane to obtain the rotation matrix R and displacement t of each matching element in the element set; and during each local iterative optimization operation, calculating the local rigid transformation matrix T based on the obtained rotation matrix R and displacement t. 局 And based on T 局 The minimum distance to all current elements is evaluated and recorded, along with the number of iterations. The local iterative optimization operation ends when the number of iterations reaches the preset threshold during parameter initialization or when the minimum distance is not greater than the distance convergence threshold. Then, the optimal local rigid transformation matrix T is extracted from all local iterative optimization operations. 局 Solve for all interior points the global rigid transformation matrix T. 全 Finally, based on the global rigid transformation matrix T 全 Distance evaluation is performed on all extracted matching elements to obtain new inliers, and image fusion registration is performed based on these new inliers. In the specific implementation, the element set can be checked early on to ensure it meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane. Only when the minimum requirements are met can the rigid transformation estimation steps be performed, avoiding unnecessary calculations when the basic requirements are not met and optimizing computational complexity. Furthermore, local iterative optimization is continuously performed during each iteration, eliminating matching elements that do not meet the matching requirements, improving image registration accuracy, and demonstrating strong robustness. Additionally, by monitoring the iteration count and distance convergence value during the local iterative optimization process, the local iterative optimization operation ends when a set threshold for the number of iterations or the distance convergence value is reached, thus controlling the computation to be completed within a reasonable timeframe. Attached Figure Description

[0020] Figure 1 This is a flowchart of an optional embodiment of the image registration method of the present invention.

[0021] Figure 2 This is a schematic diagram of an optional embodiment of the image registration device of the present invention. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following illustrative embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention. Moreover, the embodiments and features in the embodiments of the present invention can be combined with each other unless otherwise specified.

[0023] like Figure 1 As shown, an optional embodiment of the present invention provides an image registration method, including the following steps: S10: Initialize the system parameters, including at least initializing the rigid transformation matrix T and setting the iteration number threshold and distance convergence threshold; S20: Extract an element set containing at least two matching elements from the video images provided by the vehicle-mounted camera, and determine whether the element set meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane. S30: Perform local iterative optimization on the set of elements that meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane to obtain the rotation matrix R and displacement t of each matching element in the set of elements; S40: During each local iteration optimization operation, the local rigid transformation matrix T is calculated based on the obtained rotation matrix R and displacement t. 局 And based on T 局 The minimum distance of all current elements is evaluated and recorded, and the number of iterations is also recorded. The local iterative optimization operation ends when the number of iterations reaches the preset iteration threshold during parameter initialization or when the minimum distance is not greater than the distance convergence threshold. S50: Extract the optimal local rigid transformation matrix T from all local iterative optimization operations. 局 Solve for all interior points the global rigid transformation matrix T. 全 ;as well as S60: Based on the global rigid transformation matrix T 全 Distance evaluation is performed on all extracted matching elements to obtain new inliers, and image fusion registration is performed based on the new inliers.

[0024] This invention first initializes the system parameters, including at least initializing the rigid transformation matrix T and setting an iteration threshold and a distance convergence threshold. Then, it extracts an element set containing at least two matching elements from video images acquired by a vehicle-mounted camera, and determines whether the element set meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane. Next, it performs local iterative optimization on the element set that meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane to obtain the rotation matrix R and displacement t of each matching element in the element set. During each local iterative optimization operation, the local rigid transformation matrix T is calculated based on the obtained rotation matrix R and displacement t. 局 And based on T 局 The minimum distance to all current elements is evaluated and recorded, along with the number of iterations. The local iterative optimization operation ends when the number of iterations reaches the preset threshold during parameter initialization or when the minimum distance is not greater than the distance convergence threshold. Then, the optimal local rigid transformation matrix T is extracted from all local iterative optimization operations. 局 Solve for all interior points the global rigid transformation matrix T. 全 Finally, based on the global rigid transformation matrix T 全 Distance evaluation is performed on all extracted matching elements to obtain new inliers, and image fusion registration is performed based on these new inliers. In the specific implementation, the element set can be checked early on to ensure it meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane. Only when the minimum requirements are met can the rigid transformation estimation steps be performed, avoiding unnecessary calculations when the basic requirements are not met and optimizing computational complexity. Furthermore, local iterative optimization is continuously performed during each iteration, eliminating matching elements that do not meet the matching requirements, improving image registration accuracy, and demonstrating strong robustness. Additionally, by monitoring the iteration count and distance convergence value during the local iterative optimization process, the local iterative optimization operation ends when a set threshold for the number of iterations or the distance convergence value is reached, thus controlling the computation to be completed within a reasonable timeframe.

[0025] In an optional embodiment of the present invention, the parameter initialization further includes: setting a matching line parallel angle threshold, an interior point attribution threshold, a confidence level, and an execution termination state quantity. In this embodiment, the following parameters are pre-set to optimize algorithm performance during parameter initialization: setting a matching line parallel angle threshold to determine whether two lines or line segments in the matching elements are parallel; if the angle θ between the two lines is less than the matching line parallel angle threshold, they are considered parallel; setting an interior point attribution threshold to determine whether a point is sufficiently close to a line to be considered an interior point of that line; if the distance from the point to the line is less than or equal to this threshold, the point is considered an interior point of the line; setting a confidence level and an execution termination state quantity; these can be flags such as 0, 1, and 2 (for failure, success, or failure to meet conditions) to indicate the judgment status of the algorithm execution feedback result.

[0026] In an optional embodiment of the present invention, the at least two matching elements include two sets of non-overlapping matching points. During local iterative optimization, the two lines are estimated using the two sets of non-overlapping matching points, and the rotation matrix R and displacement t are calculated according to the following formula: (Formula 1) Where θ represents the angle θ formed between the two lines at the two sets of matching points. and Let t1 and t2 represent the average values ​​of the translation vectors t1 and t2 at the two sets of matching points, respectively.

[0027] In this embodiment of the invention, when the two matching elements include two sets of non-overlapping matching points: since two matching points can determine a straight line, then the two sets of non-overlapping matching points have two straight lines. Through basic mathematical operations, the angle θ between these two straight lines can be obtained, and thus the rotation matrix R can be calculated. The calculation of the displacement t is determined by the translation vectors t1 and t2 of the two sets of non-overlapping matching points and the rotation matrix R. The rotation matrix R and the displacement t can be obtained by formula (1). In the specific implementation process, the at least two matching elements include two sets of non-overlapping matching points, and the element set also meets the minimum requirements for rigid transformation matrix estimation based on the 2D plane. Rigid transformation matrix estimation can be performed on the 2D plane based on the two sets of non-overlapping matching points, effectively completing image registration.

[0028] In an optional embodiment of the present invention, the global rigid transformation matrix T is solved using the singular value decomposition algorithm for all interior points. 全 In this embodiment of the invention, there are multiple matching elements, including multiple sets of non-overlapping matching points. Based on these matching elements, an additional global rigid transformation matrix T is performed on all interior points. 全The solution is obtained by using the singular value decomposition algorithm, which effectively implements the global optimization steps and improves the accuracy and robustness of the rigid transformation matrix.

[0029] In an optional embodiment of the present invention, the at least two matching elements include a set of matching points and a set of matching lines. During local iterative optimization, the straight line is estimated using the set of matching points, and the rotation matrix R and displacement t are calculated according to the following formula: (Formula 2) Where θ represents the angle θ between the line estimated by a set of matching points and the matching line. and Let t1 and t2 represent the translation vectors t1 and t2 at a set of matching points, respectively.

[0030] If noise exists in the video image, subsequent registration may be inaccurate if the matching elements are only feature points. In this embodiment of the invention, matching lines with obvious features are introduced to form diverse feature elements (matching points and matching lines), providing more constraints than matching between matching points, facilitating the solution of the rigid transformation matrix, effectively completing image registration, and exhibiting strong robustness. In specific implementation, when at least two matching elements include a set of matching points and a set of matching lines, the element set also meets the minimum requirements for rigid transformation matrix estimation based on the 2D plane. Rigid transformation matrix estimation can be performed on the 2D plane based on two sets of non-overlapping matching points according to formula (2), and the endpoints on the matching lines are not required to overlap, reducing the computational load of matching. It can also effectively complete image registration.

[0031] In an optional embodiment of the present invention, the global rigid transformation matrix T is solved for all interior points. 全 For multiple sets of matching points, the singular value decomposition algorithm is used to solve for the angle θ1 formed between the two lines on the two sets of matching points; for multiple sets of matching lines, multiple angles θ are calculated, and the median filtering is applied to the multiple angles θ to obtain the average angle θ2; for multiple sets of matching lines, the intersection point between each pair of matching lines is determined, and the displacement t of each set of matching lines is calculated based on the intersection point. i (i=1,2,…N), and t is the translation of each matching line. i1 t i2 Equal to the average value of the included angles θ1 and θ2. Then, the rotation matrix R and displacement t are calculated according to the following formulas: (Formula 3).

[0032] In this embodiment of the invention, there are multiple matching elements, including multiple sets of matching points and at least multiple sets of matching lines. Based on these matching elements, an additional global rigid transformation matrix T is performed on all interior points. 全 The solution can be obtained by using formula (3), which can effectively solve the global rigid transformation matrix T. 全 Implement global optimization steps to improve the accuracy and robustness of the rigid transformation matrix.

[0033] In an optional embodiment of the present invention, the at least two matching elements further include two sets of non-parallel matching lines. During the local iterative optimization operation, the intersection points t1 and t2 are obtained from the two sets of non-parallel matching lines, and the included angles θ1 and θ2 are calculated from the two sets of non-parallel matching lines. The average angles of the included angles θ1 and θ2 are then calculated. Then, the rotation matrix R and displacement t are calculated according to the following formulas: (Formula 4).

[0034] If noise exists in the video image, subsequent registration may be inaccurate if the matching elements are only feature points. In this embodiment of the invention, by introducing matching lines with obvious features, a variety of feature elements (matching lines and matching lines) can be formed, providing more constraints than the matching between matching points, facilitating the solution of the rigid transformation matrix, effectively completing image registration, and exhibiting strong robustness. In specific implementation, when at least two matching elements include two sets of non-parallel matching lines, the element set also meets the minimum requirements for rigid transformation matrix estimation based on the 2D plane. Rigid transformation matrix estimation can be performed on the 2D plane based on the two sets of non-parallel matching lines according to formula (4), and it is not required that the endpoints on the matching lines coincide, reducing the computational load of matching. It can also effectively complete image registration.

[0035] In an optional embodiment of the present invention, the global rigid transformation matrix T is solved for all interior points. 全 For multiple sets of matching lines, multiple included angles θ are calculated, and median filtering is applied to these multiple included angles θ to obtain the average value of the included angles θ. For multiple sets of matching lines, determine the intersection point between each pair of matching lines, and calculate the displacement t of each set of matching lines based on the intersection point. i (i=1,2,…N), and t is the translation of each matching line. i1 t i2 Then, calculate the rotation matrix R and displacement t according to the following formulas: (Formula 5).

[0036] Because multiple sets of matching lines calculate multiple included angles Due to potential interference from noise or other factors, directly using these included angles to solve for the global rigid transformation matrix T is not feasible. 全 This may lead to inaccurate results, therefore the angle of the matching line is also important. The average value is obtained by filtering, and the average value is obtained by filtering out outliers. Then solve for the global rigid transformation matrix T 全 This improves the accuracy of image registration. In this embodiment of the invention, there are multiple matching elements, including multiple sets of matching lines. Based on these matching elements, an additional global rigid transformation matrix T is performed on all interior points. 全 The global rigid transformation matrix T can be solved using formula (5). 全 This effectively implements global optimization steps, improving the accuracy and robustness of image registration.

[0037] On the other hand, such as Figure 2 As shown, this embodiment of the invention further provides an image registration device 7, which is connected to a vehicle-mounted camera 8. The image registration device 7 includes a processor 70, a memory 72, and a computer program stored in the memory 72 and configured to be executed by the processor 70. When the processor 70 executes the computer program, it implements the image registration method as described in any of the above embodiments.

[0038] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 72 and executed by the processor 70 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the image registration device 7.

[0039] The image registration device 7 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The image registration device 7 may include, but is not limited to, a processor 70 and a memory 72. Those skilled in the art will understand that the schematic diagram is merely an example of the image registration device 7 and does not constitute a limitation on the image registration device 7. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the image registration device 7 may also include input / output devices, network access devices, buses, etc.

[0040] The processor 70 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The processor 70 is the control center of the image registration device 7, connecting all parts of the image registration device 7 via various interfaces and lines.

[0041] The memory 72 can be used to store the computer program and / or modules. The processor 70 implements various functions of the image registration device 7 by running or executing the computer program and / or modules stored in the memory 72 and calling the data stored in the memory 72. The memory 72 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as image recognition function, image overlay function, etc.), etc.; the data storage area may store data (such as graphic data, etc.) created based on the use of the vehicle memory parking mapping device 3. In addition, the memory 72 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0042] If the functions described in the embodiments of the present invention are implemented in the form of software functional modules or units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the implementation of all or part of the processes in the methods of the above embodiments of the present invention can also be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 70, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0043] In an optional embodiment of the present invention, the vehicle-mounted camera is a fisheye camera. In this embodiment, the vehicle-mounted camera is a fisheye camera, which has a large field of view and captures more image information. When performing global and local matching of the current image frame, it has more point and line features available for filtering, which can reduce the probability of mismatches in similar scenes and improve the reliability of parking trajectory planning. In specific implementation, the original image undergoes distortion correction and is scaled to 640*480.

[0044] In another aspect, embodiments of the present invention provide a readable storage medium, the readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the readable storage medium is located to perform the image registration method as described in any of the preceding claims.

[0045] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0046] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many variations under the guidance of the present invention without departing from the spirit and scope of the claims. These variations are all within the scope of protection of the present invention.

Claims

1. An image registration method, characterized in that, The method includes: The system parameters are initialized, including at least initializing the rigid transformation matrix T and setting the iteration number threshold and the distance convergence threshold; Extract an element set containing at least two matching elements from video images captured by an onboard camera, and determine whether the element set meets the minimum requirements for rigid transformation matrix estimation based on a 2D plane. The rotation matrix R and displacement t of each matching element in the set of elements are obtained by performing local iterative optimization operation on the set of elements that meet the minimum requirements for rigid transformation matrix estimation based on 2D plane. During each local iterative optimization operation, the local rigid transformation matrix T is calculated based on the obtained rotation matrix R and displacement t. 局 And based on T 局 The minimum distance of all current elements is evaluated and recorded, and the number of iterations is also recorded. The local iterative optimization operation ends when the number of iterations reaches the preset iteration threshold during parameter initialization or when the minimum distance is not greater than the distance convergence threshold. Extract the optimal local rigid transformation matrix T from all local iterative optimization operations. 局 Solve for all interior points the global rigid transformation matrix T. 全 ;as well as Based on the global rigid transformation matrix T 全 Distance evaluation is performed on all extracted matching elements to obtain new inliers, and image fusion registration is performed based on the new inliers.

2. The image registration method as described in claim 1, characterized in that, The parameter initialization also includes setting the matching line parallel angle threshold, interior point attribution threshold, confidence level, and execution termination state quantity.

3. The image registration method as described in claim 1, characterized in that, The at least two matching elements include two sets of non-coincident matching points. During the local iterative optimization operation, the two lines are estimated using the two sets of non-coincident matching points, and the rotation matrix R and displacement t are calculated according to the following formula: Where θ represents the angle θ formed between the two lines at the two sets of matching points. and Let t1 and t2 represent the average values ​​of the translation vectors t1 and t2 at the two sets of matching points, respectively.

4. The image registration method as described in claim 3, characterized in that, The global rigid transformation matrix T is solved using the singular value decomposition algorithm for all interior points. 全 .

5. The image registration method as described in claim 1, characterized in that, The at least two matching elements include a set of matching points and a set of matching lines. During local iterative optimization, the straight line is estimated using the set of matching points, and the rotation matrix R and displacement t are calculated according to the following formulas: Where θ represents the angle θ between the line estimated by a set of matching points and the matching line, and t1 and t2 represent the vectors of a set of matching points.

6. The image registration method as described in claim 5, characterized in that, Solve for the global rigid transformation matrix T for all interior points 全 For multiple sets of matching points, the singular value decomposition algorithm is used to solve for the angle θ1 formed between the two lines at the two sets of matching points; for multiple sets of matching lines, multiple angles θ are calculated, and the median filtering is applied to the multiple angles θ to obtain the average angle θ2; for multiple sets of matching lines, the intersection point between each pair of matching lines is determined, and the displacement t of each set of matching lines is calculated based on the intersection point. i Let i = 1, 2, ..., N, and denote the translation of each matching line as t. i1 t i2 Take the average value of the included angles θ1 and θ2. Then, the rotation matrix R and displacement t are calculated according to the following formulas: 。 7. The image registration method as described in claim 1, characterized in that, The at least two matching elements include two sets of non-parallel matching lines. During the local iterative optimization operation, the intersection points t1 and t2 are obtained from the two sets of non-parallel matching lines, and the included angles θ1 and θ2 are calculated from the two sets of non-parallel matching lines. The average angle of included angles θ1 and θ2 is then calculated. Then, the rotation matrix R and displacement t are calculated according to the following formulas: 。 8. The image registration method as described in claim 7, characterized in that, Solve for the global rigid transformation matrix T for all interior points 全 For multiple sets of matching lines, multiple included angles θ are calculated, and median filtering is applied to these multiple included angles θ to obtain the average value of the included angles θ. For multiple sets of matching lines, determine the intersection point between each pair of matching lines, and calculate the displacement t of each set of matching lines based on the intersection point. i Let i = 1, 2, ..., N, and denote the translation of each matching line as t. i1 t i2 Then, the rotation matrix R and displacement t are calculated according to the following formulas: 。 9. An image registration device, connected to a vehicle-mounted camera, characterized in that, The image registration apparatus includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the image registration method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the image registration method as described in any one of claims 1 to 8.

Images