Multi-view camera relative pose estimation method based on imu rotation axis direction prior

Abstract

The present application relates to the technical field of machine vision, and provides a multi-view camera relative pose estimation method based on IMU rotation axis direction prior, comprising: depth parameterization is carried out to the translation vector formed by a multi-view camera system from a first time to a second time, and the generalized epipolar constraint of the multi-view camera system is reconstructed based on the depth parameter; the relative pose estimation problem of the multi-view camera in the multi-view camera system is converted into a single variable cubic polynomial root seeking by using the rotation axis direction prior information provided by an inertial measurement unit and combining the reconstructed generalized epipolar constraint; after single variable root seeking, the rotation matrix and the translation vector of the multi-view camera system from the first time to the second time are solved based on the root seeking result, and the relative pose estimation result is obtained. While maintaining competitive estimation accuracy, the calculation efficiency can be significantly improved, and the present application provides an efficient and practical scheme for the implementation of multi-view vision positioning in an automatic driving system.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method for estimating the relative pose of a multi-view camera based on the prior knowledge of the rotation axis direction of an IMU. Background Technology

[0002] Relative pose estimation is a core problem in computer vision, playing a crucial role in applications such as Simultaneous Localization and Mapping (SLAM), Structure-of-Motion (SfM), and autonomous driving. The core of this task lies in estimating the relative position and pose relationships between two or more views, and its accuracy and efficiency directly affect the subsequent visual localization and 3D reconstruction results. In practical applications, inaccurate feature matching and mismatches are the main challenges affecting the robustness of algorithms. To address this, researchers often combine relative pose estimation methods with robust estimation frameworks such as RANSAC to effectively suppress the interference of outliers. However, the number of iterations in RANSAC increases exponentially with the minimum number of samples required by the model, resulting in high computational costs for methods relying on a large number of matching points, making it difficult to meet real-time requirements. This challenge is particularly prominent in dynamic and complex autonomous driving scenarios. The presence of dynamic objects such as vehicles and pedestrians easily introduces a large number of anomalous feature matches, while the system must operate stably under strict real-time constraints. Traditional methods often struggle to meet the extremely high processing frequency requirements while ensuring estimation accuracy.

[0003] Based on the number of cameras used, relative pose estimation can be divided into monocular and multi-view systems. Multi-view systems typically consist of multiple rigidly connected cameras, offering significant advantages over monocular systems: firstly, through simultaneous observation from multiple perspectives, a wider field of view and richer environmental information can be obtained; secondly, by utilizing pre-calibrated fixed baselines between cameras, the absolute scale of the scene can be directly recovered, thus overcoming the scale blur problem inherent in monocular vision. These characteristics make multi-view systems more suitable for autonomous driving applications where reliability and accuracy are extremely critical.

[0004] Despite the significant advantages of multi-camera systems, their relative pose estimation faces more complex geometric constraints and a higher computational burden. For example, linear methods in multi-camera systems typically require at least 17 pairs of matching points, far more than the 8 pairs required by monocular systems. Within the RANSAC framework, the increase in the number of matching points leads to an exponential increase in the number of iterations, making computational efficiency a bottleneck for practical applications in autonomous driving. Therefore, traditional pose estimation methods often suffer from high computational complexity and reliance on a large number of feature matches, making it difficult to meet the stringent speed constraints of autonomous driving scenarios. Summary of the Invention

[0005] The purpose of this invention is to solve at least one technical problem in the background art and to provide a method for estimating the relative pose of a multi-view camera based on the prior knowledge of the rotation axis direction of an IMU.

[0006] To achieve the above objectives, this invention provides a multi-camera relative pose estimation method based on IMU rotation axis prior, comprising: The translation vector formed by the multi-view camera system from the first time step to the second time step is parameterized with depth, and the generalized epipolar constraints of the multi-view camera system are reconstructed based on the depth parameters. By utilizing the prior information on the rotation axis direction provided by the inertial measurement unit and combining it with the reconstructed generalized epipolar constraints, the relative pose estimation problem of the multi-camera system in a multi-camera system is transformed into a single-variable cubic polynomial root-finding problem. After finding the roots of the single variable, the rotation matrix and translation vector of the multi-camera system from the first time step to the second time step are solved based on the root results to obtain the relative pose estimation results.

[0007] According to one aspect of the present invention, the depth parameterization of the translation vector formed by the multi-view camera system from the first time moment to the second time moment includes: Motion platform equipped with multi-view cameras from time to time Exercise 3D points exist Always being photographed It was observed that, in Always being photographed It was observed that the motion platform was in and The coordinate systems for each time point are denoted as follows: and To simplify the derivation, the world coordinate system is used. The origin is set at a three-dimensional point. and its coordinate axis direction is Platform coordinate system at any moment Alignment; Define the transformations between the coordinate systems, including: from the camera coordinate system to its respective motion platform The transformation is From the camera coordinate system to the motion platform The transformation is ,in and Here is the corresponding rotation matrix. and The corresponding translation vector; the motion platform moves from the coordinate system Move to coordinate system The relative pose transformation is ,in For rotation matrix, It is a translation vector; due to the world coordinate system With coordinate system Alignment, with the rotation matrix between them being the identity matrix. Set up a world coordinate system To coordinate system The transformation is From the world coordinate system To coordinate system The transformation is ; Based on the above definition, the following transformation relationship can be obtained: (1) in, Represents the transformation matrix The homogeneous form of it can be obtained from equation (1): (2) In the generalized camera model, the matching features corresponding to the 3D point P It can be viewed as two Plück lines and The intersection point; among which, Represents the unit direction vector, by Give, Represents a moment vector; i, j∈n; according to Plück line geometry, any point on the line can be parameterized as: (3) in, For depth parameters, It is the set of real numbers; due to the three-dimensional points Located at the intersection of two Plück lines, its coordinates can be parameterized along each of the two Plück lines. Let... and Representing three-dimensional points respectively In the coordinate system of the motion platform and coordinate system The coordinates in the matrix are given by: (4) From this we can obtain and Regarding depth parameters and The expression: (5) Equation (5) shows that the translation vector and It can be determined by two depth parameters. and express.

[0008] According to one aspect of the present invention, the reconstruction based on the generalized epipolar constraint of a multi-view camera system with depth parameters includes: Pluke's straight line From the coordinate system Transform to coordinate system At that time, the transformed coordinates are obtained. ; (6) Plücke Straight Line and Pluke's straight line In the same coordinate system The necessary and sufficient condition for the intersection of the two sides is: (7) Substituting equation (6) into equation (7), we obtain the generalized epipolar constraint: (8) Substituting further into equation (2), we get: (9) in, This indicates that the vector is transformed into an antisymmetric matrix; finally, substituting equation (5) into equation (9) yields the result based on the depth parameter. and Generalized pole constraint: (10).

[0009] According to one aspect of the present invention, the method of utilizing prior information on the rotation axis direction provided by the inertial measurement unit, combined with the reconstructed generalized epipolar constraints, to transform the relative pose estimation problem of a multi-camera system in a multi-camera system into a single-variable cubic polynomial root-finding problem includes: Using the rotation axis direction measurement data of the motion platform provided by the inertial measurement unit, when the rotation axis direction is known, the rotation is described using the axis angle representation method, and the corresponding rotation matrix satisfies the Rodrigues formula: (11) in, It is a known direction vector of the unit rotation axis. It refers to the magnitude of the rotation angle; for small rotation angles, the first-order Taylor approximation is applied: and At this point, the Rodriguez formula simplifies to: (12) Substituting the approximate expression (11) into the generalized epipolar constraint (10), we get: (13) After simplification, the original nonlinear equation can be linearized as follows: (14) Among them, coefficient to It is calculated directly from the observation data and the known direction of the rotation axis. and For the unknown variables to be solved; Using three sets of point correspondences that are not at the origin, the following homogeneous linear equation system can be constructed: (15) Equation (15) can be expressed in matrix form: (16) The condition for the existence of a non-zero solution to the system of equations (16) is that the determinant of its coefficient matrix is ​​zero: (17) This yields a value regarding the rotation angle. The cubic polynomial equation system (16) has at most three real roots.

[0010] According to one aspect of the present invention, after the univariate root calculation, the rotation matrix and translation vector of the multi-camera system from the first time moment to the second time moment are solved based on the root calculation result to obtain the relative pose estimation result, including: Calculate the rotation angle Then, rotate the angle. and the known rotation axis direction vector Substituting into equation (11), the rotation matrix is ​​reconstructed. ; Calculate the rotation angle Then, substitute it into the matrix defined by equation (16). In the solution, the depth parameter can be obtained by taking the right singular vector with the minimum singular value through singular value decomposition. and , depth parameters and Substituting into equation (5), the translation vector is calculated. and Finally, the translation vector is recovered using the following formula. : (18).

[0011] To achieve the above objectives, the present invention also provides a multi-camera relative pose estimation system based on IMU rotation axis orientation prior, comprising: The constraint reconstruction module performs depth parameterization on the translation vector formed by the multi-camera system from the first time step to the second time step, and reconstructs the generalized epipolar constraints of the multi-camera system based on the depth parameters. The problem transformation module utilizes the prior information on the rotation axis direction provided by the inertial measurement unit and combines it with the reconstructed generalized epipolar constraints to transform the relative pose estimation problem of the multi-camera system into a single-variable cubic polynomial root-finding problem. The relative pose estimation result acquisition module, after univariate root calculation, solves the rotation matrix and translation vector of the multi-camera system from the first time moment to the second time moment based on the root calculation result, and obtains the relative pose estimation result.

[0012] To achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the multi-camera relative pose estimation method based on IMU rotation axis direction prior as described above.

[0013] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-camera relative pose estimation method based on IMU rotation axis direction prior as described above.

[0014] According to the present invention, this invention innovatively integrates the prior information on the rotation axis direction provided by the inertial measurement unit (IMU) and combines it with the first-order approximation characteristic of small rotation angles between frames, simplifying the relative pose estimation problem of multi-view cameras into a solution process requiring only four sets of matching points. Furthermore, this problem is transformed into a root-finding problem of a single-variable cubic polynomial, which significantly reduces computational complexity compared to the common eighth-order polynomial solution path in existing methods. Thanks to the simplification of the number of matching points and the improvement in computational efficiency, the method of this invention can significantly reduce the number of iterations when integrated into the RANSAC framework, thus making it highly suitable for applications with high real-time requirements such as autonomous driving.

[0015] This invention introduces an inertial measurement unit (IMU) and integrates it with a vision system as a feasible approach. While cameras provide rich visual information, their measurements are susceptible to interference from factors such as occlusion and changes in lighting. In contrast, the IMU can independently output high-frequency angular velocity and acceleration, achieving short-time, high-precision motion tracking; the two are naturally complementary. Furthermore, autonomous driving scenarios offer unique conditions for algorithm simplification: due to the smooth motion of the vehicle and the high frame rate of the camera, inter-frame rotation is typically small, making first-order rotation approximation possible, thus significantly simplifying the pose estimation problem. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a multi-camera relative pose estimation method based on IMU rotation axis orientation prior according to an embodiment of the present invention; Figure 2 This diagram schematically illustrates a coordinate system diagram of a multi-view camera system according to an embodiment of the present invention. Detailed Implementation

[0017] The invention will now be discussed with reference to exemplary embodiments. It should be understood that the described embodiments are merely intended to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0018] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment".

[0019] Figure 1 The flowchart schematically illustrates a multi-camera relative pose estimation method based on IMU rotation axis orientation prior according to an embodiment of the present invention. Figure 1 As shown, in this embodiment, the multi-camera relative pose estimation method based on IMU rotation axis direction prior includes: The translation vector formed by the multi-view camera system from the first time step to the second time step is parameterized with depth, and the generalized epipolar constraints of the multi-view camera system are reconstructed based on the depth parameters. By utilizing the prior information on the rotation axis direction provided by the inertial measurement unit and combining it with the reconstructed generalized epipolar constraints, the relative pose estimation problem of the multi-camera system in a multi-camera system is transformed into a single-variable cubic polynomial root-finding problem. After finding the roots of the single variable, the rotation matrix and translation vector of the multi-camera system from the first time step to the second time step are solved based on the root results to obtain the relative pose estimation results.

[0020] Furthermore, according to one embodiment of the present invention, depth parameterization is performed on the translation vector formed by the multi-view camera system from the first time moment to the second time moment, including: Motion platform equipped with multi-view cameras from time to time Exercise 3D points exist Always being photographed It was observed that, in Always being photographed It was observed that the motion platform was in and The coordinate systems for each time point are denoted as follows: and To simplify the derivation, the world coordinate system is used. The origin is set at a three-dimensional point. and its coordinate axis direction is Platform coordinate system at any moment Alignment; Define the transformations between the coordinate systems, including: from the camera coordinate system to its respective motion platform The transformation is From the camera coordinate system to the motion platform The transformation is ,in and Here is the corresponding rotation matrix. and The corresponding translation vector; the motion platform moves from the coordinate system Move to coordinate system The relative pose transformation is ,in For rotation matrix, It is a translation vector; due to the world coordinate system With coordinate system Alignment, with the rotation matrix between them being the identity matrix. Set up a world coordinate system To coordinate system The transformation is From the world coordinate system To coordinate system The transformation is The relationships between the coordinate systems are as follows: Figure 2 As shown.

[0021] Based on the above definition, the following transformation relationship can be obtained: (1) in, Represents the transformation matrix The homogeneous form of it can be obtained from equation (1): (2) In the generalized camera model, the matching features corresponding to the 3D point P It can be viewed as two Plück lines and The intersection point; among which, Represents the unit direction vector, by Give, Represents a moment vector; i, j∈n; according to Plück line geometry, any point on the line can be parameterized as: (3) in, For depth parameters, It is the set of real numbers; due to the three-dimensional points Located at the intersection of two Plück lines, its coordinates can be parameterized along each of the two Plück lines. Let... and Representing three-dimensional points respectively In the coordinate system of the motion platform and coordinate system The coordinates in the matrix are given by: (4) From this we can obtain and Regarding depth parameters and The expression: (5) Equation (5) shows that the translation vector and It can be determined by two depth parameters. and This indicates that the degree of freedom of the problem is significantly reduced, laying the foundation for efficient solution in the future.

[0022] Furthermore, according to one embodiment of the present invention, reconstruction based on the generalized epipolar constraint of a multi-view camera system with depth parameters includes: Pluke's straight line From the coordinate system Transform to coordinate system At that time, the transformed coordinates are obtained. ; (6) Plücke Straight Line and Pluke's straight line In the same coordinate system The necessary and sufficient condition for the intersection of the two sides is: (7) Substituting equation (6) into equation (7), we obtain the generalized epipolar constraint: (8) Substituting further into equation (2), we get: (9) in, This indicates that the vector is transformed into an antisymmetric matrix; finally, substituting equation (5) into equation (9) yields the result based on the depth parameter. and Generalized pole constraint: (10).

[0023] This generalized epipolar constraint equation establishes the intrinsic geometric relationship between image observation, relative pose, and scene depth, laying a theoretical foundation for efficient solution by combining IMU prior information.

[0024] Furthermore, according to one embodiment of the present invention, utilizing the prior information on the rotation axis direction provided by the inertial measurement unit, combined with the reconstructed generalized epipolar constraints, the relative pose estimation problem of the multi-camera system in a multi-camera system is transformed into a single-variable cubic polynomial root-finding problem, including: Using the rotation axis direction measurement data of the motion platform provided by the inertial measurement unit, when the rotation axis direction is known, the rotation is described using the axis angle representation method, and the corresponding rotation matrix satisfies the Rodrigues formula: (11) in, It is a known direction vector of the unit rotation axis. It refers to the magnitude of the rotation angle; for small rotation angles, the first-order Taylor approximation is applied: and At this point, the Rodriguez formula simplifies to: (12) Substituting the approximate expression (11) into the generalized epipolar constraint (10), we get: (13) After simplification, the original nonlinear equation can be linearized as follows: (14) Among them, coefficient to It is calculated directly from the observation data and the known direction of the rotation axis. and For the unknown variables to be solved; Using three sets of point correspondences that are not at the origin, the following homogeneous linear equation system can be constructed: (15) It can be observed that each item in the matrix is The linear expression of equation (15) is expressed in matrix form: (16) The condition for the existence of a non-zero solution to the system of equations (16) is that the determinant of its coefficient matrix is ​​zero: (17) This yields a value regarding the rotation angle. The cubic polynomial equation system (16) has at most three real roots.

[0025] In real-world autonomous driving scenarios, the motion of the platform is continuous and smooth, resulting in relatively small rotation angles between adjacent frames. This invention fully leverages this characteristic, combining prior information on the rotation axis direction provided by the IMU to effectively simplify the pose estimation problem.

[0026] Further, according to one embodiment of the present invention, after univariate root finding, the rotation matrix and translation vector of the multi-camera system from the first time moment to the second time moment are solved based on the root finding results to obtain the relative pose estimation result, including: Calculate the rotation angle Then, rotate the angle. and the known rotation axis direction vector Substituting into equation (11), the rotation matrix is ​​reconstructed. ; Calculate the rotation angle Then, substitute it into the matrix defined by equation (16). In the solution, the depth parameter can be obtained by taking the right singular vector with the minimum singular value through singular value decomposition. and , depth parameters and Substituting into equation (5), the translation vector is calculated. and Finally, the translation vector is recovered using the following formula. : (18).

[0027] According to the above-described scheme of this invention, the invention innovatively integrates the prior information on the rotation axis direction provided by the inertial measurement unit (IMU) and combines it with the first-order approximation characteristic of small rotation angles between frames, simplifying the relative pose estimation problem of multi-view cameras into a solution process requiring only four sets of matching points. Furthermore, this problem is transformed into a root-finding problem of a single-variable cubic polynomial, which significantly reduces computational complexity compared to the common eighth-order polynomial solution path in existing methods. Thanks to the simplification of the number of matching points and the improvement in computational efficiency, the method of this invention can significantly reduce the number of iterations when integrated into the RANSAC framework, thus making it highly suitable for applications with high real-time requirements such as autonomous driving.

[0028] This invention introduces an inertial measurement unit (IMU) and integrates it with a vision system as a feasible approach. While cameras provide rich visual information, their measurements are susceptible to interference from factors such as occlusion and changes in lighting. In contrast, the IMU can independently output high-frequency angular velocity and acceleration, achieving short-time, high-precision motion tracking; the two are naturally complementary. Furthermore, autonomous driving scenarios offer unique conditions for algorithm simplification: due to the smooth motion of the vehicle and the high frame rate of the camera, inter-frame rotation is typically small, making first-order rotation approximation possible, thus significantly simplifying the pose estimation problem.

[0029] Furthermore, to achieve the above objectives, the present invention also provides a multi-view camera relative pose estimation system based on IMU rotation axis direction prior, comprising: The constraint reconstruction module performs depth parameterization on the translation vector formed by the multi-camera system from the first time step to the second time step, and reconstructs the generalized epipolar constraints of the multi-camera system based on the depth parameters. The problem transformation module utilizes the prior information on the rotation axis direction provided by the inertial measurement unit and combines it with the reconstructed generalized epipolar constraints to transform the relative pose estimation problem of the multi-camera system into a single-variable cubic polynomial root-finding problem. The relative pose estimation result acquisition module, after univariate root calculation, solves the rotation matrix and translation vector of the multi-camera system from the first time moment to the second time moment based on the root calculation result, and obtains the relative pose estimation result.

[0030] The multi-camera relative pose estimation system based on IMU rotation axis direction prior according to the present invention can realize the multi-camera relative pose estimation method based on IMU rotation axis direction prior. The specific process steps are as described above and will not be repeated here.

[0031] Furthermore, to achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the multi-camera relative pose estimation method based on the IMU rotation axis direction prior as described above.

[0032] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-camera relative pose estimation method based on IMU rotation axis direction prior as described above.

[0033] This invention proposes a highly efficient minimum solver for multi-camera relative pose estimation, specifically for applications with extremely high real-time requirements, such as autonomous driving. This solver fully utilizes prior information on the rotation axis direction provided by the IMU, simplifying the relative pose estimation problem of multi-camera systems to a problem solvable with only four sets of feature matching points. By innovatively introducing a translation vector modeling method based on depth parameterization, combined with the first-order approximation characteristic of small inter-frame rotation angles, the originally high-degree-of-freedom pose estimation problem is transformed into a root-finding problem of a single-variable cubic polynomial, significantly reducing computational complexity. Simulation results show that, compared with current mainstream algorithms, the method of this invention significantly improves computational efficiency while maintaining competitive estimation accuracy. This method provides an efficient and practical technical solution for the implementation of multi-camera visual localization in autonomous driving systems, possessing strong engineering application value and promising prospects for widespread application.

[0034] Those skilled in the art will recognize that the modules and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0035] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and equipment can be referred to the corresponding process in the foregoing method implementation, and will not be repeated here.

[0036] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0037] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs.

[0038] In addition, the functional modules in the embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0039] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the sending / receiving methods of various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0040] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

[0041] It should be understood that the sequence number of each step in the invention and its embodiments does not absolutely imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Claims

1. A multi-camera relative pose estimation method based on IMU rotation axis prior, characterized in that, include: The translation vector formed by the multi-view camera system from the first time step to the second time step is parameterized with depth, and the generalized epipolar constraints of the multi-view camera system are reconstructed based on the depth parameters. By utilizing the prior information on the rotation axis direction provided by the inertial measurement unit and combining it with the reconstructed generalized epipolar constraints, the relative pose estimation problem of the multi-camera system in a multi-camera system is transformed into a single-variable cubic polynomial root-finding problem. After univariate root finding, the rotation matrix and translation vector of the multi-camera system from the first time step to the second time step are solved based on the root finding results to obtain the relative pose estimation results; The depth parameterization of the translation vector formed by the multi-view camera system from the first time point to the second time point includes: Motion platform equipped with multi-view cameras from time to time Exercise 3D point P in Always being photographed It was observed that, in Always being photographed It was observed that the motion platform was in and The coordinate systems for each time point are denoted as follows: and To simplify the derivation, the world coordinate system is used. The origin is set at point P in three dimensions, and its coordinate axes are aligned with... Platform coordinate system at any moment Alignment; Define the transformations between the coordinate systems, including: from the camera coordinate system to its respective motion platform The transformation is From the camera coordinate system to the motion platform The transformation is ,in and Each is a corresponding camera and camera The rotation matrix, and Each is a corresponding camera and camera Translation vector; the motion platform from the coordinate system Move to coordinate system The relative pose transformation is Where R is the rotation matrix and t is the translation vector; due to the world coordinate system With coordinate system Alignment, with the rotation matrix between them being the identity matrix. Let the world coordinate system W be the coordinate system. The transformation is From world coordinate system W to coordinate system The transformation is ; Based on the above definition, the following transformation relationship can be obtained: （1） in, Represents the transformation matrix The homogeneous form of it can be obtained from equation (1): （2） In the generalized camera model, the matching features corresponding to the 3D point P It can be viewed as two Plück lines and The intersection point; among which, Represents the unit direction vector, by Give, Represents a moment vector; According to Plücke line geometry, any point on a straight line can be parameterized as: （3） in, For depth parameters, Let be the set of real numbers; since the three-dimensional point P lies at the intersection of two Plück lines, its coordinates can be parameterized along the two Plück lines respectively. Let be... and These represent the coordinate systems of the three-dimensional point P on the motion platform. and coordinate system The coordinates in the matrix are given by: （4） From this we can obtain and Regarding depth parameters and The expression: （5） Equation (5) shows that the translation vector and It can be determined by two depth parameters. and express.

2. The multi-camera relative pose estimation method based on IMU rotation axis direction prior as described in claim 1, characterized in that, The reconstruction based on the generalized epipolar constraints of a multi-view camera system with depth parameters includes: Pluke's straight line From the coordinate system Transform to coordinate system At that time, the transformed coordinates are obtained. ; （6） Plücke Straight Line and Pluke's straight line In the same coordinate system The necessary and sufficient condition for the intersection of the two sides is: （7） Substituting equation (6) into equation (7), we obtain the generalized epipolar constraint: （8） Substituting further into equation (2), we get: （9） in, This indicates that the vector is transformed into an antisymmetric matrix; finally, substituting equation (5) into equation (9) yields the result based on the depth parameter. and Generalized pole constraint: （10）。 3. The multi-camera relative pose estimation method based on IMU rotation axis direction prior as described in claim 2, characterized in that, The method utilizes prior information on the rotation axis direction provided by the inertial measurement unit, combined with the reconstructed generalized epipolar constraints, to transform the relative pose estimation problem of multi-camera systems into a single-variable cubic polynomial root-finding problem, including: Using the rotation axis direction measurement data of the motion platform provided by the inertial measurement unit, when the rotation axis direction is known, the rotation is described using the axis angle representation method, and the corresponding rotation matrix satisfies the Rodrigues formula: （11） Where n is the known direction vector of the unit rotation axis, and w is the magnitude of the rotation angle; for the case of small rotation angles, the first-order Taylor approximation is applied: and At this point, the Rodriguez formula simplifies to: （12） Substituting the approximate expression (11) into the generalized epipolar constraint (10), we get: （13） After simplification, the original nonlinear equation can be linearized as follows: （14） Among them, coefficient to It is calculated directly from the observation data and the known direction of the rotation axis. and For the unknown variables to be solved; Using three sets of point correspondences that are not at the origin, the following homogeneous linear equation system can be constructed: （15） Equation (15) can be expressed in matrix form: （16） The condition for the existence of a non-zero solution to the system of equations (16) is that the determinant of its coefficient matrix is ​​zero: （17） This yields a cubic polynomial equation for the rotation angle w, and the system of equations (16) has at most three real roots.

4. The multi-camera relative pose estimation method based on IMU rotation axis direction prior as described in claim 3, characterized in that, After the univariate root-finding, the rotation matrix and translation vector of the multi-camera system from the first time moment to the second time moment are solved based on the root-finding results to obtain the relative pose estimation results, including: After obtaining the rotation angle w, substitute the rotation angle w and the known rotation axis direction vector n into equation (11) to reconstruct the rotation matrix R; After obtaining the rotation angle w, substitute it into the matrix defined by equation (16). In the solution, the depth parameter can be obtained by taking the right singular vector with the minimum singular value through singular value decomposition. and , depth parameters and Substituting into equation (5), the translation vector is calculated. and Finally, the translation vector is recovered using the following formula. : （18）。 5. A multi-camera relative pose estimation system based on IMU rotation axis orientation prior, implementing the multi-camera relative pose estimation method based on IMU rotation axis orientation prior as described in any one of claims 1-4, characterized in that, include: The constraint reconstruction module performs depth parameterization on the translation vector formed by the multi-camera system from the first time step to the second time step, and reconstructs the generalized epipolar constraints of the multi-camera system based on the depth parameters. The problem transformation module utilizes the prior information on the rotation axis direction provided by the inertial measurement unit and combines it with the reconstructed generalized epipolar constraints to transform the relative pose estimation problem of the multi-camera system into a single-variable cubic polynomial root-finding problem. The relative pose estimation result acquisition module, after univariate root calculation, solves the rotation matrix and translation vector of the multi-camera system from the first time moment to the second time moment based on the root calculation result, and obtains the relative pose estimation result.

6. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the multi-camera relative pose estimation method based on IMU rotation axis orientation prior as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the multi-camera relative pose estimation method based on IMU rotation axis direction prior as described in any one of claims 1-4.

Images