Method for spatial positioning of intersecting optical axis multi-fish-eye camera

Abstract

The application discloses a kind of intersecting optical axis multi-fish-eye camera space positioning method, utilizes the target position measurement and corresponding error obtained by fish-eye camera, combines multi-sensor fusion technology, and the position optimal estimation based on world coordinate system is carried out to the spatial target point appearing in multi-camera image and the corresponding measurement error is generated.Meanwhile, by comparing the size of actual error and theoretical error, multi-target matching and error matching avoidance can be carried out in the form of hypothesis testing.

Description

Technical Field

[0001] This invention belongs to the field of fisheye camera technology, specifically relating to a spatial positioning method for multiple fisheye cameras with intersecting optical axes. Background Technology

[0002] Currently, algorithms for obtaining the spatial position of target points using fisheye cameras are mainly divided into two categories: The first category uses deep learning and other algorithms with a single fisheye camera to estimate the spatial position of target points in a single image by using methods such as contour, occlusion, color gradient, and position in the image. The disadvantages are poor interpretability, unreliability of the algorithm theory, and great susceptibility to environmental influences. The second category uses a dual fisheye camera optical system arranged in the form of parallel optical axes. The disadvantage of this optical system is that when measuring the position of any point in space, it is necessary to perform spatial rotation of the optical system, and it cannot perform position estimation for any space under any pose, which has an inherent disadvantage.

[0003] In other fisheye camera systems involving non-parallel optical axis arrangements, most projects aim to use a surround-view setup for stitching panoramic images or performing visual SLAM, such as two fisheye cameras for 360-degree fisheye optical construction for equirectangular projection of panoramic images and dual-fisheye omnidirectional stemo; fisheye cameras and wide-angle cameras for stitching videos from a fisheye lens camera and a wide-angle lens camera for telepresence robots; four fisheye cameras for SLAM (simultaneous localization and mapping) 3D visual perception for self-driving cars using a multi-camera system: Calibration, mapping, localization, and obstacle detection; and four fisheye cameras + deep learning methods for nearby vehicle heading perception: Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras. These projects do not emphasize the importance of angle measurements along intersecting optical axes and do not provide theoretical errors for these measurements. Summary of the Invention

[0004] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing a spatial positioning method for multi-fisheye cameras with intersecting optical axes. This method utilizes the target position measurement and corresponding error obtained by the fisheye camera, combined with multi-sensor fusion technology, to perform optimal position estimation of spatial target points appearing in multi-fisheye camera images based on the world coordinate system and generate corresponding measurement errors. Simultaneously, by comparing the actual error with the theoretical error, multi-target matching and error matching avoidance are performed through hypothesis testing, thereby solving the technical problem of spatial positioning of multi-fisheye cameras with intersecting optical axes.

[0005] The present invention adopts the following technical solution:

[0006] A spatial positioning method for a multi-fisheye camera with intersecting optical axes, characterized by comprising the following steps:

[0007] S1. Simultaneously acquire images using multiple fisheye cameras, perform target detection on the acquired images, and determine the position of the target object in the images obtained by each fisheye camera;

[0008] S2. Based on the position of the target object in the images obtained by each fisheye camera obtained in step S1, obtain the spatial angle measurement and corresponding error of the target object relative to the coordinate system of each fisheye camera.

[0009] S3. Perform coordinate transformation and error propagation on the spatial angle measurement and corresponding error of the target object in each fisheye camera coordinate system obtained in step S2 to obtain the spatial angle measurement and corresponding error of the target object in a unified world coordinate system.

[0010] S4. Perform multi-sensor information fusion on the target spatial angle measurement and error in the unified world coordinate system obtained in step S3 to obtain the fused target spatial angle measurement and corresponding error relative to the origin of the world coordinate system.

[0011] S5. Based on the fused spatial angle measurement of the target object relative to the origin of the world coordinate system and the corresponding error obtained in step S4, perform spatial positioning.

[0012] Specifically, in step S1, there are at least two fisheye cameras, which are arranged in an intersecting optical axis manner.

[0013] Specifically, in step S2, the pixels corresponding to the target object are segmented using an image segmentation method, and the position of the center of the corresponding region in the image is calculated. Then, based on the continuous imaging pattern of the fisheye camera, the discrete feature map of the example feature image is identified, and the spatial angle measurement of the target object relative to each fisheye camera coordinate system is calculated. The radial difference change of the spatial angle near the corresponding target point is used as the corresponding error of the spatial angle measurement.

[0014] Specifically, in step S3, the spatial angle measurement and corresponding error of the target object relative to each fisheye camera coordinate system obtained in step S2 are transformed by coordinate transformation. The optical axis direction of each fisheye camera is rotated to the Z-axis direction of the world coordinate system, and the X-axis direction of each fisheye camera is rotated to the X-axis direction of the world coordinate system. The spatial angle measurement and corresponding error of the corresponding line in the world coordinate system are obtained. Combined with the known optical center position of the fisheye camera, the coordinate system is unified, and the corresponding light ray is described in the world coordinate system.

[0015] Furthermore, during coordinate transformation, angular-radian transformation or quaternions are used to perform three-dimensional rotation of mathematical vectors; when calculating the corresponding measurement error, the rotation matrix during coordinate transformation and the equation during angular-radian transformation are used for error propagation.

[0016] Specifically, in step S4, a multi-sensor fusion approach is used to estimate the optimal spatial point location, as follows:

[0017] Through coordinate transformation and error propagation, multiple measurements and corresponding errors in the fisheye camera coordinate system are transformed to the world coordinate system. Combined with the determined position of the fisheye camera in the world coordinate system, the linear equation measurement and its corresponding error passing through the target point in the world coordinate system are obtained. Given multiple measurements and corresponding errors, the covariance matrix between measurements is analyzed to obtain an optimal estimate of the spatial point position and its fused variance. The optimal estimate is the spatial point position with the highest confidence generated by the sensor data, which is used as the fused spatial angle measurement and corresponding error of the target object relative to the origin of the world coordinate system.

[0018] Furthermore, the covariance matrix is ​​divided into three types: covariance matrix of 0, covariance matrix of non-zero and known, and covariance matrix of unknown. For the case of covariance of 0, a simple convex combination fusion method is adopted, which uses the corresponding errors obtained from each camera to perform a weighted average of the reciprocal parameters of each measurement. For the case of covariance matrix of non-zero and known, the Bar-Shalom-Campo method is adopted. For the case of covariance matrix of unknown, a conservative estimation method is adopted, which performs a worst-case analysis on the corresponding measurements and corresponding errors obtained from each camera based on accuracy and confidence.

[0019] Specifically, in step S5, the three-dimensional Hough transform is used to convert between points and lines, transforming the multi-line collinearity problem into a multi-point collinearity problem; optimization criteria are selected and the kernel function is changed to address different sensor types and different data error distributions.

[0020] Furthermore, by using hypothesis testing, uncertainty is set, and a matching check is performed using measurements and variances from different sources.

[0021] Furthermore, in multi-target matching, the measurement association method is used to cross-apply measurement combinations and combined with visual features to finally match multiple measurements of a spatial point from different fisheye cameras.

[0022] Compared with the prior art, the present invention has at least the following beneficial effects:

[0023] A spatial positioning method using a multi-fisheye camera with intersecting optical axes is proposed. This method utilizes an optical system of multi-fisheye cameras arranged with intersecting optical axes, which improves the visible field of the system, reduces the control requirements on the hardware platform on which the system is located, and makes full use of sensor errors during camera imaging. Through tools such as coordinate transformation, quaternion tools, multi-source information fusion algorithms, and three-dimensional Hough transform, the target point position is optimally estimated based on the system error. Furthermore, a series of related measurement association and multi-target matching functions have been developed, which improves the application prospects of the system.

[0024] Furthermore, in step S1, this optical system is inspired by the visual system of fish and arranges fisheye cameras with intersecting optical axes. Its advantage is that the visible field between the cameras is wide and the system can locate the sphere in space without relying on the rotation of the system itself, which reduces the difficulty of dynamic control of the system.

[0025] Furthermore, in step S2, the system first performs image processing on each fisheye camera individually. This aims to clarify the logic from local to global perspectives and provide a feasible interface for future distributed spatial positioning vision systems composed of different visual sensors with a wider spectrum. Specifically, the method of selecting the centroid of the joint region in image segmentation reduces the error in describing the target object's position in the camera coordinate system, making the algorithm more stable.

[0026] Furthermore, in step S3, the system unifies the coordinates of the information in the local fisheye camera coordinate systems, thereby completing the data preprocessing work and providing standardized data for the next step of data fusion.

[0027] Furthermore, in the process of considering camera errors in this system, spatial angle measurements will be cross-transferred through coordinate transformation formulas, quaternion rotation formulas, etc., and the corresponding errors will be transmitted through the partial differential form of the formulas, which are quite different from each other.

[0028] Furthermore, in step S4, the measurement and corresponding error values ​​of each fisheye camera obtained after standardization in the same world coordinate system are used to perform measurement information fusion for the error, thereby obtaining the measurement and corresponding error of the target point-optical center line based on the system error.

[0029] Furthermore, considering that the hardware components of this optical system are often concentrated and fixed in the same physical system, and the selection of fisheye cameras is also largely similar, this optical system needs to perform covariance measurement to characterize the degree of mutual influence of errors between various parts of the overall system.

[0030] Furthermore, during spatial positioning, the spatial positioning algorithm of this system mainly performs three-dimensional Hough transform on lines and points, transforming the "multiple lines coinciding at a point" problem into the "multiple points collinear" problem, which improves the universality of the algorithm and the ease of calculation; and allows for the selection of optimization criteria to change the optimization kernel function, thereby dealing with different sensor types and different data error distributions.

[0031] Furthermore, measurement correlation is a method used by this system to eliminate potential measurement errors or excessive measurement errors in practical engineering scenarios. Following the hypothesis testing principle, it first performs hypothesis testing analysis on the target point locations fused from the normal process. By setting confidence levels, it analyzes the presence of measurement errors or excessive measurement errors during the fusion process. If confidence errors exist, it uses the concept of controlled variables to filter measurements, thereby eliminating potential measurement errors or excessive measurement errors. Further, multi-target matching is a solution for multi-target spatial positioning problems. Essentially, it performs unsupervised clustering classification of measurements belonging to different targets and then performs measurement correlation and spatial positioning within these groups. It utilizes cross-validation to minimize the confidence between measurement groups, and the combinations with concentrated confidence within each group are fused to form the corresponding spatial point locations.

[0032] In summary, the present invention has clear steps and strong logic. By utilizing spatial coordinate transformation and error-focused optimization logic, it eliminates the transformation error between steps and makes the optimal estimate of the system error of the visual sensor-camera.

[0033] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0034] Figure 1 A dual fisheye camera system with intersecting optical axes;

[0035] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0038] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0039] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" relationship.

[0040] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0041] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0042] This invention provides a spatial positioning method using multiple fisheye cameras with intersecting optical axes. Utilizing target position measurements and corresponding errors obtained from fisheye cameras, and combining this with multi-sensor fusion technology, the method performs optimal position estimation of spatial target points appearing in multi-camera images based on the world coordinate system, generating corresponding measurement errors. Furthermore, by comparing the actual errors with the theoretical errors, it enables multi-target matching and error matching avoidance through hypothesis testing.

[0043] Please see Figure 2 The present invention provides a spatial positioning method for a multi-fisheye camera with intersecting optical axes, comprising the following steps:

[0044] S1. Simultaneously acquire images using multiple fisheye cameras, perform target detection on the acquired images, and obtain the position of the target object in the images obtained by each fisheye camera;

[0045] Please see Figure 1 The optical system consists of multiple fisheye cameras arranged along intersecting optical axes. Their spatial distribution, number, and angles can be modified according to specific functional requirements. Therefore, this invention uses the smallest unit—an intersecting optical axis dual fisheye camera system—for illustration.

[0046] In setting the physical coordinate system of an optical system, both the world coordinate system and the camera coordinate system are generally involved.

[0047] In the optical system of this invention, for the world coordinate system, the origin of the world coordinate system is established with the intersection of the two optical axes as the point of intersection, the plane determined by the two optical axes as the xy plane, and the direction of its normal vector as the z-axis.

[0048] For the camera coordinate system, the camera center is taken as the origin, the optical axis is taken as the z-axis, and the direction perpendicular to the corresponding optical axis in the xy plane of the world coordinate system is taken as the x-axis. The above x-axis, optical axis and world coordinate system z-axis form a right-handed rectangular coordinate system. The direction of the normal vector of the world coordinate system xy plane is taken as the y-axis, and the camera coordinate system is also a right-handed rectangular coordinate system.

[0049] Therefore, the transformation matrix for different camera coordinate systems in the world coordinate system is relatively simple, improving computational efficiency.

[0050] S2. Utilizing the characteristic of fisheye camera imaging changing with angle, based on the position of the target obtained in step S1 in each image, obtain the spatial angle measurement and corresponding error of the target relative to each fisheye camera coordinate system, that is, the target-optical center line measurement and corresponding error.

[0051] This system employs image segmentation methods based on color filtering and neural networks to segment the pixels corresponding to the target object and calculate the position of the center of the corresponding region in the image. Subsequently, based on the continuous imaging pattern of the fisheye camera, it identifies the discrete feature map of the example feature image and calculates the spatial angle measurement of the target object relative to each fisheye camera coordinate system. The radial difference of the spatial angle near the target point is used as the corresponding error of the spatial angle measurement.

[0052] S3. Perform coordinate transformation and error propagation on the spatial angle measurement and corresponding error of the target object in each fisheye camera coordinate system obtained in step S2 to obtain the target object-optical center line measurement and corresponding error in a unified world coordinate system.

[0053] Furthermore, for multiple fisheye cameras arranged in space, tools such as quaternions / transformation matrices are used to perform equivalent spatial rotation coordinate transformations.

[0054] Generally, a fisheye camera produces a circular image, reflecting the visual content of a hemispherical region in space with the camera's optical axis as its rotation axis. The image coordinate system typically uses the xy-axis of the camera coordinate system as its planar coordinate system direction. A key characteristic is that the distance of a pixel in the image from the center of the circle is a function of the angle (latitude) between that point and the shooting direction / optical axis, while the angle relative to the x-axis of the circle is a function of the direction (longitude) after the point is projected onto the imaging plane. As the angle between the line connecting the object and the camera center and the camera's z-axis increases, the degree of compression / distortion increases. This application's definition of measurement errors for different angles primarily focuses on the different errors in the camera's AD quantization process caused by different degrees of distortion. Finally, by mapping a point in the circular image to a point on the unit circle, the spatial angle measurement of that point relative to the camera coordinate system is obtained. Furthermore, the theoretical error of the spatial slope measurement is obtained through systematic identification of the imaging function R = f(θ) or by using discrete features.

[0055] S4. Perform multi-sensor information fusion on the target spatial angle measurement and error in the unified world coordinate system obtained by each fisheye camera in step S3, and obtain the fused target spatial angle measurement and corresponding error relative to the origin of the world coordinate system.

[0056] In a multi-fisheye system, if multiple multi-fisheye cameras have acquired images of a spatial point, then a multi-sensor fusion approach is used to optimally estimate the spatial point's location, as follows:

[0057] First, the multiple measurements and corresponding errors in the fisheye camera coordinate system are transformed to the world coordinate system through coordinate transformation and error propagation.

[0058] Based on the determined position of the fisheye camera in the world coordinate system, the equation of the straight line passing through the target point in the world coordinate system and its corresponding error are obtained. With multiple measurements and corresponding errors, the covariance between measurements is analyzed and divided into three types: covariance matrix is ​​0, covariance matrix is ​​not 0 and is known, and covariance is unknown.

[0059] For cases where the covariance is 0, a simple convex combination fusion method is adopted, which uses the corresponding errors obtained from each camera to perform a weighted average of the reciprocal parameters of each measurement.

[0060] For cases where the covariance matrix is ​​not zero and is known, the Bar-Shalom-Campo method is used to perform a matrix-weighted average based on the covariance matrix of multiple cameras.

[0061] When the covariance matrix is ​​unknown, a conservative estimation method is adopted. The worst-case analysis is usually performed on the corresponding measurements and corresponding errors obtained by each camera according to the required accuracy and confidence level.

[0062] Finally, an optimal estimate of the spatial point location and its fused variance are obtained, which is smaller than the variance of each sensor before fusion.

[0063] S5. Based on the fused spatial angle measurement of the target object relative to the origin of the world coordinate system and the corresponding error obtained in step S4, perform spatial positioning, measurement association and multi-target matching.

[0064] When using the fused spatial angle measurement of the target object relative to the origin of the world coordinate system and the corresponding error for spatial positioning, the three-dimensional Hough transform is mainly used to transform the point and line parameters. Optimization criteria can be selected, such as WLS optimization, MLE optimization, MAP optimization, MMSE optimization, etc., and the kernel function of the optimization can be changed accordingly to cope with different sensor types and different data error distributions.

[0065] In terms of applications, this invention can be used for measurement correlation and multi-target matching.

[0066] In measurement correlation, hypothesis testing is used to set uncertainty and use measurements and variances from different sources to perform matching checks.

[0067] Measurement correlation is a method used in this system to eliminate potential measurement errors or excessive measurement errors in real-world engineering scenarios. It follows the hypothesis testing principle, first performing hypothesis testing analysis on the target point locations fused from the normal process. Then, by setting confidence levels, it analyzes the presence of measurement errors or excessive measurement errors during the fusion process. If confidence level errors exist, it uses the concept of controlled variables to filter measurements, thereby eliminating potential measurement errors or excessive measurement errors.

[0068] In multi-target matching, measurement association algorithms can be used to cross-reference measurement combinations and combined with visual features to finally match multiple measurements about a spatial point from different fisheye cameras.

[0069] Multi-target matching is a solution proposed by this system for multi-target spatial localization problems. Essentially, it involves unsupervised clustering of measurements belonging to different targets, followed by measurement association and spatial localization within these groups. Utilizing the cross-validation approach, it minimizes the inter-group confidence and merges combinations with high intra-group confidence to determine the location of the corresponding spatial point.

[0070] In summary, this invention presents a spatial localization method using intersecting optical axes and a multi-fisheye camera. The spatial point localization algorithm, with angle as the primary entry point, incorporates a multi-sensor fusion algorithm to obtain the optimal measurement estimate for the spatial point. This estimate achieves an error smaller than the variance of any single measurement before fusion, thus improving estimation accuracy. Furthermore, subsequent algorithms are proposed for measurement correlation and multi-target matching. In application, the intersecting optical axis arrangement concept can be used to mimic the vision of ordinary fish and can also be applied to robots to achieve distance measurement of any point in any space from any orientation.

[0071] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A spatial positioning method for a multi-fisheye camera with intersecting optical axes, characterized in that, Includes the following steps: S1. Simultaneously acquire images using multiple fisheye cameras, perform target detection on the acquired images, and determine the position of the target object in the images obtained by each fisheye camera; S2. Based on the position of the target object in the images obtained by each fisheye camera obtained in step S1, the corresponding pixels of the target object are segmented using an image segmentation method, and the position of the center of the corresponding region in the image is calculated. Then, based on the continuous imaging law of the fisheye camera, the discrete feature map of the identification or example feature image is used to calculate the spatial angle measurement of the target object relative to each fisheye camera coordinate system, and the radial difference change of the spatial angle near the corresponding target point is used as the corresponding error of the corresponding spatial angle measurement. S3. Perform coordinate transformation on the spatial angle measurement and corresponding error of the target object relative to the coordinate systems of each fisheye camera obtained in step S2. Rotate the optical axis direction of each fisheye camera to the Z-axis direction of the world coordinate system, and rotate the X-axis direction of each fisheye camera to the X-axis direction of the world coordinate system. This yields the spatial angle measurement and corresponding error of the line from the optical center of the fisheye camera to the target object in the world coordinate system. Combine this with the known position of the optical center of the fisheye camera to unify the coordinate system. Describe the light ray from the optical center of the fisheye camera to the target object in the world coordinate system. During the coordinate transformation, use angular radian transformation or quaternions to perform three-dimensional rotation of the mathematical vector. When calculating the corresponding error, use the rotation matrix during coordinate transformation and the equation during angular radian transformation to propagate the error. S4. Perform multi-sensor information fusion on the target object spatial angle measurement and error obtained in step S3 under the unified world coordinate system to obtain the fused target object spatial angle measurement and corresponding error relative to the origin of the world coordinate system. Optimal estimation of the spatial point position is then performed using multi-sensor fusion. Specifically: Through coordinate transformation and error propagation, multiple measurements and corresponding errors in the fisheye camera coordinate system are transformed to the world coordinate system. Combined with the determined position of the fisheye camera in the world coordinate system, the linear equation measurement and its corresponding error passing through the target point in the world coordinate system are obtained. Given multiple measurements and corresponding errors, the covariance matrix between measurements is analyzed to obtain an optimal estimate of the spatial point position and its fused variance. The optimal estimate is the spatial point position with the highest confidence generated using sensor data, which is used as the fused target object spatial angle measurement and corresponding error relative to the origin of the world coordinate system. S5. Based on the fused spatial angle measurement of the target object relative to the origin of the world coordinate system and the corresponding error obtained in step S4, perform spatial positioning.

2. The spatial positioning method for a multi-fisheye camera with intersecting optical axes according to claim 1, characterized in that, In step S1, there are at least two fisheye cameras, which are arranged in a manner with intersecting optical axes.

3. The spatial positioning method for a multi-fisheye camera with intersecting optical axes according to claim 1, characterized in that, The covariance matrix is ​​categorized into three types: covariance matrix of 0, covariance matrix of non-zero and known, and covariance matrix of unknown. For the case of covariance matrix of 0, a simple convex combination fusion method is used, which uses the corresponding errors obtained from each camera to perform a weighted average of the reciprocal parameters of each measurement. For the case of covariance matrix of non-zero and known, the Bar-Shalom-Campo method is used. For the case of covariance matrix of unknown, a conservative estimation method is used, which performs a worst-case analysis of the corresponding measurements and corresponding errors obtained from each camera based on accuracy and confidence.

4. The spatial positioning method for a multi-fisheye camera with intersecting optical axes according to claim 1, characterized in that, In step S5, the three-dimensional Hough transform is used to convert between points and lines, transforming the multi-line collinearity problem into a multi-point collinearity problem; optimization criteria are selected and the kernel function is changed to address different sensor types and different data error distributions.

5. The spatial positioning method for a multi-fisheye camera with intersecting optical axes according to claim 4, characterized in that, By using hypothesis testing, uncertainty is set, and a matching check is performed using measurements and variances from different sources.

6. The spatial positioning method for a multi-fisheye camera with intersecting optical axes according to claim 4, characterized in that, In multi-target matching, a measurement association method is used to cross-apply measurement combinations and combined with visual features to finally match multiple measurements of a spatial point from different fisheye cameras.

Images