Relocation result screening method and hardware based on multi-view geometric and spatial distribution consistency
By integrating a multidimensional screening model that combines multi-view geometry and spatial distribution consistency, the problem of erroneous result screening in multi-view vision systems is solved, achieving efficient and accurate relocation result screening. It is suitable for resource-constrained or real-time applications in multi-view vision systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-03-31
AI Technical Summary
Existing technologies lack effective methods for uniformly measuring the spatial distribution and geometric consistency of multi-view observation data in multi-view vision systems, resulting in insufficient ability to filter out erroneous results, especially in complex environments where it is difficult to select the globally correct solution.
A multidimensional screening model is adopted to integrate multi-view geometric and spatial distribution consistency features. Candidate poses are scored by local density features, global clustering features, reprojection consistency features and epipolar constraint consistency features to select the optimal pose.
It significantly improves the robustness and accuracy of relocation results, can quickly locate the correct region in complex environments, is suitable for resource-constrained or real-time systems, and balances high accuracy with high real-time performance.
Smart Images
Figure CN121767752A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of general image data processing or generation, and in particular to a method and hardware for screening relocation results based on the consistency of multi-view geometry and spatial distribution. Background Technology
[0002] In the field of visual relocalization technology, systems often generate multiple candidate poses with similar confidence levels but different spatial locations through methods such as image feature matching and point cloud registration. This phenomenon is particularly pronounced when the environment has repetitive textures, dynamic occlusion, or drastic lighting changes, making single confidence ranking unreliable and prone to selecting a locally optimal solution rather than a globally correct one.
[0003] In multi-view vision systems, such as binoculars, fisheye binoculars, or multi-camera arrays, the above problems become even more complex.
[0004] Existing solutions to this problem have certain limitations. Traditional screening methods typically rely solely on matching scores or confidence levels generated internally by the algorithm. However, these scores cannot effectively characterize the overall geometric plausibility of a candidate pose across all camera views in a multi-view system, and lack the ability to discern erroneous results hidden in high scores from a single view. Existing solutions fail to systematically introduce and quantify the inherent strong geometric constraints of multi-view systems (such as cross-view reprojection consistency and epipolar constraints) as core evaluation metrics, and lack the ability to cross-validate using multi-view observation data, resulting in insufficient immunity to erroneous results. When candidate results originate from different sensors, algorithms, or fusion nodes, they form a set with varying characteristics and quality. Traditional methods lack a standardized framework that can uniformly measure the credibility of such heterogeneous results in terms of spatial distribution and geometric consistency. Although some complex global optimization or dense validation methods can improve accuracy, their enormous computational overhead makes it difficult to meet the online requirements of latency-sensitive applications such as real-time robotics and augmented reality. Summary of the Invention
[0005] This invention solves the problems existing in the prior art and provides a method and hardware for screening relocation results based on the consistency of multi-view geometry and spatial distribution, so as to achieve accurate and robust screening of correct relocation results.
[0006] The technical solution adopted in this invention is a relocalization result screening method based on the consistency of multi-view geometry and spatial distribution. The method obtains multiple candidate results representing the pose of a multi-view vision system generated by a relocalization algorithm; scores all the candidate results using a multi-dimensional screening model that integrates the consistency of multi-view geometry and spatial distribution, and selects the optimal pose.
[0007] Preferably, the multidimensional screening model integrates aggregated confidence features and geometric fit features; the aggregated confidence features are obtained by evaluating each candidate result based on spatial distribution statistics, and the geometric fit features are obtained by evaluating each candidate result based on the geometric constraints of the multi-view vision system.
[0008] Preferably, the clustering reliability features based on spatial distribution statistics include local density features and global clustering features; the local density features are associated with the distance between each candidate result and its K nearest neighbors; for clustering all candidate results, the global clustering features are associated with the relationship between each candidate result and its cluster center.
[0009] Preferably, the geometric fit features based on the geometric constraints of the multi-view vision system include reprojection consistency features and epipolar constraint consistency features; the poses of the candidate results are transformed to the coordinate systems of each camera in the multi-view system, and the reprojection consistency features are obtained based on the reprojection error of the 3D map points on each camera image; for the pose of each candidate result, the matching feature point pairs between different camera views in the multi-view system are obtained, and the epipolar constraint consistency features are statistically obtained based on the error of each matching point pair satisfying the epipolar geometric constraints.
[0010] Preferably, the multidimensional screening model uses a weighted summation to fuse aggregated credible features and geometric fit features, wherein the weight coefficient corresponding to the geometric fit feature is not less than the weight coefficient of the aggregated credible feature.
[0011] Preferably, the weighting coefficients are adjusted based on one or more of the following: texture richness of the current environment, proportion of dynamic objects, or lighting conditions.
[0012] Preferably, the multidimensional screening model is updated based on the statistical relocation results and the contribution of different features to the correctness of the results in historical relocation tasks.
[0013] A relocalization result filtering system for a multi-view vision system that implements the relocalization result filtering method based on the consistency of multi-view geometry and spatial distribution, comprising:
[0014] The data acquisition module is used to acquire multiple candidate relocation results;
[0015] The spatial distribution analysis module is used to obtain the clustering credibility features of candidate results;
[0016] The multi-view geometric verification module is used to obtain the geometric fit features of candidate results;
[0017] The feature fusion and decision module is used to fuse reliable features and geometrically consistent features and output the screening results.
[0018] A multi-view vision positioning device, comprising:
[0019] A multi-view camera array for acquiring environmental images;
[0020] A processor; and
[0021] A memory that stores computer programs;
[0022] When the computer program is executed by the processor, the device performs the relocation result filtering method based on multi-view geometry and spatial distribution consistency.
[0023] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for filtering relocation results based on the consistency of multi-view geometry and spatial distribution.
[0024] This invention relates to a method and hardware for screening relocalization results based on the consistency of multi-view geometry and spatial distribution. The method acquires multiple candidate results representing the pose of a multi-view vision system, generated by a relocalization algorithm. All candidate results are scored using a multi-dimensional screening model that integrates the consistency of multi-view geometry and spatial distribution to select the optimal pose. The system includes a data acquisition module, a spatial distribution analysis module, a multi-view geometry verification module, and a feature fusion and decision module. The multi-view vision localization device includes a multi-view camera array, a processor, and a memory, with a computer program executing the method in the memory. A medium is also provided to implement the method.
[0025] The beneficial effects of this invention are as follows:
[0026] (1) Treat the candidate result set as a whole for group analysis. By mining its spatial distribution pattern, the result that best conforms to statistical consistency can be identified, which significantly reduces the risk of misselection of a single erroneous result and greatly enhances the overall robustness of the system in complex environments.
[0027] (2) Breaking through the traditional paradigm of relying on a single metric, it deeply integrates the spatial distribution statistical characteristics with the multi-view geometric consistency characteristics. The former efficiently reveals the aggregation pattern of candidate results in space and quickly locks the potential correct area, while the latter uses the inherent strong geometric constraints of multi-view to conduct rigorous physical verification of candidate poses, thereby forming cross-validation, improving the interpretability and credibility of the screening process, achieving accurate screening effect, and finally the output pose has higher consistency guarantee in both statistical meaning and physical geometry.
[0028] (3) By using rapid spatial analysis in the early stage to narrow down the scope of fine geometric verification, and giving higher weight to geometric features in the fusion stage, the accuracy of the method at the core is ensured. The method can run stably on resource-constrained embedded platforms or real-time systems that require high-frequency positioning, effectively balancing high precision and high real-time performance. Attached Figure Description
[0029] Figure 1 This is a flowchart of the method of the present invention;
[0030] Figure 2 This is a schematic block diagram of the system structure in this invention. Detailed Implementation
[0031] The present invention will be further described in detail below with reference to embodiments, but the scope of protection of the present invention is not limited thereto.
[0032] This invention relates to a method for filtering relocalization results based on the consistency of multi-view geometry and spatial distribution. The method obtains multiple candidate results representing the pose of a multi-view vision system generated by a relocalization algorithm; scores all the candidate results using a multi-dimensional filtering model that integrates the consistency of multi-view geometry and spatial distribution, and selects the optimal pose.
[0033] In this invention, N candidate relocalization results generated by the multi-view relocalization algorithm are first obtained. Each result represents the pose of the multi-view system (such as the main camera or the system center) in three-dimensional space, including position coordinates (x, y, z) and rotational attitude (such as quaternion).
[0034] The multidimensional screening model integrates aggregated confidence features and geometric fit features; the aggregated confidence features are obtained by evaluating each candidate result based on spatial distribution statistics, and the geometric fit features are obtained by evaluating each candidate result based on the geometric constraints of the multi-view vision system.
[0035] Clustering reliability features based on spatial distribution statistics include local density features. and global clustering features The local density feature is associated with the distance between each candidate result and its K nearest neighbors; for clustering all candidate results, the global clustering feature is associated with the relationship between each candidate result and its cluster center;
[0036] Specifically, local density features Used to quantify the degree of clustering of each candidate result within its local neighborhood; for the candidate result set Each of its candidate results A SE(3) pose can be associated. The Euclidean distance between the defined results satisfies the following condition:
[0037]
[0038] For each candidate result Calculate the average distance to its K nearest neighbors.
[0039]
[0040] in, for The set of K nearest neighbors;
[0041] By using the max-min normalization method, the average distance Transformed into local density features ,
[0042]
[0043] It directly reflects local density; a higher value indicates a denser region. Correct results are usually clustered in high-density areas.
[0044] Global clustering features This is used to evaluate the structural position of each candidate result in the global distribution. Cluster analysis is performed on the candidate result set (e.g., using the density-based DBSCAN algorithm). Candidate results belonging to a specific cluster after clustering are then... Calculate its distance to the cluster center. Similarly, the centrality score is obtained using the max-min normalization method. ,
[0045]
[0046] The score reflects the degree of deviation of the result from the center of its cluster. The higher the score, the closer the result is to its cluster center, the more representative it is in the global distribution, and usually the more reliable it is.
[0047] Geometric fit features based on geometric constraints of multi-view vision systems include reprojection consistency features. Consistency characteristics with polar constraints The poses of the candidate results are transformed to the coordinate systems of each camera in the multi-view system, and reprojection consistency features are obtained based on the reprojection errors of the 3D map points on each camera image. For the pose of each candidate result, obtain its matching feature point pairs across different camera views in the multi-view system. Based on the error of each matching point pair satisfying the epipolar geometric constraints, statistically obtain the epipolar constraint consistency features. .
[0048] Specifically, reprojection consistency features Used to quantify the overall fit between candidate poses and multi-view observation data. For each candidate pose... This is then transformed to the coordinate system of each camera in the multi-view system. For known 3D map points... The reprojection error on each camera image plane c is calculated, defined as the two-dimensional pixel coordinates of the map point after pose transformation and camera projection, compared with the pixel coordinates observed in the actual image. The Euclidean distance between them
[0049]
[0050] in, Let be the projection function of camera c;
[0051] For each candidate pose For a single camera view c, calculate the average reprojection error of all visible 3D points.
[0052]
[0053] in, In position The number of 3D points visible in view c;
[0054] By performing comprehensive statistical processing on the single-view errors of all camera views, the reprojection consistency characteristics of the candidate pose are obtained. The comprehensive statistical processing includes, but is not limited to, taking the mean, median, or truncated mean.
[0055] Reprojection consistency features It directly measures the geometric rationality of pose interpretation of multi-view observation data, reflects the degree of fit between candidate poses and multi-view observation data, and the lower the error, the higher the consistency.
[0056] Polar constraint consistency feature Cross-validation is performed using epipolar geometry relationships between different views in a multiview system. For each candidate pose... Based on the derived relative poses between the multi-camera systems, the fundamental matrix is calculated. For a set of feature point pairs that are successfully matched between the left and right views (or any two views) and (corresponding to the same three-dimensional point) ), calculate its epipolar error, which characterizes whether the matching point pair satisfies the epipolar geometric constraint. The degree of accuracy; in this embodiment, a widely used first-order geometric approximation is used for measurement.
[0057]
[0058] By statistically averaging the epipolar errors of all matching point pairs under the candidate pose, the epipolar constraint consistency feature is obtained. Polar constraint consistency characteristics The geometric consistency reflecting the correspondence between candidate poses and images across multiple views provides strong cross-validation.
[0059] When the system performs relocalization, numerous feature points have been detected and described from the left and right camera images using feature extraction algorithms (such as SIFT and ORB), and multiple feature point pairs are constructed through feature matching. For each candidate pose, the system processes all successfully matched left and right camera image feature point pairs using the full-frame image data acquired by the multi-camera system at the current moment. For a candidate pose currently being evaluated, based on this candidate pose and the known calibration parameters of the binocular cameras (such as the relative positional relationship between them), the position of the epipolar line corresponding to a point on the right camera image plane at this moment can be calculated on the left camera image plane. Based on the candidate pose and epipolar geometry, the epipolar line corresponding to the right point on the left image can be calculated. Then, the vertical distance from the point actually observed by the left camera to this theoretical epipolar line L is checked, which is the epipolar error of the point pair. When the candidate pose is completely correct and the feature matching is error-free, this distance should approach zero. For all matching values, a feature score is obtained from the batch error; this feature ensures high accuracy in relocalization screening.
[0060] The multidimensional screening model uses a weighted summation to fuse aggregated credible features and geometric fit features, where the weight coefficients of the geometric fit features are not less than the weight coefficients of the aggregated credible features.
[0061] Calculate the overall consistency score for each candidate result i Using the weighted summation method,
[0062]
[0063] in, These are weighting coefficients, and are typically assigned multi-view geometric features. A higher weight is given to highlight the importance of multi-view constraints.
[0064] Based on the overall consistency score Sort and filter, then output based on a strategy, including but not limited to:
[0065] Direct output, that is, selecting the candidate result with the highest score as the final output;
[0066] Fine-grained verification involves selecting the Top-K results and feeding them into a more precise pose optimizer for final decision-making.
[0067] The final output is the optimal relocation result of the multi-view system after filtering.
[0068] The weighting coefficients are adjusted based on one or more of the following factors: texture richness of the current environment, proportion of dynamic objects, or lighting conditions.
[0069] The statistical relocation results are used to update the multidimensional screening model based on the contribution of different features to the correctness of the results in historical relocation tasks.
[0070] In the actual fusion process, it is generally necessary to set up an adaptive fusion decision module, which dynamically fuses the spatial aggregation credibility feature and the multi-view geometric fit feature based on the current scene attributes or historical performance data, and generates a comprehensive consistency score for each pose.
[0071] By upgrading fixed weighting coefficients to dynamic adaptive adjustment or online learning updates, the system can adapt to different scenarios, making the screening more intelligent and achieving dynamic optimization.
[0072] This invention also relates to a relocalization result filtering system for a multi-view vision system that implements the aforementioned relocalization result filtering method based on the consistency of multi-view geometry and spatial distribution, comprising:
[0073] The data acquisition module is used to obtain multiple candidate relocation results, which are N in this case.
[0074] The spatial distribution analysis module is used to obtain the clustering credibility features of candidate results, i.e., to calculate... , ;
[0075] The multi-view geometric verification module is used to obtain the geometric fit features of candidate results, i.e., to calculate... , ;
[0076] The feature fusion and decision module is used to fuse reliable features and geometrically consistent features and output the screening results.
[0077] The present invention also relates to a multi-view vision positioning device, comprising:
[0078] A multi-view camera array for acquiring environmental images;
[0079] A processor; and
[0080] A memory that stores computer programs;
[0081] When the computer program is executed by the processor, the device performs the relocation result filtering method based on multi-view geometry and spatial distribution consistency.
[0082] The present invention also relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for filtering relocation results based on the consistency of multi-view geometry and spatial distribution.
[0083] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0084] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0087] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0088] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for filtering relocation results based on the consistency of multi-view geometry and spatial distribution, characterized in that: The method obtains multiple candidate results representing poses of the multi-view vision system generated by a relocalization algorithm; scores all the candidate results with a multi-dimensional screening model fusing multi-view geometry and spatial distribution consistency, and screens an optimal pose. 2.The method of claim 1, wherein: The multi-dimensional screening model fuses aggregated reliable features and geometric consistent features; the aggregated reliable features are obtained based on spatial distribution statistics for each candidate result, and the geometric consistent features are obtained based on geometric constraints of the multi-view vision system for each candidate result. 3.The method of claim 2, wherein: The aggregated reliable features based on spatial distribution statistics include local density features and global clustering features; The local density features are associated with distances of each candidate result and its K nearest neighbors; the global clustering features are associated with relationships of each candidate result and its cluster center.
4. The method of claim 2, wherein the method further comprises: The geometric consistent features based on geometric constraints of the multi-view vision system include re-projection consistency features and epipolar constraint consistency features; The re-projection consistency features are obtained based on re-projection errors of three-dimensional map points on images of each camera of the multi-view system; For each candidate result, matched feature point pairs between different camera views of the multi-view system are obtained, and the epipolar constraint consistency features are obtained based on errors of each matched point pair satisfying epipolar geometric constraints.
5. The method of claim 2, wherein the method further comprises: The multi-dimensional screening model fuses the aggregated reliable features and the geometric consistent features by weighted summation, and a weight coefficient corresponding to the geometric consistent features is not less than a weight coefficient of the aggregated reliable features.
6. The method of claim 5, wherein the method further comprises: The weight coefficients are adjusted based on one or more of texture richness, dynamic object proportion, or illumination conditions of the current environment.
7. The method of claim 1, wherein the method further comprises: The relocalization results are statistically analyzed, and the multi-dimensional screening model is updated based on contributions of different features in historical relocalization tasks to correctness of the results.
8. A relocalization result screening system of a multi-view vision system implementing the relocalization result screening method based on multi-view geometry and spatial distribution consistency according to any one of claims 1-7. The method comprises: a data acquisition module configured to acquire multiple candidate relocalization results; a spatial distribution analysis module configured to obtain aggregated reliable features of the candidate results; a multi-view geometry verification module configured to obtain geometric consistent features of the candidate results; a feature fusion and decision module configured to fuse the reliable features and the geometric consistent features and output screening results.
9. A multi-camera visual positioning device, characterized by: The method comprises: a multi-view camera array configured to capture environment images; a processor; and a memory storing a computer program; when the computer program is executed by the processor, the device is caused to perform the relocalization result screening method based on multi-view geometry and spatial distribution consistency according to any one of claims 1-7.
10. A computer readable storage medium having stored thereon a computer program, characterized in that: The program is executed by the processor to implement the relocalization result screening method based on multi-view geometry and spatial distribution consistency according to any one of claims 1-7.