A 3D measurement system and method based on machine vision

Abstract

The present invention relates to the field of three-dimensional vision technology, and provides a machine vision-based three-dimensional measurement system and method, comprising a three-dimensional measurement platform, a trajectory graph construction module, a translation constraint construction module, a constraint function construction module, and a three-dimensional measurement module. The trajectory graph construction module is used to construct a trajectory graph for a preset three-dimensional scene. The translation constraint construction module is used to construct a first constraint for the target relative translation of a camera node pair, and a second constraint for the target feature trajectory of the camera node and the three-dimensional point. The constraint function construction module is used to construct a first objective function corresponding to the first constraint, and a second objective function corresponding to the second constraint. The three-dimensional measurement module is used to solve the first objective function and the second objective function with the goal of minimizing the sum of the constraint errors of the first and second constraints, thereby obtaining the relative translation of the camera node and the position of the three-dimensional point. The present invention improves the estimation accuracy and robustness in low-parallax scenes.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional vision technology, and in particular to a three-dimensional measurement system and method based on machine vision. Background Art

[0002] In the field of 3D vision, accurately estimating camera poses and generating scene point clouds from image sets are fundamental tasks, with wide applications in areas such as autonomous driving, augmented reality, and neural radiance. Generally speaking, structure-from-motion algorithms have emerged as a common and effective approach to achieving these goals.

[0003] The main approaches to camera pose estimation are incremental. Although they excel in accuracy and robustness against outliers, they are sensitive to changes in the order in which images are registered, potentially leading to error accumulation and drift. Furthermore, repeated nonlinear bundle adjustments significantly impact efficiency, making them unsuitable for large-scale scenes. To address these issues in incremental methods, a global approach has been proposed that registers all cameras by estimating their global rotation and translation from their relative poses. Subsequently, the scene structure is triangulated and optimized. Since only a single bundle adjustment optimization is required, efficiency is significantly improved and a uniform error distribution across all cameras is achieved.

[0004] However, global translation estimation is more difficult than global rotation estimation due to its sensitivity to low-parallax feature matching and scale uncertainty. Methods that rely solely on relative translation are limited to cameras registered in parallel rigid graphs and suffer from degradation problems when the cameras undergo collinear motion. Even if the camera motion trajectories are nearly collinear, slight perturbations in the relative translation can lead to significant changes in the estimated camera position, making it impossible to achieve accurate estimation using relative translation alone. Summary of the Invention

[0005] The present invention provides a three-dimensional measurement system and method based on machine vision, which can improve the estimation accuracy and robustness in low-parallax scenes.

[0006] The technical solution of the present invention to solve the above technical problems is as follows:

[0007] A three-dimensional measurement system based on machine vision, comprising a three-dimensional measurement platform, a trajectory map building module, a translation constraint building module, a constraint function building module and a three-dimensional measurement module;

[0008] The three-dimensional measurement platform is respectively connected to the trajectory graph construction module, the translation constraint construction module, the constraint function construction module and the three-dimensional measurement module, and is used to store and manage the data of each module;

[0009] The trajectory graph construction module is used to construct a trajectory graph in a preset three-dimensional scene; the trajectory graph includes camera nodes, three-dimensional points, edges between camera node pairs, and edges between the camera nodes and the three-dimensional points;

[0010] The translation constraint construction module is used to construct a first constraint on the relative translation of the target of the camera node pair, and to construct a second constraint on the target feature trajectory of the camera node and the three-dimensional point;

[0011] The constraint function construction module is used to construct a first objective function corresponding to the first constraint, and to construct a second objective function corresponding to the second constraint;

[0012] The three-dimensional measurement module is used to solve the first objective function and the second objective function with the goal of minimizing the sum of the constraint errors of the first constraint and the second constraint, and obtain the relative translation of the camera node and the position of the three-dimensional point.

[0013] The present invention also provides a three-dimensional measurement method based on machine vision, which is characterized by comprising:

[0014] Constructing a trajectory graph in a preset three-dimensional scene; the trajectory graph includes camera nodes, three-dimensional points, edges between camera node pairs, and edges between the camera nodes and the three-dimensional points;

[0015] Constructing a first constraint on the relative translation of the target of the camera node pair, and constructing a second constraint on the target feature trajectory of the camera node and the three-dimensional point;

[0016] Constructing a first objective function corresponding to the first constraint, and constructing a second objective function corresponding to the second constraint;

[0017] With the goal of minimizing the sum of constraint errors of the first constraint and the second constraint, the first objective function and the second objective function are solved to obtain the relative translation of the camera node and the position of the three-dimensional point.

[0018] According to the three-dimensional measurement method based on machine vision provided by the present invention, the method further includes:

[0019] Filter out the first target three-dimensional point and the first target camera node pair with feature matching;

[0020] Determining a normal vector of an epipolar plane between the first target 3D point and the first target camera node pair, and determining a weight corresponding to the normal vector based on a parallax angle between the first target 3D point and the first target camera node pair;

[0021] Obtaining a target relative translation of the camera node pair by an iterative reweighted least squares method based on the normal vector and a weight corresponding to the normal vector;

[0022] Among them, the global posture of the camera R i ,t i Describes the rotation matrix R of camera i in the global coordinate system i and the translation vector t i , global pose R j ,t j Describes the rotation matrix R of camera j in the global coordinate system j and the translation vector t j , relative posture R ij ,t ij Describes the rotation matrix R of camera i relative to camera j ij and the translation vector t ij ; The global pose R of the camera i ,t i and relative posture R ij ,t ij Satisfies the following equation:

[0023]

[0024] in, is the rotation matrix R i The bias matrix, is the rotation matrix R j The bias matrix, v ij Represents a relative translation in the global coordinate system.

[0025] According to the machine vision-based three-dimensional measurement method provided by the present invention, the step of screening out a first target three-dimensional point and a first target camera node pair with matching features includes:

[0026] Filter out the first three-dimensional point and the first camera node pair with feature matching;

[0027] Determine the disparity angle between the first three-dimensional point and the first camera node pair for each feature match;

[0028] Based on the parallax angle, a first target three-dimensional point and a first target camera node pair of feature matching is filtered out from the first three-dimensional point and the first camera node pair of feature matching.

[0029] According to the machine vision-based three-dimensional measurement method provided by the present invention, constructing a first objective function corresponding to the first constraint includes:

[0030] A first objective function corresponding to the first constraint is constructed based on the target relative translation of the camera node pair and a cross product of the position difference between two camera nodes in the camera node pair.

[0031] According to the machine vision-based three-dimensional measurement method provided by the present invention, constructing a second objective function corresponding to the second constraint includes:

[0032] A second objective function corresponding to the second constraint is constructed based on the target feature trajectory of the camera node and the three-dimensional point, and the cross product of the position difference between the camera node and the three-dimensional point.

[0033] According to the three-dimensional measurement method based on machine vision provided by the present invention, the target feature trajectory of the camera node and the three-dimensional point is obtained by:

[0034] Filter out the second target three-dimensional point and the second target camera node pair with feature matching;

[0035] Constructing a feature trajectory between the second target three-dimensional point and the second target camera node pair based on a set-finding algorithm;

[0036] sorting all the feature trajectories in descending order based on the maximum viewing angle difference of the feature trajectories between the second target three-dimensional point and the second target camera node pair;

[0037] Target feature trajectories that meet the first requirement are screened out from all the feature trajectories in sequence until the number of times each camera node is covered by all the target feature trajectories reaches the second requirement.

[0038] According to the machine vision-based three-dimensional measurement method provided by the present invention, solving the first objective function and the second objective function to obtain the relative translation of the camera node and the position of the three-dimensional point includes:

[0039] Optimizing the first objective function and the second objective function under the L1 norm to obtain a first optimization result;

[0040] The first optimization result is optimized by using a third objective function based on an angle to obtain the relative translation of the camera node and the position of the three-dimensional point.

[0041] The present invention also provides an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing any of the above-mentioned three-dimensional measurement methods based on machine vision.

[0042] The present invention also provides a non-transitory computer-readable storage medium, characterized in that a computer software program is stored in the storage medium, and when the computer software program is executed by a processor, it implements any of the above-mentioned three-dimensional measurement methods based on machine vision.

[0043] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned three-dimensional measurement methods based on machine vision.

[0044] The present invention has the following beneficial effects: by constructing a trajectory graph for a preset 3D scene, including camera nodes, 3D points, edges between camera node pairs, and edges between camera nodes and 3D points; constructing a first constraint on the target relative translation of the camera node pairs, and constructing a second constraint on the target feature trajectory of the camera nodes and the 3D points; constructing a first objective function corresponding to the first constraint, and constructing a second objective function corresponding to the second constraint; solving the first and second objective functions with the goal of minimizing the sum of the constraint errors of the first and second constraints, and obtaining the relative translation of the camera nodes and the positions of the 3D points. The present invention improves the estimation accuracy and robustness in low-parallax scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the structure of the three-dimensional measurement system based on machine vision provided by the present invention;

[0046] Figure 2 Schematic diagram of the process of the three-dimensional measurement method based on machine vision provided by the present invention;

[0047] Figure 3 This is a schematic diagram of a scenario of the three-dimensional measurement method based on machine vision provided by the present invention;

[0048] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0052] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0053] Reference Figure 1 , Figure 1 This is a structural diagram of a three-dimensional measurement system based on machine vision provided by the present invention. The three-dimensional measurement system based on machine vision includes a three-dimensional measurement platform, a trajectory map construction module, a translation constraint construction module, a constraint function construction module and a three-dimensional measurement module;

[0054] Among them, the three-dimensional measurement center is connected to the trajectory graph construction module, translation constraint construction module, constraint function construction module and three-dimensional measurement module respectively, and is used to store and manage the data of each module.

[0055] In one embodiment, the trajectory graph construction module may construct a trajectory graph in a preset 3D scene, wherein the trajectory graph includes camera nodes, 3D points, edges between camera node pairs, and edges between camera nodes and 3D points.

[0056] In one embodiment, the translation constraint construction module may construct a first constraint on the relative translation of the target of the camera node pair, and a second constraint on the target feature trajectory of the camera node and the three-dimensional point.

[0057] In one embodiment, the constraint function construction module may construct a first objective function corresponding to the first constraint, and construct a second objective function corresponding to the second constraint.

[0058] In one embodiment, the 3D measurement module may solve the first objective function and the second objective function with the goal of minimizing the sum of the constraint errors of the first constraint and the second constraint to obtain the relative translation of the camera node and the position of the 3D point.

[0059] The embodiments of the present invention construct a trajectory graph for a preset 3D scene, including camera nodes, 3D points, edges between pairs of camera nodes, and edges between camera nodes and 3D points; construct a first constraint for the target relative translation of the camera node pairs, and a second constraint for the target feature trajectories of the camera nodes and 3D points; construct a first objective function corresponding to the first constraint, and a second objective function corresponding to the second constraint; and solve the first and second objective functions with the goal of minimizing the sum of the constraint errors of the first and second constraints to obtain the relative translation of the camera nodes and the positions of the 3D points. The present invention improves estimation accuracy and robustness in low-parallax scenes.

[0060] In an alternative embodiment, referring to Figure 2 , Figure 2 : is a flow chart of a three-dimensional measurement method based on machine vision provided by the present invention, the method comprising:

[0061] Step 110: construct a trajectory graph in a preset 3D scene, wherein the trajectory graph includes camera nodes, 3D points, edges between camera node pairs, and edges between camera nodes and 3D points;

[0062] Step 120 , constructing a first constraint on the relative translation of the target of the camera node pair, and constructing a second constraint on the target feature trajectory of the camera node and the three-dimensional point;

[0063] Step 130: constructing a first objective function corresponding to the first constraint, and constructing a second objective function corresponding to the second constraint;

[0064] Step 140 , with the goal of minimizing the sum of the constraint errors of the first constraint and the second constraint, solve the first objective function and the second objective function to obtain the relative translation of the camera node and the position of the three-dimensional point.

[0065] In this embodiment, a scene and trajectory graph G = {V∪P,E v ∪E p}, where V represents a set of cameras, each node represents a camera, P represents a set of 3D points, each node represents a 3D point, and E v represents the set of edges connecting camera nodes, i.e., camera-to-camera relationships, E p Represents the set of edges connecting the camera node and the 3D point node, that is, the characteristic ray from the camera to the 3D point.

[0066] Among them, the global posture of the camera Ri ,t i Describes the rotation matrix R of camera i in the global coordinate system i and the translation vector t i , global pose R j ,t j Describes the rotation matrix R of camera j in the global coordinate system j and the translation vector t j , and the relative posture R ij ,t ij It describes the rotation matrix R of camera i relative to camera j ij and the translation vector t ij Here, the global pose of any other camera can be calculated by combining the relative pose with the global pose of the reference camera. Specifically, the global pose of the camera R i ,t i and relative posture R ij ,t ij Satisfies the following equation:

[0067]

[0068] in, is the rotation matrix R i The bias matrix, is the rotation matrix R j The bias matrix, v ij Represents a relative translation in the global coordinate system.

[0069] In addition, suppose that from the three-dimensional point P k The emitted light generates n projected feature points on n camera planes. For a feature point, its coordinates in the local normalized camera coordinate system are expressed as Where k is the index of the feature trajectory or 3D point, i is the index of the camera, and x is the ki is the horizontal coordinate of the feature point in the local normalized camera coordinate system, y ki is the ordinate of the feature point in the local normalized camera coordinate system.

[0070] Then in the global camera coordinate system, the three-dimensional point P k The relationship between and camera i satisfies:

[0071]

[0072] Among them, T i represents the camera position of camera i, f ki Represents the distance from camera i to 3D point P k The normalized characteristic rays of .

[0073] In this embodiment, not only the relative translation in the trajectory graph is used to provide direct constraints between cameras, but also more reliable feature trajectories are selected to provide constraints between cameras and three-dimensional points.

[0074] Here, in order to improve the accuracy of the relative translation, the coplanar constraint in epipolar geometry is used in this embodiment to re-estimate the relative translation, and the target relative translation is obtained to provide a direct constraint between cameras.

[0075] In addition, the re-estimated target relative translation is used to filter out feature matches that violate coplanarity or cheirality constraints, and a union-find algorithm is used to construct feature trajectories. Because many feature trajectories may contain a high proportion of characteristic ray outliers, this embodiment only selects a portion of the target feature trajectories to improve efficiency and robustness.

[0076] Here, let C be the first constraint from camera to camera, and P be the second constraint from camera to 3D point. Then the first objective function constructed is The second objective function constructed is The relative translation of the camera node and the position of the three-dimensional point can be further solved by the following formula:

[0077]

[0078] Where p represents the optimization norm and ρ(·) represents the robust estimation function.

[0079] The method provided by an embodiment of the present invention constructs a trajectory graph for a preset three-dimensional scene, the trajectory graph including camera nodes, three-dimensional points, edges between camera node pairs, and edges between camera nodes and three-dimensional points; constructs a first constraint on the target relative translation of the camera node pairs, and a second constraint on the target feature trajectory of the camera nodes and the three-dimensional points; constructs a first objective function corresponding to the first constraint, and a second objective function corresponding to the second constraint; and solves the first and second objective functions with the goal of minimizing the sum of the constraint errors of the first and second constraints to obtain the relative translation of the camera nodes and the positions of the three-dimensional points. This method improves estimation accuracy and robustness in low-parallax scenes.

[0080] Based on any of the above embodiments, the method further includes:

[0081] Filter out the first target three-dimensional point and the first target camera node pair with feature matching;

[0082] Determine a normal vector of an epipolar plane between the first target 3D point and the first target camera node pair, and determine a weight corresponding to the normal vector based on a parallax angle between the first target 3D point and the first target camera node pair;

[0083] Based on the normal vector and the weight corresponding to the normal vector, the target relative translation of the camera node pair is obtained by iterative reweighted least squares method.

[0084] It should be noted that when two cameras observe the same 3D point from different angles, the 3D point, its projections on the image planes of the two cameras, and the origins of the two cameras all lie on the same plane. This coplanarity condition is known as the coplanar constraint. In this embodiment, to improve the accuracy of relative translation, the coplanar constraint from epipolar geometry is used for re-estimation.

[0085] In epipolar geometry, the coplanar constraint between the first target 3D point and the first target camera node pair for each feature matching can be expressed as X kj ·(t ij ×R ij X ki )=0. Given the global camera pose, the coplanar constraint can be rewritten as:

[0086]

[0087] Among them, X kj Represents the coordinate X of a feature point in the camera coordinate system kj =(x kj ,y kj ,1), where k is the index of the feature trajectory or 3D point, j is the index of the camera, x kj is the horizontal coordinate of the feature point in the camera coordinate system, y kj is the vertical coordinate of the feature point in the camera coordinate system. kj Represents the distance from camera j to 3D point P k From the above formula, we can see that each relative translation can be re-estimated using the normal vector of the epipolar plane, which is given by f ki ×f kj Due to the inaccuracies of camera intrinsic parameters and global rotation, the normal vector estimated from the eigenray inevitably has some angular errors.

[0088] In this embodiment, the angle error is decomposed along the normal vector direction and the epipolar plane direction. Since the error along the epipolar plane direction does not affect the direction of the normal vector, in this embodiment, only the error in the normal vector direction is considered.

[0089] like Figure 3 As shown, the two characteristic rays f ki ,f kj Triangulate a 3D point P with a parallax angle α k A small angular error θ in the normal direction causes the ki to f' ki By deviating from f'ki Mark a point A on the top and draw a vertical line from point A to f ki , intersecting it at point B. Then, draw another perpendicular line from point B to f kj , intersecting it at point C. Since the angle error θ is along the normal vector direction, line AB is perpendicular to the plane {P k -BC}. Therefore, line P k C is perpendicular to the plane {ABC}, which means that the angle γ between lines AC and BC is equal to the angular error of the normal vector. Since the angle γ is expected to be small, sinγ is approximately equal to tanγ, which satisfies the following formula:

[0090]

[0091] From the above formula, we can see that when the angle error θ is fixed, the angle errors of the normal vector, sinγ and sinα, are inversely proportional. This means that the larger the parallax angle, the more accurate the normal vector. Therefore, in this embodiment, a reasonable weight || f is retained for the normal vector of the epipolar plane of the first target 3D point and the first target camera node pair for each feature match. ki ×f kj ||2=sinα. Then, the IRLS (Iterative Reweighted Least Squares) method is used to estimate the relative translation. Specifically, the corresponding target relative translation is obtained by the following formula:

[0092]

[0093] Among them, v ij is the relative translation vector to be estimated, f ki ,f kj are respectively related to the three-dimensional point P k Corresponding to the two characteristic rays, ρ is a robust estimated kernel function.

[0094] In this embodiment, the Cauchy kernel ρ(ε)=log(β 2 +ε 2 ) is used as the kernel function of robust estimation, which is used to define the weight of each observation, where ε represents the residual of each observation and β represents the error loss bandwidth. Further, combined with the weight function Adjust the weight of each observation.

[0095] Specifically, during the iteration process, according to the current v ij The estimated value and the observed value calculate the residual ε, then adjust the weight according to φ(ε), and re-estimate v ijThis iterative process is repeated to minimize the weighted residual sum of squares until a certain convergence condition is met.

[0096] Furthermore, in some embodiments, screening out the first target three-dimensional point and the first target camera node pair with feature matching includes:

[0097] Filter out the first three-dimensional point and the first camera node pair with feature matching;

[0098] Determine the disparity angle between the first three-dimensional point and the first camera node pair for each feature match;

[0099] Based on the disparity angle, a first target three-dimensional point and a first target camera node pair of feature matching are screened out from the first three-dimensional point and the first camera node pair of feature matching.

[0100] It should be noted that when the normal vector error at low parallax angles becomes very large, estimating relative translation or verifying feature matching based on coplanarity consistency becomes ineffective. Therefore, in this embodiment, a threshold is pre-set, and the relative translation between the first target 3D point and the first target camera node pair of feature matching is re-estimated for parallax angles α greater than the set threshold.

[0101] In this embodiment, the relative translation is re-estimated using the coplanarity constraint in epipolar geometry to improve the accuracy of the relative translation. Furthermore, by analyzing the influence of the parallax angle, unstable feature matching is filtered out, thereby enhancing the robustness of the re-estimation.

[0102] Based on any of the foregoing embodiments, constructing a first objective function corresponding to the first constraint includes:

[0103] A first objective function corresponding to the first constraint is constructed based on the target relative translation of the camera node pair and the cross product of the position difference between the two camera nodes in the camera node pair.

[0104] Among them, the constraints from camera to camera and from camera to 3D point can be expressed as the following formula:

[0105] s i -s j =||s i -s j ||2·s ij , where s i ,s j represents a camera or a 3D point, s ij Indicates from s j to s i A known normalized vector of , such as an eigenray or a relative translation.

[0106] In some embodiments, the cross product form ||s ij×(s i -s j )||2 constructs the objective function, or it can be constructed in the form of scale||s i -s j -λ ij s ij ||2Construct the objective function, where λ ij is a scale variable.

[0107] The inequality constraint in the form of a cross product provides an erroneous feasible region, leading to a biased solution. Therefore, this embodiment uses the target relative translation v based on the camera node pair. ij , and the position difference between the two camera nodes in the camera node pair (t i -t j ) is used to construct the first objective function to obtain better convergence.

[0108] Based on any of the foregoing embodiments, constructing a second objective function corresponding to the second constraint includes:

[0109] A second objective function corresponding to the second constraint is constructed based on the target feature trajectory of the camera node and the three-dimensional point, and the cross product of the position difference between the camera node and the three-dimensional point.

[0110] In this embodiment, similar to the first objective function, by using the target feature trajectory f of the camera node and the three-dimensional point ki The position difference between the camera node and the 3D point (P k -t i ) to construct the second objective function to obtain better convergence.

[0111] Based on any of the above embodiments, the target feature trajectory of the camera node and the three-dimensional point is obtained by:

[0112] Filter out the second target three-dimensional point and the second target camera node pair with feature matching;

[0113] Constructing a feature trajectory between the second target three-dimensional point and the second target camera node pair based on a set-finding algorithm;

[0114] sorting all feature trajectories in descending order based on the maximum viewing angle difference of the feature trajectories between the second target 3D point and the second target camera node pair;

[0115] Target feature tracks that meet the first requirement are screened out from all feature tracks in sequence until each camera node is covered by all target feature tracks for a number of times that meets the second requirement.

[0116] After reestimating the relative translation, feature matches that violate coplanarity or cheirality constraints are removed. The coplanarity constraint means that the matched feature points should lie on the same plane, while the cheirality constraint ensures that the 3D point is in front of the camera, not behind it. Feature matches that violate these constraints are usually caused by incorrect matches or noise and are therefore removed. Only the second target 3D point and second target camera node pairs of feature matches that do not violate coplanarity or cheirality constraints are retained.

[0117] Then, a Union-Find algorithm is used to construct a feature trajectory between the second target three-dimensional point and the second target camera node pair.

[0118] Here, since many feature trajectories in the feature-matched second target 3D point and second target camera node pair may contain a high proportion of feature ray outliers, this embodiment may select some target feature trajectories to improve efficiency and robustness.

[0119] Specifically, all feature tracks are sorted in descending order according to their maximum parallax angle. In this way, feature tracks with larger parallax angles and more reliable features will be ranked at the front. Then, in order to ensure that each camera is fully covered, the sorted feature tracks are checked in turn to determine whether they can establish a connection with those cameras that are currently under-covered. If so (i.e., the first requirement), the feature track is added to the feature track subset. The above process is continued until each camera can be covered by the target feature track in the selected feature track subset at least N times (i.e., the second requirement).

[0120] In this embodiment, a reliable target feature trajectory is selected through the above method.

[0121] Based on any of the above embodiments, solving the first objective function and the second objective function to obtain the relative translation of the camera node and the position of the three-dimensional point includes:

[0122] Optimize the first objective function and the second objective function under the L1 norm to obtain a first optimization result;

[0123] By using the third objective function based on the angle, the first optimization result is optimized to obtain the relative translation of the camera node and the position of the three-dimensional point.

[0124] In this embodiment, to avoid redundant and incorrect constraints from characteristic rays, only camera-to-camera inequality constraints are used to eliminate scale and direction ambiguity. In addition, to improve robustness, the objective function is optimized under the L1 norm, specifically referring to the following formula:

[0125]

[0126]

[0127] Among them, ∑ i∈V t i = 0 is used to eliminate the internal position, v ij ·(t i -t j )≥1 is used to eliminate scale ambiguity.

[0128] Furthermore, since the above optimization results may lead to a bias towards certain camera positions or directions, this embodiment further uses an unbiased angle-based third objective function to further optimize the current optimization results. The third objective function no longer directly depends on the size of the vector (i.e., the L1 or L2 norm), but rather on the angular relationship between the vectors. Specific reference is made to the following formula:

[0129]

[0130]

[0131] In each iterative optimization, Then the IRLS method is used to optimize the third objective function, which will not be described in detail here.

[0132] See also Figure 4 , Figure 4 Schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented:

[0133] Constructing a trajectory graph in a preset three-dimensional scene; the trajectory graph includes camera nodes, three-dimensional points, edges between camera node pairs, and edges between the camera nodes and the three-dimensional points;

[0134] Constructing a first constraint on the relative translation of the target of the camera node pair, and constructing a second constraint on the target feature trajectory of the camera node and the three-dimensional point;

[0135] Constructing a first objective function corresponding to the first constraint, and constructing a second objective function corresponding to the second constraint;

[0136] With the goal of minimizing the sum of constraint errors of the first constraint and the second constraint, the first objective function and the second objective function are solved to obtain the relative translation of the camera node and the position of the three-dimensional point.

[0137] See also Figure 5 , Figure 5 Schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, the following steps are implemented:

[0138] Constructing a trajectory graph in a preset three-dimensional scene; the trajectory graph includes camera nodes, three-dimensional points, edges between camera node pairs, and edges between the camera nodes and the three-dimensional points;

[0139] Constructing a first constraint on the relative translation of the target of the camera node pair, and constructing a second constraint on the target feature trajectory of the camera node and the three-dimensional point;

[0140] Constructing a first objective function corresponding to the first constraint, and constructing a second objective function corresponding to the second constraint;

[0141] With the goal of minimizing the sum of constraint errors of the first constraint and the second constraint, the first objective function and the second objective function are solved to obtain the relative translation of the camera node and the position of the three-dimensional point.

[0142] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. That is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0143] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A three-dimensional measurement method based on machine vision, characterized in that: It includes a 3D measurement platform, a trajectory map building module, a translation constraint building module, a constraint function building module and a 3D measurement module; The three-dimensional measurement platform is respectively connected to the trajectory graph construction module, the translation constraint construction module, the constraint function construction module and the three-dimensional measurement module, and is used to store and manage the data of each module; The trajectory graph construction module is used to construct a trajectory graph in a preset three-dimensional scene; the trajectory graph includes camera nodes, three-dimensional points, edges between camera node pairs, and edges between the camera nodes and the three-dimensional points; The translation constraint construction module is used to construct a first constraint on the relative translation of the target of the camera node pair, and to construct a second constraint on the target feature trajectory of the camera node and the three-dimensional point; The constraint function construction module is used to construct a first objective function corresponding to the first constraint, and to construct a second objective function corresponding to the second constraint; The three-dimensional measurement module is configured to solve the first objective function and the second objective function with the goal of minimizing the sum of the constraint errors of the first constraint and the second constraint, and obtain the relative translation of the camera node and the position of the three-dimensional point; The method comprises: constructing a trajectory graph under a preset three-dimensional scene; the trajectory graph includes camera nodes, three-dimensional points, edges between camera node pairs, and edges between the camera nodes and the three-dimensional points; Constructing a first constraint on the relative translation of the target of the camera node pair, and constructing a second constraint on the target feature trajectory of the camera node and the three-dimensional point; Constructing a first objective function corresponding to the first constraint, and constructing a second objective function corresponding to the second constraint; With the goal of minimizing the sum of the constraint errors of the first constraint and the second constraint, solving the first objective function and the second objective function to obtain the relative translation of the camera node and the position of the three-dimensional point; The method further comprises: Filter out the first target three-dimensional point and the first target camera node pair with feature matching; Determining a normal vector of an epipolar plane between the first target 3D point and the first target camera node pair, and determining a weight corresponding to the normal vector based on a parallax angle between the first target 3D point and the first target camera node pair; Obtaining a target relative translation of the camera node pair by an iterative reweighted least squares method based on the normal vector and a weight corresponding to the normal vector; Among them, the global posture of the camera R i , t i Describes the rotation matrix R of camera i in the global coordinate system i and the translation vector t i , global pose R j ,t j Describes the rotation matrix R of camera j in the global coordinate system j and the translation vector t j , relative posture R ij , t ij Describes the rotation matrix R of camera i relative to camera j ij and the translation vector t ij ; The global pose R of the camera i , t i and relative posture R ij , t ij Satisfies the following equation: in, is the rotation matrix R i The bias matrix, is the rotation matrix R j The bias matrix, v ij Represents a relative translation in the global coordinate system.

2. The three-dimensional measurement method based on machine vision according to claim 1, characterized in that: The step of screening out the first target three-dimensional point and the first target camera node pair with feature matching includes: Filter out the first three-dimensional point and the first camera node pair with feature matching; Determine the disparity angle between the first three-dimensional point and the first camera node pair for each feature match; Based on the parallax angle, a first target three-dimensional point and a first target camera node pair of feature matching is filtered out from the first three-dimensional point and the first camera node pair of feature matching.

3. The three-dimensional measurement method based on machine vision according to claim 1, characterized in that: The constructing a first objective function corresponding to the first constraint includes: A first objective function corresponding to the first constraint is constructed based on the target relative translation of the camera node pair and a cross product of the position difference between two camera nodes in the camera node pair.

4. The three-dimensional measurement method based on machine vision according to claim 1, characterized in that: The constructing a second objective function corresponding to the second constraint includes: A second objective function corresponding to the second constraint is constructed based on the target feature trajectory of the camera node and the three-dimensional point, and the cross product of the position difference between the camera node and the three-dimensional point.

5. The three-dimensional measurement method based on machine vision according to claim 1, characterized in that: The target feature trajectory of the camera node and the three-dimensional point is obtained by: Filter out the second target three-dimensional point and the second target camera node pair with feature matching; Constructing a feature trajectory between the second target three-dimensional point and the second target camera node pair based on a set-finding algorithm; sorting all the feature trajectories in descending order based on the maximum viewing angle difference of the feature trajectories between the second target three-dimensional point and the second target camera node pair; Target feature trajectories that meet the first requirement are screened out from all the feature trajectories in sequence until the number of times each camera node is covered by all the target feature trajectories reaches the second requirement.

6. The three-dimensional measurement method based on machine vision according to any one of claims 2 to 5, characterized in that: Solving the first objective function and the second objective function to obtain the relative translation of the camera node and the position of the three-dimensional point includes: Optimizing the first objective function and the second objective function under the L1 norm to obtain a first optimization result; The first optimization result is optimized by using a third objective function based on an angle to obtain the relative translation of the camera node and the position of the three-dimensional point.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the three-dimensional measurement method based on machine vision as described in any one of claims 2 to 6 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the three-dimensional measurement method based on machine vision as claimed in any one of claims 2 to 6 is implemented.

Images