High-precision three-dimensional measurement method, system and medium based on multi-view machine vision

Abstract

The application discloses a high-precision three-dimensional measurement method and system based on multi-view machine vision and a medium, and particularly relates to the technical field of multi-view three-dimensional optical measurement, and is used for solving the problem of scale inconsistency of reconstructed point clouds when a global motion structure-from-motion method is used in a camera network with weak connection or low parallax; a camera view graph is constructed by screening camera pairs with consistent translation directions, a scale transfer bottleneck edge connecting different camera groups and having no alternative path is identified, a conflict injection edge is positioned in a closed loop and is jointly marked as a fragile edge, the relative translation vector module length of the fragile edge is adjusted by using the scale reference of a non-fragile edge, and then translation constraints and relative rotation are input into global translation solving to obtain a global position and generate a consistent-scale three-dimensional point cloud.

Description

Technical Field

[0001] This invention relates to the field of multi-view vision 3D optical measurement technology, and in particular to a high-precision 3D measurement method, system and medium based on multi-view machine vision. Background Technology

[0002] Acquiring high-precision 3D topography of an object surface can be achieved by simultaneously acquiring images from different perspectives using multiple cameras. Algorithms then transform the 2D information from each perspective into a 3D point cloud in a unified coordinate system. Estimating the precise relative position and pose between the cameras is crucial for ensuring the final measurement accuracy. Structure-of-motion (SOP) algorithms are a common method for recovering camera pose and scene structure from image sequences. Global methods utilize the relative pose relationships between all camera pairs to construct a view graph, and then solve for the global pose of all cameras at once, avoiding error accumulation caused by sequential registration and resulting in higher reconstruction efficiency. However, when applying global methods to visual measurements, the baseline length and parallax between camera pairs in the view graph are often uneven. The relative translation estimation of cameras with small viewpoint changes inherently carries significant directional uncertainty. When these inherently inaccurate relative translations are used as constraints in the global solution, current practices often treat all constraints equally, affecting the quality of the overall translation solution.

[0003] Existing global translation estimation methods lack a mechanism to adjust the weights of different translation constraints based on local geometric connectivity when dealing with situations where the connection strength of camera pairs in the view image varies. This leads to scale inconsistency deviations in the reconstructed point clouds of some regions after global pose calculation. This is more pronounced when there are weak connections or low parallax camera pairs in the camera network, affecting the accuracy of the overall 3D reconstruction in visual measurements. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a high-precision three-dimensional measurement method, system, and medium based on multi-view machine vision.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: High-precision 3D measurement methods based on multi-view machine vision include: S1: Acquire images of the object under test from multiple cameras simultaneously from different perspectives using a multi-view machine vision system; S2: Calculate the relative rotation and relative translation directions for each camera pair that has a co-view relationship between the images of the object under test. Based on the consistency of the translation directions between the camera pair and the surrounding camera pairs, filter the camera pairs to be constructed and construct a camera view graph with the camera as the node and the relative rotation and relative translation directions of the camera pairs to be constructed as the edges. S3: Identify edges in the camera view graph that connect different camera groups and have no alternative paths, and use them as bottleneck edges for scale propagation; S4: For a closed loop containing a scale transfer bottleneck edge, calculate the consistency deviation of the relative translation direction of adjacent edges within the closed loop. Mark the edge with the largest increase in deviation among the continuous edge segments where the consistency deviation increases monotonically as the conflict injection edge, and mark the conflict injection edge and the scale transfer bottleneck edge together as the brittle edge. S5: For each brittle edge, based on the scale reference determined by the non-brittle edge that shares the camera with the brittle edge, adjust the magnitude of the relative translation vector of the brittle edge to obtain the corrected relative translation vector; S6: Input the relative translation vector, the corrected relative translation vector, and the relative rotation of the non-brittle edges into the global translation solver to obtain the global position of each camera and generate a 3D point cloud with a consistent scale.

[0006] Furthermore, a multi-view machine vision system acquires images of the object under test simultaneously from different perspectives using multiple cameras, including: Multiple cameras are arranged around the object being measured, so that the field of view of each camera covers the measured area of ​​the object and the fields of view of adjacent cameras overlap. Send synchronization trigger signals to multiple cameras to control the exposure of multiple cameras at the same time; It receives image data synchronously output from multiple cameras and uses it as an image of the object under test.

[0007] Furthermore, for each camera pair with a co-view relationship among the images of the measured object, the relative rotation and relative translation directions are calculated. Based on the consistency of the translation directions between the camera pair and surrounding camera pairs, camera pairs to be constructed are selected. A camera view graph is constructed with cameras as nodes and the relative rotation and relative translation directions of the camera pairs to be constructed as edges, including: Extract feature matching points between images of the object under test, and calculate the relative rotation and relative translation directions of each camera pair that have a co-view relationship based on the feature matching points; For each camera pair, obtain the neighboring camera pairs that share a camera with that camera pair and whose relative translation direction has been calculated, and construct a local translation direction field from the relative translation directions of the neighboring camera pairs. The relative translation direction of the camera pair is converted into a unit vector. The relative translation direction of each neighboring camera pair in the local translation direction field is converted into a unit vector. The spherical angular distance between the unit vector of the camera pair and the unit vector of each neighboring camera pair is calculated. The average of all spherical angular distances is taken as the direction difference. When the direction difference is less than the preset consistency range, the camera pair is determined as the camera pair to be constructed. Using the camera as a node and the relative rotation and relative translation directions of the camera pair to be constructed as edges, generate a camera view graph.

[0008] Furthermore, edges in the camera view graph that connect different camera groups and have no alternative paths are identified as scale transfer bottleneck edges, including: Iterate through each edge in the camera view graph and determine whether removing the current edge separates the camera view graph into two unconnected camera groups. If removing the current edge results in the camera view graph being separated into two unconnected camera groups, then the current edge is determined to be an edge connecting different camera groups. For edges identified as connecting different camera groups, check if there are other paths in the camera view graph that connect the cameras at both ends of the edge; If there is no other path connecting the cameras at both ends of the edge, then the edge is identified as a scale transfer bottleneck edge.

[0009] Furthermore, for a closed loop containing a scale transfer bottleneck edge, the consistency deviation of the relative translation direction of adjacent edges within the closed loop is calculated. The edge with the largest increase in deviation among continuous edge segments where the consistency deviation monotonically increases is marked as a conflict injection edge. Both the conflict injection edge and the scale transfer bottleneck edge are marked as brittle edges, including: Extract all closed loops containing the scale transfer bottleneck edge from the camera view graph; For each closed loop, calculate the angle between the relative translation directions recorded on the two adjacent sides along the loop direction, as the consistent deviation of the translation directions of the adjacent sides; Identify continuous edge segments within a closed loop where the consistency deviation continues to increase, and mark the edge corresponding to the adjacent edge that causes the largest increase in consistency deviation in the continuous edge segment as a conflict injection edge. Both conflict-injected edges and scale-transfer bottleneck edges are labeled as brittle edges.

[0010] Further, the angle between the relative translation directions recorded by the two adjacent sides is calculated sequentially, including: in the direction of the closed loop, for the adjacent first side and second side, the relative translation direction of the first side is projected onto the local coordinate system with the origin of the second side as the origin, and the angle between the projection direction and the relative translation direction of the second side is calculated.

[0011] Furthermore, for each brittle edge, based on the scale reference determined by the non-brittle edges that share the camera with the brittle edge, the magnitude of the relative translation vector of the brittle edge is adjusted to obtain the corrected relative translation vector, including: For each brittle edge, identify the non-brittle edges that share a camera with it and are not marked as brittle edges; extract the magnitude of the relative translation vector of the non-brittle edges as the scale reference. Keeping the direction of the relative translation vector of the brittle edge unchanged, the magnitude of the relative translation vector of the brittle edge is replaced with the scale reference to obtain the corrected relative translation vector.

[0012] Furthermore, the relative translation vectors of the non-brittle edges, the corrected relative translation vectors, and the relative rotations are input into the global translation solver to obtain the global positions of each camera, generating a consistent-scale 3D point cloud, including: Extract the relative translation direction of non-brittle edges that are not marked as brittle edges in the camera view graph, and combine the relative translation direction of the non-brittle edges with the original relative translation magnitude of the non-brittle edges to form the relative translation vector of the non-brittle edges. The relative translation vector, the corrected relative translation vector, and the relative rotation of the non-brittle edges are used as a set of constraints. Global translation is solved on the set of constraints to obtain the global position of each camera. Triangulation is performed based on the global position of each camera and the feature matching points between the images of the object being measured to generate a 3D point cloud of consistent scale.

[0013] On the other hand, the present invention provides a high-precision three-dimensional measurement system based on multi-view machine vision, comprising: The image acquisition module is used to acquire images of the object under test simultaneously from different perspectives by multiple cameras through a multi-view machine vision system. The view graph construction module is used to calculate the relative rotation and relative translation directions for each camera pair that has a co-view relationship between the images of the measured object. Based on the consistency of the translation direction between the camera pair and the surrounding camera pairs, it filters the camera pairs to be constructed and constructs a camera view graph with the camera as the node and the relative rotation and relative translation directions of the camera pairs to be constructed as the edges. The bottleneck edge recognition module is used to identify edges in the camera view that connect different camera groups and have no alternative paths, and to pass these edges as the scale. The brittle edge marking module is used to calculate the consistency deviation of the relative translation direction of adjacent edges in a closed loop containing a scale transfer bottleneck edge. The edge with the largest increase in deviation among the continuous edge segments with monotonically increasing consistency deviation is marked as a conflict injection edge, and the conflict injection edge and the scale transfer bottleneck edge are jointly marked as brittle edges. The vector correction module is used to adjust the magnitude of the relative translation vector of each brittle edge based on the scale reference determined by the non-brittle edges that share the camera with the brittle edge, so as to obtain the corrected relative translation vector. The point cloud generation module is used to input the relative translation vectors, corrected relative translation vectors, and relative rotations of non-brittle edges into the global translation solver to obtain the global positions of each camera and generate a 3D point cloud of consistent scale.

[0014] On the other hand, the present invention provides a medium on which a program or instruction is stored, which, when executed by a processor, implements a high-precision three-dimensional measurement method based on multi-view machine vision.

[0015] The beneficial effects of this invention are: 1. By combining topological analysis of the view graph with the evaluation of the consistency deviation of the relative translation direction within the closed loop, the scale transfer bottleneck edge and conflict injection edge are identified from the camera network connection relationship. The relative translation vector magnitude of the brittle edge is adjusted in a targeted manner, rather than treating the translation constraints of all camera pairs equally or simply eliminating them. While maintaining the high efficiency of the global method in solving the problem once, the translation constraints entering the global translation solution tend to be consistent in scale. This reduces the spread of scale deviations caused by local weak connections, low parallax, or orientation anomalies along the camera network, thereby improving the scale consistency and accuracy of visual measurement 3D reconstruction as a whole.

[0016] 2. During the view map construction stage, the consistency of the translation direction between the camera pair and the surrounding camera pairs is checked to select camera pairs with reliable directions to participate in the map construction. This reduces the number of abnormally oriented edges entering the subsequent process from the source. In the processing of brittle edges, the orientation information of the brittle edges is retained and only their modulus is replaced. This maintains the topological integrity of the camera view map and restores the scale of the brittle edges with the help of reliable scale benchmarks of non-brittle edges. This makes the final generated 3D point cloud have a consistent spatial scale throughout the measurement area, which is beneficial for high-precision 3D detection of the shape of large-sized objects in industrial vision measurement scenarios. Attached Figure Description

[0017] Figure 1 This is a flowchart of the high-precision three-dimensional measurement method based on multi-view machine vision according to the present invention; Figure 2 This is a schematic diagram of the high-precision three-dimensional measurement system based on multi-view machine vision according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: Figure 1 This invention presents a high-precision 3D measurement method based on multi-view machine vision, comprising: S1: Acquire images of the object under test from multiple cameras simultaneously from different perspectives using a multi-view machine vision system; S2: Calculate the relative rotation and relative translation directions for each camera pair that has a co-view relationship between the images of the object under test. Based on the consistency of the translation directions between the camera pair and the surrounding camera pairs, filter the camera pairs to be constructed and construct a camera view graph with the camera as the node and the relative rotation and relative translation directions of the camera pairs to be constructed as the edges. S3: Identify edges in the camera view graph that connect different camera groups and have no alternative paths, and use them as bottleneck edges for scale propagation; S4: For a closed loop containing a scale transfer bottleneck edge, calculate the consistency deviation of the relative translation direction of adjacent edges within the closed loop. Mark the edge with the largest increase in deviation among the continuous edge segments where the consistency deviation increases monotonically as the conflict injection edge, and mark the conflict injection edge and the scale transfer bottleneck edge together as the brittle edge. S5: For each brittle edge, based on the scale reference determined by the non-brittle edge that shares the camera with the brittle edge, adjust the magnitude of the relative translation vector of the brittle edge to obtain the corrected relative translation vector; S6: Input the relative translation vector, the corrected relative translation vector, and the relative rotation of the non-brittle edges into the global translation solver to obtain the global position of each camera and generate a 3D point cloud with a consistent scale.

[0020] In the specific implementation of the embodiment corresponding to S1, multiple cameras are arranged around the object under test, such that the field of view of each camera covers the test area of ​​the object, and the fields of view of adjacent cameras share a common viewing range in the overlapping area. When arranging multiple cameras, the arrangement position of the multiple cameras and the orientation of the optical axes of the multiple cameras are determined according to the size of the test area of ​​the object and the field of view angle of each camera. This ensures that any test point on the test area of ​​the object is observed simultaneously by at least two cameras, and that the area of ​​the common viewing range of adjacent cameras in the overlapping area is not less than 10% of the field of view area of ​​a single camera. This ensures that a sufficient number of feature matching points can be extracted between the images of the object under test subsequently acquired synchronously from different perspectives.

[0021] The multiple cameras can be arranged in any of the following ways: a ring arrangement, a spherical arrangement, or a truss arrangement along the outline of the measured area of ​​the object being measured. The number of cameras can be any number greater than or equal to two, such as four, eight, or sixteen. The specific number is determined by the coverage requirements of the measured area of ​​the object being measured and the field of view of each camera.

[0022] When the measured area of ​​the object is the outer side panel of a car body, the length dimension of the outer side panel is approximately 2 meters and the height dimension is approximately 1.5 meters. Eight cameras are arranged in two rows and four columns along the length and height of the outer side panel. Each camera is about 1.5 meters away from the surface of the outer side panel, and the distance between adjacent cameras is about 0.8 meters. This ensures that the field of view of each camera completely covers the measured area of ​​the outer side panel, and the area of ​​the common visible range of adjacent cameras in the overlapping area is greater than 15% of the field of view of a single camera. This ensures that a sufficient number of evenly distributed feature matching points can be extracted in subsequent steps.

[0023] A synchronous trigger signal is sent to multiple cameras to control them to expose simultaneously. The synchronous trigger signal is generated by a synchronous trigger signal source independent of the multiple cameras. The synchronous trigger signal source is connected to each of the multiple cameras via a wired trigger cable or a wireless trigger module. The synchronous trigger signal source is configured in hardware trigger mode. In hardware trigger mode, the synchronous trigger signal source simultaneously sends trigger pulses to each of the multiple cameras in a single trigger mode or a fixed frequency trigger mode. Each of the multiple cameras starts exposure within the same clock cycle after receiving the trigger pulse. The trigger delay error of the synchronous trigger signal source is determined by the hardware performance parameters of the synchronous trigger signal source. The trigger delay error is less than the minimum exposure time of each of the multiple cameras. The exposure time of each of the multiple cameras is preset according to the lighting conditions of the measured object and the aperture parameters of each of the multiple cameras. The trigger delay error is much smaller than the typical value of the exposure time of each camera. For example, if the typical exposure time of each camera is 10 milliseconds, the trigger delay error is less than 1 millisecond, ensuring that the time deviation between the images of the measured object acquired synchronously by multiple cameras is negligible relative to the dynamic change rate of the measured object scene.

[0024] The system receives image data synchronously output from multiple cameras as images of the object under test. After each camera completes its exposure, it synchronously transmits the acquired image data to the data receiving device via a data transmission interface. The data receiving device groups and stores the received multi-frame image data according to the unique identifier of each camera. Each group of image data corresponds to one frame of the object under test acquired by a single camera. The data transmission interface is selected to support multi-channel parallel transmission. Multi-channel parallel transmission ensures that multi-frame image data can be transmitted from multiple cameras to the data receiving device simultaneously without channel conflicts. The object under test images obtained according to the above steps are multi-frame two-dimensional digital images. Each frame of the multi-frame two-dimensional digital images records the two-dimensional grayscale information or two-dimensional color information of the measured area of ​​the object under test under the corresponding viewpoint of the corresponding camera. Since adjacent cameras have a common field of view in the overlapping area, at least one pair of two-dimensional digital images in the multi-frame two-dimensional digital images contains the imaging content of the same local area of ​​the measured area of ​​the object under test. These pairs of two-dimensional digital images have a co-viewing relationship.

[0025] In the measurement scenario of the side panel of the car body, eight cameras simultaneously output eight frames of images of the object under test. The resolution of each frame of the object under test is 2448 pixels by 2048 pixels. The common visible range between the object under test images acquired by adjacent cameras in the overlapping area corresponds to an image strip area with a width of about 200 pixels in the object under test image. This common visible range provides a sufficient data basis for subsequent feature matching.

[0026] A multi-view machine vision system consists of multiple cameras arranged around the object under test, a synchronous trigger signal source independent of the cameras, and a data receiving device. The field of view of each camera covers the area under test of the object, and adjacent cameras share a common field of view in overlapping areas. The synchronous trigger signal source is connected to each camera via a wired trigger cable or a wireless trigger module. The synchronous trigger signal source is configured in hardware trigger mode, simultaneously sending trigger pulses to each camera using either a single trigger or a fixed-frequency trigger method. Each camera initiates exposure within the same clock cycle of receiving the trigger pulse. After each camera completes exposure, it synchronously transmits the acquired image data to the data receiving device via a data transmission interface. The data receiving device groups and stores the received multi-frame image data according to the unique identifier of each camera, with each group corresponding to a single frame of two-dimensional digital image acquired by a single camera.

[0027] In the specific implementation of the embodiment corresponding to S2, feature matching points are extracted between the images of the object under test. Based on the feature matching points, the relative rotation and relative translation directions of each camera pair with a co-view relationship are calculated. The feature matching point is the pixel coordinate pair corresponding to the same physical point on the measured area of ​​the object under test in different images of the object under test. The extraction of feature matching points adopts the scale-invariant feature transform algorithm. The specific steps are as follows: feature points are extracted for each frame of the object under test and the feature descriptor of each feature point is calculated. For any two frames of the object under test with a co-view relationship, the similarity distance of the feature descriptors between the two frames of the object under test is calculated. For each feature point, the candidate matching point with the smallest similarity distance is found in another frame of the object under test. The ratio of the nearest neighbor distance to the second nearest neighbor distance is calculated. The ratio is compared with the matching threshold. When the ratio is less than the matching threshold, the corresponding pair of feature points is determined as feature matching points. At the same time, bidirectional matching verification is performed on the two frames of the object under test. Only when the constraint that the ratio is less than the matching threshold is satisfied in both directions is a pair of feature matching points finally retained.

[0028] The matching threshold is determined by analyzing the distribution of the ratio of the nearest neighbor distance to the second nearest neighbor distance of all candidate feature point pairs. The ratios of all candidate feature point pairs are statistically analyzed and a ratio distribution histogram is plotted. If the ratio distribution histogram shows a clear bimodal shape, the matching threshold is set to the ratio corresponding to the valley position between the two peaks. If the ratio distribution histogram shows a unimodal shape, the matching threshold is set to the minimum ratio that ensures the proportion of candidate feature point pairs with a ratio less than the matching threshold is not less than 60% of the total number of candidate feature point pairs, thereby adapting to the texture characteristics of the current object image being measured.

[0029] For each camera pair with a shared viewing relationship, the fundamental matrix or essential matrix of the camera pair is calculated based on all feature matching points between the two frames of the measured object images of the camera pair. Singular value decomposition is then performed on the essential matrix to extract the relative rotation and relative translation directions of the camera pair. The relative rotation is represented by a 3x3 orthogonal matrix, indicating the rotation relationship of one camera coordinate system relative to the other. The relative translation direction is represented by a 3-dimensional unit vector, indicating the translation direction of the optical center of one camera relative to the optical center of the other. The magnitude of the 3-dimensional unit vector is 1, representing only the direction information of the translation and not the length information.

[0030] For each camera pair, obtain the neighboring camera pairs that share a camera and whose relative translation direction has been calculated. The relative translation directions of the neighboring camera pairs constitute the local translation direction field. Each camera among multiple cameras is uniquely identified by Ci, where i is the camera's index, a positive integer, and its value ranges from 1 to the number of cameras. For a camera pair Ci-Cj consisting of the i-th camera and the j-th camera, among all camera pairs with calculated relative translation directions, find other camera pairs containing the i-th camera and other camera pairs containing the j-th camera. These found other camera pairs containing the i-th camera and other camera pairs containing the j-th camera are considered neighboring camera pairs of camera pair Ci-Cj. Other camera pairs containing the i-th camera are denoted as Ci-Ck, where k is a positive integer and k is not equal to i, and other camera pairs containing the j-th camera are denoted as Cj-Cm, where m is a positive integer and m is not equal to j. The relative translation directions of all found neighboring camera pairs are aggregated to form the local translation direction field of camera pair Ci-Cj. The size of the local translation direction field is determined by the number of other cameras connected to the i-th and j-th cameras. When the i-th and j-th cameras each have multiple connected cameras, the local translation direction field contains the relative translation directions of multiple neighboring camera pairs. When the i-th or j-th camera is only connected to the other camera, the local translation direction field contains fewer or no relative translation directions of other neighboring camera pairs. In this case, the orientation consistency of camera pair Ci-Cj lacks sufficient reference, the reliability of subsequent orientation difference calculations is reduced, and camera pair Ci-Cj is difficult to determine as the camera pair to be constructed.

[0031] The direction difference between the relative translation direction of the camera pair and the relative translation direction of neighboring camera pairs in the local translation direction field is calculated. When the direction difference is less than a preset consistency range, the camera pair is determined as a camera pair to be constructed. The direction difference is calculated as follows: the relative translation direction of the camera pair is converted into a unit vector, the relative translation direction of each neighboring camera pair in the local translation direction field is converted into a unit vector, the spherical angular distance between the unit vector of the camera pair and the unit vector of each neighboring camera pair is calculated, and the average of all spherical angular distances is taken as the direction difference. The spherical angular distance is calculated as: d = arccos(|v1·v2|); where d is the spherical angular distance, v1 is the unit vector obtained by converting the relative translation direction of the camera pair, v2 is the unit vector obtained by converting the relative translation direction of the neighboring camera pairs, · is the vector dot product operation, | is the absolute value operation, and arccos is the inverse cosine function. The unit of the spherical angular distance d is radians, and the value range is between 0 and π / 2.

[0032] Since relative translation direction only represents directional information, the relative translation direction of a camera pair is physically equivalent to its opposite relative translation direction. Therefore, the absolute value of the dot product is taken before calculating the inverse cosine, ensuring that the spherical angular distance remains within the range of 0 to π / 2. This avoids calculation errors caused by reversed directions due to different naming orders between the two cameras. When the local translation direction field contains the relative translation directions of multiple neighboring camera pairs, the spherical angular distance is calculated for each neighboring camera pair, resulting in multiple spherical angular distances. The arithmetic mean of these multiple spherical angular distances is used as the directional difference. The preset consistency range is set based on the uncertainty of the relative translation direction of the camera pair itself. The uncertainty of the relative translation direction of the camera pair itself is determined by the number of feature matching points used in calculating the relative translation direction and the spatial uniformity of the feature matching points.

[0033] The preset consistency range is set as follows: The statistical variance of the backprojection error of the camera to the corresponding feature matching points is calculated. The statistical variance of the backprojection error reflects the overall positioning accuracy of the feature matching points. The larger the statistical variance, the less accurate the positioning of the feature matching points, and the higher the uncertainty of the relative translation direction itself. The spatial distribution uniformity of the camera to the corresponding feature matching points is calculated. The spatial distribution uniformity is obtained by mapping the feature matching points onto the image plane of the measured object and dividing it into multiple grids, and then statistically analyzing the proportion of the number of grids containing feature matching points to the total number of grids. The higher the proportion, the more uniform the spatial distribution of the feature matching points. A two-dimensional lookup table is jointly established based on the statistical variance of the backprojection error and the spatial distribution uniformity of the feature matching points. The two input dimensions of the two-dimensional lookup table are the statistical variance of the backprojection error and the spatial distribution uniformity of the feature matching points, respectively. The output of the two-dimensional lookup table is the angle value of the preset consistency range. The angle value in the two-dimensional lookup table is pre-calibrated using statistical methods so that the preset consistency range takes a smaller angle value when the number of feature matching points and the spatial distribution uniformity are both good, and a larger angle value when the number of feature matching points is small or the spatial distribution uniformity is poor.

[0034] When the directional difference is less than the preset consistency range, it indicates that the relative translation direction of the camera pair is consistent with the relative translation direction of neighboring camera pairs in the local translation direction field. The relative translation direction of the camera pair is reliable, and therefore the camera pair is determined as a camera pair to be constructed. Conversely, when the directional difference is greater than or equal to the preset consistency range, the relative translation direction of the camera pair is unreliable, and the camera pair is not included in the camera pair to be constructed.

[0035] The local translation orientation field consistency screening utilizes the physical principle that the translation orientation field in a multi-camera network should satisfy local consistency. In a multi-view machine vision system, there are definite spatial geometric relationships between multiple cameras. When all camera positions and optical axis orientations are fixed, the relative translation direction between any two cameras does not exist in isolation, but rather constitutes a continuous translation orientation field together with the relative translation directions of other surrounding camera pairs. The translation orientation field should possess smoothness and consistency within a local region because adjacent camera pairs observe adjacent local areas of the same measured object; the change in translation direction between cameras should be gradual rather than abrupt. If the relative translation direction of a certain camera pair deviates significantly from the relative translation directions of surrounding camera pairs, the relative translation direction of the camera pair is likely to introduce a large orientation estimation error due to the existence of outliers in the feature matching points on which the camera pair depends, or the excessively small disparity between the two frames of the measured object images corresponding to the camera pair.

[0036] In existing technologies, after calculating the relative translation direction of all camera pairs, all of them are directly included in the camera view graph as camera pairs to be constructed, without filtering out camera pairs with abnormal orientations. This results in the presence of unreliable edges in the subsequently constructed camera view graph. These unreliable edges propagate errors during global translation calculation, causing inconsistencies in the scale of the reconstructed point cloud. By comparing the consistency of the relative translation direction of each camera pair with the relative translation directions of its neighboring camera pairs in its local translation direction field, reliable camera pairs to be constructed are selected, reducing the number of abnormal edges in the camera view graph from the source and providing a cleaner set of camera pairs to be constructed for subsequent steps.

[0037] A camera view graph is generated, using cameras as nodes and the relative rotation and translation directions of the camera pairs to be constructed as edges. The camera view graph is an undirected graph structure. Each node in the camera view graph corresponds to one of multiple cameras. Each edge in the camera view graph connects two nodes and corresponds to a camera pair to be constructed. The relative rotation and translation directions of the camera pair to be constructed are recorded on each edge. The generation process of the camera view graph is as follows: Initialize an empty undirected graph structure; traverse all camera pairs to be constructed; for each camera pair to be constructed, add an edge between its two corresponding nodes, and record the relative rotation and translation directions of the camera pair to be constructed as attributes on the edge. Once all camera pairs to be constructed have been added, the camera view graph is completed. The camera view graph records the global connection topology between multiple cameras. Subsequent steps will use the camera view graph to identify scale-transfer bottleneck edges and mark brittle edges.

[0038] In the specific implementation of the embodiment corresponding to S3, scale transfer bottleneck edges are identified in the camera view graph. The camera view graph is generated by S2 and contains an undirected graph structure with cameras as nodes and the relative rotation and relative translation directions of the camera pairs to be constructed as edges. In the camera view graph, multiple cameras are connected by edges to form an overall connection topology. When there is an edge in the camera view graph that connects different camera groups, and this edge is the only edge connecting different camera groups, this edge is identified as a scale transfer bottleneck edge. The meaning of the scale transfer bottleneck edge is: during the global translation solution, the scale information between different camera groups can only be transferred through the scale transfer bottleneck edge. Once the relative translation vector corresponding to the scale transfer bottleneck edge has a scale error, the scale error of the scale transfer bottleneck edge cannot be verified or corrected through other redundant paths, resulting in scale inconsistency between different camera groups. This manifests as a scale abrupt change at the boundary between the point clouds reconstructed by different camera groups, i.e., regional point cloud scale splitting.

[0039] Traverse each edge in the camera view graph and determine whether removing the current edge separates the camera view graph into two disconnected camera groups. A camera group is a connected subgraph within the camera view graph, where at least two nodes are connected by a path, and different camera groups are not connected by any path. Removing the current edge involves temporarily deleting it from the camera view graph while keeping all other nodes and edges unchanged. The method to determine whether the camera view graph is separated into two unconnected camera groups after removing the current edge is as follows: Starting from the node corresponding to either of the cameras at the two ends of the current edge, a breadth-first search is used to traverse all reachable nodes. The breadth-first search starts from the starting node, visits the nodes directly connected to the starting node layer by layer, and then visits the nodes connected to the directly connected nodes, until no new nodes can be reached. If the node corresponding to the other camera at the two ends of the current edge can be reached by the breadth-first search, then the camera view graph remains connected after removing the current edge, and the current edge is not an edge connecting different camera groups. If the node corresponding to the other camera at the two ends of the current edge cannot be reached by the breadth-first search, then the camera view graph is separated into two unconnected camera groups after removing the current edge, and the current edge is an edge connecting different camera groups.

[0040] If removing the current edge results in the camera view graph splitting into two unconnected camera groups, then the current edge is determined to be an edge connecting different camera groups. Based on this, for edges determined to connect different camera groups, further checks are performed to see if there are other paths in the camera view graph connecting the two ends of this edge. Other paths refer to continuous pathways in the camera view graph formed by one or more edges connected end-to-end, with the starting and ending points of the continuous pathway being the two ends of the current edge, respectively. The method for checking for other paths is as follows: after removing the current edge in the camera view graph, starting from the node corresponding to any one of the two cameras at the ends of the current edge, a breadth-first search is used to traverse all reachable nodes. If, under the condition of removing the current edge, a path can still be found from one camera at the end of the current edge to the other camera at the other end, then this path is an other path. Since it was determined in the previous step that removing the current edge splits the camera view graph into two unconnected camera groups, and there is no path connecting the two camera groups, there is no such other path, and the current edge is identified as a scale transfer bottleneck edge.

[0041] The identification of scale transfer bottleneck edges leverages the topological characteristics of the camera view graph. In the camera view graph, the connectivity of edges determines the path for scale information to be transferred between different cameras. For an edge connecting different camera groups, if this edge is the only channel connecting two camera groups, the scale relationship between the two camera groups depends entirely on the relative translation vector of this edge. In actual measurements, the camera pair to be constructed corresponding to this edge may have significant uncertainty in the calculation result of the relative translation direction due to small disparity between the two frames of the measured object image or insufficient number of feature matching points, leading to scale errors when subsequently recovering the magnitude of the relative translation vector. If there are redundant edges between two camera groups, i.e., multiple edges connecting the two camera groups, the scale error can be mitigated through cross-validation of different edges and will not be concentrated on a single edge. However, when there is no redundancy, the scale transfer bottleneck edge becomes the only propagation path for scale error. The scale error of the scale transfer bottleneck edge will be fully propagated to one of the two camera groups connected by the bottleneck edge, causing an overall scale deviation in one camera group, ultimately resulting in scale inconsistency deviations in the reconstructed point cloud of some regions after global pose calculation. By identifying scale transfer bottlenecks in advance, the bottlenecks and their associated fragile connections can be addressed in subsequent steps to prevent scale errors from spreading along non-redundant topological paths, thereby improving the scale consistency of the overall 3D reconstruction from visual measurements.

[0042] In the specific implementation of the embodiment corresponding to S4, all closed loops containing the scale transfer bottleneck edge are extracted from the camera view diagram. For each closed loop, the consistency deviation of the relative translation direction of adjacent edges within the closed loop is calculated. Continuous edge segments with monotonically increasing consistency deviations are identified. Among these continuous edge segments, the edge with the largest increase in deviation is located and marked as the conflict injection edge. The conflict injection edge and the scale transfer bottleneck edge are jointly marked as brittle edges. This step starts from the topology of the camera view diagram and uses the changing trend of the relative translation direction within the closed loop to trace the injection source of scale contradictions, providing a precise location basis for scale correction in subsequent steps. Existing technologies, when evaluating the consistency of closed loops, typically only check whether the loop closure error is within the tolerance range, focusing on the overall error rather than the distribution characteristics of the error within the loop. This step differs from this approach. This step delves into the interior of the closed loop, analyzing the distribution pattern of the consistency deviation between the relative translation directions of adjacent edges, and locating the conflict injection edge that causes scale contradictions from the changing trend of the deviation. This allows subsequent scale correction to target the source of scale error rather than all edges.

[0043] Extract all closed loops containing the scale-transfer bottleneck edge from the camera view graph. A closed loop is a closed path in the camera view graph formed by connecting at least three edges end-to-end, with the starting and ending points of the closed path being the same camera node. The extraction method is as follows: using the scale-transfer bottleneck edge as the starting edge, perform a depth-first search in the camera view graph to find all closed paths that start from one camera at the starting edge, pass through other edges in the camera view graph, and reach the other camera at the starting edge, without repeatedly passing through the same camera. Each closed path that meets this condition is a closed loop containing the scale-transfer bottleneck edge. The depth-first search is performed on the connected region of the camera view graph containing all camera nodes connected to the scale-transfer bottleneck edge. When the camera view graph is large and contains multiple closed loops, the extracted closed loops containing the scale-transfer bottleneck edge constitute a set of closed loops, where each closed loop in the set contains the scale-transfer bottleneck edge.

[0044] For each closed loop, the angle between the relative translation directions recorded on adjacent edges is calculated sequentially along the loop's circumduction direction, serving as the consistency deviation of the translation directions of adjacent edges. The circumduction direction of the closed loop is an ordered arrangement of successive nodes within the loop. The circumduction direction starts from one end of the scale transfer bottleneck edge, passes through all nodes sequentially along the closed loop, and returns to the starting camera, forming the closed loop. In the circumduction direction of the closed loop, adjacent edges are those sharing a camera within the closed loop, and these two edges have a sequential order in the circumduction direction. The specific method for calculating the angle between the relative translation directions recorded for adjacent sides is as follows: In the direction of the closed loop, for the adjacent first and second sides, a local coordinate system is defined with the origin of the second side as its origin. The origin of the local coordinate system is the optical center position of the camera corresponding to the origin of the second side. The three coordinate axes of the local coordinate system are respectively taken as the horizontal, vertical, and optical axis directions of the camera corresponding to the origin of the second side. The optical axis direction points from the camera corresponding to the origin of the second side to the measured area of ​​the object. The horizontal and vertical directions are extracted from the factory calibration parameters of the camera corresponding to the origin of the second side or the relative rotation matrix obtained in S2. The relative translation direction of the first side is projected onto the local coordinate system to obtain the projection direction vector.

[0045] Calculate the angle between the projection direction vector and the relative translation direction of the second side. The angle is calculated as: θ = arccos(|vp·u|); where θ is the angle between the projection direction vector and the relative translation direction of the second side, vp is the projection direction vector, u is the unit vector obtained by transforming the relative translation direction of the second side, · is the vector dot product operation, || is the absolute value operation, and arccos is the inverse cosine function. The dimension of the angle θ is degrees. This angle is the consistency deviation between adjacent first and second sides. Perform the angle calculation sequentially on each pair of adjacent sides in the closed loop to obtain a sequence of consistency deviations for all adjacent sides in the closed loop. The consistency deviation sequence is an ordered arrangement of angle values, and the order of the arrangement is consistent with the direction of the closed loop.

[0046] The establishment of the local coordinate system utilizes the optical geometric parameters of the camera corresponding to the starting point of the second side. The horizontal, vertical, and optical axis directions of each camera can be extracted using the camera's factory calibration parameters or the relative rotation matrix obtained in S2. Projecting the relative translation direction of the first side into the local coordinate system transforms the relative translation direction corresponding to the first side from the local reference frame defined by the camera itself to the observation reference frame of the second side camera, allowing the angle comparison between the two relative translation directions from different camera pairs under a unified reference frame. The physical basis for using this projection operation instead of directly calculating the spherical angular distance between the two relative translation direction vectors is that the relative translation direction of the first side represents the directional relationship between the two cameras at the two ends of the first side, and this directional relationship is not represented differently in different reference frames. After transforming it to the local reference frame of the starting camera of the second side, both relative translation directions are expressed from the observation perspective of the same camera. The angle at this time reflects the directional difference between the two relative translation directions when observed from the starting camera of the second side, and the dimensional consistency is naturally satisfied.

[0047] Identifying continuous edge segments with continuously increasing consistent deviations within a closed loop. A continuous edge segment with continuously increasing consistent deviations refers to a sequence of adjacent edges within the closed loop where the consistent deviation value increases strictly monotonically as the sequence progresses, and the consistent deviation of each pair of adjacent edges is greater than that of the previous pair. The length of the continuous edge segment is at least two pairs of adjacent edges, i.e., at least three edges are involved. The specific method for identifying continuous edge segments with continuously increasing consistent deviations is as follows: traverse the consistent deviation sequence within the closed loop. Starting from the first consistent deviation value, compare adjacent consistent deviation values ​​sequentially. If a subsequent consistent deviation value is consistently greater than the previous one, the sequence of adjacent edges satisfying this condition is identified as a continuous edge segment. If, during the traversal, a consistent deviation value is found that is not greater than the previous one, the identification of the current continuous edge segment is terminated, and the search for a new continuous edge segment restarts from the point of interruption. A closed loop may contain zero, one or more consecutive edge segments. The number of consecutive edge segments depends on the distribution of the consistency deviation within the closed loop.

[0048] For identified continuous edge segments, the edge corresponding to the adjacent edge that causes the largest increase in consistency deviation is marked as a conflict-injecting edge. The increase in consistency deviation is the difference in consistency deviation between two adjacent pairs of edges in the continuous edge segment, calculated by subtracting the consistency deviation of the previous pair from the consistency deviation of the later pair of adjacent edges. The position with the largest increase in consistency deviation corresponds to the inflection point with the largest rate of change in consistency deviation. The accumulation rate of consistency deviation is fastest at the inflection point, making it the most likely location to introduce scale inconsistencies. The edge corresponding to an adjacent edge refers to the latter of the two adjacent edges, i.e., the edge located later in the circumferential direction. Marking the edge corresponding to the adjacent edge with the largest increase in consistency deviation in a continuous edge segment as a conflict-injecting edge implies that, based on the cumulative deviation pattern of the closed loop, the conflict-injecting edge is the key edge causing scale conflicts within the closed loop, and the relative translation vector corresponding to the conflict-injecting edge may contain unreliable scale information.

[0049] Both conflict-injected edges and scale-transfer bottleneck edges are labeled as brittle edges. The resulting set of brittle edges comprises two types: scale-transfer bottleneck edges, identified topologically as vulnerable channels for scale transfer; and conflict-injected edges, identified as sources of scale inconsistencies from the perspective of geometric deviation propagation. Both types of edges severely impact the scale consistency of the global translation solution. In subsequent step S5, the magnitudes of the relative translation vectors for these two types of brittle edges will be adjusted to ensure that the scale of the 3D point cloud tends to be consistent throughout the entire camera network.

[0050] In the specific implementation of the embodiment corresponding to S5, for each brittle edge marked in S4, the magnitude of the relative translation vector of the brittle edge is adjusted based on the scale reference determined by non-brittle edges that share a camera with the brittle edge and are not marked as brittle edges, to obtain a corrected relative translation vector. Brittle edges include two types of edges: scale transfer bottleneck edges and conflict injection edges. The relative translation directions of these two types of edges have been calculated in S2. However, since the camera pairs corresponding to these two types of edges are judged to have unreliable direction consistency in S2 or identified as sources of scale inconsistencies in S4, the magnitude of the relative translation vector of these two types of edges may have large errors. Therefore, it is necessary to adjust the magnitude of the relative translation vector of these two types of edges. Non-brittle edges refer to edges that are not marked as brittle edges in the camera view diagram. The relative translation vector of non-brittle edges has more reliable magnitude information because the camera pairs corresponding to non-brittle edges are judged to have reliable direction consistency in S2 and are not identified as conflict injection edges in S4. The physical basis for repairing the unreliable modulus of brittle edges using reliable modulus information of non-brittle edges is that in a multi-view machine vision system, when multiple camera pairs sharing the same camera observe the same object and the same area being measured, the translation scale between the multiple camera pairs should be consistent, because the spatial distance between the same camera and different adjacent cameras changes continuously within the same local area, and there are no abrupt jumps.

[0051] For each brittle edge, identify the non-brittle edges that share a camera with the brittle edge and are not marked as brittle. A brittle edge connects two camera nodes, which are the first-end camera and the second-end camera of the brittle edge, respectively. Non-brittle edges that share a camera with a brittle edge are those edges in the camera view graph that are connected to the first-end camera of the brittle edge and are not marked as brittle, and those edges that are connected to the second-end camera of the brittle edge and are not marked as brittle. The specific method for identifying non-brittle edges is as follows: traverse all edges in the camera view graph connected to the first-end camera of the brittle edge, and filter out those edges that are not marked as brittle, as the candidate set of non-brittle edges; similarly, traverse all edges in the camera view graph connected to the second-end camera of the brittle edge, and filter out those edges that are not marked as brittle, also as the candidate set of non-brittle edges. From the set of candidate non-brittle edges, the candidate non-brittle edge that shares a camera with the brittle edge and has the highest reliability of the magnitude of the relative translation vector is selected as the final non-brittle edge. The reliability of the magnitude of the relative translation vector is evaluated by the number of feature matching points corresponding to the candidate non-brittle edge and the spatial distribution uniformity of the feature matching points. The more feature matching points and the higher the spatial distribution uniformity, the higher the reliability of the magnitude of the relative translation vector. For example, when the camera at the first end of the brittle edge is connected to two candidate non-brittle edges, one of which corresponds to 120 pairs of feature matching points with a spatial distribution uniformity of 75%, and the other of which corresponds to 40 pairs of feature matching points with a spatial distribution uniformity of 35%, then the candidate non-brittle edge with 120 pairs of feature matching points and a spatial distribution uniformity of 75% is selected as the final non-brittle edge. If there are no edges not marked as brittle edges among the edges connected to the first and second cameras of the brittle edge, then there is no reliable scale reference in the local regions where the two cameras of the brittle edge are located. In this case, the magnitude of the relative translation vector of the brittle edge is kept as the original value calculated in S2 and is not adjusted.

[0052] The magnitude of the relative translation vector of the non-brittle edge is extracted as the scale reference. The relative translation vector of the non-brittle edge is recorded on the edge of the camera view in S2. The relative translation vector consists of two parts: the relative translation direction and the magnitude of the relative translation vector. The relative translation direction represents the directional relationship between the cameras at both ends of the non-brittle edge, and the magnitude of the relative translation vector represents the estimated spatial distance between the cameras at both ends of the non-brittle edge. The relative translation vector of the non-brittle edge is read from the camera view, decomposed into a direction component and a magnitude component, and the magnitude component is extracted as the scale reference. The scale reference is a positive scalar value, with units of length, and its value reflects the actual spatial distance between the cameras at both ends of the non-brittle edge. Since the camera pairs corresponding to non-brittle edges are determined to be directionally consistent and reliable in S2, the number of feature matching points corresponding to non-brittle edges is sufficient and spatially uniform, the error in estimating the magnitude of the relative translation vector of non-brittle edges is small, the scale benchmark has high reliability, and it can provide an effective scale reference for adjusting the magnitude of brittle edges.

[0053] Keeping the direction of the relative translation vector of the brittle edge unchanged, the magnitude of the relative translation vector of the brittle edge is replaced with the scale reference to obtain the corrected relative translation vector. The direction of the relative translation vector of the brittle edge is the relative translation direction of the brittle edge calculated in S2. The relative translation direction is a unit vector, reflecting the translation direction relationship between the cameras at both ends of the brittle edge. The original magnitude of the relative translation vector of the brittle edge is calculated together with the relative translation direction in S2, but since the brittle edge is marked as brittle, the original magnitude is considered unreliable. The corrected relative translation vector is obtained by combining the relative translation direction of the brittle edge as the corrected direction and the scale reference as the corrected magnitude. The direction of the corrected relative translation vector is the same as the relative translation direction of the brittle edge, and the magnitude of the corrected relative translation vector is equal to the scale reference. The modified relative translation vector retains the original orientation information of the brittle edge and repairs the magnitude of the brittle edge by using the reliable scale information of the non-brittle edge, so that the modified brittle edge can provide a scale-consistent translation constraint in the global translation solution.

[0054] By using the scale reference of non-brittle edges to repair the module length of brittle edges, the problem of topological breakage in the camera view graph caused by directly removing all brittle edges is avoided. If brittle edges are directly removed, the two camera groups connected by the scale transfer bottleneck edge will completely lose scale connection, and the scale of the point clouds reconstructed by the two camera groups will be completely independent and cannot be unified. If conflict injection edges are directly removed, the missing scale constraint in the closed loop will lead to a decrease in loop closure performance. By retaining the orientation information of brittle edges and only replacing the module length, the scale error introduced by brittle edges is eliminated while maintaining the topological integrity of the camera view graph, thereby obtaining a scale-consistent global position solution in the subsequent global translation solution of S6.

[0055] In the specific implementation of the embodiment corresponding to S6, the relative translation direction recorded in the camera view image for non-brittle edges not marked as brittle edges is extracted. The relative translation direction of the non-brittle edge is combined with the original relative translation magnitude of the non-brittle edge to form the relative translation vector of the non-brittle edge. The relative translation direction recorded on each edge in the camera view image is calculated in S2, and the relative translation direction is a unit vector. For each non-brittle edge, the relative translation direction of the non-brittle edge is read from the camera view image, and the original relative translation magnitude of the non-brittle edge is also read from the camera view image. The original relative translation magnitude is calculated synchronously with the relative translation direction in S2, and the original relative translation magnitude is recorded in the attributes of the corresponding edge in the camera view image.

[0056] The relative translation vector of a non-brittle edge is obtained by multiplying the relative translation direction of the non-brittle edge by its original relative translation magnitude. The product is the relative translation vector of the non-brittle edge. The relative translation vector of a non-brittle edge is a three-dimensional vector. The direction of the relative translation vector is equal to the relative translation direction of the non-brittle edge, and the magnitude of the relative translation vector is equal to the original relative translation magnitude of the non-brittle edge. The relative translation vector of the non-brittle edge matches the characteristic that the non-brittle edge was determined to have reliable directional consistency in S2 and was not marked as a brittle edge in S4. The direction and magnitude information of the relative translation vector of the non-brittle edge remain unchanged from their original calculated values.

[0057] The relative translation vectors of non-brittle edges, the corrected relative translation vectors, and the relative rotations are used as a constraint set. A global translation solution is performed on this constraint set to obtain the global position of each camera. The constraint set contains three types of constraints: Type 1 is the relative translation vector of non-brittle edges, providing a translation constraint unaffected by brittle edges and maintaining the original calculated magnitude; Type 2 is the corrected relative translation vector, obtained in S5 by keeping the direction of the relative translation vector of brittle edges unchanged and replacing its magnitude with a scale reference, providing a magnitude-corrected translation constraint; Type 3 is the relative rotation, calculated synchronously with the relative translation direction in S2, providing rotational constraints between camera pairs. Each constraint in the constraint set corresponds to an edge in the camera view graph, and the constraint takes the form of a relative pose relationship between two camera nodes, defined jointly by relative rotation and relative translation.

[0058] The global translation solution is achieved by minimizing the global translation error function, which is: E=Σ||Rijti-tj|| 2Where E is the global translation error, Rij is the relative rotation from the i-th camera to the j-th camera in the camera pair, ti is the global position of the i-th camera, tj is the global position of the j-th camera, Σ represents the summation of all constraints in the constraint set, || || 2 This represents the square of the Euclidean distance between the vectors. The output of the global translation solution is: t = argminE; where t is the set of global positions of all cameras, argmin represents the t that minimizes E, and E is the global translation error. The global position of each camera is a three-dimensional vector, representing the spatial coordinates of the camera's optical center in a unified world coordinate system. Since the corrected relative translation vectors included in the constraint set have been scaled, the scale relationship between camera groups at both ends of the brittle edge is consistent with the scale relationship established by the non-brittle edge. Therefore, the global positions of all cameras obtained from the global translation solution satisfy scale consistency across all cameras.

[0059] The global translation solution utilizes the magnitude correction results for brittle edges in S5, achieving scale uniformity between brittle and non-brittle edges. Existing techniques treat the relative translation vectors of all camera pairs as equal inputs to the global translation solution. Because the magnitude of the relative translation vectors corresponding to brittle edges is unreliable, the global position output by the global translation solution exhibits scale inconsistencies between different camera groups connected by brittle edges. In this new approach, the magnitude of the relative translation vectors corresponding to brittle edges is replaced by the scale reference of non-brittle edges. The translation constraints provided by brittle edges in the global translation solution are consistent in scale with those of non-brittle edges. The scales of different camera groups are coordinated through a unified scale system established by both brittle and non-brittle edges.

[0060] Triangulation is performed based on the global positions of each camera and the feature matching points between the images of the object under test to generate a 3D point cloud of consistent scale. For each pair of feature matching points between the images of the object under test, the feature matching point corresponds to the same physical point on the measured area of ​​the object. Based on the global positions and relative rotations of the two cameras corresponding to the image of the object under test where the feature matching point is located, a triangulation algorithm is used to calculate the 3D spatial coordinates of the physical point. The input to the triangulation algorithm is the projection matrix of each of the two cameras. The projection matrix is ​​obtained by combining the camera's global position, relative rotation, and camera intrinsic parameter matrix. The camera intrinsic parameter matrix includes the focal length and principal point coordinates, and is obtained through pre-calibration of the cameras. The output of the triangulation algorithm is the 3D spatial coordinates of the physical point corresponding to the feature matching point. Triangulation is performed on all feature matching points, and the set of 3D spatial coordinates corresponding to all feature matching points constitutes the 3D point cloud. Because the global positions of each camera obtained by global translation solution have scale consistency, the 3D point cloud generated by triangulation has a consistent scale across the reconstructed areas of different camera groups. The overall scale of the 3D point cloud is consistent with the scale reference, and the 3D point cloud can accurately reflect the real spatial shape and size information of the measured object and the measured area.

[0061] It is worth noting that the relative translation direction calculated in S2 is a unit vector, reflecting only the direction information of the translation but not the length information. In S5, by replacing the magnitude of the relative translation vector of the brittle edge with a scale reference, a corrected relative translation vector containing scale information is obtained. The corrected relative translation vector contains both direction and magnitude information. The relative translation vector of the non-brittle edge is also obtained by combining the relative translation direction with the original relative translation magnitude, containing both direction and magnitude information.

[0062] Example 2: Figure 2 A schematic diagram of the high-precision 3D measurement system based on multi-view machine vision of the present invention is provided. The high-precision 3D measurement system based on multi-view machine vision includes: The image acquisition module is used to acquire images of the object under test simultaneously from different perspectives by multiple cameras through a multi-view machine vision system. The view graph construction module is used to calculate the relative rotation and relative translation directions for each camera pair that has a co-view relationship between the images of the measured object. Based on the consistency of the translation direction between the camera pair and the surrounding camera pairs, it filters the camera pairs to be constructed and constructs a camera view graph with the camera as the node and the relative rotation and relative translation directions of the camera pairs to be constructed as the edges. The bottleneck edge recognition module is used to identify edges in the camera view that connect different camera groups and have no alternative paths, and to pass these edges as the scale. The brittle edge marking module is used to calculate the consistency deviation of the relative translation direction of adjacent edges in a closed loop containing a scale transfer bottleneck edge. The edge with the largest increase in deviation among the continuous edge segments with monotonically increasing consistency deviation is marked as a conflict injection edge, and the conflict injection edge and the scale transfer bottleneck edge are jointly marked as brittle edges. The vector correction module is used to adjust the magnitude of the relative translation vector of each brittle edge based on the scale reference determined by the non-brittle edges that share the camera with the brittle edge, so as to obtain the corrected relative translation vector. The point cloud generation module is used to input the relative translation vectors, corrected relative translation vectors, and relative rotations of non-brittle edges into the global translation solver to obtain the global positions of each camera and generate a 3D point cloud of consistent scale.

[0063] Example 3: The present invention provides a medium on which a program or instruction is stored, and when the program or instruction is executed by a processor, a high-precision three-dimensional measurement method based on multi-view machine vision is realized.

[0064] The calculations involved in the embodiments are all dimensionless numerical calculations, and the preset parameters and thresholds in the calculations are set by those skilled in the art according to the actual situation.

[0065] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0066] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wireless or wired transmission; wired transmission methods include optical fiber, twisted pair, coaxial cable, etc.; wireless transmission includes infrared, microwave, etc. Computer-readable storage media can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0067] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0069] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0070] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0071] If a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0073] In conclusion, the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-precision three-dimensional measurement method based on multi-view machine vision, characterized in that, include: S1: Acquire images of the object under test from multiple cameras simultaneously from different perspectives using a multi-view machine vision system; S2: Calculate the relative rotation and relative translation directions for each camera pair that has a co-view relationship between the images of the object under test. Based on the consistency of the translation directions between the camera pair and the surrounding camera pairs, filter the camera pairs to be constructed and construct a camera view graph with the camera as the node and the relative rotation and relative translation directions of the camera pairs to be constructed as the edges. S3: Identify edges in the camera view graph that connect different camera groups and have no alternative paths, and use them as bottleneck edges for scale propagation; S4: For a closed loop containing a scale transfer bottleneck edge, calculate the consistency deviation of the relative translation direction of adjacent edges within the closed loop. Mark the edge with the largest increase in deviation among the continuous edge segments where the consistency deviation increases monotonically as the conflict injection edge, and mark the conflict injection edge and the scale transfer bottleneck edge together as the brittle edge. S5: For each brittle edge, based on the scale reference determined by the non-brittle edge that shares the camera with the brittle edge, adjust the magnitude of the relative translation vector of the brittle edge to obtain the corrected relative translation vector; S6: Input the relative translation vector, the corrected relative translation vector, and the relative rotation of the non-brittle edges into the global translation solver to obtain the global position of each camera and generate a 3D point cloud of consistent scale.

2. The high-precision three-dimensional measurement method based on multi-view machine vision according to claim 1, characterized in that, A multi-view machine vision system acquires images of the object under test simultaneously from different perspectives using multiple cameras, including: Multiple cameras are arranged around the object being measured, so that the field of view of each camera covers the measured area of ​​the object and the fields of view of adjacent cameras overlap. Send synchronization trigger signals to multiple cameras to control the exposure of multiple cameras at the same time; It receives image data synchronously output from multiple cameras and uses it as an image of the object under test.

3. The high-precision three-dimensional measurement method based on multi-view machine vision according to claim 1, characterized in that, For each camera pair with a co-view relationship among the images of the measured object, calculate the relative rotation and relative translation directions. Based on the consistency of the translation directions between the camera pair and surrounding camera pairs, select the camera pairs to be constructed. Construct a camera view graph with cameras as nodes and the relative rotation and relative translation directions of the camera pairs to be constructed as edges, including: Extract feature matching points between images of the object under test, and calculate the relative rotation and relative translation directions of each camera pair that have a co-view relationship based on the feature matching points; For each camera pair, obtain the neighboring camera pairs that share a camera with that camera pair and whose relative translation direction has been calculated, and construct a local translation direction field from the relative translation directions of the neighboring camera pairs. The relative translation direction of the camera pair is converted into a unit vector. The relative translation direction of each neighboring camera pair in the local translation direction field is converted into a unit vector. The spherical angular distance between the unit vector of the camera pair and the unit vector of each neighboring camera pair is calculated. The average of all spherical angular distances is taken as the direction difference. When the direction difference is less than the preset consistency range, the camera pair is determined as the camera pair to be constructed. Using the camera as a node and the relative rotation and relative translation directions of the camera pair to be constructed as edges, generate a camera view graph.

4. The high-precision three-dimensional measurement method based on multi-view machine vision according to claim 1, characterized in that, Identify edges in the camera view graph that connect different camera groups and have no alternative paths, and use them as scale transfer bottleneck edges, including: Iterate through each edge in the camera view graph and determine whether removing the current edge separates the camera view graph into two unconnected camera groups. If removing the current edge results in the camera view graph being separated into two unconnected camera groups, then the current edge is determined to be an edge connecting different camera groups. For edges identified as connecting different camera groups, check if there are other paths in the camera view graph that connect the cameras at both ends of the edge; If there is no other path connecting the cameras at both ends of the edge, then the edge is identified as a scale transfer bottleneck edge.

5. The high-precision three-dimensional measurement method based on multi-view machine vision according to claim 1, characterized in that, For a closed loop containing a scale transfer bottleneck edge, calculate the consistency deviation of the relative translation direction of adjacent edges within the closed loop. Mark the edge with the largest increase in deviation among continuous edge segments where the consistency deviation monotonically increases as the conflict injection edge. Both the conflict injection edge and the scale transfer bottleneck edge are marked as brittle edges, including: Extract all closed loops containing the scale transfer bottleneck edge from the camera view graph; For each closed loop, calculate the angle between the relative translation directions recorded on the two adjacent sides along the loop direction, as the consistent deviation of the translation directions of the adjacent sides; Identify continuous edge segments within a closed loop where the consistency deviation continues to increase, and mark the edge corresponding to the adjacent edge that causes the largest increase in consistency deviation in the continuous edge segment as a conflict injection edge. Both conflict-injected edges and scale-transfer bottleneck edges are labeled as brittle edges.

6. The high-precision three-dimensional measurement method based on multi-view machine vision according to claim 5, characterized in that, The calculation of the angle between the relative translation directions recorded on two adjacent sides in sequence includes: in the direction of the closed loop, for the adjacent first side and second side, project the relative translation direction of the first side onto the local coordinate system with the origin of the second side as the origin, and calculate the angle between the projection direction and the relative translation direction of the second side.

7. The high-precision three-dimensional measurement method based on multi-view machine vision according to claim 1, characterized in that, For each brittle edge, based on the scale reference determined by the non-brittle edges that share the camera with the brittle edge, the magnitude of the relative translation vector of the brittle edge is adjusted to obtain the corrected relative translation vector, including: For each brittle edge, identify the non-brittle edges that share a camera with it and are not marked as brittle edges; extract the magnitude of the relative translation vector of the non-brittle edges as the scale reference. Keeping the direction of the relative translation vector of the brittle edge unchanged, the magnitude of the relative translation vector of the brittle edge is replaced with the scale reference to obtain the corrected relative translation vector.

8. The high-precision three-dimensional measurement method based on multi-view machine vision according to claim 1, characterized in that, The relative translation vectors of non-brittle edges, the corrected relative translation vectors, and the relative rotations are input into the global translation solver to obtain the global positions of each camera, generating a consistent-scale 3D point cloud, including: Extract the relative translation direction of non-brittle edges that are not marked as brittle edges in the camera view graph, and combine the relative translation direction of the non-brittle edges with the original relative translation magnitude of the non-brittle edges to form the relative translation vector of the non-brittle edges. The relative translation vector, the corrected relative translation vector, and the relative rotation of the non-brittle edges are used as a set of constraints. Global translation is solved on the set of constraints to obtain the global position of each camera. Triangulation is performed based on the global position of each camera and the feature matching points between the images of the object being measured to generate a 3D point cloud of consistent scale.

9. A high-precision three-dimensional measurement system based on multi-view machine vision, used to implement the high-precision three-dimensional measurement method based on multi-view machine vision as described in any one of claims 1-8, characterized in that, include: The image acquisition module is used to acquire images of the object under test simultaneously from different perspectives by multiple cameras through a multi-view machine vision system. The view graph construction module is used to calculate the relative rotation and relative translation directions for each camera pair that has a co-view relationship between the images of the measured object. Based on the consistency of the translation direction between the camera pair and the surrounding camera pairs, it filters the camera pairs to be constructed and constructs a camera view graph with the camera as the node and the relative rotation and relative translation directions of the camera pairs to be constructed as the edges. The bottleneck edge recognition module is used to identify edges in the camera view that connect different camera groups and have no alternative paths, and to pass these edges as the scale. The brittle edge marking module is used to calculate the consistency deviation of the relative translation direction of adjacent edges in a closed loop containing a scale transfer bottleneck edge. The edge with the largest increase in deviation among the continuous edge segments with monotonically increasing consistency deviation is marked as a conflict injection edge, and the conflict injection edge and the scale transfer bottleneck edge are jointly marked as brittle edges. The vector correction module is used to adjust the magnitude of the relative translation vector of each brittle edge based on the scale reference determined by the non-brittle edges that share the camera with the brittle edge, so as to obtain the corrected relative translation vector. The point cloud generation module is used to input the relative translation vectors, corrected relative translation vectors, and relative rotations of non-brittle edges into the global translation solver to obtain the global positions of each camera and generate a 3D point cloud of consistent scale.

10. A medium, characterized in that, The medium stores a program or instructions, which, when executed by a processor, implement the high-precision three-dimensional measurement method based on multi-view machine vision as described in any one of claims 1-8.

Images