A mobile swing-type multi-camera-based vision measurement method

Abstract

The application provides a mobile swing type multi-camera based visual measurement method, and belongs to the field of precise visual measurement of spatial three-dimensional coordinates of intelligent manufacturing of super large size targets. In view of the problems of the existing visual measurement method, such as the large number of images, the complex network structure, the easy divergence of calculation, the low efficiency, the precision reduction and the like in the measurement of the spatial three-dimensional coordinates of the super large target, the application is based on a mobile truss, an electrical control cabinet and a swing type multi-camera mobile visual measurement device of multiple visual measurement units. The light source and the motion device are added on the basis of the industrial camera, so that the camera has the swing motion function. Then, the mobile truss is combined to build a mobile measurement system. For the large size target, the automatic visual measurement process is realized. Based on the global rapid orientation method of a large number of image fusion, the estimation method based on the local to global optimization strategy is researched, and the efficient and high-precision global image orientation is realized.

Description

Technical Field

[0001] This invention relates to the field of precision visual measurement technology for three-dimensional spatial coordinates in intelligent manufacturing, and in particular to a visual measurement method based on a moving sweeping multi-camera system. Background Technology

[0002] With the iterative development of industrial manufacturing technology, my country is transforming from a manufacturing giant to a manufacturing powerhouse, rapidly moving towards a digital manufacturing model. Geometric appearance control and inspection of industrial products are crucial means to achieve "quality control" in the manufacturing powerhouse. Coordinate measurement provides an important technical means for geometric appearance control and inspection of industrial products. The continuous improvement of the performance and output of new high-end equipment has placed high-precision, wide-range, and high-efficiency technical requirements on large-size measurement technology. Photogrammetry, based on image technology, has the characteristics of high efficiency, rich information perception, and scalable measurement range, making it an important means to realize coordinate measurement of large-size industrial objects.

[0003] Currently, coordinate measurement of large targets, especially ultra-large long linear (narrow) targets such as ship decks, high-speed railway tracks, and particle accelerators, faces new challenges due to the complexity of the environment and the shape of the target. New demands are being placed on measurement accuracy, measurement range, automation, and intelligence. Currently, mobile single-camera measurement requires manual operation and is difficult to adapt to automated measurement needs. Fixed multi-camera measurement is limited by fixed measurement angles and camera fields of view, with the measurement range proportional to the number of cameras, making it difficult to apply to the measurement of ultra-large targets. Furthermore, target measurement yields a large number of images; directly orienting all images can easily lead to slow or divergent nonlinear optimization solutions, and may even result in orientation failure.

[0004] Commonly used visual measurement methods often suffer from poor measurement efficiency and low automation when measuring large targets. They also suffer from computational divergence and low efficiency due to the large number of photographic images and complex network structures. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention aims to propose a measurement device for large-sized targets, a multi-camera vision measurement device capable of measuring three-dimensional coordinates of targets with a large measurement range, high measurement accuracy, and high degree of automation and intelligence, thereby solving the problems existing in the background art.

[0006] The method of this invention adds a light source and motion device to an industrial camera, enabling the camera to have a sweeping motion function. At the same time, it is combined with a mobile truss to build a mobile measurement system. For large-sized targets, it realizes a visual measurement process with automated functions. It also proposes a global fast orientation method for a large number of images based on sub-image fusion, and studies an estimation method based on a local to global optimization strategy, realizing efficient and high-precision global image orientation.

[0007] The above-mentioned technical objective of the present invention is achieved through the following technical means:

[0008] A visual measurement method based on a moving sweeping multi-camera system includes the following steps:

[0009] S1: Arrange multiple target point reflective markers and target mounts for placing the laser tracker target ball and photogrammetry ball in the target area, and use the target point reflective markers and target mounts as control points;

[0010] S2: Arrange reflective markers on the target area and use the reflective markers as the target points for measurement;

[0011] S3: After measuring the laser tracker target ball with the laser tracker, replace the laser tracker target ball with a photogrammetry ball, and use visual measurement methods to measure the global control point position coordinates to establish a global coordinate system;

[0012] S4: Photogrammetry is performed sequentially on all reflective markers within the target area using a mobile visual measurement device based on a sweeping multi-camera system to obtain measurement images;

[0013] S5: The measured images in S4 above are quickly oriented using a local map fusion-based image fast orientation method. Local map construction and local map fusion are performed respectively to achieve global map construction and obtain all image spatial attitude orientation parameters.

[0014] S6: Based on the image space attitude orientation parameters obtained in S5 above, the three-dimensional point coordinates of the target are calculated using bundle adjustment based on the target reflective markers and their matching relationships.

[0015] To achieve the above objectives, a further preferred embodiment of the present invention is as follows: the mobile vision measurement device based on a sweeping multi-camera system described in step S4 includes a mobile truss, an electrical control cabinet, and multiple vision measurement units. The multiple vision measurement units are mounted on the crossbeams of the mobile truss. Each vision measurement unit includes a camera, a motion mechanism, and a light source. The vision measurement unit is electrically connected to a synchronous triggering and power supply unit.

[0016] Furthermore, the control points in S1 above are arranged in the measurement area according to the principle of uniform distribution.

[0017] A further preferred embodiment of the present invention is as follows: The camera mounted on the movable truss in S4 above realizes sweeping photography and changes the camera measurement angle through a motion mechanism. During measurement, the movable truss moves along the target area to different positions and performs sweeping measurements at each position in sequence.

[0018] A further preferred embodiment of the present invention is as follows: S5 specifically includes the following steps:

[0019] S51: Perform image processing on all images to identify and extract feature point information;

[0020] S52: Based on the overlap between the measured images, construct a local map X by region. L ;

[0021] S53: After constructing local maps from all images, a global map X is built by sequentially fusing the local maps using a sub-map fusion algorithm. G This yields all the image space attitude orientation parameters.

[0022] A further preferred embodiment of the present invention is as follows: S52 specifically includes the following steps:

[0023] S521: Define the pose of the starting image as the initial pose of the local map, define the pose of the last image as the ending pose of the local map, and the last image of the local map will be used as the starting image of the next local map.

[0024] Establish the objective function using collinearity equations:

[0025]

[0026] In the formula, p ij Let P be the image point coordinates of the target point on the image. i M represents the corresponding three-dimensional point coordinates. i =(R i ,t i Let Q be the map pose and Q be the covariance matrix.

[0027] S522: Using the feature point information extracted in S51 above, calculate the fundamental matrix F between images according to the eight-point method, and obtain the essential matrix E using the fundamental matrix F and camera intrinsic parameters.

[0028] E=K T FK (2)

[0029] The pose M′=R[I|T] between any two images can be solved using the essential matrix E;

[0030] S523: Obtain the relative pose of a pair of images through S522 above, and measure the three-dimensional feature points P of the pair of images. i In each image, there is an equation p i =MX i ,p i ′=M′X i Where M and M′ are the pose matrices corresponding to two images, and the set yields the information about P. i The linear equation system AP i =0;

[0031] For each image point, by eliminating the homogeneous scaling factor in the equation using the vector cross product, we obtain the following three equations, two of which are linearly independent:

[0032]

[0033] Where, m iT Let P be the row vector corresponding to the pose matrix M. The above equation is linear on the P component, from which we can obtain AP. i =0 system of equations:

[0034]

[0035] By performing SVD decomposition on A in the two equations obtained from each image, the unit singular vector corresponding to the minimum singular value is the solution P;

[0036] S524: Combining the objective function from step S521, nonlinear optimization is performed using the Gauss-Newton method, and the local map feature X is obtained using the bundle adjustment method. L Information matrix I of local map L .

[0037] A further preferred embodiment of the present invention is as follows: S53 specifically includes the following steps:

[0038] S531: Complete the construction of all local maps using the above S52, and use... The following formula represents the estimation of the local map state vector:

[0039]

[0040]

[0041] In the formula, This represents the state vector estimation for each image that makes up the local map. This represents the pose of the image in the local map, and the estimation of the local map state vector. The coordinate system is established on the pose of the initial image in the atlas;

[0042] S532: Global map features are denoted as X G (k), where k represents a subgraph that has been merged with a number of elements 1, ..., k. The specific expression is written as:

[0043]

[0044] In the formula, It is local Figure 1 The coordinates of the included feature points in the global coordinate system. It is contained in the local area Figure 2 In but not included in the local Figure 1 The coordinates of the feature points in the global coordinate system. It is included in the local map k but not included in the local map. Figure 1 The coordinates of the feature points in k-1 in the global coordinate system, and This represents the ending pose of local map i (1≤i≤k) and the starting pose of local map i+1 (1≤i≤k-1);

[0045] When performing sub-graph fusion, the information vector i(k) and the information matrix I of the kth local map are required. L (k), i(k) and I L (k) and global state vector estimation The following relationship exists between the covariance matrix P(k) and the corresponding covariance matrix:

[0046]

[0047] S533: Using a sub-map fusion algorithm, the features of the constructed local map are sequentially combined. Integrating into global map features X G (k).

[0048] A further preferred embodiment of the present invention is as follows: S533 specifically includes the following steps:

[0049] S5331: Matching feature points;

[0050] (1) Find local map sets that may have overlapping relationships in the constructed global map;

[0051] (2) Find possible feature points to be matched in the local map set;

[0052] (3) Restore the pose at the end of the global map The relevant covariance submatrix is ​​obtained by calculating the covariance matrix P(k) and extracting the required rows.

[0053] (4) After obtaining the state estimate and covariance matrix of the feature points to be matched, the nearest neighbor algorithm is used to match the feature points.

[0054] S5332: Obtain the new information vector i(k) and information matrix I L (k) and I L (k) The matrix L(k) of the Cholesky decomposition;

[0055] (1) Add new feature points to the global map in the global coordinate system. and ending pose

[0056] (2) Estimating pose using a global map State estimation of overlapping local atlases: In the global coordinate system, calculate the initial 3D coordinates of all newly matched feature points and the final pose of local map k+1;

[0057] (3) Add the calculation results to the global map state vector estimation A new global map state vector estimate is obtained.

[0058] (4) For information vector i(k) and information matrix I L Adding zeros to (k) and L(k) increases the matrix dimension, resulting in a new information vector i(k)′ and an information matrix I. L (k)'、L(k)′;

[0059] S5333: Use the least squares method to calculate the global map state vector estimate.

[0060] (1) Calculate the information matrix I(k+1) and the information vector i(k+1):

[0061]

[0062] In the formula, It is matrix H k+1 About X G (k) in Jacobian matrix at z k+1 To end the pose Observed values;

[0063] (2) Calculate the information matrix I L Cholesky decomposition of (k+1):

[0064]

[0065] In the formula, Ω is a non-zero symmetric matrix, derived from... Sure;

[0066] (3) Recover the state vector estimation of the global map

[0067]

[0068] Solving equation (11) above, we get...

[0069] (4) Repeat (1)-(3) until convergence.

[0070] A further preferred embodiment of the present invention is as follows: In S6 above, the coordinates of the control points and the global image space attitude orientation parameters are taken as known quantities, and the three-dimensional coordinates of the measurement target points arranged in S2 above are calculated by the bundle adjustment model.

[0071] By employing the above-described technical means, this invention has the following beneficial technical effects compared to the prior art:

[0072] 1. This invention combines a mobile truss with a multi-camera system, and integrates a host computer, synchronous control box, power supply and other equipment in a control cabinet. For large targets, such as ship structures, high-speed railway tracks and particle accelerators, it realizes automated and convenient high-precision measurement.

[0073] 2. This invention utilizes a motion mechanism to provide the camera with multiple shooting angles, which improves measurement efficiency and measurement range while ensuring measurement accuracy.

[0074] 3. By measuring global control points in the measurement site using a laser tracker and constructing a global coordinate system through an adjustment model, the overall measurement error can be effectively controlled.

[0075] 4. To address the problems of computational divergence and low efficiency caused by the large number of images and complex network structure in sweeping photography, this invention proposes a fast global orientation method for a large number of images based on subgraph fusion and an estimation method based on a local-to-global optimization strategy. Compared with traditional handheld single-camera measurement, this method saves more time. At the same time, the number of images measured by this invention is much higher than that of handheld methods. Handheld methods measure 600-700 images, while this invention measures 1500-2000 images, thus achieving higher accuracy and realizing efficient and high-precision global orientation while reducing errors. Attached Figure Description

[0076] Figure 1 This is a schematic diagram of the structure of the mobile vision measurement device based on a sweeping multi-camera according to the present invention;

[0077] Figure 2 This is a schematic diagram of the measurement of a large target using the mobile vision device based on a sweeping multi-camera according to the present invention.

[0078] Figure 3This is an illustration of the algorithm for global orientation of a large number of images based on subgraph fusion in this invention;

[0079] Figure 4 This is an explanatory diagram of the radius definition for local maps in this invention;

[0080] In the diagram: 1. Movable truss; 2. Electrical control cabinet; 3. Vision measurement unit. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0082] Reference Figure 1-4 This invention proposes a visual measurement method based on a moving sweeping multi-camera system, comprising the following steps:

[0083] S1: Arrange multiple target point reflective markers and target mounts for placing the laser tracker target ball and photogrammetry ball in the target area, and use the target point reflective markers and target mounts as control points;

[0084] S2: Arrange reflective markers on the target area and use the reflective markers as the target points for measurement;

[0085] S3: After measuring the laser tracker target ball with the laser tracker, replace the laser tracker target ball with a photogrammetry ball, and use visual measurement methods to measure the global control point position coordinates to establish a global coordinate system;

[0086] S4: Photogrammetry is performed sequentially on all reflective markers within the target area using a mobile visual measurement device based on a sweeping multi-camera system to obtain measurement images;

[0087] S5: The measured images in S4 above are quickly oriented using a local map fusion-based image fast orientation method. Local map construction and local map fusion are performed respectively to achieve global map construction and obtain all image spatial attitude orientation parameters.

[0088] S6: Based on the image space attitude orientation parameters obtained in S5 above, the three-dimensional point coordinates of the target are calculated using bundle adjustment based on the target reflective markers and their matching relationships.

[0089] Example 1:

[0090] like Figure 1As shown, a mobile vision measurement device based on a sweeping multi-camera system includes a mobile truss 1, an electrical control cabinet 2, and multiple vision measurement units 3. The electrical control cabinet 2 integrates the system's synchronization triggering unit, power supply unit, and host computer. The multiple vision measurement units 3 are mounted on the crossbeams of the mobile truss 1. Each vision measurement unit 3 includes a camera, a motion mechanism, and a light source. The vision measurement units 3 are electrically connected to the synchronization triggering and power supply units. Specifically: the light source enables the camera to measure the reflective characteristics of the target; the motion mechanism drives the camera to perform a sweeping motion through its own rotation, providing multi-angle shooting for the camera; the synchronization triggering and power supply units supply power to the vision measurement units 3 and, in conjunction with the motion mechanism, trigger the camera to acquire images after it moves to the measurement position. The synchronization triggering and power supply units, in conjunction with the motion structure, complete the overall automated measurement; the image data measured by the camera is transmitted to the computer for processing via a wired network.

[0091] Specifically, the vision measurement unit 3 consists of a motion mechanism and a light source, which are connected to an industrial camera through a specially designed camera housing for vision measurement. The motion mechanism drives the camera to perform a sweeping motion by rotating itself, providing the camera with more shooting angles. The synchronous triggering and power supply unit supplies power to multiple units (camera part and light source) of the vision measurement unit 3, and works with the motion mechanism to trigger the camera to perform photogrammetry after the camera moves to the measurement position. The image data measured by the camera is sent to the computer for processing via a wired network.

[0092] like Figure 1 As shown, the camera is installed inside a specially designed camera housing, the light source is installed at the front of the camera and connected to the lens at the camera lens, and the motion mechanism is connected to the top of the camera housing through a rigid connector; a complete set of mobile vision measurement devices based on sweeping multi-camera has only one synchronous triggering and power supply unit. One synchronous triggering and power supply unit has multiple connection ports (8) that can connect multiple cameras. The synchronous triggering and power supply unit is used to expand the measurement range of the camera at a fixed position.

[0093] The motion mechanism uses a servo module and has a rotating mechanism that can rotate around a fixed axis. When the motion mechanism rotates, it can drive the camera part connected by the connecting block to perform a sweeping motion. It is powered separately and directly connected to the host computer.

[0094] The synchronous triggering and power supply unit mainly includes a power supply section and a synchronous triggering section. The internal circuit board is the main component, and the outer casing has a switch, a frequency adjustment knob, and eight aviation plugs. The power supply section can provide a fixed voltage power supply for the camera and the light source. A single synchronous triggering section can synchronously output eight trigger signals at six adjustable frequencies to trigger the camera 1 to take pictures. The synchronous control box of the camera 1 is designed to be cascaded using an external interrupt hardware cascading method, so that the synchronous control box has the function of multi-level series use.

[0095] The synchronous triggering and power supply unit is integrated in the electrical control cabinet 2 and is directly connected to the vision measurement unit 3 via a trigger line. The vision measurement unit 3 supplies power to the light source, the camera, and the trigger signal via branch wiring harnesses. Figure 1 As shown, the electrical control cabinet 2 is equipped with control buttons, which can be used by the operator to control the device during measurement.

[0096] Example 2:

[0097] The mobile vision measurement device based on a sweeping multi-camera system from Embodiment 1 is applied to the 3D measurement of large-sized targets, such as... Figure 2 As shown, the specific measurement method includes the following steps:

[0098] S1: As Figure 2 As shown, multiple target reflective markers and target mounts for placing the laser tracker target ball and photogrammetry ball are arranged in the test area according to the principle of uniform distribution. The above part serves as control points. The laser tracker is located outside the measurement area by default (the position of the target mount can be observed). The role of the laser tracker is to provide accurate three-dimensional coordinate references for other target reflective markers in the control field.

[0099] Among them: the target is Figure 2 The small cylinders in the target reflective markers include coded points with numbered information. Figure 2 (The black squares in the middle) and the circular non-coded points without numbering information ( Figure 2 (a single point in the middle);

[0100] S2: Also place reflective markers on the target and use these reflective markers as target points for measurement.

[0101] S3: After measuring the target ball with a laser tracker, replace the target ball with a photogrammetry ball, and use visual measurement methods to measure the coordinates of the global control points and establish a global coordinate system O-XYZ.

[0102] The process of measuring the coordinates of control points is as follows: First, a laser tracker target ball is placed inside the target holder. The three-dimensional coordinates of the target ball are measured using the laser tracker. Then, it is replaced with a photogrammetric ball (the coordinates of the target ball are the coordinates of the photogrammetric ball). Then, a visual measurement method (close-range photogrammetry) is used to photograph the reflective markers of the target points in the area to be measured and the photogrammetric ball. The coordinates of the photogrammetric ball are used as constants. The coordinates of the remaining reflective markers of the target points are solved using spatial resection and spatial anterior intersection, thus completing the coordinate measurement of all control points. The photogrammetric ball acts similarly to a reflective marker, providing reflective markers for visual measurement. At the same time, its center can be considered to be consistent with the laser tracker target ball. The points measured by the two methods can be regarded as a single point.

[0103] S4: As Figure 3 As shown, a mobile visual measurement device based on a sweeping multi-camera system sequentially performs photogrammetric measurements on all reflective markers within the target area. During the measurement, the mobile truss 1 moves the device along the target area to different stations, and sequentially performs sweeping measurements at each station.

[0104] S5: As Figure 3 As shown, a rapid orientation method based on local map fusion is used to quickly orient the measured images. Local map construction and local map fusion are performed separately to achieve global map construction and obtain all spatial pose orientation parameters of the images.

[0105] Specifically, image processing is performed on all images, including image preprocessing, edge extraction, circle center extraction, image feature point matching, and coded feature recognition. Feature point information is identified and extracted, such as all coded points with numbered information and circular non-coded points without numbered information in the images.

[0106] like Figure 3 As shown, local maps X will be constructed by region based on the overlap relationship between the measured images. L ;

[0107] Specifically, the pose of the starting image is defined as the initial pose of the local map, and the pose of the last image is defined as the ending pose of the local map. At the same time, the last image of the local map will be used as the starting image of the next local map.

[0108] Establish the objective function using collinearity equations:

[0109]

[0110] In the formula, p ij Let P be the image point coordinates of the target point on the image. i M represents the corresponding three-dimensional point coordinates. i =(R i ,t i Let Q be the map pose and Q be the covariance matrix, which describes the relative uncertainty of the observations.

[0111] Taking the processing of two images as an example, using the feature point information extracted above, the fundamental matrix F between the images is calculated according to the eight-point method. Then, the essential matrix E is obtained using the F matrix and the camera intrinsic parameters.

[0112] E=K T FK (2)

[0113] After obtaining the essential matrix E, the pose between the two images is then solved, M′=R[I|T].

[0114] After obtaining the relative pose of a pair of images, for the three-dimensional feature point P measured from the pair of images... i In each image, there is an equation p i =MX i ,p i ′=M′X i Where M and M′ are the pose matrices corresponding to two images, the set yields information about P. i The linear equation system AP i =0.

[0115] For each image point, the homogeneous scaling factor in the equation is eliminated by the cross product of vectors, resulting in three equations, two of which are linearly independent.

[0116] Taking the first image as an example, p i ×(MP i The equation ) = 0 corresponds to the following three equations:

[0117]

[0118] Where, m iT Let P be the row vector corresponding to the pose matrix M. The above equation is linear on the P component, from which we can obtain AP. i =0 system of equations:

[0119]

[0120] Each image yields two equations. SVD decomposition is performed on A, and the unit singular vector corresponding to the minimum singular value is the solution P. Using bundle adjustment and the aforementioned objective function, nonlinear optimization is performed using the Gauss-Newton method to obtain the local map features X. L And related information matrix I L (The inverse of the covariance matrix).

[0121] like Figure 3 As shown, after constructing a local map using all the images, a global map X is built by sequentially fusing the local maps using a submap fusion algorithm. G This allows us to obtain all the spatial attitude orientation parameters of the images.

[0122] Specifically, after completing the construction of all local maps, use Represents the local map state vector X L The estimation includes feature point information, image pose information, etc.

[0123]

[0124]

[0125] Local map state vector estimation The coordinate system is established on the pose of the initial image of the atlas.

[0126] The global map feature is denoted as X. G (k), where k represents the number of subgraphs that have been merged by a ratio of 1:k. The specific expression is written as:

[0127]

[0128] In the formula, It is local Figure 1 The coordinates of the included feature points in the global coordinate system; It is contained in the local area Figure 2 In but not included in the local Figure 1 The coordinates of the feature points in the global coordinate system; similarly, It is included in the local map k but not included in the local map. Figure 1 The coordinates of the feature points in k-1 in the global coordinate system; while This represents the ending pose of local map i (1≤i≤k) and the starting pose of local map i+1 (1≤i≤k-1).

[0129] When performing sub-graph fusion, the information vector i(k) and the information matrix I of the kth local map are required. L (k), i(k) and I L (k) and global state vector estimation The following relationship exists between the covariance matrix P(k) and the corresponding covariance matrix:

[0130]

[0131] The sub-map fusion algorithm sequentially integrates the features of the constructed local map. Integrating into global map features X G (k).

[0132] Specifically, taking the fusion of local map k+1 as an example, the specific algorithm for subgraph fusion is introduced step by step:

[0133] Step 1: Matching Feature Points

[0134] (1) Find local map sets that may overlap in the constructed global map. Define the radius of each local map as the maximum distance from the origin of its initial pose to the feature points measured within that local map, as follows: within the possible estimation error range, in the global coordinate system, if the distance between the origins of the initial poses of two local maps is greater than the sum of the radii of the two maps, then the two local maps are determined to have no overlap; otherwise, the two local maps are determined to have an overlap.

[0135] (2) Find possible feature points to be matched in the local map set: within the allowable error range, if the feature point is within the local map set at position k+1 of the starting pose. If the distance to a point is less than the radius of the local map k+1, then the point is considered a possible feature point to be matched and is added to the set of points to be matched.

[0136] (3) Restore the pose at the end of the global map The relevant covariance submatrix is ​​obtained by calculating the covariance matrix P(k) and extracting the required rows.

[0137] (4) After obtaining the state estimate and covariance matrix of the feature points to be matched, the nearest neighbor algorithm is used to match the feature points.

[0138] Step 2: Obtain the new information vector i(k) and information matrix I L (k) and I L (k) The matrix L(k) of the Cholesky decomposition;

[0139] (1) Add new feature points to the global map in the global coordinate system. and ending pose

[0140] (2) Estimating pose using a global map State estimation of overlapping local atlases: In the global coordinate system, calculate the initial 3D coordinates of all newly matched feature points and the final pose of local map k+1;

[0141] (3) Add the calculation results to the global map state vector estimation A new global map state vector estimate is obtained.

[0142] (4) For information vector i(k) and information matrix I L Adding zeros to (k) and L(k) increases the matrix dimension, resulting in a new information vector i(k)′ and an information matrix I. L (k)'、L(k)′;

[0143] Step 3: Use the least squares method to calculate the global map state vector estimate.

[0144] (1) Calculate the information matrix I L (k+1) and information vector i(k+1):

[0145]

[0146] In the formula, It is matrix H k+1 About XG (k) in Jacobian matrix at z k+1 To end the pose Observed values;

[0147] (2) Calculate the information matrix I L Cholesky decomposition of (k+1):

[0148]

[0149] In the formula, Ω is a non-zero symmetric matrix, derived from... Sure;

[0150] (3) Recover the state vector estimation of the global map

[0151]

[0152] Solving equation (11) above, we get...

[0153] (4) Repeat (1)-(3) until convergence.

[0154] S6: Using bundle adjustment, based on the spatial attitude orientation parameters of the image obtained in step S5, calculate the three-dimensional point coordinates of the target based on the reflective markers and their matching relationships.

[0155] Specifically, the coordinates of the control points and the global image orientation parameters are used as known quantities, and the three-dimensional coordinates of the measurement target points arranged in step S2 are calculated using the bundle adjustment model.

[0156] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. For those skilled in the art, the technical solutions and concepts of the present invention can have various equivalent substitutions and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be covered within the scope of protection of the present invention.

Claims

1. A visual measurement method based on a moving sweeping multi-camera system, characterized in that, Includes the following steps: S1: Arrange multiple target point reflective markers and target mounts for placing the laser tracker target ball and photogrammetry ball in the target area, and use the target point reflective markers and target mounts as control points; S2: Arrange reflective markers on the target area and use the reflective markers as measurement target points; S3: After measuring the laser tracker target ball with the laser tracker, replace the laser tracker target ball with a photogrammetry ball, and use visual measurement methods to measure the global control point position coordinates to establish a global coordinate system; S4: Photogrammetry is performed sequentially on all reflective markers within the target area using a mobile vision measurement device based on a sweeping multi-camera system to obtain measurement images; S5: The measured images in S4 above are quickly oriented using a local map fusion-based image fast orientation method. Local map construction and local map fusion are performed respectively to achieve global map construction and obtain all image spatial attitude orientation parameters. S51: Perform image processing on all images to identify and extract feature point information; S52: Construct local maps by region based on the overlap between the measured images. ; S521: Define the pose of the starting image as the initial pose of the local map, define the pose of the last image as the ending pose of the local map, and the last image of the local map will be used as the starting image of the next local map. Establish the objective function using collinearity equations: （1） In the formula, The image coordinates of the target point on the image. For the corresponding 3D point coordinates, Let Q be the map pose and Q be the covariance matrix. S522: Using the feature point information extracted in S51 above, calculate the fundamental matrix F between images according to the eight-point method, and obtain the essential matrix E using the fundamental matrix F and camera intrinsic parameters. （2） Solving the pose between any two images using the essential matrix E ; S523: Obtain the relative pose of a pair of images through S522 above, and measure the three-dimensional feature points of the pair of images. There is an equation in each image. ,in These are the pose matrices corresponding to two images, and the set yields information about... linear equation system ; For each image point, by eliminating the homogeneous scaling factor in the equation using the vector cross product, we obtain the following three equations, two of which are linearly independent: （3） in, Let M be the row vector corresponding to the pose matrix M. The above equation is linear on the P component, and we can obtain... System of equations: （4） By performing SVD decomposition on A in the two equations obtained from each image, the unit singular vector corresponding to the minimum singular value is the solution P; S524: Combining the objective function from step S521, perform nonlinear optimization using the Gauss-Newton method, and obtain local map features using bundle adjustment. Information matrix of local maps ; S53: After constructing local maps from all images, a global map is built by sequentially fusing the local maps using a sub-map fusion algorithm. This yields all the image space attitude orientation parameters; S531: Complete the construction of all local maps using the above S52, and use... The following formula represents the estimation of the local map state vector: （5） （6） In the formula, This represents the state vector estimation for each image that makes up the local map. This represents the pose of the image in the local map, and the estimation of the local map state vector. The coordinate system is established on the pose of the initial image in the atlas; S532: Global map features are denoted as , k represents the subgraphs that have been merged with a number of elements 1, ..., k. The specific expression is written as: （7） In the formula, These are the coordinates of the feature points contained in local map 1 in the global coordinate system. These are the coordinates of feature points contained in local map 2 but not in local map 1, in the global coordinate system. It refers to the coordinates of feature points contained in local map k but not in local maps 1, ..., k-1 in the global coordinate system. This represents the ending pose of local map i (1≤i≤k) and the starting pose of local map i+1 (1≤i≤k-1); When performing subgraph fusion, information vectors are required. and the information matrix of the kth local map , and With global state vector estimation and the corresponding covariance matrix The following relationship exists between them: （8） S533: Using a sub-map fusion algorithm, the features of the constructed local map are sequentially combined. Integrate into global map features ; S6: Based on the spatial attitude orientation parameters of the image obtained in S5 above, the three-dimensional point coordinates of the target are calculated using bundle adjustment based on the target reflection markers and their matching relationships.

2. The visual measurement method based on a moving sweeping multi-camera according to claim 1, characterized in that, The mobile vision measurement device based on sweeping multi-camera described in step S4 includes a mobile truss (1), an electrical control cabinet (2), and multiple vision measurement units (3). The multiple vision measurement units (3) are installed on the crossbeams of the mobile truss. Each vision measurement unit (3) includes a camera, a motion mechanism, and a light source. The vision measurement unit (3) is electrically connected to the synchronous triggering and power supply unit.

3. The visual measurement method based on a moving sweeping multi-camera according to claim 1, characterized in that, The control points in S1 above are arranged in the measurement area according to the principle of uniform distribution.

4. The visual measurement method based on a moving sweeping multi-camera according to claim 1, characterized in that, The camera installed on the mobile truss (1) above uses a motion mechanism to perform sweeping photography and change the camera measurement angle. During measurement, the mobile truss (1) moves along the target area to different stations and performs sweeping measurements at each station in sequence.

5. The visual measurement method based on a moving sweeping multi-camera according to claim 1, characterized in that, The above S533 specifically includes the following steps: S5331: Matching feature points; (1) Find local map sets that may have overlapping relationships in the constructed global map; (2) Find possible feature points to be matched in the local map set; (3) Restore the pose at the end of the global map The relevant covariance submatrix is ​​obtained by calculating the covariance matrix. Extract the required rows to obtain the covariance submatrix; (4) After obtaining the state estimate and covariance matrix of the feature points to be matched, the nearest neighbor algorithm is used to match the feature points; S5332: Obtain a new information vector Information matrix And to Cholesky decomposition of matrices ; (1) Add new feature points to the global map in the global coordinate system. and ending pose ; (2) Estimating pose using a global map State estimation of overlapping local atlases: In the global coordinate system, calculate the initial 3D coordinates of all newly matched feature points and the final pose of local map k+1; (3) Add the calculation results to the global map state vector estimation. This yields a new global map state vector estimate. ; (4) Information vector Information matrix , Adding zeros increases the matrix dimension, resulting in a new information vector. Information matrix , ; S5333: Use the least squares method to calculate the global map state vector estimate. ; (1) Calculate the information matrix and information vector : （9） In the formula, It is a matrix about exist Jacobian matrix at the location, To end the pose Observed values; (2) Calculate the information matrix Cholesky decomposition: （10） In the formula, It is a non-zero symmetric matrix, by Sure; (3) Estimating the state vector of the global map : （11） Solving equation (11) above, we get ; (4) Repeat (1)-(3) until convergence.

6. The visual measurement method based on a moving sweeping multi-camera according to claim 1, characterized in that, In S6 above, the coordinates of the control points and the global image space attitude orientation parameters are taken as known quantities, and the three-dimensional coordinates of the measurement target points arranged in S2 above are calculated by the bundle adjustment model.

Images