Reconfigurable large-size fringe projection optical three-dimensional measurement system

By combining a local measurement system with a global positioning network, along with automated guided vehicles and collaborative robots, the problem of high-precision, flexible optical 3D measurement of large and complex components was solved, achieving scalability and flexibility of the measurement space.

CN119022826AActive Publication Date: 2024-11-26BEIHANG UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411174297.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-26
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Existing stripe projection systems are insufficient to meet the requirements of in-situ, large-size, high-precision, and flexible optical three-dimensional measurement of large and complex components, and traditional systems lack scalability and flexibility.

Method used

A combination of a local measurement system and a global positioning network is adopted. The local measurement system is used for high-precision dense point cloud data acquisition, while the global positioning network is used for large-size sparse point cloud measurement and stitching. The scalability and flexibility of the measurement space are achieved through an automated guided vehicle and a collaborative robot.

Benefits of technology

It achieves high-precision, flexible optical three-dimensional measurement of large and complex components, and can adjust the measurement space according to the measurement object to meet the measurement needs of different product models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119022826A_ABST
    Figure CN119022826A_ABST
Patent Text Reader

Abstract

A reconfigurable large-size fringe projection optical three-dimensional measurement system comprises a local measurement system, an automatic guide vehicle, a collaborative robot and a global positioning subsystem, the local measurement system and the global positioning subsystem are both binocular fringe projection optical three-dimensional measurement systems, the local measurement system is installed at the tail end of the collaborative robot, and the automatic guide vehicle is installed at the tail end of the collaborative robot. The collaborative robot is fixed on the automated guided vehicle, and the global positioning subsystem is networked to form a global positioning network. The method is used for flexible high-precision splicing measurement of large-size parts, a local measurement system is carried by an automatic guided vehicle and a collaborative robot to move to a required viewpoint position, and a cascade measurement space formed by a plurality of global positioning subsystems is used for jointly tracking and positioning a local measurement point cloud; each global positioning subsystem serves as a global positioning network node, flexible configuration layout and online global camera calibration can be realized according to the size, placement and in-situ measurement requirements of a measured object by following a system reconstruction principle, the measurement requirements of large complex components of different models are met, the expansibility of a measurement space is effectively improved, and the measurement efficiency is improved. Therefore, reconfigurable large-size fringe projection optical three-dimensional measurement is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a reconfigurable large-size stripe projection optical three-dimensional measurement system, which can realize high-precision flexible three-dimensional measurement of large and complex components without spraying or marking points. The present invention belongs to the field of machine vision. Background Art

[0002] With the widespread application of large and complex personalized components in industry, their optical 3D measurement has become a difficult problem in modern manufacturing. The fringe projection system has attracted wide attention due to its dense point cloud, high precision, and easy manufacturing integration; however, large and complex components often require in-situ measurement. Due to the different linear dimensions, placement and methods of workpieces, the traditional workstation-type fixed measurement space system lacks scalability and is difficult to meet the in-situ measurement needs of different models of products. This poses a challenge to the measurement space and flexibility of the current fringe projection system.

[0003] Existing systems can be divided into fixed multi-view sensor type and mobile measurement type: the fixed multi-view sensor type combines multiple sensors together to measure objects simultaneously or sequentially from different viewpoints. The relative position of each sensor is fixed and needs to be calibrated in advance. The measurement range is limited and the viewing angle is fixed. The mobile measurement type, with the help of prior information from auxiliary equipment or tracking systems, unifies the single-field local measurement data of different positions through point cloud stitching, but the single tracking system limits the freedom of movement and lacks scalability. Therefore, the existing systems and measurement methods are difficult to meet the needs of industrial manufacturing.

[0004] In view of the above problems, the present invention proposes a reconfigurable large-size fringe projection three-dimensional measurement system, which realizes large-size, high-precision, flexible optical three-dimensional measurement based on the phase matching marker-free stitching principle. Summary of the invention

[0005] In order to solve the problem of in-situ, large-size, high-precision, flexible optical three-dimensional measurement of large and complex components, the present invention proposes a reconfigurable large-size fringe projection three-dimensional measurement system.

[0006] The basic principle of the present invention is to adopt the solution of local measurement system and global positioning network. The local measurement system can obtain local high-precision measurement point cloud, while the global positioning network is used for positioning and splicing. The whole system adopts the independent subsystem design concept, that is, each measurement sensor (local measurement system, positioning subsystem) can work independently. In addition, in order to ensure the scalability of the measurement space, the global positioning network is composed of multiple subsystems, and the size of the measurement space can be changed by increasing or decreasing the number of subsystems and the layout flexibility.

[0007] The technical problem solved by the present invention is to solve the problem of in-situ, large-size, high-precision, flexible optical three-dimensional measurement of large and complex components.

[0008] The technical solution of the present invention is: a reconfigurable large-size fringe projection optical three-dimensional measurement system, which includes a local measurement system, an automatic guided vehicle and a collaborative robot, and a global positioning subsystem:

[0009] The local measurement system is a small field of view, high spatial resolution binocular fringe projection optical 3D measurement system, which is used for single field of view, high precision, dense point cloud data acquisition;

[0010] The automatic guided vehicle and the collaborative robot are used to carry the local measurement system and move the local measurement system to a planned measurement viewpoint position;

[0011] The global positioning subsystem is a large field of view, low spatial resolution binocular fringe projection optical three-dimensional measurement system, which can be used as a binocular fringe projection system for large-scale sparse point cloud measurement, and can also be networked to form a global positioning network to establish a global coordinate system for tracking and locating local measurement field point clouds and transforming them into a global coordinate system to achieve high-precision splicing measurement of large-scale parts.

[0012] The global positioning network is composed of no less than one global positioning subsystem, and the spatial layout of the global positioning subsystem can be adjusted according to the measurement requirements of the measurement object. After the global positioning network is reconfigured, each global positioning subsystem projects grating phase shift fringes and uses the phase splicing method to calculate the rigid body transformation of the adjacent system to achieve global camera online calibration. After the global camera is calibrated online, the cameras of different global positioning subsystems in the global positioning network can be combined into a binocular tracking system to track and locate the local measurement system.

[0013] The reconfigurable large-size fringe projection optical three-dimensional measurement system can perform high-precision splicing measurement of large-size parts, and includes the following steps:

[0014] (1) Configure the system according to the in-situ measurement requirements of the measurement object;

[0015] (2) Online calibration of the global positioning network to establish a global coordinate system;

[0016] (3) The AGV and collaborative robot drive the local measurement system to the initial measurement viewpoint position;

[0017] (4) The local measurement system performs precise three-dimensional measurement, projects grating phase-shifted fringe images, and performs global positioning network tracking and shooting;

[0018] (5) stitching the measurement results of the local measurement system into the global coordinate system;

[0019] (6) The automated guided vehicle and the collaborative robot carrying the local measurement system move to the next measurement viewpoint and repeat steps 4 and 5 until the measurement is completed.

[0020] The system configuration must follow the principle of system reconstruction, reasonably layout the global positioning subsystem, ensure that the common field of view between different global positioning subsystems is not less than one-third of the field of view of a single global positioning subsystem, and that all key dimensions of the measured object are visible to more than two global cameras, the global camera optical axis angle range is 30° to 120°, and the angle between the camera optical axis and the surface normal of the measured object is less than 45°. Set a suitable working distance for the global positioning subsystem to ensure that the local measurement system projected fringes observed by the global camera meet the sampling points of no less than 15 camera pixels per cycle.

[0021] When the local measurement system projects grating phase-shifted fringes, all cameras in the global positioning network shoot synchronously; a pair of tracking cameras in the global positioning network serve as optimal tracking cameras, and a phase stitching method is used to achieve stitching positioning of the local measurement system.

[0022] The optimal tracking camera is selected based on (1) the maximum effective point principle and (2) the optimal structural parameter principle.

[0023] According to the maximum effective point number principle, when multiple cameras track the grating phase shift fringes projected by the local measurement system, in the absolute phase domain, the camera whose absolute phase overlaps more with the absolute phase of the left camera of the local measurement system is selected as the optimal tracking camera.

[0024] The optimal structural parameter principle simulates the random errors of structural parameters of all binocular combinations in the global positioning network. When multiple cameras track the grating phase-shifted fringes projected by the local measurement system, the camera combination with the smallest average error is selected as the optimal tracking camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flowchart of the high-precision splicing measurement of large-sized parts by the reconfigurable large-size fringe projection optical three-dimensional measurement system of the present invention.

[0026] Figure 2 It is a schematic diagram of the reconfigurable large-size fringe projection optical three-dimensional measurement system of the present invention.

[0027] The symbols in the accompanying drawings are:

[0028] 1- global positioning subsystem No. 1, 2- camera No. 1, 3- projector No. 1, 4- camera No. 2, 5- global positioning subsystem No. 2, 6- camera No. 3, 7- projector No. 2, 8- camera No. 4, 9- global positioning subsystem No. 3, 10- camera No. 5, 11- projector No. 3, 12- camera No. 6, 13- global positioning subsystem No. 4, 14- camera No. 7, 15- projector No. 4, 16- camera No. 8, 17- local measurement system, 18- collaborative robot, 19- automatic guided vehicle, 20- measurement object. DETAILED DESCRIPTION

[0029] In order to better understand the present invention, the technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0030] See attached Figure 1 , a reconfigurable large-size fringe projection optical 3D measurement system can be used for high-precision splicing measurement of large-size parts, including the following steps:

[0031] 1. Before starting the measurement, according to the size, position and in-situ measurement requirements of the measured object, follow the principle of system reconstruction, flexibly adjust the number and spatial layout of the global positioning subsystems, ensure that the common field of view between different global positioning subsystems is not less than one-third of the field of view of a single global positioning subsystem, and that all key dimensions of the measured object are visible to more than two global cameras, the global camera optical axis angle range is 30° to 120°, and the angle between the camera optical axis and the surface normal of the measured object is less than 45°. Set a suitable working distance for the global positioning subsystem to ensure that the local measurement system projected fringes observed by the global camera meet the sampling points of no less than 15 camera pixels per cycle.

[0032] See attached Figure 2 A local measurement system and four global positioning subsystems are built, all of which are independent fringe projection binocular systems. Among them, the local measurement system is installed at the end of the collaborative robot and carried by an automatic guided vehicle. The global positioning subsystem is distributed at the corners of a 4m×2m rectangular measurement space.

[0033] 2. The global positioning subsystem forms a global positioning network through networking. Set the coordinate system of camera 1 of global positioning subsystem 1 as the global coordinate system, obtain the rigid body transformation relationship between the coordinate systems of cameras 2 to 8 in the global positioning network and the global coordinate system, and establish a unified global coordinate system.

[0034] Each global positioning subsystem obtains the rigid body transformation between its left and right cameras through binocular calibration, and obtains the extrinsic parameter matrix and its intrinsic parameter matrix. The rigid body transformation between global positioning subsystems is obtained through global camera online calibration. When the kth global positioning subsystem (k = 1, 2, 3, 4) projects the stripe image, the k+1th subsystem synchronously collects the image. and (For brevity, subscript c is omitted) The camera rigid body transformation relationship between the global measurement subsystems is obtained by binocular positioning. Obtained by the phase stitching method, the camera rigid body transformation relationship It can be calculated by the following formula

[0035]

[0036] Thus, by projecting the fringe image through the kth system, the transformation relationships from the three cameras to the left camera of the kth system are obtained respectively. Therefore, the rigid body transformation relationship from each camera coordinate system to the global coordinate system can be obtained by the following formula:

[0037]

[0038] If there is enough common field of view between global positioning subsystem 1 and global positioning subsystem 4, then there is a closed loop in the global positioning network. The fringe pattern is projected by global positioning subsystem 4, and the image is collected synchronously by global positioning subsystem 1 to obtain By utilizing the rigid body transformation relationship at the closed loop, the pose graph is optimized and the overall adjustment is performed to reduce the cumulative error.

[0039] 3. The measurement process is initialized. The automated guided vehicle carries the collaborative robot and the local measurement system and moves to the set initial measurement position. The collaborative robot controls the end movement to move the local measurement system to the initial measurement viewpoint position and waits for the measurement to begin.

[0040] 4. The local measurement system performs precise three-dimensional measurement. The projector projects horizontal and vertical phase-shifted fringe images, and the local measurement system camera and all cameras in the global positioning network shoot synchronously.

[0041] According to the principle of maximum effective points, the first three cameras with the most absolute phase matching points with the left camera of the local measurement system are selected as candidate tracking cameras, and the camera with the most matching points is selected as the main tracking camera. According to the principle of optimal structural parameters, the optimal tracking camera is selected through structural parameter random error simulation.

[0042] The binocular 3D reconstruction process can be expressed as follows:

[0043] X=f stereo (A l ,R l ,t l ,A r ,R r ,t r ,p l ,p r )

[0044] Considering the two-dimensional phase matching error δ~N(0,Σ), the binocular three-dimensional stereo reconstruction process can be expressed as

[0045] X'=f stereo (A l ,R l ,t l ,A r ,R r ,t r ,p l+δ l ,p r +δ r )

[0046] The coordinate error of 3D reconstruction can be expressed as

[0047] e=||X-X'||

[0048] Sample a set of 3D points from the measurement space, and for each sampled point, project it onto all tracking cameras. After adding phase perturbations to the image points, any two tracking cameras can reconstruct the 3D coordinates and calculate the coordinate error, and calculate the average 3D coordinate error of all sampled points. For all tracking camera combinations, select the camera combination with the smallest average error as the optimal tracking camera.

[0049] Three candidate tracking cameras are selected according to the principle of maximum valid points. The main tracking camera and the other two candidate tracking cameras can form a tracking camera combination for splicing positioning. According to the principle of optimal structural parameters, the camera combination with the smallest average error is selected as the optimal tracking camera.

[0050] 5. The local measurement system reconstructs the final high-precision dense measurement point cloud, and the optimal tracking camera in the global positioning network performs marker-free splicing positioning based on phase matching. The pose relationship between the optimal tracking cameras is obtained through the online calibration of the global positioning network in step 2. The rigid body transformation between the local measurement system and the optimal tracking camera can be obtained based on absolute phase matching, and the measurement results of the local measurement system can be further spliced ​​into the global coordinate system.

[0051] 6. The automated guided vehicle and collaborative robot carry the local measurement system and move to the next measurement viewpoint. Steps 4 and 5 are repeated to perform local high-precision measurement and splicing positioning until the measurement is completed.

Claims

1. Reconfigurable large-size fringe projection optical three-dimensional measurement system, characterized in that: Including local measurement system, automatic guided vehicle and collaborative robot, global positioning subsystem: The local measurement system is a small field of view, high spatial resolution binocular fringe projection optical 3D measurement system, which is used for single field of view, high precision, dense point cloud data acquisition; The automatic guided vehicle and the collaborative robot are used to carry the local measurement system and move the local measurement system to a planned measurement viewpoint position; The global positioning subsystem is a large field of view, low spatial resolution binocular fringe projection optical three-dimensional measurement system, which can be used as a binocular fringe projection system for large-scale sparse point cloud measurement, and can also be networked to form a global positioning network to establish a global coordinate system for tracking and locating local measurement field point clouds and transforming them into a global coordinate system to achieve high-precision splicing measurement of large-scale parts.

2. The reconfigurable large-size fringe projection optical three-dimensional measurement system according to claim 1, characterized in that: In the above 1, the global positioning network is composed of no less than one global positioning subsystem, and the spatial layout of the global positioning subsystem can be adjusted according to the measurement requirements of the measurement object and in accordance with the system reconstruction principle.

3. The system reconstruction principle according to claim 2, characterized in that: The global positioning subsystems should be arranged reasonably to ensure that the common field of view between different global positioning subsystems is not less than one-third of the field of view of a single global positioning subsystem, that all the key dimensions of the measured object are visible to more than two global cameras, that the angle of the global camera optical axis ranges from 30° to 120°, and that the angle between the camera optical axis and the surface normal of the measured object is less than 45°. Set a suitable working distance for the global positioning subsystem to ensure that the projected fringes of the local measurement system observed by the global camera meet the sampling points of no less than 15 camera pixels per cycle.

4. The reconfigurable large-size fringe projection optical three-dimensional measurement system according to claim 1, characterized in that: In the above 2, after the global positioning network is reconfigured, each global positioning subsystem projects grating phase-shifted stripes and uses the phase stitching method to calculate the rigid body transformation of adjacent systems, thereby realizing online calibration of the global camera and constructing a unified global coordinate system.

5. The reconfigurable large-size fringe projection optical three-dimensional measurement system according to claim 1, characterized in that: In the above 4, after the global camera is calibrated online, the cameras of any different global positioning subsystems in the global positioning network can be combined into a binocular tracking system. During the high-precision splicing measurement of large-sized parts, while the local measurement system projects grating phase-shifted fringes, all cameras in the global positioning network shoot synchronously; the optimal tracking camera is selected from the global positioning network, and the measurement results of the local measurement system are tracked and positioned using the phase splicing method.

6. The optimal tracking camera selection criteria according to claim 5 adopt (1) the maximum effective point principle and (2) the optimal structural parameter principle.

7. According to the maximum effective point number principle in claim 6, it is characterized in that: When multiple cameras track the grating phase-shift fringes projected by the local measurement system, in the absolute phase domain, the camera whose absolute phase overlaps more with the absolute phase of the left camera of the local measurement system is selected as the optimal tracking camera.

8. According to the optimal structural parameter principle in claim 6, it is characterized in that: By simulating the random errors of structural parameters of all binocular combinations in the global positioning network, when multiple cameras track the grating phase-shifted stripes projected by the local measurement system, the camera combination with the smallest average error is selected as the optimal tracking camera.

Citation Information

Patent Citations

  • Optics-based quick pipeline detection method

    CN102538677A

  • Large-size part three-dimensional measurement splicing method based on phase matching

    CN113432550A

  • Three-dimensional measurement method and measurement system for large complex part

    CN117232388A

  • System for minimizing error in construction site data based on gcp

    KR102508936B1

  • Precision source-tracing method for precise control field of indoor space measurement and positioning system

    WO2015074357A1