A multi-faceted mirror based reflective omnidirectional vision three-dimensional measurement system

Abstract

The application relates to a multi-mirror-based reflective omnidirectional vision three-dimensional measurement system, which realizes monocular three-dimensional measurement by combining camera image feature extraction with depth data, and belongs to the field of vision measurement. The system mainly comprises an image acquisition module, a reflection module, a distance measurement module, an image processing and three-dimensional reconstruction module and a supporting and fixing mechanism. The reflection module reflects omnidirectional 360-degree measurement target light into a camera, the image acquisition module images the reflected light, obtains omnidirectional images and transmits the images to a computer, the distance measurement module obtains depth data of a measurement target to the system and transmits the data to the computer, the image processing and three-dimensional reconstruction module in the computer carries out noise reduction and feature extraction operations on the images, three-dimensional measurement is realized in combination with the depth data, the modules are connected through the supporting and fixing mechanism, and the reflective omnidirectional vision three-dimensional measurement system is formed. The system calibrates internal and external parameters with the aid of a laser tracker, and uses the calibrated internal and external parameters to realize high-precision three-dimensional measurement of a measurement target point. The measurement result can be used for quality evaluation of surface development of a large weak-texture structure and vibration feedback suppression, and has important significance for mechanical manufacturing and the national defense industry.

Description

TECHNICAL FIELD

[0001] The application relates to a multi-mirror-based reflective omnidirectional vision three-dimensional measurement system, adopts a near-far range composite measurement architecture, uses a mirror and a camera to acquire an omnidirectional 360-degree field of view range, realizes high-precision calibration of internal and external parameters of the system through a laser tracker, and realizes real-time three-dimensional reconstruction of a target to be measured in combination with target depth data acquired by a ranging system. The application mainly belongs to the field of vision measurement. BACKGROUND

[0002] In the field of national defense industry and mechanical manufacturing in China, real-time three-dimensional measurement of large weak-textured structural parts such as antennas has important significance, which can not only ensure the quality and performance of the structural parts during work, but also can analyze the working state in real time when the structural parts are flexibly vibrated. For large weak-textured structural parts, the field of view range of the traditional three-dimensional measurement method is limited, and the field of view splicing problem is usually involved. Some classic methods, such as multi-camera network measurement method, will introduce coordinate system conversion error, and the use of a rotary table for rotary scanning will introduce additional rotary positioning error. The 8 reflective omnidirectional vision method uses a mirror to reflect the target scene, avoids introducing additional error, and can greatly improve the measurement efficiency without losing accuracy.

[0003] The reflective omnidirectional vision three-dimensional measurement technology is currently widely used in the fields of robot navigation, virtual reality, automatic driving and the like. Common mirror shapes include spherical mirror, conical mirror, parabolic mirror and plane mirror. The curved mirror will cause significant distortion and distortion of the camera image, and the traditional image processing algorithm is no longer applicable. Therefore, the application adopts a plane mirror to reflect the target scene, a plurality of plane mirrors are arranged in the shape of a dodecahedron to form a near-range measurement system and a long-range measurement system, and an omnidirectional target from near to far is captured at one time, and after imaging in the camera, the image is transmitted to a computer, and high-precision three-dimensional measurement is realized through an algorithm. SUMMARY

[0004] The technical problem to be solved by the application is that large weak-textured structural parts usually have large surface size, contain folded and unfolded structures, and have limited control point pasting positions, and there are problems of high difficulty, low measurement accuracy and poor real-time performance in three-dimensional measurement. The application provides a multi-mirror-based reflective omnidirectional vision three-dimensional measurement system, which realizes one-time acquisition of an omnidirectional target scene through a mirror, realizes high-precision calibration of internal and external parameters through a laser tracker, and realizes high-precision real-time three-dimensional reconstruction of a target in combination with a ranging system.

[0005] The application relates to a multi-mirror-based reflective omnidirectional vision three-dimensional measurement system, which mainly comprises the following parts:

[0006] Image acquisition module (1): composed of a short-range camera and a long-range camera, arranged in an up-down structure, the short-range camera is located at the bottom and is responsible for imaging the target scene within a range of 3-18 meters, the long-range camera is located at the top and is responsible for imaging the target scene within a range of 15-50 meters, and the measurement accuracy at 50 meters is better than 3mm;

[0007] Reflection module (2): composed of a polyhedral mirror, a plurality of plane mirrors are arranged in a dodecahedron shape at a fixed angle, responsible for reflecting the light of the target scene, the reflected light is captured by the image acquisition module to realize omnidirectional 360° imaging;

[0008] Distance measuring module (3): composed of a laser range finder, responsible for obtaining the distance from the system to the target measurement point, and converting it into depth data to reconstruct the three-dimensional coordinates of the target measurement point;

[0009] Image processing and three-dimensional reconstruction module (4): mainly composed of a computer, which processes the camera image through the self-developed related software, such as noise reduction and feature extraction, and calculates the three-dimensional coordinates of the target measurement point using three-dimensional reconstruction algorithm;

[0010] Supporting and fixing mechanism (5): mainly composed of upper and lower bottom plates, camera fixing plates, supporting rods and transmission cables, responsible for supporting the system structure, fixing other modules, and stably installing the system at the measurement position.

[0011] The technical solution of the present application is: a reflection type omnidirectional vision three-dimensional measurement system based on a multi-faceted mirror, characterized by: mainly composed of an image acquisition module (1), a reflection module (2), a distance measuring module (3), an image processing and three-dimensional reconstruction module (4) and a supporting and fixing mechanism (5). The system uses a planar target and a laser tracker to realize the calibration of internal and external parameters before measurement. The light from the target measurement point is reflected onto the reflection module, reflected by the mirror into the image acquisition module, and then transmitted to the computer through the data line after imaging by the camera. The distance measuring module obtains the distance between the system and the target measurement point as the depth value, which is transmitted to the computer through the data line. The image processing and three-dimensional reconstruction module is used to realize three-dimensional measurement of the target measurement point.

[0012] The internal and external parameter calibration method of the system is a planar target calibration method assisted by a laser tracker. A five-circle planar target is placed in different positions and poses, a camera is used to shoot the planar target image, the center feature is extracted in the image, and the camera internal parameter matrix is calculated, including the equivalent focal length, the principal point coordinates and the distortion parameters. The laser tracker is used to obtain the spatial three-dimensional coordinates of the standard target ball arranged in the scene, and the two-dimensional pixel coordinates of the standard target ball are obtained by the system, combined with the camera internal parameters, and the PnP algorithm is used to calculate the system external parameters, i.e. the position and pose of the virtual camera in the world coordinate system.

[0013] The system three-dimensional reconstruction method of the present application, through the laser range finder, obtains the depth data of the target measurement point to the system, and the conversion relationship between the ranging system coordinate system and the world coordinate system is completed in advance by the laser tracker calibration. High-reflective circular marker points are pasted on the target to be measured, the omnidirectional target scene is imaged using the system, the center two-dimensional coordinates of the circular marker points are extracted, and the spatial three-dimensional coordinates of the center of the marker points are calculated in combination with the depth data. The conversion between the pixel coordinate system, the image coordinate system, the camera coordinate system and the world coordinate system is involved in the calculation process, and the internal and external parameters of the system can be known to realize high-precision three-dimensional reconstruction.

[0014] Compared with the prior art, the present application has the following advantages:

[0015] (1) The present application uses a mirror to reflect the light of the target scene, which can obtain an omnidirectional 360° target at one time, avoids introducing factors such as multi-view field splicing error and rotation error that affect the measurement accuracy, adopts an innovative compound system architecture combining long-range and short-range, and simultaneously realizes long-range and short-range measurement, with the advantages of high precision, high efficiency, small size and easy installation.

[0016] (2) The present application adopts a laser tracker assisted internal and external parameter calibration method, which does not need additional projection light sources and can complete calibration under laboratory environment light source conditions. The calibration accuracy of the laser tracker is high, the operation is simple, after the global coordinate system is established, a plurality of virtual camera coordinate systems are unified under the global coordinate system at one time, the conversion relationship between the plurality of virtual camera coordinate systems is obtained, and individual calibration of each camera is not needed, thereby improving the calibration efficiency.

[0017] In summary, the present application provides a reflective omnidirectional vision three-dimensional measurement system based on a multi-faceted mirror, which uses a laser tracker to assist in calibrating system parameters and ranging system positions, uses a multi-faceted mirror to realize one-time collection and fast and accurate three-dimensional reconstruction of omnidirectional measurement targets, and the results can be used to judge the vibration suppression or unfolding surface evaluation of large weak texture structure parts. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The present application provides a reflective omnidirectional vision three-dimensional measurement system based on a multi-faceted mirror, which uses a laser tracker to assist in calibrating system parameters and ranging system positions, uses a multi-faceted mirror to realize one-time collection and fast and accurate three-dimensional reconstruction of omnidirectional measurement targets, and the results can be used to judge the vibration suppression or unfolding surface evaluation of large weak texture structure parts.

[0019] Figure 2 The present application provides a reflective omnidirectional vision three-dimensional measurement system based on a multi-faceted mirror, which uses a laser tracker to assist in calibrating system parameters and ranging system positions, uses a multi-faceted mirror to realize one-time collection and fast and accurate three-dimensional reconstruction of omnidirectional measurement targets, and the results can be used to judge the vibration suppression or unfolding surface evaluation of large weak texture structure parts.

[0020] Figure 3The figure is a calibration scene schematic diagram of the present application, in which 6 is a laser tracker, and 7 is a standard target ball. DETAILED DESCRIPTION

[0021] The technical solutions of the present application are further described below in combination with the drawings and specific embodiments.

[0022] The present application proposes a reflection type omnidirectional vision three-dimensional measurement system based on a multi-faceted mirror, the system structure is shown in Figure 1 The figure is a calibration scene schematic diagram of the present application, in which 6 is a laser tracker, and 7 is a standard target ball.

[0023] The present application proposes a reflection type omnidirectional vision three-dimensional measurement system based on a multi-faceted mirror, the system structure is shown in Figure 2 The figure is a calibration scene schematic diagram of the present application, in which 6 is a laser tracker, and 7 is a standard target ball.

[0024] The present application proposes a reflection type omnidirectional vision three-dimensional measurement system based on a multi-faceted mirror, the system structure is shown in

[0025]

[0026] Among them, (u0, v0) is the image principal point coordinates, that is, the optical axis projection coordinates in the image pixel coordinate system. (fx, fy) is the equivalent focal length. R is the rotation matrix, and T is the translation vector. Then the coordinates of the mark point circle center in the camera coordinate system are

[0027]

[0028] The coordinates in the world coordinate system are

[0029]

[0030] The spatial three-dimensional coordinates of the center of the mark point can be obtained, the above calculation is performed on all the circular mark points, and the three-dimensional reconstruction of all the mark points is completed, and the measurement result data can be used for judging the real-time deformation condition and vibration feedback of the target structure.

[0031] The system calibration method disclosed by the application is divided into internal parameter calibration and external parameter calibration. According to a camera ideal pinhole model, the relationship between a spatial point P=(X W ,Y W ,Z W ) in a world coordinate system and its pixel coordinates (u,v) on a camera image plane can be expressed as

[0032]

[0033] Wherein, s is a proportional coefficient, M1 and M2 are respectively an internal parameter matrix and an external parameter matrix of the system, and γ is a non-perpendicular factor, and usually γ=0. The normalized image coordinates (x,y) of the spatial point P=(X C ,Y C ,Z C ) in the camera coordinate system are

[0034]

[0035] When calibrating the internal parameters of the system, the world coordinate system is established on a plane target, and the target is placed at the world coordinate system Z=0, so that

[0036]

[0037] The homography matrix is set as

[0038]

[0039] Wherein, s' is a constant factor. In the above formula, the calculation of H is to minimize the residual error between the actual image coordinates m i and the image coordinates m i calculated in the calibration, and the objective function is

[0040]

[0041] After the optimal solution of the homography matrix H is calculated, the internal parameter matrix M1 of the system can be solved.

[0042] The system's external parameter calibration uses a laser tracker (6) to establish a global coordinate system, and uses a standard target ball (7) to align each virtual camera coordinate system to the global coordinate system, thereby establishing the pose relationship between the virtual cameras. The system external parameter calibration scene is arranged as shown in Figure 3 The specific operation steps are as follows:

[0043] 1. Fix the standard target ball seat of the laser tracker within the system field of view, and use the laser tracker to obtain the ball center coordinates {P1, P2, P3,..., Pn} of the target ball in the world coordinate system. n};

[0044] 2. Replace the standard target ball on the ball seat with a conversion ball of the same ball diameter, use the system described in the application to shoot the conversion ball image, extract the center coordinates {m1, m2, m3,..., mn} of the conversion ball in the omnidirectional image, and sequentially correspond to the target ball of the laser tracker. n};

[0045] 3. Use the n-point perspective pose solving method (Perspective-n-Point, PnP) to calculate the pose conversion relationship between the virtual camera coordinate system and the world coordinate system, obtain the absolute pose of all virtual cameras in the world coordinate system, and then calculate the pose relationship of each virtual camera relative to other virtual cameras to obtain the external parameter matrix M2.

[0046] The PnP algorithm in step 3 above refers to using n feature points in the world coordinate system and their corresponding image points in the image to calculate the projection relationship, thereby obtaining the camera pose. Using the above method, the reflective omnidirectional vision three-dimensional measurement system calibration can be realized.

Claims

1. A multi-faceted mirror based reflective omni-directional vision three-dimensional measurement system, characterized in that, The system comprises the following modules: An image acquisition module (1) composed of a short-range camera and a long-range camera, responsible for image acquisition of the target to be measured, the short-range camera located at the lower layer of the system, with a working distance range of 3-18 meters, and the long-range camera located at the upper layer of the system, with a working distance range of 15-50 meters, and the measurement accuracy at 50 meters better than 3mm; A reflection module (2) composed of a polyhedral mirror, in the shape of a dodecahedron, responsible for reflecting the light of the target to be measured into the camera, following the principle of optical plane reflection, wherein the six mirrors located at the upper part are long-range mirrors, and the six mirrors located at the lower part are short-range mirrors; A distance measurement module (3) composed of a laser range finder, fixed at the top of the system, responsible for distance measurement of the target to be measured, and the obtained depth distance information used for three-dimensional reconstruction of the target to be measured; An image processing and three-dimensional reconstruction module (4) composed of a computer and related software, responsible for noise reduction and feature extraction of the images of the target to be measured taken by the camera, and three-dimensional reconstruction combined with the depth information of the target to be measured; A support and fixing mechanism (5) composed of upper and lower bottom plates, camera fixing plates, support rods, and transmission cables, responsible for supporting the system structure, fixing the image acquisition module and the reflection module, and transmitting the camera images to the computer.

2. The mirror-based reflective omni-directional vision three-dimensional measurement system according to claim 1, wherein: The light emitted by the target to be measured is reflected by the polyhedral mirror and captured by the camera, transmitted to the computer, and processed by software for image processing and three-dimensional reconstruction, combined with the depth information obtained by the distance measurement module, to obtain the spatial three-dimensional coordinates of the target to be measured, according to the optical reversibility principle, each camera is reflected by six plane mirrors to form six camera virtual images, equivalent to a virtual camera imaging 360° around the target to be measured, and three-dimensional reconstruction is realized in six directions at the same time through one shooting.

3. The mirror-based reflective omni-directional vision three-dimensional measurement system according to claim 1, wherein: The internal parameters of the system are calibrated using a five-circle plane target, several target images are taken at different positions and attitudes, and the equivalent focal length, principal point coordinates, and distortion parameters are calculated using the target images; the external parameters of the system are calibrated using a laser tracker (6), several standard target balls (7) are arranged within the working range, the three-dimensional coordinates of the target ball centers are obtained using the laser tracker, the two-dimensional pixel coordinates of the target ball centers are obtained using the system, and the external parameters of the system, i.e. the conversion matrix between the system coordinate system and the world coordinate system, are calculated using the PnP algorithm.

4. The mirror-based reflective omni-directional vision three-dimensional measurement system according to claim 1, wherein: The system realizes monocular three-dimensional reconstruction through internal and external parameters and depth values of the target to be measured, takes a circular marker point pasted on the target to be measured, extracts the center of the marker point in the image, obtains the pixel coordinates of the center of the marker point, obtains the depth value of the center of the marker point using the distance measurement system, calculates the coordinates of the center of the marker point in the camera coordinate system using the internal parameters of the system, calculates the three-dimensional coordinates of the center of the marker point in the world coordinate system using the external parameters of the system, and judges the real-time position change of the target to be measured through the spatial position of the marker point.

Images