Intelligent scanning method based on two-axis turntable and computer-readable storage medium

Through the intelligent sweeping method of the two-axis turntable, point cloud data can be obtained and completed in real time, and the problem of data missing in the two-axis turntable measurement system is solved, the integrity and measurement accuracy of the three-dimensional model are improved, and the interaction and real-time nature are enhanced. .

CN114066983BActive Publication Date: 2024-11-15XINTUO 3D TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111371122.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-18
Publication Date
2024-11-15
Estimated Expiration
2041-11-18

AI Technical Summary

Technical Problem

The existing two-axis turntable measurement system has data missing problems in the data acquisition stage, which leads to large measurement deviations in the post-processing stage, especially in narrow areas such as holes and inner cavity, which are difficult to obtain a complete three-dimensional model.

Method used

Using intelligent complementary scanning method based on two-axis rotary tables, the parameter calibration and hand-eye calibration of the binocular structured light three-dimensional scanner can be used to obtain the missing areas of the point cloud model in real time, and the UI window is used to control the synchronous motion of the two-axis rotary tables, collect and supplement it. Point cloud data in all missing areas realizes the integrity of the three-dimensional digital model.

Benefits of technology

It improves the integrity and measurement accuracy of point cloud data, reduces the impact of the post-processing stage, enhances interactivity and real-timeness, and can provide higher quality model data in the point cloud acquisition stage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114066983B_ABST
    Figure CN114066983B_ABST
Patent Text Reader

Abstract

The present invention discloses an intelligent scanning method based on a two-axis turntable and a computer-readable storage medium, comprising S1, calibrating the parameters of a binocular structured light three-dimensional scanner; S2, performing hand-eye calibration based on the two-axis turntable to obtain a calibration result matrix of the spatial relative position relationship between the scanner and the two-axis turntable base; S3, using the calibrated scanner to collect point cloud data of a three-dimensional digital model of a target object to obtain a point cloud model; S4, searching for missing areas of the point cloud model in a computer UI window and aligning the window view with the missing area, obtaining the target posture of the point cloud model displayed in the window in real time, and controlling the synchronous movement of the turntable according to the calibration result matrix and the target posture; S5, collecting a single point cloud under the target posture corresponding to the point cloud of the missing area, reconstructing the single point cloud and aligning it with the point cloud data collected by S3 to complete the point cloud data of the missing area; S6, repeating S4 to S5 until all the point cloud data of the missing areas of the point cloud model are completed, and a complete three-dimensional digital model of the target object for deviation detection is obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of visual 3D measurement technology, and in particular to an intelligent re-scanning method based on a two-axis turntable and a related computer-readable storage medium. Background Art

[0002] Optical 3D measurement technology is widely used in multiple industrial production stages such as reverse design, inspection and quality control. In the process of product manufacturing, accurate inspection is a key link in product quality control. Digital inspection of complex surface products based on 3D CAD models measures the surface model data of processed product parts and performs registration and deviation analysis with the original design CAD model to obtain inspection results for quality control.

[0003] Surface structured light free-form surface measurement is widely used in many fields such as reverse engineering, mold design, industrial inspection, quality control, cultural relics protection, medical imaging, agricultural surveying and mapping, underwater detection, etc. due to its advantages of fast, full-field, non-contact, high-density point cloud, on-site measurement, and non-cooperative targets. With the rapid development of the manufacturing industry, the product production cycle is shortened, and high-precision and high-efficiency quality inspection solutions can effectively cope with the challenges of product inspection and quality control brought about by industrial changes.

[0004] By combining robotic equipment and turntable equipment, many highly integrated and automated visual 3D measurement solutions have been derived. Among them, the robot-based automated measurement system is used for the three-dimensional measurement of larger-sized, industrial-grade parts. For model repair and inspection in the desktop and light industrial fields, the two-axis turntable system is more capable.

[0005] At present, the existing two-axis turntable measurement system usually uses viewpoint planning or manual teaching to perform measurements. That is, the two-axis turntable movement is controlled by inputting the rotation angles of the two axes, and the target object point cloud is measured using a binocular structured light three-dimensional scanner. This measurement method has the problem of missing data, and the existing methods cannot effectively solve this problem. At present, most methods solve this problem by sampling the point cloud, encapsulating it into a grid, and filling the holes in the grid model during the data post-processing stage. However, the three-dimensional digital model obtained by the grid hole filling technology is not a real processing model, but a model obtained after algorithm calculation and optimization, and cannot reflect the real appearance of the model well, especially in small areas such as holes and cavities. The difference between the model obtained by post-processing and the real model is greater. Therefore, in the point cloud acquisition stage, it is necessary to obtain more complete model data to reduce the impact of post-processing on the model. Summary of the invention

[0006] In view of this, the present invention proposes an intelligent supplementary scanning method based on a two-axis turntable, aiming to solve the technical problem that the current two-axis turntable measurement system has missing data and cannot effectively compensate for it in the post-processing stage, resulting in large measurement deviations.

[0007] The present invention proposes the following technical solutions to solve the above technical problems:

[0008] An intelligent scanning method based on a two-axis turntable comprises the following steps: S1, calibrating parameters of a binocular structured light 3D scanner; S2, performing hand-eye calibration based on the two-axis turntable to obtain a calibration result matrix of the spatial relative position relationship between the binocular structured light 3D scanner and the two-axis turntable base; S3, scanning a target object using the binocular structured light 3D scanner calibrated in step S1 to collect point cloud data of a 3D digital model of the target object to obtain a point cloud model; S4, searching for a missing area of ​​the point cloud model in a UI window on a computer, aligning the window view with the missing area, and obtaining a UI in real time. I, the target posture of the point cloud model displayed in the point cloud model window, and the two-axis turntable is controlled to move synchronously according to the calibration result matrix and the target posture, wherein the UI refers to the user interface; S5, collecting a single point cloud under the target posture corresponding to the point cloud of the missing area, reconstructing the single point cloud and aligning it with the point cloud data collected in step S3 to complete the point cloud data of the missing area; S6, repeating steps S4 to S5 until the point cloud data of all missing areas of the point cloud model are completed, and a complete three-dimensional digital model of the target object is obtained, wherein the complete three-dimensional digital model is used for deviation detection.

[0009] Furthermore, step S1 includes: calibrating internal and external parameters of the camera of the binocular structured light 3D scanner.

[0010] Furthermore, the calibration of the internal and external parameters of the camera of the binocular structured light 3D scanner in step S1 includes: S11, photographing the calibration plate from different angles and positions to obtain a calibration image, wherein the calibration plate is configured with coded marker points and non-coded marker points; S12, identifying the calibration image to obtain the two-dimensional image coordinates of the coded marker points and the two-dimensional image coordinates of the non-coded marker points; performing three-dimensional reconstruction on the coded marker points and the non-coded marker points respectively to obtain the three-dimensional space coordinates of the coded marker points and the three-dimensional space coordinates of the non-coded marker points; S13, based on the two-dimensional image coordinates and the three-dimensional space coordinates of the coded marker points, as well as the two-dimensional image coordinates and the three-dimensional space coordinates of the non-coded marker points, performing overall bundle adjustment iterative optimization on the internal and external parameters of the camera, with minimizing the reprojection error of the camera as the target loss function, and adding scale parameters to constrain the internal and external parameters of the camera.

[0011] Furthermore, in step S2, the hand-eye calibration is performed using the binocular structured light 3D scanner as an eye and the two-axis turntable base as a hand; wherein the coordinate system of the binocular structured light 3D scanner is set at the optical center of the reference camera, the coordinate system of the two-axis turntable base is set at the base of the two-axis turntable, and the mathematical model used for calibration is an eye-outside-hand model.

[0012] Furthermore, the hand-eye calibration based on the two-axis turntable in step S2 includes: S21, using a standard calibration plate configured with coded markers and non-coded markers for calibration; during the calibration process, the spatial relative position relationship between the binocular structured light 3D scanner and the base of the two-axis turntable remains unchanged, and the spatial relative position relationship between the coordinate system of the two-axis turntable table and the standard calibration plate remains unchanged, so that the standard calibration plate is fixed on the two-axis turntable table and moves with the two-axis turntable during the calibration process;

[0013] S22, control the rotation of the two axes of the two-axis turntable to drive the standard calibration plate to different positions and postures in space, and use the binocular structured light 3D scanner to shoot the standard calibration plate in the process to obtain multiple calibration images at different positions and postures; S23, perform binocular matching on the calibration image, and then perform 3D reconstruction, and calculate the relative external parameters of the 3D coordinate points of the reconstructed landmarks and the known global points of the world coordinate system of the standard calibration plate; S24, through the robot DH modeling method, regard the two-axis turntable as a two-axis robot, and establish the basic The conversion relationship between the base coordinate system and the table coordinate system of the two-axis turntable; by using the Rodriguez formula and the rotation angles of the two axes of the two-axis turntable, the conversion matrix from the base coordinate system of the two-axis turntable to the table coordinate system of the two-axis turntable at different positions and postures of the two-axis turntable table is calculated; S25, the relative external parameters obtained in step S23 and the conversion matrix obtained in step S24 are used as data input for the hand-eye calibration process, and the hand-eye calibration algorithm is used for solving to obtain the calibration result matrix from the optical center of the reference camera to the base coordinate system of the two-axis turntable.

[0014] Furthermore, in the process of hand-eye calibration in step S2, for the calibration image captured by the binocular structured light 3D scanner, the following hand-eye relationship equation is satisfied:

[0015] AX=XB

[0016] The hand-eye relationship equation indicates that for each calibration posture X, there are different A matrices and B matrices. The A matrix represents the external parameter matrix obtained by camera calibration, and the B matrix represents the transformation matrix from the base coordinate system to the table coordinate system of the two-axis turntable at each position. The B matrix is ​​obtained through DH modeling of the two-axis turntable.

[0017] Furthermore, obtaining the B matrix through DH modeling of the two-axis turntable includes: setting respective coordinate systems for the connecting rods of the two-axis turntable, determining the parameters of the kinematic model of the two-axis turntable by moving the coordinate system, and converting the parameters of the kinematic model into a transformation matrix B from the two-axis turntable base coordinate system to the two-axis turntable table top coordinate system according to the three-dimensional rigid body transformation in space.

[0018] Furthermore, step S3 specifically includes: S31, placing the target object at the center of the table top of the two-axis turntable and fixing it to ensure that the relative position relationship between the target object and the two-axis turntable remains unchanged during the scanning process; S32, controlling the two-axis turntable to move to different positions and postures, and using the binocular structured light three-dimensional scanner to scan the target object at each position and posture to obtain multiple sets of point cloud data; S33, using a marker-free splicing method to splice the multiple sets of point cloud data to obtain three-dimensional digital model point cloud data of the target object.

[0019] Furthermore, step S4 specifically includes: S41, setting a reference posture of the scanned point cloud in the coordinate system of the two-axis turntable base, and recording the reference transformation matrix from the two-axis turntable table coordinate system to the two-axis turntable base coordinate system at this time; S42, regarding the computer screen as a virtual camera, moving the mouse to rotate the view of the point cloud model in the UI window to find the missing area of ​​the point cloud model, and aligning the window view with the missing area, and recording the initial coordinates and the terminal coordinates of the mouse moving in the two-dimensional plane; S43, using the OpenGL view matrix transformation relationship, converting the two-dimensional coordinates of the mouse into three-dimensional coordinates in three-dimensional space; S44, according to the definition of a three-dimensional sphere, the three-dimensional coordinates of the initial coordinates and the terminal coordinates are distributed on the spherical surface of the three-dimensional sphere; respectively constructing the center of the three-dimensional sphere and the initial coordinates The first vector connected, the center of the three-dimensional sphere and the second vector connected to the termination coordinate, the rotation axis and the angle are obtained by calculating the first vector and the second vector; S45, the rotation axis and the angle obtained in step S44 are used as input, and the target pose estimation matrix of the missing area is calculated by using the Rodriguez formula; S46, the target pose estimation matrix calculated in step S45 is converted into the rotation angle of the two axes of the two-axis turntable using the two-axis turntable DH model; S47, the two-axis turntable is controlled to move according to the rotation angle calculated in step S46, at this time, the pose of the target object in the optical center coordinate system of the reference camera of the binocular structured light three-dimensional scanner is consistent with the pose of the point cloud model view in the UI window, that is, the synchronous movement of the two-axis turntable and the point cloud model is completed.

[0020] Furthermore, step S5 specifically includes: S51, using the binocular structured light 3D scanner to collect a single point cloud of the target object in the target pose; S52, according to the target pose estimation matrix calculated in step S45 and the reference transformation matrix recorded in step S41, calculating the transformation relationship between the single point cloud and the point cloud model, so as to transform the scanned single point cloud from the target pose in the optical center coordinate system of the reference camera to the reference pose in the optical center coordinate system of the reference camera, and obtain the point cloud data after the scan of the missing area.

[0021] Compared with the prior art, the beneficial effect of the technical solution of the present invention is that: by using a binocular structured light 3D scanner to scan the target object and introducing a two-axis turntable, there is no need to manually adjust the position and posture of the binocular structured light 3D scanner. The acquisition of model point clouds from different perspectives can be achieved by controlling the rotation of the two-axis turntable through software on the PC. On this basis, the point cloud model view is adjusted through the software UI window to simulate the perspective of the camera scanning the target object, and the missing area of ​​the point cloud model can be visually viewed. The two-axis turntable is automatically controlled to move to a position corresponding to the posture when the missing area is aligned with the observation perspective, and the binocular structured light 3D scanner is used to collect data. The missing point cloud data is repaired in the point cloud acquisition stage to achieve intelligent scanning, improve interactivity, ease of use and real-time performance, and its measurement target posture estimation accuracy is high, which can improve the model integrity in the point cloud data acquisition stage and provide a higher quality model for point cloud data post-processing. In addition, a binocular structured light high-precision 3D scanner can be used, which has the advantages of high imaging accuracy (single-frame scanning accuracy can reach 0.01mm), fast point cloud acquisition speed (single-frame scanning speed is less than 1s), high acquisition quality, strong resistance to ambient light interference, etc. when scanning the target object, and has high adaptability to black and reflective objects.

[0022] In a further technical solution, a binocular camera system calibration method based on photogrammetry is used to accurately obtain the calibration parameters of the binocular structured light 3D scanner and the two-axis turntable, which solves the problems of non-convergence and poor robustness of the traditional calibration calculation process; at the same time, it greatly simplifies the calibration process, and the reprojection error of the calibration method is within 0.05 pixels, which meets the measurement accuracy of the system.

[0023] The present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the aforementioned intelligent re-scanning method can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of an intelligent re-scanning method based on a two-axis turntable in an embodiment of the present invention.

[0025] Figure 2It is a schematic diagram of a process for calibrating the internal and external parameters of a camera of a binocular structured light 3D scanner in an embodiment of the present invention.

[0026] Figure 3 It is a schematic diagram of a method for calibrating the internal and external parameters of a camera of a binocular structured light 3D scanner in an embodiment of the present invention.

[0027] Figure 4 It is a schematic diagram of a standard calibration plate configured with coded marker points and non-coded marker points and a scale Scale used for reconstruction in an embodiment of the present invention.

[0028] Figure 5 Schematic diagram of a hand-eye calibration method based on a two-axis turntable in an embodiment of the present invention.

[0029] Figure 6 It is a schematic diagram of a method for collecting point cloud data of a three-dimensional digital model of a target object to be scanned in an embodiment of the present invention.

[0030] Figure 7 It is a scanning effect diagram of the pump body model point cloud including the missing area obtained by preliminary scanning in an embodiment of the present invention.

[0031] Figure 8 It is a schematic diagram of a method in which a computer acquires the target pose of a point cloud model displayed in a UI window in real time and controls the synchronous movement of a two-axis turntable in an embodiment of the present invention.

[0032] Fig. 9 It is a schematic diagram of the OpenGL view display matrix transformation process in an embodiment of the present invention.

[0033] Fig.10 It is a schematic diagram of a method for converting two-dimensional coordinates into three-dimensional space in an embodiment of the present invention.

[0034] Fig.11 It is a schematic diagram of a method for collecting and completing missing area point cloud data in an embodiment of the present invention.

[0035] Fig.12 It is a scanning effect diagram of the pump body model point cloud after completing the missing area through intelligent supplementary scanning in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the technical problems, technical solutions and beneficial effects to be solved by the embodiments of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] refer to Figure 1 The embodiment of the present invention provides an intelligent re-scanning method based on a two-axis turntable, comprising the following steps S1 to S6:

[0038] S1. Calibrate parameters of a binocular structured light 3D scanner; wherein the parameter calibration refers to calibrating internal and external parameters of a camera of the binocular structured light 3D scanner;

[0039] S2. Perform hand-eye calibration based on a two-axis turntable to obtain a calibration result matrix of the spatial relative position relationship between the binocular structured light 3D scanner and the two-axis turntable base;

[0040] S3, scanning the target object using the binocular structured light 3D scanner calibrated in step S1 to collect point cloud data of the 3D digital model of the target object to obtain a point cloud model;

[0041] S4, searching for the missing area of ​​the point cloud model in the UI window on the computer, aligning the window view with the missing area, acquiring the target pose of the point cloud model displayed in the UI window in real time, and controlling the synchronous movement of the two-axis turntable according to the calibration result matrix and the target pose; wherein the UI refers to a user interface (User Interface);

[0042] S5, collecting a single point cloud in the target posture corresponding to the point cloud of the missing area, reconstructing the single point cloud and registering it with the point cloud data collected in step S3 to complete the point cloud data of the missing area;

[0043] S6. Repeat steps S4 to S5 until the point cloud data of all missing areas of the point cloud model are completed to obtain a complete three-dimensional digital model of the target object, wherein the complete three-dimensional digital model is used for deviation detection.

[0044] In one embodiment of the present invention, the three-dimensional digital model is a pump body CAD model, and CAD (Computer Aided Design) is computer-aided design. In one embodiment of the present invention, by using a binocular structured light high-precision three-dimensional scanner, the imaging accuracy is high, the point cloud acquisition speed is fast, the acquisition quality is high, the ability to resist ambient light interference is strong, and it has high adaptability to black and reflective objects. The single-frame scanning accuracy can reach 0.01mm, and the single-frame scanning speed is less than 1s.

[0045] Specifically, the calibration of the internal and external parameters of the camera of a binocular structured light 3D scanner refers to the process of determining its internal and external orientation parameters. The accuracy of the camera parameters directly affects the accuracy of the point cloud reconstruction of the 3D scanner. In one embodiment of the present invention, a standard calibration plate equipped with coded marker points and non-coded marker points is used for calibration, and the calibration plate is photographed from different angles and positions to obtain a certain number of calibration pictures; then the marker point image uv coordinates in the calibration picture are identified, and based on the principle of photogrammetry, the marker points are three-dimensionally reconstructed to obtain their accurate three-dimensional space coordinates; then, based on the marker point two-dimensional image uv coordinates and three-dimensional space coordinates, all camera parameters in the pinhole imaging camera model are adjusted by overall bundle adjustment iterative optimization, with minimization of the camera reprojection error as the target loss function, and scale parameters (image and real dimension scaling parameters) are added for constraints to finally obtain accurate internal and external parameters of the 3D scanner camera.

[0046] This calibration method does not require a high-precision calibration plate. It only needs to use the accurate distance between any pair of marking points on the calibration plate as a scale to accurately calculate the internal and external orientation parameters of the camera, thereby realizing the calibration of the camera.

[0047] like Figure 2 As shown, it is a schematic diagram of the process of calibrating the internal and external parameters of the camera of the binocular structured light 3D scanner in an embodiment of the present invention, including processes A01 to A05: A01, collecting calibration pictures; A02, identifying landmark points; A03, reconstructing the 3D coordinates of the landmark points; A04, iterative optimization of bundle adjustment; A05, adding scale parameters. Figure 3 Schematic diagram of a method for calibrating the internal and external parameters of a camera of a binocular structured light 3D scanner according to an embodiment of the present invention. Figure 2 and Figure 3 , calibrating the internal and external parameters of the camera of the binocular structured light 3D scanner includes the following steps S11 to S13:

[0048] S11. Photograph the calibration plate from different angles and positions to obtain calibration images; wherein the calibration plate is a standard calibration plate, which is provided with coded marker points and non-coded marker points.

[0049] S12, recognize the calibration image to obtain the two-dimensional image coordinates of the coded marker points and the two-dimensional image coordinates of the non-coded marker points; perform three-dimensional reconstruction on the coded marker points and the non-coded marker points respectively to obtain the three-dimensional space coordinates of the coded marker points and the three-dimensional space coordinates of the non-coded marker points. Specifically, first perform three-dimensional reconstruction on the coded marker points to obtain their accurate three-dimensional space coordinates, and then reconstruct the non-coded marker points to obtain their accurate three-dimensional space coordinates.

[0050] S13, based on the two-dimensional image coordinates and three-dimensional space coordinates of the coded mark points, as well as the two-dimensional image coordinates and three-dimensional space coordinates of the non-coded mark points, the internal and external parameters of the camera are adjusted by overall bundle adjustment iteration optimization, with minimizing the reprojection error of the camera as the target loss function, and adding scale parameters to constrain the internal and external parameters of the camera. Wherein, when optimizing, the coordinates, external parameter matrix and internal parameter matrix as well as distortion and other parameters are taken into consideration, bundle optimization is performed, the reprojection error is minimized, and the optimal internal and external parameters of the camera are obtained after iteration.

[0051] like Figure 4 , which is a schematic diagram of a standard calibration plate configured with coded marker points and non-coded marker points and a scale Scale for three-dimensional reconstruction in an embodiment of the present invention.

[0052] The purpose of hand-eye calibration based on the two-axis turntable is to obtain the spatial relative position relationship between the binocular structured light 3D scanner and the turntable base. It is essentially the calibration of the conversion relationship between the two coordinate systems. Its accuracy directly affects the target pose acquisition accuracy and the stitching accuracy of the scanned point cloud of the intelligent fill scan. The binocular structured light 3D scanner is used as the eye (Eye), and its coordinate system is set at the optical center of the reference camera. In one embodiment of the present invention, the reference camera is the left camera; the turntable base is used as the hand (Hand), and its coordinate system is set at the turntable base. Its mathematical model is the eye-outside-hand model. It can be understood that in the intelligent fill scan, the computer screen is regarded as a virtual camera, and the scanning target pose of the missing area point cloud is obtained in the optical center coordinate system of the left camera. It is necessary to convert the target pose to the coordinate system of the two-axis turntable base through hand-eye calibration, and then convert the transformation matrix describing the target pose into the angle of the two axes of the two-axis turntable through the two-axis turntable DH model, so as to drive the model to a position consistent with the viewing angle of the computer screen.

[0053] like Figure 5 As shown, the hand-eye calibration of the two-axis turntable includes the following steps S21 to S25:

[0054] S21. Calibrate using a standard calibration plate equipped with coded marking points and non-coded marking points; during the calibration process, the spatial relative position relationship between the binocular structured light 3D scanner and the two-axis turntable base remains unchanged, and the spatial relative position relationship between the coordinate system of the two-axis turntable table surface and the standard calibration plate remains unchanged so that the standard calibration plate is fixed on the two-axis turntable table surface during the calibration process and moves with the two-axis turntable.

[0055] S22, control the rotation of the two axes of the two-axis turntable to drive the standard calibration plate to different positions and postures in space, use the binocular structured light 3D scanner to shoot the standard calibration plate in the process, repeat this process to obtain multiple calibration images in different positions and postures. In one embodiment, 10 to 15 frames of calibration images are collected.

[0056] S23, performing binocular matching on the calibration image, and then performing three-dimensional reconstruction, and calculating the relative external parameters of the three-dimensional coordinate points of the reconstructed landmark points and the known global points of the world coordinate system of the standard calibration plate.

[0057] S24: By using the robot DH modeling method, the two-axis turntable is regarded as a two-axis robot, and a conversion relationship from the base coordinate system of the two-axis turntable to the table coordinate system of the two-axis turntable is established; by using the Rodriguez formula and the rotation angles of the two axes of the two-axis turntable, the conversion matrix from the base coordinate system of the two-axis turntable to the table coordinate system of the two-axis turntable at different positions and postures of the two-axis turntable table is calculated. Among them, the DH modeling method refers to a robot modeling method proposed by Denavit and Hartenberg, which is mainly used in robot kinematics.

[0058] S25. Use the relative external parameters obtained in step S23 and the transformation matrix obtained in step S24 as data input for the hand-eye calibration process, solve them using the hand-eye calibration algorithm, and obtain a calibration result matrix from the optical center of the reference camera to the two-axis turntable base coordinate system.

[0059] Specifically, since the calibration plate is fixed on the two-axis turntable surface during the hand-eye calibration process, the transformation matrix between the calibration plate and the two-axis turntable surface remains unchanged when the two-axis turntable rotates. For the calibration image taken by the binocular structured light 3D scanner, the following equation can be listed:

[0060] AX=XB (1)

[0061] This equation is the hand-eye relationship equation, which means that for each calibration pose X, there are different A matrices and B matrices. The A matrix represents the external parameter matrix obtained by camera calibration, and the B matrix represents the transformation matrix from the base coordinate system of the two-axis turntable to the table coordinate system at each position. The B matrix is ​​obtained through the DH modeling of the two-axis turntable. The left and right sides of the equation represent that no matter where the two-axis turntable moves, the transformation matrix from the two-axis turntable table to the calibration plate remains unchanged.

[0062] Specifically, by setting respective coordinate systems for the connecting rods of the two-axis turntable, the parameters of the turntable kinematic model are determined by moving the coordinate systems, and the parameters of the kinematic model are converted into a transformation matrix from the two-axis turntable base coordinate system to the two-axis turntable table top coordinate system according to the three-dimensional rigid body transformation in space.

[0063] The specific steps are as follows:

[0064] 1. Establish the coordinate system of each component of the two-axis turntable. The steps for establishing the coordinate system are as follows:

[0065] 1.1、Establish z i Axis: The coordinate system of link i is established at joint i, z i The axis direction is along the axis of joint i.

[0066] 1.2. Establish x i Axis: Along z i Axis and z i+1 The direction of the common perpendicular line of the axis points to z i+1 axis.

[0067] 1.3、Build y i Axis: Through x i Axis and z i Axis, determined using the right-hand rule.

[0068] 2. The following steps can be used to determine the four parameters of the DH model (θ i , d i , a i , α i ):

[0069] 2.1. Around x i Axis rotation torsion angle α i-1 , so that the moved z i Axis and z i+1 Axes coincide.

[0070] 2.2, along x i Axis moving rod length a i-1 , so that the origin of the coordinate system of link i and the origin of the coordinate system of link i+1 are at z i+1 Axis direction.

[0071] 2.3, along z i Axis moving distance d i , so that x i Axis and x i+1 The axes lie in the same plane.

[0072] 2.4. Around z i Axis rotation angle θ i , so that x i Axis and x i+1 Axes coincide.

[0073] According to the above steps, combined with the structural parameters of the two-axis turntable, the corresponding DH parameters can be calculated.

[0074] 3. According to the connecting rod coordinate system transformation matrix shown in formula (2), the obtained DH parameters are substituted, and then the transformation matrix from the turntable base coordinate system to the turntable table coordinate system is obtained by matrix multiplication through formula (3). This matrix is ​​the B matrix required for hand-eye calibration.

[0075]

[0076]

[0077] In the embodiment of the present invention, the calibration processes of step S1 and step S2 do not require the production of calibration reference objects with high production costs. Only a standard calibration plate equipped with marker points is used, and a certain number of calibration images of the calibration plate are collected from different angles and positions. By identifying the marker points, reconstructing the three-dimensional coordinates of the marker points, optimizing the overall bundle adjustment, adding scale parameters and internal and external parameters of the camera, the re-projection error is iteratively minimized, thereby obtaining accurate relative position parameters of the binocular structured light camera and hand-eye calibration, thereby ensuring the quality of point cloud acquisition and the stitching accuracy of the supplementary scanned point cloud.

[0078] After completing the calibration of step S1 and step S2, Figure 6 As shown, the three-dimensional digital model point cloud data of the target object (pump body parts) is collected. The three-dimensional digital model data collection steps of the pump body parts include S31 to S33:

[0079] S31, placing the pump body part at the center of the two-axis turntable and fixing it to ensure that the relative position relationship between the pump body part and the turntable remains unchanged during the scanning process;

[0080] S32, controlling the two-axis turntable to move to different positions and postures, and using the binocular structured light 3D scanner to scan the pump body parts at each position and posture to obtain multiple sets of point cloud data;

[0081] S33. The plurality of groups of point cloud data are spliced ​​together in a non-marking point splicing method to obtain three-dimensional digital model point cloud data of the pump body parts, or a point cloud model.

[0082] like Figure 7 The figure shows the point cloud data of the 3D digital model of the pump body parts obtained by scanning. Due to the complex features of the pump body model, the small number of planned positions, and the common occlusion of binocular structured light scanning, the matching failure and inability to reconstruct the pump body point cloud model is incomplete. The intelligent re-scanning method of the present invention can effectively solve this problem.

[0083] Figure 8 Schematic diagram of a method for a computer to obtain the target position of a point cloud model displayed in a UI window in real time and control the synchronous movement of a two-axis turntable in an embodiment of the present invention. Figure 8As shown, searching for missing areas of the point cloud model in a UI window on a computer, aligning the window view with the missing areas, acquiring the target pose of the point cloud model displayed in the UI window in real time, and controlling the synchronous movement of the two-axis turntable according to the calibration result matrix and the target pose, including the following steps S41 to S47:

[0084] S41, setting a reference pose of the scanned point cloud in the two-axis turntable base coordinate system, and recording the reference transformation matrix from the two-axis turntable table top coordinate system to the base coordinate system at this time;

[0085] S42, regarding the computer screen as a virtual camera, moving the mouse to rotate the view of the point cloud model in the UI window to find the missing area of ​​the point cloud model, and aligning the window view with the missing area (i.e., adjusting the posture of the point cloud model in the UI window of the computer software by mouse operation, aligning the missing area with the perspective of human eye observation, which is equivalent to the rotation of the turntable driving the model to move to the position corresponding to the camera shooting), and recording the initial coordinates and the end coordinates of the mouse movement in the two-dimensional plane;

[0086] S43, using the OpenGL view matrix transformation relationship, converting the two-dimensional coordinates of the mouse into three-dimensional coordinates in three-dimensional space;

[0087] Specifically, in order to transform one coordinate system (3D space) to another coordinate system (2D screen), several transformation matrices are required, namely the model matrix, view matrix, and projection matrix. The description of a vertex in the model point cloud data in the 3D world is transformed to the computer screen and displayed after rendering. Fig. 9 As shown in the figure, the conversion process is as follows: First, the vertex coordinates start in the local space, called local coordinates, then go through world coordinates, observation coordinates, clipping coordinates and finally end with screen coordinates. Local coordinates are the coordinates of the object relative to the local origin, and after conversion to world coordinates, they are at the world origin. The coordinates observed from the perspective of a camera or an observer are observation coordinates, which are then converted to clipping coordinates through perspective projection, and finally the clipping coordinates are converted to the coordinates on the final two-dimensional screen through viewport transformation.

[0088] Using the OpenGL view transformation relationship, converting the two-dimensional coordinates of the mouse to three-dimensional coordinates in three-dimensional space is essentially the inverse transformation of OpenGL to display the model of the three-dimensional world on the computer. And because in intelligent scanning, the reference coordinate system describing the model point cloud is the camera coordinate system, and the camera coordinate system is the world coordinate system scanned by the two-axis turntable, that is, the initial position of the model point cloud coordinate system in the real world coordinate system is in the observation space, and the three-dimensional transformation matrix of the real world is the matrix in the observation space, and there is no need to further inversely transform to world coordinates and local coordinates. Therefore, Fig.10 As shown, the specific steps of converting the mouse from two-dimensional space to three-dimensional space are as follows:

[0089] (1) OpenGL first associates a coordinate with each point on the screen through viewport conversion in screen space, and maps the screen coordinate back to 2D standardized device coordinates through internal parameters;

[0090] (2) Then perform the inverse perspective division operation on the 2D normalized device coordinates, that is, multiply each coordinate component by the homogeneous component to map the screen coordinates back to the clipping space;

[0091] (3) OpenGL maps the clipping coordinates in the clipping space back to the observation space through the projection matrix. The coordinates in the observation space are the required coordinates in the three-dimensional space.

[0092] S44. According to the definition of a three-dimensional sphere, the three-dimensional coordinates of the initial coordinates and the final coordinates are distributed on the spherical surface; a vector connecting the center of the three-dimensional sphere and the initial coordinates, and a vector connecting the center of the three-dimensional sphere and the final coordinates are respectively constructed, and the rotation axis and the rotation angle are obtained through vector calculation; the details are as follows:

[0093] (1) The vector connecting the center of the three-dimensional ball and the initial coordinates is denoted as vector The vector connecting the center of the three-dimensional ball and the terminal coordinates is denoted as vector

[0094] (2) Rotation axis and the rotation angle θ can be calculated by the following equations (4) and (5) respectively:

[0095]

[0096]

[0097] S45, taking the rotation axis and the rotation angle obtained in step S44 as input, and using the Rodriguez formula to calculate the target pose estimation matrix R of the missing area, as follows:

[0098] R=cosθI+(1-cosθ)nn T +sinθn Λ

[0099] Where I represents the unit matrix, n represents the unit length vector of the rotation axis, the symbol Λ represents the transformation symbol from the vector to the antisymmetric matrix, and n Λ Represents the antisymmetric matrix of vector n;

[0100] S46, converting the target pose estimation matrix calculated in step S45 into the rotation angles of the two axes of the two-axis turntable using the two-axis turntable DH model;

[0101] S47. Control the two-axis turntable to move at the rotation angle calculated in step S46 through software on the computer. At this time, the position of the target object in the optical center coordinate system of the reference camera of the binocular structured light 3D scanner is consistent with the position of the point cloud model view in the UI window, thereby completing the synchronous movement of the two-axis turntable and the point cloud model.

[0102] Fig.11 Schematic diagram of a method for collecting and completing missing area point cloud data in an embodiment of the present invention. Fig.11 As shown in the figure, when the turntable moves synchronously to the position consistent with the computer UI window Figure 1 After the target pose is obtained, a single point cloud under the target pose is collected, and the scanning result of the missing point cloud at the position is completed by reconstructing the single point cloud and aligning it with the previously scanned point cloud model, including the following steps:

[0103] S51, using the binocular structured light 3D scanner to collect a single point cloud of the target object in the target posture;

[0104] S52. Calculate the transformation relationship between the single point cloud and the point cloud model according to the target pose estimation matrix calculated in step S45 and the reference transformation matrix recorded in step S41, so as to transform the scanned single point cloud from the target pose in the reference camera optical center coordinate system to the reference pose in the reference camera optical center coordinate system, and obtain the point cloud data after the scan of the missing area.

[0105] Finally, steps S4 to S5 are repeated until the point cloud data of all missing areas are completed to obtain the point cloud data of the complete three-dimensional digital model of the pump body parts. The complete three-dimensional digital model is used for deviation detection. The pump body point cloud model obtained after the intelligent scanning is completed is as follows Fig.12 shown.

[0106] The intelligent re-scanning method based on a two-axis turntable proposed in an embodiment of the present invention has good interactivity, ease of use, and real-time performance, and has high accuracy in estimating the position and pose of the measured target. It can improve the model integrity in the point cloud data acquisition stage and provide a higher quality model for point cloud data post-processing.

[0107] Another embodiment of the present invention further proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the intelligent scanning method based on a two-axis turntable in the aforementioned embodiment can be implemented. Among them, the computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier, which carries a readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable storage medium can send, propagate or transmit a program for use by or in combination with an instruction execution system, device or device. The program code contained in the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency, etc., or any suitable combination of the above.

[0108] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. An intelligent scanning method based on a two-axis turntable, characterized in that: The steps include: S1. Calibrate the parameters of the binocular structured light 3D scanner; S2. Perform hand-eye calibration based on a two-axis turntable to obtain a calibration result matrix of the spatial relative position relationship between the binocular structured light 3D scanner and the two-axis turntable base; S3, scanning the target object using the binocular structured light 3D scanner calibrated in step S1 to collect point cloud data of the 3D digital model of the target object to obtain a point cloud model; S4, searching for the missing area of ​​the point cloud model in the UI window on the computer, aligning the window view with the missing area, acquiring the target pose of the point cloud model displayed in the UI window in real time, and controlling the synchronous movement of the two-axis turntable according to the calibration result matrix and the target pose; wherein the UI refers to a user interface; S5, collecting a single point cloud in the target posture corresponding to the point cloud of the missing area, reconstructing the single point cloud and registering it with the point cloud data collected in step S3 to complete the point cloud data of the missing area; S6, repeating steps S4 to S5 until the point cloud data of all missing areas of the point cloud model are completed to obtain a complete three-dimensional digital model of the target object, wherein the complete three-dimensional digital model is used for deviation detection; Step S4 includes: S41, setting a reference pose of the scan point cloud in the two-axis turntable base coordinate system, and recording the reference transformation matrix from the two-axis turntable table top coordinate system to the two-axis turntable base coordinate system at this time; S42, regarding the computer screen as a virtual camera, moving the mouse to rotate the view of the point cloud model in the UI window to find the missing area of ​​the point cloud model, aligning the window view with the missing area, and recording the initial coordinates and the end coordinates of the mouse movement in the two-dimensional plane; S43, using the OpenGL view matrix transformation relationship, converting the two-dimensional coordinates of the mouse into three-dimensional coordinates in three-dimensional space; S44, according to the definition of the three-dimensional sphere, the three-dimensional coordinates of the initial coordinates and the final coordinates are distributed on the spherical surface of the three-dimensional sphere; a first vector connecting the center of the three-dimensional sphere and the initial coordinates and a second vector connecting the center of the three-dimensional sphere and the final coordinates are respectively constructed, and a rotation axis and a rotation angle are obtained by calculating the first vector and the second vector; S45, taking the rotation axis and the rotation angle obtained in step S44 as input, and using the Rodriguez formula to calculate the target pose estimation matrix of the missing area; S46, converting the target pose estimation matrix calculated in step S45 into the rotation angles of the two axes of the two-axis turntable using the two-axis turntable DH model; S47, controlling the two-axis turntable to move at the rotation angle calculated in step S46, at which point the position of the target object in the optical center coordinate system of the reference camera of the binocular structured light 3D scanner is consistent with the position of the point cloud model view in the UI window, thereby completing the synchronous movement of the two-axis turntable and the point cloud model.

2. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 1, characterized in that: Step S1 includes: calibrating the internal and external parameters of the camera of the binocular structured light 3D scanner.

3. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 2, characterized in that: The step S1 of calibrating the internal and external parameters of the camera of the binocular structured light 3D scanner includes: S11, photographing the calibration plate from different angles and positions to obtain calibration images, wherein the calibration plate is configured with coded marker points and non-coded marker points; S12, identifying the calibration image to obtain the two-dimensional image coordinates of the coded marker points and the two-dimensional image coordinates of the non-coded marker points; performing three-dimensional reconstruction on the coded marker points and the non-coded marker points respectively to obtain the three-dimensional space coordinates of the coded marker points and the three-dimensional space coordinates of the non-coded marker points; S13. Based on the two-dimensional image coordinates and three-dimensional space coordinates of the coded mark points, as well as the two-dimensional image coordinates and three-dimensional space coordinates of the non-coded mark points, an overall bundle adjustment iterative optimization is performed on the internal and external parameters of the camera, with minimizing the reprojection error of the camera as the target loss function, and a scale parameter is added as a constraint to obtain the internal and external parameters of the camera.

4. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 1, characterized in that: In step S2, the hand-eye calibration is performed using the binocular structured light 3D scanner as the eye and the two-axis turntable base as the hand; wherein the coordinate system of the binocular structured light 3D scanner is set at the optical center of the reference camera, the coordinate system of the two-axis turntable base is set at the base of the two-axis turntable, and the mathematical model used for calibration is the eye-outside-hand model.

5. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 4, characterized in that: The hand-eye calibration based on the two-axis turntable in step S2 includes: S21, using a standard calibration plate equipped with coded mark points and non-coded mark points for calibration; during the calibration process, the spatial relative position relationship between the binocular structured light 3D scanner and the two-axis turntable base remains unchanged, and the spatial relative position relationship between the coordinate system of the two-axis turntable table and the standard calibration plate remains unchanged, so that the standard calibration plate is fixed on the two-axis turntable table and moves with the two-axis turntable during the calibration process; S22, controlling the rotation of the two axes of the two-axis turntable to drive the standard calibration plate to different positions and postures in space, and using the binocular structured light 3D scanner to photograph the standard calibration plate during the process to obtain multiple calibration images at different positions and postures; S23, performing binocular matching on the calibration image, and then performing three-dimensional reconstruction, and calculating the relative external parameters of the three-dimensional coordinate points of the reconstructed landmark points and the known global points of the world coordinate system of the standard calibration plate; S24. By using the robot DH modeling method, the two-axis turntable is regarded as a two-axis robot, and a conversion relationship from the base coordinate system of the two-axis turntable to the table coordinate system of the two-axis turntable is established; by using the Rodriguez formula and the rotation angles of the two axes of the two-axis turntable, a conversion matrix from the base coordinate system of the two-axis turntable to the table coordinate system of the two-axis turntable at different positions and postures of the two-axis turntable table is calculated; S25. Use the relative external parameters obtained in step S23 and the transformation matrix obtained in step S24 as data input for the hand-eye calibration process, solve them using the hand-eye calibration algorithm, and obtain a calibration result matrix from the optical center of the reference camera to the two-axis turntable base coordinate system.

6. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 5, characterized in that: During the hand-eye calibration process in step S2, for the calibration image captured by the binocular structured light 3D scanner, the following hand-eye relationship equation is satisfied: AX=XB The hand-eye relationship equation indicates that for each calibration posture X, there are different A matrices and B matrices. The A matrix represents the external parameter matrix obtained by camera calibration, and the B matrix represents the transformation matrix from the base coordinate system to the table coordinate system of the two-axis turntable at each position. The B matrix is ​​obtained through DH modeling of the two-axis turntable.

7. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 6, characterized in that: Obtaining the B matrix through DH modeling of a two-axis rotary table includes: By setting respective coordinate systems for the connecting rods of the two-axis turntable, the parameters of the kinematic model of the two-axis turntable are determined by moving the coordinate systems, and the parameters of the kinematic model are converted into a transformation matrix B from the two-axis turntable base coordinate system to the two-axis turntable table top coordinate system according to the three-dimensional rigid body transformation in space.

8. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 1, characterized in that: Step S3 specifically includes: S31, placing the target object at the center of the table of the two-axis turntable and fixing it to ensure that the relative position relationship between the target object and the two-axis turntable remains unchanged during the scanning process; S32, controlling the two-axis turntable to move to different positions and postures, and using the binocular structured light 3D scanner to scan the target object at each position and posture to obtain multiple sets of point cloud data; S33, splicing the multiple groups of point cloud data using a marker-free splicing method to obtain three-dimensional digital model point cloud data of the target object.

9. The intelligent re-scanning method based on a two-axis turntable as claimed in claim 1, characterized in that: Step S5 specifically includes: S51, using the binocular structured light 3D scanner to collect a single point cloud of the target object in the target position; S52. Calculate the transformation relationship between the single point cloud and the point cloud model according to the target pose estimation matrix calculated in step S45 and the reference transformation matrix recorded in step S41, so as to transform the scanned single point cloud from the target pose in the reference camera optical center coordinate system to the reference pose in the reference camera optical center coordinate system, and obtain the point cloud data after the scan of the missing area.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the intelligent re-scanning method as described in any one of claims 1 to 9 can be implemented.