A three-coordinate machine intelligent composite joint inspection system and method based on stereo vision

Abstract

The present invention discloses an intelligent composite joint inspection system and method for a three-coordinate machine based on stereo vision. A multi-degree-of-freedom robot carries a stereo camera to obtain point cloud data of a workpiece to be measured; the posture of the workpiece to be measured is feature matched with the system workpiece, the specifications and models are identified, and the scanning path of the multi-degree-of-freedom robot is planned. The three-coordinate machine unit accurately locates the structural features of the workpiece to be measured; it is judged whether the scanning path planning is successful; the posture of the three-coordinate machine unit measuring the structural features of the workpiece to be measured is accurately located through feature extraction; the three-dimensional point cloud obtained by the stereo camera and the high-precision measurement data of the three-coordinate machine unit are integrated to perform high-precision three-dimensional reconstruction of the workpiece to be measured. The present invention intelligently guides the three-coordinate machine probe to automatically perform measurement program posture correction to adapt to the processing deviation or deformation of the parts, and can adapt to the needs of rapid and automatic production change, which can ensure the precise adaptation of the workpiece to be measured and quickly adapt to the production change of the workpiece to be measured; it is used to guide assembly, grinding, welding and other processes.

Description

Technical Field

[0001] The present invention belongs to the field of measuring equipment, and in particular relates to a flexible composite joint inspection system and method using a non-contact structured light stereo camera combined with a three-coordinate machine. Background Art

[0002] Measurement technology has advanced rapidly in recent years, and fast, accurate 3D measurement has become a crucial component of the manufacturing process, encompassing design, development, processing, testing, and quality assurance. CMMs, with their high accuracy, adaptability, and robust performance, have become the most commonly used 3D precision measurement equipment in the manufacturing industry, finding widespread use in aerospace, automotive, shipbuilding, equipment manufacturing, and mold making.

[0003] However, with the rapid development of the manufacturing industry, the shortcomings of the three-coordinate machine have gradually become more prominent: (1) As a point-by-point measurement device, the three-coordinate machine needs to rely on manual teaching programming to guide the measuring probe for measurement, which is labor-intensive, has low measurement efficiency, and is not intelligent enough; (2) Programming needs to be based on the morphological characteristics and measurement items of the object being measured. The object being measured needs to be equipped with fixtures and other tools to strictly ensure the consistency of each placement position before it can be applied. If the processing quality and size of different batches of the same part are different, when the difference is large, the original measurement program cannot be applied, and re-teaching programming is required; (3) For the case where the three-coordinate machine can program the measurement trajectory based on the part digital model, when the part processing accuracy is not high or deformation occurs after processing, the actual size of the part is significantly different from the digital model, resulting in the measurement program being unapplicable; (4) The three-coordinate machine program is specialized. When the object being measured is replaced, it is necessary to manually call the specialized program according to the specifications and models of the object being measured, which makes it impossible to achieve rapid and automatic production change. Especially for small batches and multiple varieties, the disadvantages of conventional three-coordinate machines are more obvious.

[0004] Therefore, there is an urgent need for a high-efficiency and highly intelligent three-coordinate measurement system that does not require special fixtures to strictly ensure the placement posture, can adapt to the dimensional deviation and deformation of different batches of processing, and can achieve rapid and automated production change. Summary of the Invention

[0005] In order to solve the above-mentioned defects in the prior art, the purpose of the present invention is to provide an intelligent flexible composite joint inspection system and method for a three-coordinate machine based on non-contact stereo vision, which integrates non-contact stereo vision with a contact three-coordinate machine. While retaining the high measurement accuracy of the three-coordinate machine, it can intelligently guide the three-coordinate machine probe to automatically correct the measurement program posture. It does not require special tooling for strict positioning, can adapt to the processing deviation or deformation of parts, and can identify the specifications and models of the workpiece to be measured, automatically call the contact measurement program, and meet the needs of rapid and automatic production change.

[0006] The present invention is achieved through the following technical solutions.

[0007] In one aspect, the present invention provides a three-coordinate machine intelligent composite joint inspection method based on stereo vision, comprising:

[0008] The multi-degree-of-freedom robot carries a stereo camera to scan the workpiece to be measured at the initial position to obtain point cloud data of the surface of the workpiece to be measured;

[0009] Through point cloud data processing, the position and posture of the workpiece to be measured on the three-dimensional coordinate machine workbench are obtained, and feature matching is performed with the existing workpiece digital model information of the system to identify the specifications and models of the workpiece to be measured. The system then calls the three-dimensional coordinate machine unit measurement program corresponding to the workpiece to be measured;

[0010] The system plans the motion scanning path of the multi-degree-of-freedom robot and stereo camera, and the three-coordinate machine unit accurately locates the structural features of the workpiece to be measured;

[0011] If the system's autonomous scanning path planning is successful, the current placement of the workpiece to be measured satisfies the measurement operation of the three-dimensional coordinate machine unit; if the system's automatic scanning path planning fails, the system prompts you to manually change the position of the workpiece to be measured;

[0012] The system controls the multi-degree-of-freedom robot carrying a stereo camera to scan the workpiece to be measured along the planned path to obtain point cloud information. After the acquisition is completed, the multi-degree-of-freedom robot carrying the stereo camera returns to the initial position;

[0013] Combined with the acquired point cloud data of the workpiece to be measured, the position and posture of the structural features of the workpiece to be measured are accurately located by the three-dimensional coordinate machine unit through feature extraction;

[0014] The system corrects the measurement point posture of the three-dimensional coordinate machine unit according to the actual posture information of the three-dimensional coordinate machine unit, and guides the three-dimensional coordinate unit measurement probe to the specified position to perform the measurement operation;

[0015] The system integrates the three-dimensional point cloud obtained by the stereo camera and the high-precision measurement data of the three-coordinate machine unit to perform high-precision three-dimensional reconstruction of the workpiece to be measured.

[0016] Preferably, the stereo camera uses a structured light stereo camera to scan the workpiece to be measured, actively projects a grating, collects the grating fringes modulated by the workpiece surface, and solves and generates surface point cloud data of the workpiece to be measured.

[0017] Preferably, the local point cloud data collected by the stereo camera is used to perform feature matching on the position of the workpiece to be measured on the three-coordinate machine workbench with the existing workpiece digital model information of the system, so as to obtain the specifications and model of the workpiece to be measured and automatically call the contact measurement program of the three-coordinate machine.

[0018] Preferably, feature matching is to use a rigid body transformation matrix to make two point sets in different coordinate systems coincide with each other, including coarse registration and fine registration;

[0019] Rough registration, which performs rough matching by extracting geometric features or contour curves of point clouds;

[0020] Precise registration is to establish the mapping relationship between corresponding point pairs by using the closest point iterative method (ICP) algorithm when the initial positions of the two point cloud data sets are known. The least squares method is used to iteratively solve the final optimized rigid body transformation matrix to obtain high matching accuracy.

[0021] As a preference, the closest point iterative method ICP algorithm is used to establish a mapping relationship between corresponding point pairs, including two sampling data of the same point with a distance of 0 after point set alignment, and point sets P and Q of the same object that are not in the same coordinate system, and to find the rigid body transformation between the two point sets so that the sum of the distances between the corresponding points of the two point sets is minimized.

[0022] Preferably, the precise registration is to search the system model library for the part closest to the workpiece to be measured, determine the specifications and models of the workpiece to be measured, and then call the three-coordinate machine contact measurement program of the workpiece to be measured.

[0023] Preferably, the system solves the true posture of the workpiece based on the point cloud of the workpiece to be measured collected by the stereo camera, and automatically plans the motion scanning path of the multi-degree-of-freedom robot and the stereo camera. The stereo camera is perpendicular to the target surface to be collected, and when within the measurement range, the relative posture relationship between the stereo camera and the target position when collecting the point cloud is obtained.

[0024] Preferably, the system plans the motion scanning path of the multi-degree-of-freedom robot and the stereo camera, and adopts the RRT-Connect algorithm for path planning; a collision detection algorithm is used to detect each point on the planned path. If a collision occurs, the point is discarded and the path point is reselected until a collision-free path connecting the starting point and the end point is obtained.

[0025] In another aspect, the present invention provides a three-coordinate machine intelligent composite joint inspection system based on stereo vision used in the method, comprising:

[0026] The multi-degree-of-freedom robot unit moves with the stereo vision unit according to the instructions of the industrial control computer to obtain point cloud data of the surface of the workpiece to be measured;

[0027] Stereo vision unit, which obtains point cloud data of the workpiece to be measured through a stereo camera;

[0028] The three-coordinate machine unit accurately locates the structural features of the workpiece to be measured;

[0029] Industrial control computers plan the motion scanning paths of multi-degree-of-freedom robots and stereo cameras, control the motion of multi-degree-of-freedom robots, collect information from stereo cameras, and perform measurement operations using three-coordinate machine units.

[0030] Preferably, the industrial control computer includes:

[0031] Offline calibration module, used to calibrate the stereo camera and the multi-degree-of-freedom robot, and obtain the calibration transformation matrix of the stereo camera, the multi-degree-of-freedom robot and the three-coordinate machine unit coordinate system;

[0032] A stereo camera acquisition module is used to control the stereo camera to acquire point cloud data on the surface of the workpiece to be measured;

[0033] Path planning module, used to plan the stereo camera acquisition view angle planning and the motion path planning of the multi-degree-of-freedom robot;

[0034] The stereo vision intelligent recognition module processes and analyzes point cloud data, extracts the structural features of the workpiece to be measured, matches the features with the system's existing workpiece digital and analog information, and identifies the specifications and models of the workpiece to be measured;

[0035] The non-contact 3D measurement module processes and analyzes point cloud data, performs feature segmentation and extraction, calls relevant measurement algorithms, and calculates the true position and posture information of the structural features of the workpiece to be measured;

[0036] The three-coordinate machine guidance module corrects the measurement point pose information in the three-coordinate machine unit contact measurement program based on the acquired real pose information of the workpiece feature to be measured, and updates the three-coordinate machine contact measurement program;

[0037] The contact measurement module of the three-dimensional coordinate machine reads the updated contact measurement program and guides the measuring probe of the three-dimensional coordinate machine unit to the correct position after correction to perform the contact measurement operation.

[0038] The present invention adopts the above technical solution, which has the following beneficial effects:

[0039] 1. This invention uses a multi-degree-of-freedom robot carrying a 3D camera to scan the topographic features of the workpiece from multiple perspectives, acquiring 3D point cloud data of the workpiece surface. The system automatically calculates the actual position of the workpiece or measured feature, automatically corrects the point information in the CMM's contact measurement program, and guides the CMM's measuring probe to the corresponding position to perform contact measurement. The workpiece does not need to be strictly placed in a fixed position on the CMM's worktable. For workpieces with machining errors or deformation, there is no need to manually re-teach the points; the system automatically adapts.

[0040] 2. The system uses a stereo camera to collect point cloud data of the workpiece to be measured, automatically segments and extracts typical features, combines existing workpiece digital and analog information, identifies the specifications and models of the workpiece to be measured, and automatically calls the corresponding measurement program to perform the measurement operation without human intervention. It can quickly adapt to the production change of the workpiece to be measured and has a high degree of intelligence.

[0041] 3. The workpiece to be measured does not need to be strictly placed in a fixed position on the workbench of the three-dimensional coordinate machine. The system obtains the actual position of the workpiece through non-contact scanning by a stereo camera, automatically corrects the points in the measurement program to guide the three-dimensional coordinate machine measurement, reducing the requirements for manual placement of the workpiece and the corresponding tooling costs, and is more adaptable.

[0042] 4. In case of dimensional deviation or deformation after processing in different batches of the same workpiece, the system can use the surface point cloud data of the workpiece to be measured collected by the stereo camera to obtain the true shape and size of the workpiece to be measured, and automatically correct and guide the three-axis coordinate machine probe to the corresponding position to perform the measurement operation. There is no need to manually reprogram the three-axis machine, which is highly flexible.

[0043] 5. The system uses a multi-degree-of-freedom robot carrying a stereo camera to scan the workpiece from multiple perspectives. For dimensions that do not require high measurement accuracy or cannot be measured by a three-dimensional coordinate machine, the system can directly output measurement results based on three-dimensional point cloud data analysis and processing, realizing flexible composite detection that combines non-contact stereo vision with high-precision measurement of a contact three-dimensional coordinate machine.

[0044] 6. The system can integrate the 3D point cloud data obtained by the stereo camera and the high-precision measurement data of the three-dimensional coordinate machine to perform high-precision 3D reconstruction of the workpiece to be measured, which can be used for subsequent process guidance such as assembly, grinding, and welding. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute an improper limitation of the present invention. In the drawings:

[0046] Figure 1 This is a schematic diagram of the composition of a three-coordinate machine intelligent flexible composite joint inspection system based on stereo vision of the present invention;

[0047] Figure 2 It is a schematic diagram of the composition of the three-coordinate machine unit;

[0048] Figure 3 This is a block diagram of the composition of a three-coordinate machine intelligent flexible composite joint inspection system based on stereo vision of the present invention;

[0049] Figure 4 The present invention is a schematic flow chart of a working method of a three-coordinate machine intelligent flexible composite joint inspection system based on stereo vision.

[0050] Figure 5 It is a schematic diagram of the assembly hole structure of an engine cylinder body measured by a three-coordinate machine according to an embodiment of the present invention.

[0051] In the figure: 1. Multi-degree-of-freedom robot unit; 2. Stereo vision unit; 3. Three-coordinate machine unit; 4. Industrial control computer; 5. Example of workpiece to be measured; 11. Multi-degree-of-freedom robot body; 12. Mounting base; 13. Robot controller; 21. Stereo camera; 22. Camera mounting bracket; 31. Workbench; 32. Gantry; 33. Mobile bracket; 34. Measuring probe; 35. Gantry support; 36. Bottom support; 37. Three-coordinate machine controller; 41. Offline calibration module; 42. Stereo camera acquisition module; 43. Path planning module; 44. Stereo vision intelligent recognition module; 45. Non-contact three-dimensional measurement module; 46. Three-coordinate machine guidance module; 47. Three-coordinate machine contact measurement module. DETAILED DESCRIPTION

[0052] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The exemplary embodiments and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0053] like Figure 1 、 Figure 2 As shown, an embodiment of the present invention provides an intelligent, flexible, and combined inspection system for a three-dimensional coordinate machine (CCM) based on stereo vision. The system comprises a multi-DOF robot unit 1, a stereo vision unit 2, a CCM unit 3, and an industrial control computer 4. The multi-DOF robot unit 1 is fixedly positioned near the CCM unit 3 via a base 11, ensuring that the CCM unit 3's worktable is within the effective range of motion of the multi-DOF robot unit. The multi-DOF robot unit 1 is connected to the industrial control computer 4 for signal transmission, receiving coordinate information and instructions and carrying the stereo vision unit 2 to a designated location. The stereo vision unit 2 is fixedly mounted on the end flange of the multi-DOF robot unit 1 and is used to scan the topography of the workpiece to be measured and acquire information such as three-dimensional point cloud data. The stereo camera 21 on the stereo vision unit 2 is connected to the industrial control computer 4 for signal transmission of instructions and information such as three-dimensional point cloud data. The CCM unit 3 is fixedly mounted on the ground for contact measurement of relevant dimensions of the workpiece to be measured. The installation of the CCM unit 3 must meet the environmental and vibration requirements of conventional CCM installations.

[0054] like Figure 2As shown, the three-dimensional coordinate machine unit 3 includes a workbench 31, a gantry 32, a mobile bracket 33, a gantry support 35, a measuring probe 34, a bottom support 36, and a three-dimensional coordinate controller 37. The workbench 31 is located on the bottom support 36. The gantry support 35 supports the gantry 32 on the bottom support 36. The mobile bracket 33 carries the measuring probe 34 and is slidably connected to the gantry 32. The three-dimensional coordinate machine controller 37 is located on the side of the bottom support 36. The three-dimensional coordinate machine controller 37 is connected to the industrial control computer 4 for signal transmission. It receives the corrected position calculated by the system and guides the measuring probe 34 to the actual position of the measurement object for contact measurement. It also receives start and stop commands from the industrial control computer and executes the measurement operation according to the system requirements.

[0055] like Figure 3 As shown, the industrial control computer 4 is connected to the multi-degree-of-freedom robot unit 1, the stereo vision unit 2, and the three-coordinate machine unit 3 through Ethernet for transmission of information such as data and control instructions.

[0056] The multi-freedom robot unit 1 comprises a multi-freedom robot 11, a mounting base 12, and a robot controller 13. The selection of the multi-freedom robot 11 must ensure that its arm span covers the effective measurement area of ​​the CMM worktable 31. The robot mounting base 12 is designed in detail based on the range of motion of the multi-freedom robot 11 and the dimensions of the CMM unit 3, taking into account the application scenario.

[0057] The stereo vision unit 2 includes a stereo camera 21 and a camera mounting bracket 22. The stereo camera 21 is fixedly connected to the end flange of the multi-degree-of-freedom robot 11 via the camera mounting bracket 22. The stereo camera 21 can be of various types, including but not limited to structured light stereo cameras and binocular cameras. The camera mounting bracket 22 can be designed in various forms based on the multi-degree-of-freedom robot 11 and the stereo camera 2, in accordance with site requirements. However, the camera mounting bracket 22 must ensure that the multi-degree-of-freedom robot 11 does not interfere with the three-dimensional coordinate machine unit 3 when carrying the stereo camera 21.

[0058] The industrial control computer 4 controls the movement of the multi-degree-of-freedom robot 11 , the stereo camera 21 to collect information of the workpiece to be measured, and the three-coordinate machine unit 3 to perform measurement operations through system software.

[0059] like Figure 3 As shown, the industrial control computer 4 is equipped with an offline calibration module 41, a stereo camera acquisition module 42, a path planning module 43, a stereo vision intelligent recognition module 44; a non-contact three-dimensional measurement module 45, a three-coordinate machine guidance module 46 and a three-coordinate machine contact measurement module 47.

[0060] The offline calibration module 41 can be used to perform calibration operations on the stereo camera 21 and the multi-degree-of-freedom robot 1 to obtain a calibration transformation matrix of the coordinate systems of the stereo camera 21 and the multi-degree-of-freedom robot 1 .

[0061] The offline calibration module 41 can be used to perform calibration operations on the stereo camera 21 and the three-coordinate machine unit 3 to obtain a calibration transformation matrix of the coordinate system of the stereo camera 21 and the three-coordinate machine unit 3 .

[0062] The stereo camera acquisition module 42 is used to control the stereo camera to acquire point cloud data of the surface of the workpiece to be measured.

[0063] The path planning module 43 is used to plan the acquisition viewing angle of the stereo camera 21 and the motion path of the multi-degree-of-freedom robot 11.

[0064] The stereo vision intelligent recognition module 44 processes and analyzes the point cloud data to extract the structural features of the workpiece to be measured, matches the features with the existing workpiece digital and analog information in the system, and identifies the specifications and models of the workpiece to be measured.

[0065] The non-contact three-dimensional measurement module 45 processes and analyzes the point cloud data, performs feature segmentation and extraction, calls relevant measurement algorithms, and calculates the true position and posture information of the structural features of the workpiece to be measured.

[0066] The three-coordinate machine guiding module 46 automatically corrects the measurement point pose information in the contact measurement program of the three-coordinate machine unit 3 according to the acquired real pose information of the feature to be measured, and updates the contact measurement program of the three-coordinate machine.

[0067] The three-dimensional coordinate machine contact measurement module 47 reads the updated contact measurement program and guides the measuring probe 34 of the three-dimensional coordinate machine unit 3 to the correct position after correction to perform the contact measurement operation.

[0068] like Figure 4 As shown, the embodiment of the present invention also provides a three-coordinate machine intelligent composite joint inspection method based on stereo vision, comprising the following steps:

[0069] 1. Preparation

[0070] (1) Import of digital models of all measured workpieces

[0071] All mathematical models of workpieces to be measured are imported into the system database for verification of their specifications. When using stereo vision to guide the correction of contact measurement points on the CMM unit, the multi-degree-of-freedom robot, equipped with a stereo camera, first acquires a point cloud of the workpiece surface to be measured. This point cloud is then reconstructed in 3D and compared with all mathematical models of the workpiece in the system database to determine its specifications. The system then automatically retrieves the CMM unit measurement program for the corresponding workpiece.

[0072] (2) Input of stereo camera acquisition control parameters

[0073] The acquisition parameters of the stereo camera are input into the system in advance. The input acquisition control parameters are automatically called when the stereo camera shoots, so as to obtain more stable point cloud data of the workpiece to be measured. The acquisition control parameters of the stereo camera can be tested repeatedly to obtain the most suitable parameter combination.

[0074] (3) Offline calibration of stereo cameras and multi-DOF robots:

[0075] The stereo camera is fixedly mounted on the end flange of the multi-DOF robot in an eye-on-hand configuration. The coordinate transformation matrix between the stereo camera and the multi-DOF robot is calibrated offline. The calibration method involves placing a stereo cooperative target within the effective range of motion of the multi-DOF robot and keeping it stationary. The multi-DOF robot, carrying the stereo camera, is then controlled to move to different positions to capture the target from different perspectives. This generates multiple sets of point cloud data for the cooperative target in different poses, while also recording the pose information of the multi-DOF robot during camera capture. Then, based on feature matching principles and combined with the known pose information of the multi-DOF robot, the coordinate transformation matrix between the multi-DOF robot and the stereo camera is calculated, completing the calibration of the stereo camera and the multi-DOF robot.

[0076] (4) Offline calibration of stereo camera and three-coordinate machine unit:

[0077] The calibration of the stereo camera and the three-coordinate machine unit is eye-to-hand and is performed offline. The specific calibration method is as follows: a cooperative target is installed on the probe head of the three-coordinate machine unit, and the multi-degree-of-freedom robot carrying the stereo camera moves to a certain posture and remains unchanged, and collects the cooperative target point cloud; the various joint axes of the three-coordinate machine unit are controlled to transform the posture of the cooperative target, and the stereo camera collects the cooperative target point cloud at different postures, and at the same time records the posture information of each joint axis of the three-coordinate machine when the stereo camera is collecting. Then, based on the feature matching principle and combined with the known posture information of each joint axis of the three-coordinate machine unit, the coordinate transformation matrix of the three-coordinate machine unit and the stereo camera is solved to complete the calibration of the stereo camera and the three-coordinate machine unit. When controlling the transformation of each joint axis of the three-coordinate machine unit, it is necessary to ensure that the cooperative targets are within the effective field of view of the stereo camera.

[0078] 2. Stereo vision guides the three-dimensional coordinate machine to execute the measurement process

[0079] The workpiece to be measured is placed on the workbench of the three-dimensional coordinate machine unit without strict positioning. The specific process of the three-dimensional vision-guided three-dimensional coordinate machine unit to perform the measurement operation is as follows:

[0080] (1) Stereo camera collects the initial point cloud of the workpiece

[0081] The multi-degree-of-freedom robot, carrying a stereo camera, scans the workpiece 5 at its initial position to acquire point cloud data. The stereo camera uses structured light to scan the workpiece, actively projecting a grating, collecting the modulated grating fringes on the workpiece surface, and generating surface point cloud data.

[0082] (2) Obtaining the true pose

[0083] The system processes and analyzes the point cloud data collected in step (1) to preliminarily obtain the position and posture of the workpiece to be measured on the worktable of the three-coordinate machine unit.

[0084] (3) Identify the specifications and models of the parts to be tested and retrieve the measurement program

[0085] The system performs feature matching on the point cloud data obtained in step (1) with the existing workpiece digital model information of the system based on the feature matching principle to identify the specifications and models of the workpiece to be measured. According to the specifications and models of the workpiece to be measured, the system automatically calls the three-coordinate machine unit measurement program corresponding to the workpiece to be measured.

[0086] Specifically, feature matching uses the rigid body transformation matrix to make two point sets in different coordinate systems coincide with each other. It includes two steps: coarse registration and fine registration:

[0087] 1) Coarse registration is to perform rough matching by extracting the geometric features or contour curves of the point cloud.

[0088] 2) Precision registration is to establish a mapping relationship between corresponding point pairs using the closest point iterative method (ICP) when the initial positions of two point cloud datasets are known. The least squares method is then used to iteratively solve the final optimized rigid body transformation matrix to achieve higher matching accuracy. Alternatively, the system model library is used to find the part closest to the workpiece to be measured. This is used to determine the specifications and model of the workpiece to be measured, and then the contact measurement program of the three-dimensional coordinate machine is called up for the workpiece to be measured.

[0089] The calculation process of the ICP algorithm is as follows: the initial iterative position of the two point cloud data is generally based on the result of coarse registration. That is, the distance between the two sampling data at the same point on the object surface is 0 after point set registration. Translated into mathematical interpretation: given point sets P and Q of the same object in different coordinate systems, find the rigid body transformation between the two point sets so that the sum of the distances between the corresponding points of the two point sets is minimized, that is:

[0090]

[0091] Where, P i is the initial point set of reference data, Q i is the corresponding P in the target data point set iThe nearest point, R is a 3×3 rotation matrix, T is a 3×1 translation vector, R and T together constitute the entire rigid body transformation, N is the number of points in the point set, and F(R,T) represents the sum of the squares of the corresponding point distances between the two point sets after the reference point set is rotated and translated.

[0092] When performing ICP iterative calculations, you must first find a solution to a problem, then convert it into a linear optimization problem to obtain the final optimal solution. That is, the following equation:

[0093] ε(P,Q)=min(d 2 (p i ,q j )),i=1,2,3......,j=1,2,3......,

[0094] The p that minimizes ε(P,Q) in this equation i and q j This is what we are looking for. i represents the data point in the reference point set, q j represents the data points in the target point set, and ε(P,Q) indicates that the sum of the distances between the reference point set and the point pairs in the target point set is minimized.

[0095] (4) Scanning path planning

[0096] Based on the dimensions of the CMM unit requiring contact measurement and the preliminary pose information and structural features of the workpiece obtained in step 2, the system automatically plans the scanning path for the multi-degree-of-freedom robot carrying the stereo camera. Scanning path planning includes planning the stereo camera's acquisition angle of view and the multi-degree-of-freedom robot's motion path.

[0097] The acquisition viewing angle of a stereo camera refers to the distance and posture at which the stereo camera collects the point cloud of the workpiece surface.

[0098] The planning process is as follows: the position of the coordinate machine unit that needs to be measured by contact is taken as the target position G t The measurement range and field of view of the stereo camera are used as the known input information M. The purpose of the stereo camera acquisition angle planning is to ensure that the stereo camera can collect the point cloud data of the part that needs to be measured by contact. Generally speaking, when the stereo camera field of view is perpendicular to the target surface to be collected, and the target surface to be collected is within the measurement range, the quality of the obtained point cloud is relatively good. From this, we can know the relative pose relationship G between the stereo camera and the target position when collecting the point cloud. d The target position G to be collected t The preliminary pose information and structural features of the workpiece to be measured are obtained through step 2. In this way, the pose information G of the point cloud of the target position to be measured collected by the stereo camera can be obtained. m , complete the acquisition perspective planning of the stereo camera.

[0099] Path planning for a multi-degree-of-freedom robot involves determining the correct posture and path for the multi-degree-of-freedom robot to capture a point cloud from a workpiece surface using a stereo camera. The stereo camera's pose for capturing the point cloud is determined by planning the camera's acquisition angle. The coordinate transformation between the stereo camera and the multi-degree-of-freedom robot is also determined through calibration. This allows the position of the multi-degree-of-freedom robot's flange 11 to be calculated, corresponding to the point cloud captured by the stereo camera at the target location.

[0100] When there are multiple acquisition points, the scanning path of the multi-degree-of-freedom robot must also be planned. That is, a trajectory that meets the constraints must be found for the multi-degree-of-freedom robot from one acquisition point to another. Based on the trajectory, the hardware parameters of the multi-degree-of-freedom robot are combined to improve information such as acceleration and speed to form a complete trajectory. When planning the scanning path, the target point and planning constraints of the multi-degree-of-freedom robot are set. The RRT-Connect algorithm is used for path planning, and a collision detection algorithm is used to detect whether there is a collision on the planned path at each point. If a collision exists, the point is discarded and the path point is reselected until a collision-free path connecting the starting point and the end point is generated. The planning constraints here are that the stereo camera and the multi-degree-of-freedom robot will not collide or interfere with the workpiece to be measured and the three-coordinate machine unit.

[0101] (5) Determine whether the current placement posture meets the measurement requirements

[0102] If the system's autonomous scanning path planning is successful, the current placement position of the workpiece to be measured meets the measurement requirements of the three-coordinate machine unit; otherwise, if the system's automatic scanning path planning fails, the current placement position of the workpiece cannot meet the measurement requirements of the three-coordinate machine unit, and the system prompts you to manually change the position of the workpiece to be measured on the three-coordinate machine workbench.

[0103] (6) Stereo camera scans the point cloud of the workpiece to be measured

[0104] The system controls the multi-degree-of-freedom robot carrying a stereo camera to automatically scan the workpiece to be measured according to the planned path to obtain point cloud information of the workpiece to be measured. After the acquisition is completed, the multi-degree-of-freedom robot carrying the stereo camera returns to the initial position, which will not interfere with the three-coordinate machine unit when performing measurement operations.

[0105] (7) Accurate positioning of features to be measured

[0106] Combined with the workpiece point cloud data obtained in step 6, the system accurately locates the true position of the structural features that require contact measurement by the three-coordinate machine unit through feature extraction.

[0107] (8) Correct the measuring point of the three-coordinate machine unit and perform contact measurement operations

[0108] Based on the actual pose information of the feature to be measured obtained in step 7, the system automatically corrects the pose information of the measuring point in the contact measurement program of the three-dimensional coordinate machine unit, and guides the measuring probe of the three-dimensional coordinate machine unit to the correct position after correction to perform the contact measurement operation.

[0109] (9) Accurate 3D reconstruction to guide production

[0110] The system integrates the three-dimensional point cloud obtained by the stereo camera and the high-precision measurement data of the three-coordinate machine unit to perform high-precision three-dimensional reconstruction of the workpiece to be measured, which can be used for subsequent process guidance such as assembly, grinding, and welding.

[0111] The present invention is further illustrated below by means of specific examples.

[0112] Taking the measurement of the engine cylinder block as an example, the machining accuracy of the assembly holes on the cylinder block is required to be high. A three-dimensional coordinate machine is needed to measure the size information of the assembly holes on the engine cylinder block, including the position, diameter and depth of holes 1 to 13. Figure 5 There may be discrepancies between the dimensions and processing technology of the engine cylinder block in actual production, but this does not affect the interpretation of the present invention.

[0113] The implementation steps of the technical solution of the present invention are as follows:

[0114] 1. Preparation

[0115] (1) Import of all measured workpiece digital models:

[0116] Import the mathematical models of all engine blocks that need to be measured into the system;

[0117] (2) Input of stereo camera acquisition control parameters:

[0118] Input stereo camera setting parameters into the system;

[0119] (3) Offline calibration of stereo cameras and multi-DOF robots:

[0120] Complete the calibration of the stereo camera and the multi-degree-of-freedom robot, and write the calibration matrix into the system configuration file;

[0121] (4) Offline calibration of stereo camera and three-coordinate machine unit:

[0122] Complete the calibration of the stereo camera and the three-coordinate machine unit, and write the calibration matrix into the system configuration file.

[0123] 2. Stereo vision guides the three-dimensional coordinate machine unit to execute the measurement process

[0124] Place the workpiece to be measured on the workbench of the three-dimensional coordinate machine without strict positioning.

[0125] (1) Stereo camera collects the initial point cloud of the workpiece

[0126] The multi-degree-of-freedom robot carries a stereo camera and scans the workpiece to be tested at the initial position to obtain partial point cloud data of the engine cylinder.

[0127] (2) Obtaining the true pose

[0128] The system processes and analyzes the point cloud data collected in step (1) to preliminarily obtain the position P0 of the engine cylinder on the three-coordinate machine workbench.

[0129] (3) Identify the specifications and models of the parts to be tested and retrieve the measurement program

[0130] The system extracts the significant features ①, ②, ③, and ④ of the engine cylinder from the point cloud data obtained in step (1), compares and analyzes them with the existing digital and analog information of the engine cylinder based on the feature matching principle, identifies the specification model of the engine cylinder to be measured as GT1, and automatically calls the three-coordinate machine contact measurement program chengxu1 corresponding to the engine cylinder GT1.

[0131] (4) Scanning path planning

[0132] Based on the dimensions of the CMM requiring contact measurement and the preliminary pose information and structural features of the workpiece obtained in step 2, the system automatically plans the scanning path for the multi-degree-of-freedom robot carrying the stereo camera. Scanning path planning includes planning the stereo camera's acquisition angle of view and the multi-degree-of-freedom robot's motion path, generating the measurement trajectory L1.

[0133] (5) Determine whether the current placement posture meets the measurement requirements

[0134] The system automatically plans and successfully generates the measurement trajectory L1, so this step is omitted.

[0135] (6) Stereo camera scans the point cloud of the workpiece to be measured

[0136] The system controls the multi-degree-of-freedom robot carrying the stereo camera to automatically scan the engine cylinder GT1 along the measurement trajectory L1 to obtain its point cloud information. After the acquisition is completed, the multi-degree-of-freedom robot carrying the stereo camera returns to the initial position, which will not interfere with the three-dimensional coordinate machine unit when performing measurement operations.

[0137] (7) Accurate positioning of features to be measured

[0138] Because the coordinate machine unit is required to contact-measure the position, diameter, and depth of holes 1 to 13, the system automatically extracts the hole features of holes 1 to 13 based on the workpiece point cloud data obtained in step 5 and calculates the center coordinates P1 to P13 of holes 1 to 13.

[0139] (8) Correct the measuring point of the three-coordinate machine unit and perform contact measurement operations

[0140] Based on the center coordinates P1 to P13 of holes 1 to 13 obtained in step 7, the system automatically corrects the measurement point pose information of chengxu1 in the contact measurement program of the three-coordinate machine unit, guides the three-coordinate machine unit measurement probe to the correct position after correction, and performs contact measurement of the diameter size and hole depth of holes 1 to 13.

[0141] Finally, the center position, diameter and depth of holes 1 to 13 are accurately measured.

[0142] (9) Accurate 3D reconstruction to guide production

[0143] The point cloud acquired by the stereo camera can be used to reversely reconstruct the true 3D model of the engine block. When the measured pose, diameter, and depth of holes 1 to 13 do not meet the requirements and need to be re-processed using a CNC machine tool, the 3D reconstructed model and measurement data can be used to guide the CNC machine tool for fine processing.

[0144] As can be seen from the above embodiments, the present invention utilizes a multi-degree-of-freedom robot equipped with a stereo camera to measure the true pose of the workpiece under test. This pose is then transmitted to a three-dimensional coordinate machine (CMM) unit, guiding the CMM to perform contact measurement. This ensures that the workpiece can be effectively positioned even when it is not precisely positioned. The stereo camera captures a point cloud of the workpiece surface, allowing the true surface state of the workpiece to be determined for post-processing deformation or machining errors between batches. This allows the CMM unit to calculate a corrected pose and perform measurement. The system uses a feature matching method to compare and analyze the point cloud data of the workpiece to be measured, acquired by the stereo camera, with the workpiece digital model. This method obtains the specifications and models of the workpiece to be measured and automatically calls the contact measurement program of the three-dimensional coordinate machine for the workpiece of the corresponding specifications. This method requires no human intervention and can quickly adapt to the needs of changing the production of the workpiece to be measured. By combining the non-contact measurement method of the stereo camera with the contact measurement method of the three-dimensional coordinate machine unit, the system can integrate the fast and efficient characteristics of non-contact measurement with the high precision of contact measurement. For certain features with low measurement accuracy requirements, the point cloud data acquired by the stereo camera can be directly used for solution calculations. For certain key features with higher and more stringent measurement accuracy requirements, the contact measurement method of the three-dimensional coordinate machine unit can be used to obtain the data. The system uses the three-dimensional point cloud data acquired by the stereo camera, combined with the high-precision measurement data of the three-dimensional coordinate machine, to perform high-precision three-dimensional reconstruction of the workpiece to be measured, which can be used for subsequent process guidance such as assembly, grinding, and welding.

[0145] The invention is not limited to the above-mentioned embodiments. On the basis of the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and modifications to some of the technical features therein according to the disclosed technical content without creative labor, and these substitutions and modifications are all within the protection scope of the present invention.

Claims

1. A three-coordinate machine intelligent composite joint inspection method based on stereo vision, characterized in that: include: The multi-degree-of-freedom robot carries a stereo camera to scan the workpiece to be measured at the initial position to obtain point cloud data of the surface of the workpiece to be measured; Through point cloud data processing, the position and posture of the workpiece to be measured on the three-dimensional coordinate machine workbench are obtained, and feature matching is performed with the existing workpiece digital model information of the system to identify the specifications and models of the workpiece to be measured. The system then calls the three-dimensional coordinate machine unit measurement program corresponding to the workpiece to be measured; The system plans the scanning path of the multi-degree-of-freedom robot carrying the stereo camera based on the dimensions of the three-dimensional coordinate machine unit that require contact measurement; If the system's autonomous scanning path planning is successful, the current placement position of the workpiece to be measured satisfies the measurement operation of the three-coordinate machine unit; If the system fails to automatically plan the scanning path, the system will prompt you to manually change the position of the workpiece to be measured; The system controls the multi-degree-of-freedom robot carrying a stereo camera to scan the workpiece to be measured along the planned path to obtain point cloud information. After the acquisition is completed, the multi-degree-of-freedom robot carrying the stereo camera returns to the initial position; Using the local point cloud data collected by the stereo camera, the position of the workpiece on the three-dimensional coordinate machine table is matched with the existing workpiece digital model information in the system to obtain the specifications and model of the workpiece, and the contact measurement program of the three-dimensional coordinate machine is automatically called. Combined with the acquired point cloud data of the workpiece to be measured, the position and posture of the structural features of the workpiece to be measured are accurately located by the three-dimensional coordinate machine unit through feature extraction; The system corrects the measurement point posture of the three-dimensional coordinate machine unit according to the actual posture information of the three-dimensional coordinate machine unit, and guides the three-dimensional coordinate unit measurement probe to the specified position to perform the measurement operation; The system integrates the three-dimensional point cloud obtained by the stereo camera and the high-precision measurement data of the three-coordinate machine unit to perform high-precision three-dimensional reconstruction of the workpiece to be measured.

2. The three-coordinate machine intelligent composite joint inspection method based on stereo vision according to claim 1 is characterized in that: The stereo camera uses a structured light stereo camera to scan the workpiece to be measured, actively projects a grating, collects the grating fringes modulated on the workpiece surface, and solves and generates the surface point cloud data of the workpiece to be measured.

3. The three-coordinate machine intelligent composite joint inspection method based on stereo vision according to claim 1 is characterized in that: Feature matching uses the rigid body transformation matrix to make two point sets in different coordinate systems coincide with each other, including coarse registration and fine registration; Rough registration, which performs rough matching by extracting geometric features or contour curves of point clouds; Precise registration is to establish the mapping relationship between corresponding point pairs by using the closest point iterative method (ICP) algorithm when the initial positions of the two point cloud data sets are known. The least squares method is used to iteratively solve the final optimized rigid body transformation matrix to obtain high matching accuracy.

4. The three-coordinate machine intelligent composite joint inspection method based on stereo vision according to claim 3 is characterized in that: The closest point iterative method (ICP) algorithm is used to establish the mapping relationship between corresponding point pairs. This includes setting the distance between two sampling data of the same point to 0 after point set registration. The point sets P and Q of the same object are not in the same coordinate system. The rigid body transformation between the two point sets is found to minimize the sum of the distances between the corresponding points of the two point sets, that is: Where, is the initial point set of reference data, The target data points correspond to Closest, is the rotation matrix, is the translation vector, and Together they constitute the entire rigid body transformation, N is the number of points in the point set, It is the sum of the squares of the distances between the corresponding points in the two point sets after the reference point set is rotated and translated.

5. The three-coordinate machine intelligent composite joint inspection method based on stereo vision according to claim 3 is characterized in that: Precision registration is to find the part closest to the workpiece to be measured in the system model library, determine the specifications and model of the workpiece to be measured, and then call the three-coordinate machine contact measurement program of the workpiece to be measured.

6. The three-coordinate machine intelligent composite joint inspection method based on stereo vision according to claim 1 is characterized in that: Based on the point cloud of the workpiece to be measured collected by the stereo camera, the system solves the true position and posture of the workpiece, and automatically plans the motion scanning path of the multi-degree-of-freedom robot and the stereo camera. The stereo camera is perpendicular to the target surface to be collected, and when it is within the measurement range, the relative position relationship between the stereo camera and the target position when collecting the point cloud is obtained.

7. The three-coordinate machine intelligent composite joint inspection method based on stereo vision according to claim 1 is characterized in that: The system plans the motion scanning path of the multi-degree-of-freedom robot and stereo camera, and uses the RRT-Connect algorithm for path planning; a collision detection algorithm is used to detect each point on the planned path. If a collision occurs, the point is discarded and a new path point is selected until a collision-free path connecting the starting point and the end point is obtained.

8. A three-coordinate machine intelligent composite joint inspection system based on stereo vision used in the method according to any one of claims 1 to 7, characterized in that: include: The multi-degree-of-freedom robot unit moves with the stereo vision unit according to the instructions of the industrial control computer to obtain point cloud data of the surface of the workpiece to be measured; Stereo vision unit, which obtains point cloud data of the workpiece to be measured through a stereo camera; The three-coordinate machine unit accurately locates the structural features of the workpiece to be measured; Industrial control computers plan the motion scanning paths of multi-degree-of-freedom robots and stereo cameras, control the motion of multi-degree-of-freedom robots, collect information from stereo cameras, and perform measurement operations using three-coordinate machine units.

9. The three-coordinate machine intelligent composite joint inspection system based on stereo vision according to claim 8 is characterized in that: Industrial control computers include: Offline calibration module, used to calibrate the stereo camera and the multi-degree-of-freedom robot, and obtain the calibration transformation matrix of the stereo camera, the multi-degree-of-freedom robot and the three-coordinate machine unit coordinate system; A stereo camera acquisition module is used to control the stereo camera to acquire point cloud data on the surface of the workpiece to be measured; Path planning module, used to plan the stereo camera acquisition view angle planning and the motion path planning of the multi-degree-of-freedom robot; The stereo vision intelligent recognition module processes and analyzes point cloud data, extracts the structural features of the workpiece to be measured, matches the features with the system's existing workpiece digital and analog information, and identifies the specifications and models of the workpiece to be measured; The non-contact 3D measurement module processes and analyzes point cloud data, performs feature segmentation and extraction, calls relevant measurement algorithms, and calculates the true position and posture information of the structural features of the workpiece to be measured; The three-coordinate machine guidance module corrects the measurement point pose information in the three-coordinate machine unit contact measurement program based on the acquired real pose information of the workpiece feature to be measured, and updates the three-coordinate machine contact measurement program; The contact measurement module of the three-dimensional coordinate machine reads the updated contact measurement program and guides the measuring probe of the three-dimensional coordinate machine unit to the correct position after correction to perform the contact measurement operation.

Images