Non-standard special-shaped part high-precision measurement verification method based on three-dimensional vision

Abstract

The invention relates to a non-standard special-shaped part high-precision measurement and verification method based on three-dimensional vision, and belongs to the field of three-dimensional reconstruction and computer vision. Point cloud data of a non-standard special-shaped part are obtained through multi-angle scanning by means of multi-camera-projector collaborative layout in combination with a structured light coding technology and a phase shift code technology; the method comprises the following steps of: acquiring point cloud data, ensuring comprehensiveness and accuracy of the data, realizing high-precision alignment of the point cloud data through matching an underivative optimization algorithm with image feature points, and performing fusion, denoising and registration processing on the acquired point cloud data to ensure that point clouds acquired from a plurality of angles can be effectively fused so as to reconstruct a high-precision three-dimensional model. In the measurement and verification stage, the deviation between the point cloud data and the CAD model is measured through a standard coordinate system, multi-stage registration and a high-precision algorithm, rapid detection and size verification of the geometric deviation of the industrial workpiece are achieved, and reliable technical support is provided for quality control.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction and computer vision, and particularly to a high-precision measurement and verification method for non-standard shaped parts based on three-dimensional vision. Background Art

[0002] In the fields of computer vision and image processing, three-dimensional reconstruction technology has been widely applied in many fields such as virtual reality, augmented reality, and industrial inspection. Especially in the detection and verification of non-standard shaped parts, three-dimensional reconstruction technology can accurately obtain the geometric shape of an object, providing an important basis for subsequent quality analysis and process optimization.

[0003] Conventional three-dimensional reconstruction systems adopt a single-camera system. However, when facing non-standard shaped parts with complex shapes and irregular surfaces, traditional monocular systems often struggle to meet the requirements of high-precision reconstruction. Especially when the object surface has high-reflectivity or low-reflection characteristics, it is prone to incomplete or distorted image data. In addition, due to their complex shapes and diverse surface details, non-standard shaped parts often require data collection from multiple angles to ensure comprehensiveness and accuracy.

[0004] A three-dimensional reconstruction system can obtain point cloud data of an object to be measured through scanning. Point cloud reconstruction technology is one of the core technologies in three-dimensional reconstruction. However, since point cloud data usually contains noise and redundant information, how to effectively denoise and perform efficient point cloud registration and fusion has become a key issue for achieving high-precision reconstruction. Traditional three-dimensional point cloud registration methods mainly rely on feature matching and iterative optimization. When facing complex object surfaces such as non-standard shaped parts, these methods often cannot guarantee accurate registration results.

[0005] Therefore, how to optimize three-dimensional reconstruction technology to achieve efficient and high-precision three-dimensional reconstruction, measurement, and inspection of non-standard shaped parts is a difficult problem currently faced. Summary of the Invention

[0006] Aiming at the defects of the prior art, the present invention provides a method that can achieve high-precision point cloud alignment during the reconstruction of non-standard shaped parts, provide more accurate shape and size analysis, realize rapid detection of geometric deviations of industrial workpieces and size verification, and provide reliable technical support for quality control.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A high-precision measurement and verification method for non-standard shaped parts based on three-dimensional vision, comprising the following steps: Step S1: Select and configure a multi-camera three-dimensional reconstruction system; Step S2: Point cloud acquisition. Using structured light encoding technology, through the cooperation of a projector and a camera, alternately acquire the surface images of different regions of the object at regular intervals, and generate the point cloud data of the object surface through phase shift code encoding and decoding technology; Step S3: Multi-viewpoint cloud stitching. Use the method based on the derivative-free optimization algorithm (BQBYQA) and image feature point matching to accurately align the point cloud data collected from multiple viewpoints; Step S4: Deviation analysis and dimension measurement: Calculate the difference between the point cloud data and the design model to perform dimension measurement and deviation analysis of the object under test.

[0009] Furthermore, in step S1, according to the size and complexity of the non-standard shaped part, select multiple cameras for arrangement to ensure data collection from multiple viewpoints, covering all angles of the object to be measured, and each camera is equipped with a projector, and the projector corresponds to the camera.

[0010] Furthermore, in step S2, project the encoded pattern through the projector, capture and decode the deformed pattern by the camera, and calculate the three-dimensional coordinates of the object by combining the internal and external parameters of the projector and the camera; when using phase shift code encoding, use four phase shift code maps, and solve the phase through the light intensity formula; then calculate the normalized coordinates using the phase difference and the camera internal parameters, and finally obtain the point cloud data of the object surface.

[0011] Furthermore, in step S2, adopt the block sequential projection method for data collection. Divide the object to be measured into multiple regions. In each region, multiple cameras alternately and cross-collect the point cloud data of this region to ensure sufficient data coverage from different viewpoints. After completing the data collection of one region, switch to another region and use the same alternating collection method for data collection, and the collection process of each region is carried out according to the alternating cycle. Different cameras alternately collect data from different angles to ensure that all cameras can cover different viewpoints of the object and obtain sufficient point cloud data.

[0012] Furthermore, step S3 is specifically as follows: S3.1. Establish an error model: Set the initial rotation vector R and the initial displacement vector T of the template point cloud and the target point cloud, and define a reliable semi-domain diameter constraint such that a ≤ [R, T] ≤ b to limit the optimization range; S3.2. Construct an approximation function: Define the approximation function F(yi), which represents the sum of the distances between the points after the projection of two point clouds on the image plane, and is used to measure the pose difference; in the optimization process, minimize this total difference value to optimize the pose yi between the template point cloud and the target point cloud, so that the projections of the template point cloud and the target point cloud on the image are as consistent as possible; S3.3. Quadratic approximation and iterative optimization: Construct a quadratic approximation model Q i (y i) = F(y i );Update the pose parameters using the gradient and approximate Hessian matrix; During each iteration, adjust R and T within the trust region to avoid excessive parameter jumps and ensure stability; S3.4. Convergence determination and result output: Set the threshold of the error function and the maximum number of iterations, terminate the optimization according to the preset error threshold or the maximum number of iterations, output the optimal pose parameters, and complete the high-precision point cloud matching.

[0013] Furthermore, the deviation analysis and dimension measurement in step S4 specifically include the following steps: S4.1. Point cloud data preprocessing; S4.2. Point cloud data normalization and coordinate system reconstruction: Normalize the point cloud data and encode the point cloud data using the Scaffold Coordinate System (SCSS) to generate standardized point cloud data; S4.3. Comparison and matching between the designed model and the point cloud: Discretize the designed model into a point cloud format and match the designed model of the workpiece to be measured and the point cloud data using the SCSS encoding method; S4.4. Deviation analysis: Based on the SCSS ray intersection points, initially align the designed model and the point cloud data, and calculate the original pose difference between the point cloud data and the designed model; After obtaining the pose relationship between the point cloud data and the designed model, use the ICP algorithm to further optimize the alignment of the point cloud data and the designed model to ensure high-precision comparison; S4.5. Dimension measurement: Calculate the geometric deviation between the point cloud data and the designed model, and perform dimension measurement through the corresponding software to obtain the actual dimensions and deviation data of the workpiece.

[0014] Furthermore, in step S4.1, the statistical filtering method is used to denoise the point cloud data.

[0015] Furthermore, step S4.2 is specifically as follows: Normalize the point cloud data, remove obvious outliers, construct a new spatial coordinate system with the centroid of the point cloud data as the coordinate origin, called the "Scaffold Coordinate System (SCSS)", and encode the point cloud data using SCSS to generate standardized point cloud data.

[0016] Furthermore, in step S4.4, the specific method for initially aligning the designed model and the point cloud data based on the SCSS ray intersection points is as follows: Calculate the average distance of the intersection points closest to the intersection points of the SCSS rays and the workpiece surface in the workpiece point cloud data. By rotating the SCSS rays, when the average distance of the closest points is less than the set threshold, this rotation direction is considered to represent the position in the object coordinate system, and thus the original pose difference between the workpiece to be detected and the template point cloud is obtained.

[0017] Furthermore, the design model is a CAD model.

[0018] Beneficial effects:

[0019] 1. The three-dimensional system built by the present invention uses a multi-camera-projector collaborative layout to ensure that each camera can clearly capture different parts of the object. Combining structured light coding technology and phase shift code technology, point cloud data of non-standard shaped parts is obtained through multi-angle scanning, ensuring the comprehensiveness and accuracy of the data.

[0020] 2. The present invention combines the derivative-free optimization algorithm (BQBYQA) with image feature point matching to achieve high-precision alignment of point cloud data: (1) It fuses feature matching and derivative-free optimization, combines image feature point information with the derivative-free characteristics of the BQBYQA algorithm, and improves the robustness to noise and distortion; (2) It allows dynamic adjustment of the camera internal parameters during the optimization process to compensate for the influence of distortion on the point cloud coordinates; (3) It sets a reliable semi-domain constraint, and through parameter range limitation, avoids optimization divergence and enhances the algorithm stability; This method can achieve high-precision point cloud alignment under large pose differences and distortion conditions, improve the robustness to noise and distortion, overcome the problem that the traditional ICP algorithm is sensitive to the initial pose, improve the comprehensiveness of data acquisition and the reconstruction accuracy, solve the reconstruction error problem of traditional methods when facing complex shapes and irregular surfaces, and has high efficiency, can reduce the computational complexity, and is suitable for high-dimensional parameter space optimization.

[0021] 3. The present invention processes the obtained point cloud data through fusion, denoising and registration to ensure that the point clouds obtained from multiple angles can be effectively fused, and then reconstructs a high-precision three-dimensional model, which can provide more accurate shape and size analysis during the reconstruction of non-standard shaped parts, and provide reliable data support for subsequent comparison and verification with the standard CAD model; The present invention also measures the deviation between the point cloud data and the CAD model through a standardized coordinate system, multi-stage registration and high-precision algorithms during the measurement verification stage, realizes the rapid detection and size verification of the geometric deviation of industrial workpieces, and provides reliable technical support for quality control. Description of the drawings

[0022] Figure 1 Schematic diagram of the point cloud acquisition and three-dimensional reconstruction process in the present invention;

[0023] Figure 2 Schematic diagram of the pinhole imaging model;

[0024] Figure 3 Schematic diagram of the projector imaging;

[0025] Figure 4 Schematic diagram of the point cloud acquisition in the embodiment of the present invention;

[0026] Figure 5Schematic diagram of point cloud stitching in the embodiments of the present invention. Detailed implementation manners

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] A high-precision measurement and verification method for non-standard shaped parts based on three-dimensional vision of the present invention proposes an innovative multi-angle point cloud acquisition and fusion method. It optimizes the three-dimensional reconstruction of non-standard shaped parts by using multi-angle data, and improves the comprehensiveness of data acquisition and reconstruction accuracy through multi-angle scanning and point cloud data fusion technology, solving the reconstruction error problem of traditional methods when facing complex shapes and irregular surfaces. By combining the characteristics of multi-view data, it optimizes the denoising, registration and fusion processes of point clouds. Specifically, the method of the present invention includes the following steps:

[0029] S1. Selection and setting of a multi-camera three-dimensional reconstruction system;

[0030] S2. High-speed and high-precision point cloud acquisition;

[0031] S3. Multi-view point cloud stitching scheme;

[0032] S4. Deviation analysis and dimension measurement.

[0033] Due to the complex shape and diverse surface details of non-standard shaped parts, their measurement and inspection are difficult problems in the industry. The method of the present invention proposes a solution to the measurement problem of non-standard shaped parts. Of course, this method can also be used for the measurement and verification of standard parts or conventional parts. Taking the measurement of non-standard shaped parts as an example, the detailed processes of each step of the present invention will be explained below.

[0034] 1. Selection and setting of a multi-camera three-dimensional reconstruction system

[0035] According to the size and complexity of the non-standard shaped part, multiple cameras are selected and arranged to ensure data acquisition from multiple perspectives, covering all angles of the object to be measured, and each camera is equipped with a projector, and the projector corresponds to the camera.

[0036] 1.1 Arrangement and selection of cameras

[0037] In the process of three-dimensional reconstruction of non-standard shaped parts, the selection of cameras is crucial. To ensure high-precision three-dimensional reconstruction results, industrial cameras with high resolution and low noise are required. The cameras are required to have good optical performance and be able to stably provide clear images under different lighting conditions. According to the size and complexity of the non-standard shaped part, the corresponding number of cameras is selected for arrangement. Usually, 4 to 8 cameras are arranged to ensure data acquisition from multiple perspectives and cover all angles of the object. For larger non-standard shaped parts, more cameras may be required to avoid occlusion and incomplete data problems caused by a single perspective.

[0038] The position and angle of each camera should be optimized according to the shape and size of the object, and evenly distributed around the calibration scene to ensure that each camera can clearly capture different parts of the object. For non-standard shaped parts with complex shapes, special attention should be paid to the arrangement and angle of the cameras to ensure that each area can be captured by at least two cameras simultaneously to improve the integrity and accuracy of the reconstruction.

[0039] 1.2 Arrangement and Selection of Projectors

[0040] In the 3D reconstruction system built in the present invention, each camera is equipped with a high-precision projector to ensure the consistency and high precision of the point cloud data collected from multiple angles. The projector should be a device with high resolution and strong stability, which can provide clear and uniform projection patterns under different lighting conditions. The resolution and brightness of the projector need to be high enough to ensure the clarity and contrast of the pattern projection, thus ensuring the quality of the point cloud data.

[0041] The arrangement of the projector should be closely coordinated with the camera arrangement. Each projector corresponds directly to the camera, and the projection pattern should cover the surface of the object. To avoid the overlap of the projection pattern and the camera field of view, the projector should be precisely adjusted according to the spatial layout of the calibration scene. The projector should be arranged directly in front of or in a relative position to the camera to ensure the uniformity of the projection pattern without affecting the camera's viewing angle and shooting range.

[0042] Through this supporting arrangement method of the camera and the projector, it can be ensured that during the multi-angle data acquisition process, the point cloud data obtained by each camera is accurate and complete, and thus provides high-precision data support for the subsequent 3D reconstruction of non-standard shaped parts.

[0043] 2. High-Speed and High-Precision Point Cloud Acquisition

[0044] Adopt structured light encoding technology. Through the cooperation of the projector and the camera, alternately acquire the surface images of different regions of the object in cycles, and generate the point cloud data of the object surface through phase shift code encoding and decoding technology;

[0045] 2.1 Point Cloud Acquisition by Single Device

[0046] Structured light encoding technology is an important technology for obtaining the three-dimensional information of the surface of the part to be detected. The projector projects a black-and-white structured light pattern onto the surface of the part to be detected. After being photographed by the camera and decoded, the corresponding relationship between the pattern photographed by the camera and the pattern projected by the projector is obtained. Using the internal and external parameters between the projector and the camera, the distance and direction between the pattern and the camera are calculated, so as to indirectly obtain the distance between the photographed object and the camera.

[0047] In the 3D scanning system used in the present invention, the structured light pattern adopted is phase-shift code encoding. The encoded pattern is projected by a projector, and after the deformed pattern is captured by a camera and decoded, the 3D coordinates of the object are calculated by combining the internal and external parameters of the projector and the camera. When using phase-shift code encoding, four phase-shift code maps are used, and the phase is solved by the light intensity formula. Then, the normalized coordinates are calculated using the phase difference and the camera internal parameters, and finally the point cloud data of the object surface is obtained. Refer to Figure 1 as shown.

[0048] For 3D scanning based on phase-shift code encoding, the projector projects a sinusoidal grating stripe pattern. The gray value of the stripe pattern changes regularly as the number of rows increases, and the stripes with the same number of rows have the same gray value. After this pattern is projected onto the object surface, the light intensity at a certain point on the object surface captured by the camera is affected by both the pixel gray value projected by the projector to this point and the gray values of the pixels in the upper and lower rows projected by the projector to this point. Assume that two points in space are projected by the same number of pixels of the projector and imaged on the same pixel of the camera. If there is a slight difference in the positions of these two points, the gray values of their imaging pixels on the camera will also have a slight difference due to the influence of their spatial positions.

[0049] The light intensity distribution of the deformed stripes captured by the camera after the sinusoidal grating stripes are projected onto the object surface and encoded and deformed by the object surface height can be expressed as: I(x,y) = a(x,y) + b(x,y) * cos(2 * πf0x + θ(x,y) + φ) (1)

[0050] Where (x,y) represents the coordinates of the pixel point in the image coordinate system, I(x,y) is the actual light intensity (pixel gray value) of this point, a(x,y) is the light intensity without considering the external light source, b(x,y) is the reflection coefficient, representing the ability of this pixel point to become brighter (darker) under the influence of the external light, cos(2 * πf0x + θ(x,y) + φ) represents the brightness distribution of the incident picture. The incident image is a sinusoidal grating stripe pattern projected by the projector, θ represents the phase, and φ is the image phase shift value. φ remains unchanged in a single phase-shift code picture, and φ is different for different phase-shift code pictures.

[0051] In the present invention, the φ values of the four phase-shift code encoding maps used are According to the stripe light intensity distribution, the light intensity of a certain point in space irradiated by the projector with four phase-shift code maps respectively is estimated as:

[0052] In the formula, a and b respectively represent the background light intensity and the modulation amplitude, which are jointly determined by the ambient light and the projection light.

[0053] According to the trigonometric formula and the light intensity estimation, the phase θ is demodulated and solved as:

[0054] According to the definition of the camera's internal parameters:

[0055] We can obtain:

[0056] (u c . v c ): Represents the pixel coordinates on the camera imaging plane; (x c , y c , z c ): The three-dimensional coordinates of the measured point in the camera coordinate system; k cx , k cy : The focal length of the camera (in pixels); (u c0 , v c0 ): The pixel coordinates of the camera's principal point (the intersection of the optical axis and the imaging plane).

[0057] Let: Since the imaging plane equation is x = 1, so x c1 , y c1 represents the x and y coordinate points on the normalized imaging plane, and the normalized coordinates eliminate the depth effect. Simplifying formula (5) gives:

[0058] As Figure 2 shown, according to the pinhole imaging model, we know the straight line equation where the point p(x c , y c , z c ) is located:

[0059] According to the pinhole imaging model, the parametric equation can be obtained:

[0060] This parametric equation is the representation of the straight line in the camera coordinate system. To obtain the actual numerical values of each coordinate of p(x c , y c , z c ), the representation of this straight line equation in the projector coordinate system needs to be obtained.

[0061] Assume that the coordinates of the straight line in the projector coordinate system are:

[0062] Any coordinate (x p , y p , z p), according to the definition of the external parameter matrix

[0063] Substituting the external parameter matrix into equation (10), we can obtain:

[0064] Among them, P Tc : The external parameter matrix between the projector and the camera, including the rotation matrix R and the translation vector T;

[0065] (x p ,y p ,z p ): coordinates of the measured point in the projector coordinate system;

[0066] Rcpx, Rcpy, Rcpz: rotation matrix components used to describe the alignment relationship of coordinate axes;

[0067] Pcp1, Pcp2, Pcp3: Translation components, describing the offset of the coordinate system origin.

[0068] make:

[0069] The equation of the straight line in the projector coordinate system is:

[0070] Point p(x c ,y c ,z c ) is both on the straight line represented by equation (7) and on the plane projected by the projector. The equation of the straight line is known. Let the coordinates of the pixel in the projector image coordinate system be (u p , v p ), according to the definition of the projector camera internal parameters, we can get:

[0071] like Figure 3 As shown, according to the projector imaging diagram, each projection plane of the projector can use the plane equation: cosθ×y+sinθ×z=0(15)

[0072] After the phase shift code is decoded to obtain the value of θ, the light plane equation and the straight line equation in the projector coordinate system are combined to obtain:

[0073] By substituting the coordinates of each point in the image, the three-dimensional coordinates of the spatial point can be obtained.

[0074] In summary, phase information is provided through the phase shift decoding formula, the camera internal parameter model completes the two-dimensional to three-dimensional mapping, the external parameter matrix realizes the alignment of multiple coordinate systems, and the light plane equation constrains the spatial geometric relationship. Finally, the three-dimensional coordinates are solved by solving the simultaneous equations.

[0075] 2.2 Multiple devices cooperate to complete the data acquisition task

[0076] There is a certain response time required between the trigger shooting and the actual shooting by a single camera. To improve the acquisition speed of the original workpiece data, the present invention adopts a block sequential projection method for data acquisition. With the cooperation of multiple cameras, the point cloud data of different regions are alternately acquired in cycles, so as to improve the overall efficiency of data acquisition as much as possible. The specific implementation steps are as follows:

[0077] Region division and cross-acquisition: The object to be measured is divided into multiple regions (such as region A and region B). Within each region, multiple cameras alternately and cross-acquire the point cloud data of region A first to ensure sufficient data coverage from different perspectives. After the data acquisition of region A is completed, switch to region B and use the same alternate acquisition method for data acquisition.

[0078] Periodic alternate acquisition: Different cameras alternately acquire data from different angles, avoiding long-term idleness of each camera, thus effectively improving the data acquisition speed. Specifically, if each camera needs to take 17 pictures each time, for the N cameras and projectors in region A, when the projector at position 1 projects a picture, the camera at position 1 simultaneously acquires it, and when the image data is processed and transmitted at position 1, the projector at position 2 projects a picture for acquisition, and so on. The acquisition process of each region will be carried out according to the alternate cycle to ensure that all cameras can cover different perspectives of the object and obtain sufficient point cloud data.

[0079] Through this block sequential projection and alternate acquisition method, not only the overall acquisition speed is improved, but also the integrity and high precision of the point cloud data in each region are guaranteed, providing high-quality data support for subsequent three-dimensional reconstruction. This data acquisition process can obtain data quickly and effectively, and minimize the influence caused by object jitter.

[0080] 3. Multi-viewpoint cloud stitching

[0081] Matching the feature points of different images can effectively obtain the external parameters of two cameras. However, the vast majority of feature matching methods are applicable only to cameras with extremely similar poses. When the camera pose deviation is too large, feature matching becomes difficult. The commonly used ICP algorithm for point clouds can quickly find a suitable rigid transformation matrix by iteratively calculating the distances between corresponding points in two point cloud data sets, minimizing the sum of the distances between point pairs, and thus completing the calculation of the poses between the two point cloud data sets. However, the ICP algorithm is extremely sensitive to rigid transformations and noise. Due to camera distortion, the point clouds collected from two perspectives cannot completely coincide, which poses new challenges to the ICP algorithm. In the optimization algorithm, it should be appropriate to allow adjustment of the camera internal parameters to change the coordinates of each point.

[0082] The present invention proposes a splicing method based on the derivative-free optimization algorithm (BQBYQA) and image feature point matching to improve the splicing accuracy, and the process is as follows:

[0083] 3.1 Establish an error model

[0084] Let the initial rotation vector of the template point cloud and the target point cloud be R = [a1, a2, a3], and the initial displacement vector be T = [x, y, z]. Set the initial and ending trust region diameters a and b, and let the rotation vector and displacement vector satisfy a ≤ [R, T] ≤ b to limit the optimization range; on the premise that the initial pose estimates of adjacent cameras are relatively accurate, the values of a and b can be set relatively close to the [R, T] values.

[0085] 3.2 Construct an approximation function

[0086] Define the approximation function F(y i ), where the value of y i is the pose between the template point cloud and the target point cloud in the current optimization. At the current pose, the template point cloud and the target point cloud are respectively projected onto the image plane after pose transformation, and the distance differences between each pair of projected points are calculated. F(y i) ) is the sum of all these distance differences between projected points, used to measure the matching degree between the template point cloud and the target point cloud on the image plane.

[0087] In the optimization process, the pose y i is optimized by minimizing this total difference value, so that the projections of the template point cloud and the target point cloud in the image are as consistent as possible.

[0088] 3.3 Quadratic approximation and iterative optimization

[0089] According to the BQBYQA algorithm, construct the quadratic approximation model Q i (y i ) = F(y i ). This quadratic approximation model is based on multiple y iA function fitted to the corresponding function values, which has a gradient. Optimize and modify the [R, T] values along the direction of the function gradient descent to reduce the corresponding error.

[0090] In other words, the BQBYQA algorithm constructs a quadratic approximation model using the current pose difference and gradient information through an iterative update method to quickly estimate the optimal pose update direction. In each optimization step, based on the current pose parameters and error model, the gradient and approximate Hessian matrix of the pose are calculated, and using this information, the current pose parameters (rotation vector and displacement vector) are updated to minimize the error function.

[0091] In each iteration, the algorithm calculates the updated values of the rotation vector and displacement vector according to the current quadratic approximation model. Through constrained optimization, it is ensured that the updates of rotation and displacement do not exceed the set trust region diameter, avoiding unstable optimization results caused by excessive updates. As the iteration progresses, the pose parameters gradually approach the optimal solution, making the projection difference between the template point cloud and the target point cloud in the image gradually decrease, and finally converging to the global or local optimal solution.

[0092] 3.4 Convergence Criterion and Termination Condition

[0093] To ensure the convergence and efficiency of the algorithm, a threshold of the error function and a maximum number of iterations are set. When the value of the error function in the optimization process is less than the preset convergence threshold or the maximum number of iterations is reached, the algorithm stops iterating. At this time, the optimization result is the final pose parameter, and the projection difference between the template point cloud and the target point cloud is minimized, completing high-precision point cloud matching.

[0094] Final Optimization Result: After iterative optimization, the pose parameters of the template point cloud are accurately updated to make its projection on the image plane as consistent as possible with that of the target point cloud. Through this optimization process, high-precision alignment of the template point cloud and the target point cloud can be achieved, and stitching of multi-view point cloud data can be completed.

[0095] 4. Deviation Analysis and Dimension Measurement

[0096] In the industrial manufacturing process, deviation analysis and dimension measurement of the workpiece point cloud are crucial for ensuring geometric accuracy and product quality. Through comprehensive analysis of the point cloud data, the geometric deviation between the workpiece and the design model (such as the CAD model) can be accurately evaluated, providing a basis for quality control and product optimization. After completing the stitching of multi-angle point cloud data, a post-processing process is carried out to complete the deviation analysis and dimension measurement of the workpiece, which specifically includes the following steps

[0097] 4.1 Point Cloud Data Preprocessing

[0098] When preprocessing the collected point cloud data, the present invention uses the classical statistical filtering method for denoising. The statistical filtering method is based on the local neighborhood information of the point cloud data. By statistically analyzing the distances between each point and its neighborhood points, outliers and noise data are removed to ensure the accuracy of the data. After that, the data collected from multiple perspectives is registered through the above-mentioned derivative-free optimization algorithm to ensure the alignment of the data from each perspective in space, thereby ensuring the spatial consistency of the generated three-dimensional model.

[0099] 4.2. Normalization of Point Cloud Data and Reconstruction of Coordinate System

[0100] The original point cloud data is normalized to remove obvious outliers. By taking the centroid of the point cloud data as the coordinate origin, a new spatial coordinate system called the "scaffold coordinate system (SCSS)" is constructed, and the SCSS is used to encode the point cloud data to generate standardized point cloud data.

[0101] 4.3. Comparison and Matching between CAD Model and Point Cloud

[0102] For the CAD model of the workpiece, it is first discretized and converted into point cloud format, and it is ensured that it is in the same coordinate system as the collected point cloud data. The SCSS encoding method is used to match the CAD model and the collected point cloud, and then the geometric deviation between the two is calculated.

[0103] 4.4. Deviation Analysis

[0104] Deviation analysis is carried out by calculating the difference between the point cloud and the CAD model. The specific method is: through the intersection point data of the SCSS ray and the workpiece surface, the average distance of the closest point of the intersection point in the workpiece point cloud data is calculated. By rotating the SCSS ray, when the average distance of the closest point is less than the set threshold, this rotation direction is considered to represent the position in the object coordinate system, and then the original pose difference between the workpiece to be detected and the template point cloud is obtained.

[0105] After obtaining the pose relationship between the point cloud and the CAD model, the ICP (Iterative Closest Point) algorithm is used to further optimize the alignment of the point cloud data and the CAD model to ensure high-precision comparison.

[0106] 4.5. Dimension Measurement

[0107] Finally, the geometric deviation between the point cloud and the CAD model is calculated, and the actual dimensions and deviation data of the workpiece are obtained through dimension measurement using POLYWORKS software.

[0108] Embodiment

[0109] The following is an application example of the method of the present invention. In this application example, first, a multi-camera 3D reconstruction system is selected and configured, then point cloud acquisition is performed using the system, multi-viewpoint cloud stitching is carried out, and finally deviation analysis and dimension measurement are performed, as follows.

[0110] 1. Selection and setting of the multi-camera 3D reconstruction system:

[0111] Arrangement and selection of cameras

[0112] In the 3D reconstruction process of non-standard shaped parts, it is crucial to select a suitable camera. In this embodiment, the MV-CH250-90TM-C-NF industrial camera is used. This camera has the characteristics of high resolution and low noise, and can provide clear and stable images under complex lighting conditions. According to the size and shape characteristics of the non-standard shaped parts, 4 to 8 cameras are arranged to ensure that data can be collected from multiple perspectives, covering all angles of the object, and avoiding problems such as occlusion and incomplete data that may be caused by a single perspective.

[0113] Arrangement and selection of projectors

[0114] In this embodiment, each camera is equipped with a DLP4500 projector. The resolution of the DLP4500 is 1080x912, and it has good stability and high brightness, which is suitable for ensuring clear and uniform pattern projection during multi-angle acquisition. The resolution and brightness of the projector can ensure high-quality projection patterns under different lighting conditions, thereby guaranteeing the accuracy and quality of the point cloud data.

[0115] The arrangement of the projector is closely coordinated with the arrangement of the camera. Each projector corresponds directly to the camera, and the angle and position of the projector are precisely adjusted to ensure that the projected pattern completely covers the surface of the object to be measured without affecting the field of view of the camera.

[0116] Such a configured system can effectively improve the 3D reconstruction accuracy of non-standard shaped parts. Especially during the scanning process of objects with small sizes and complex shapes, it can ensure the comprehensiveness and efficiency of data acquisition.

[0117] In this embodiment, the 3D reconstruction system consists of multiple groups of point cloud acquisition devices. Each group of point cloud acquisition devices includes a camera and a projector, forming a complete acquisition device. This device uses a combination of the MV-CH250-90TM-C-NF industrial camera and the DLP4500 projector. Through precise arrangement and coordinated operation of the camera and the projector, the point cloud data of the target object can be efficiently obtained from different perspectives. The camera is responsible for capturing the structured light pattern projected by the projector, and by decoding the projected pattern, the 3D information of the object surface is obtained. The projector provides a precise structured light pattern to ensure the coverage and uniformity of the projected pattern at different angles, thereby ensuring the high quality of the point cloud data.

[0118] In a single device, the camera and the projector are fixed in a metal protective shell, with a certain distance between them. The relative positions of the camera and the projector are precisely calculated and adjusted to ensure that during the shooting process, the projected pattern does not interfere with the camera's field of view and accurate point cloud data acquisition can be achieved. Through this design, the single device can efficiently collect three-dimensional data of non-standard shaped parts, ensuring the high precision and comprehensiveness of the data.

[0119] 2. Point Cloud Acquisition

[0120] Structured Light Pattern Projection and Camera Acquisition Diagram

[0121] The projector (DLP4500) projects a preset structured light pattern (phase shift code encoding pattern) onto the surface of the object to be detected. The projected structured light stripe pattern is distorted by the geometric shape of the object surface. After the object surface is irradiated by the structured light, the MV-CH250-90TM-C-NF camera captures an image of the object surface according to the preset viewing angle, as Figure 4 (a), which shows the image of the object surface captured by the camera, including the distorted stripes projected on the object surface. Due to the height change of the object surface, the stripes will be bent and deformed, thus encoding the depth information of the object surface.

[0122] Image Decoding and Point Cloud Generation

[0123] Through the phase shift code encoding and decoding technology, the captured images will be decoded to calculate the three-dimensional coordinates of each image point. Combining the internal and external parameter information of the camera and the projector, the image points are converted into three-dimensional coordinate points to generate the point cloud data of the object surface. As Figure 4 (b) shows, it is the preliminary point cloud data generated after decoding, demonstrating the three-dimensional point cloud distribution on the object surface. The spatial position of each point is determined by the deformation of the structured light stripes, reflecting the geometric shape of the object surface.

[0124] 3. Multi-viewpoint Cloud Mosaic Scheme

[0125] In this embodiment, the BQBYQA optimization algorithm is used to accurately align the point cloud data collected from multiple viewpoints. Since there are certain pose differences and non-overlapping phenomena in the point clouds of two industrial parts during the original data acquisition process (such as Figure 5 (a) shows the original point cloud overlap diagram), traditional point cloud matching methods (such as the ICP algorithm) are easily affected by factors such as camera distortion and noise, resulting in insufficient alignment accuracy. To solve this problem, we introduce a derivative-free optimization method and combine it with image feature point matching to improve the optimization accuracy.

[0126] At the beginning of the optimization process, the initial point cloud data ( Figure 5(a)) shows a certain degree of non - coincidence. Obviously, the red point cloud and the cyan point cloud have a large non - coincidence. The algorithm of the present invention first performs iterative optimization of the pose according to the error model and the approximation function. By calculating the distance difference between each point after the projection of the template point cloud and the target point cloud, and minimizing this difference, the pose parameters are gradually updated.

[0127] As the optimization progresses, the projection error between the template point cloud and the target point cloud gradually decreases and finally converges to a relatively accurate pose (as shown in Figure 5 (b), which is the effect diagram after pose optimization), enabling the point clouds of the two industrial parts to achieve high - precision alignment.

[0128] As Figure 5 The picture of the pose after optimization as shown in

[0129] (b) shows the point cloud matching result after optimization. The projection difference between the optimized template point cloud and the target point cloud in the image is significantly reduced. The optimization result shows that using the BQBYQA algorithm for pose update can effectively reduce the error in the original acquisition, achieve more accurate point cloud fusion, and provide a reliable data basis for subsequent 3D reconstruction.

[0130] 4. Scale difference analysis and dimension measurement

[0131] First, the collected point cloud data is pre - processed, including denoising using the classical statistical filtering method. This method analyzes the local neighborhood information of the point cloud data to remove outliers and noise, ensuring the accuracy of the data quality. After that, the data collected from multiple perspectives is registered through an optimization algorithm to ensure the alignment of the data from each perspective in space, thus ensuring the spatial consistency of the generated 3D model.

[0132] After the pre - processing is completed, the SCSS (scaffold coordinate system) method is used to normalize the point cloud data. The specific steps include taking the centroid of the point cloud data as the coordinate origin, constructing a new coordinate system, and encoding the point cloud data through this coordinate system to generate standardized point cloud data. At the same time, the CAD model is also converted into a point cloud format and ensured to be in the same coordinate system as the collected point cloud data. By comparing the point cloud data with the CAD model, the geometric deviation between the two is calculated.

[0133] Next, based on the intersection data of the SCSS ray and the workpiece surface, calculate the average distance to the nearest point of this data in the workpiece point cloud. By rotating the SCSS ray, find the position where the average distance to the nearest point is less than the set threshold, and determine that this rotation direction represents the position in the workpiece coordinate system, thereby obtaining the pose difference between the workpiece and the template point cloud.

[0134] After obtaining the pose relationship between the point cloud and the CAD model, the ICP (Iterative Closest Point) algorithm is used to further optimize the alignment of the two to ensure high-precision matching. Finally, through POLYWORKS software for dimension measurement, the actual dimensions and deviation data of the workpiece are obtained, thus realizing efficient and accurate workpiece point cloud deviation analysis and dimension measurement.

[0135] Through the above method, this patent provides an efficient and accurate workpiece point cloud deviation analysis and dimension measurement solution, which can provide strong data support for quality control in industrial manufacturing.

[0136] The above are only the preferred embodiments of the present invention and do not impose any form of limitation on the present invention. Although the present invention has been disclosed above with the preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art, within the scope of the technical solution of the present invention, can make some changes or modifications to the above-disclosed technical content to form equivalent embodiments with equivalent changes. However, as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A high-precision measurement and verification method for non-standard special-shaped parts based on three-dimensional vision, characterized in that It includes the following steps: Step S1: Select and configure a multi-camera 3D reconstruction system; Step S2: Point cloud acquisition. Adopt structured light encoding technology. Through the cooperation of a projector and a camera, alternately acquire the surface images of the object in different regions periodically, and generate the point cloud data of the object surface through phase shift code encoding and decoding technology; Step S3: Multi-viewpoint cloud stitching. Use a method based on the derivative-free optimization algorithm (BQBYQA) and image feature point matching to accurately align the point cloud data acquired from multiple viewpoints; Step S4: Deviation analysis and dimension measurement: Calculate the difference between the point cloud data and the design model to perform dimension measurement and deviation analysis of the measured object.

2. The high-precision measurement and verification method for non-standard special-shaped parts based on three-dimensional vision according to claim 1, wherein In step S1, according to the size and complexity of the non-standard shaped part, select multiple cameras for arrangement to ensure data acquisition from multiple viewpoints, covering all angles of the object to be measured, and each camera is equipped with a projector, and the projector corresponds to the camera.

3. A high-precision measurement and verification method for non-standard special-shaped parts based on three-dimensional vision according to claim 1, characterized in that In step S2, project the encoded pattern through the projector, decode it after the camera captures the deformed pattern, and calculate the three-dimensional coordinates of the object by combining the internal and external parameters of the projector and the camera; when encoding the phase shift code, use four phase shift code maps and solve the phase through the light intensity formula; then calculate the normalized coordinates using the phase difference and the camera internal parameters, and finally obtain the point cloud data of the object surface.

4. A high-precision measurement and calibration method for non-standard special-shaped parts based on three-dimensional vision according to claim 3, characterized in that In step S2, adopt a block sequential projection method for data acquisition. Divide the object to be measured into multiple regions. In each region, multiple cameras alternately and cross-acquire the point cloud data of this region to ensure sufficient data coverage from different viewpoints. After completing the data acquisition of one region, switch to another region and use the same alternating acquisition method for data acquisition, and the acquisition process of each region is carried out according to the alternating cycle. Different cameras alternately acquire data from different angles to ensure that all cameras can cover different viewpoints of the object and obtain sufficient point cloud data.

5. For a high-precision measurement and verification method for non-standard shaped parts based on 3D vision according to claim 1, step S3 is specifically as follows: S3.

1. Establish an error model: Set the initial rotation vector R and the initial displacement vector T of the template point cloud and the target point cloud, and define a reliable semi-domain diameter constraint such that a ≤ [R, T] ≤ b to limit the optimization range; S3.

2. Construct an approximation function: Define an approximation function F(yi), which represents the sum of the distances between the points after the projection of two point clouds on the image plane, and is used to measure the pose difference; in the optimization process, minimize this total difference value to optimize the pose yi between the template point cloud and the target point cloud, so that the projections of the template point cloud and the target point cloud in the image are as consistent as possible; S3.3, Second-order approximation and iterative optimization: Construct a second-order approximation model Q i (y i ) = F(y i ); Update the pose parameters using the gradient and the approximate Hessian matrix; Adjust R and T within the trust region during each iteration to avoid excessive parameter jumps and ensure stability; S3.

4. Convergence determination and result output: Set the threshold of the error function and the maximum number of iterations, terminate the optimization according to the preset error threshold or the maximum number of iterations, output the optimal pose parameters, and complete the high-precision point cloud matching.

6. A high-precision measurement and calibration method for non-standard shaped parts based on three-dimensional vision according to claim 1, characterized in that The deviation analysis and dimension measurement in step S4 specifically include the following steps: S4.

1. Point cloud data preprocessing; S4.

2. Normalization of Point Cloud Data and Reconstruction of Coordinate System: Normalize the point cloud data and encode the point cloud data using the Scaffold Coordinate System (SCSS) to generate standardized point cloud data; S4.

3. Comparison and Matching between the Designed Model and the Point Cloud: Discretize the designed model into the point cloud format and use the SCSS encoding method to match the designed model of the workpiece to be measured and the point cloud data; S4.

4. Deviation Analysis: Based on the SCSS ray intersection points, preliminarily align the designed model and the point cloud data, and calculate the original pose difference between the point cloud data and the designed model; after obtaining the pose relationship between the point cloud data and the designed model, use the ICP algorithm to further optimize the alignment of the point cloud data and the designed model to ensure high-precision comparison; S4.

5. Dimension Measurement: Calculate the geometric deviation between the point cloud data and the designed model, and perform dimension measurement through the corresponding software to obtain the actual dimensions and deviation data of the workpiece.

7. A high-precision measurement and calibration method for non-standard special-shaped parts based on three-dimensional vision according to claim 6, characterized in that, In step S4.1, the statistical filtering method is used to denoise the point cloud data.

8. A high-precision measurement and verification method for non-standard special-shaped parts based on three-dimensional vision according to claim 6, characterized in that, Step S4.2 is specifically as follows: Normalize the point cloud data, remove obvious outliers, construct a new space coordinate system by taking the centroid of the point cloud data as the coordinate origin, which is called the "Scaffold Coordinate System (SCSS)", and use the SCSS to encode the point cloud data to generate standardized point cloud data.

9. A high-precision measurement and calibration method for non-standard special-shaped parts based on three-dimensional vision according to claim 6, characterized in that, In step S4.4, the specific method for preliminarily aligning the designed model and the point cloud data based on the SCSS ray intersection points is: Calculate the average distance of the nearest points of the intersection points in the workpiece point cloud data through the intersection point data of the SCSS ray and the workpiece surface. By rotating the SCSS ray, when the average distance of the nearest points is less than the set threshold, it is considered that this rotation direction represents the position in the object coordinate system, and thus the original pose difference between the workpiece to be detected and the template point cloud is obtained.

10. A high-precision measurement and calibration method for non-standard special-shaped parts based on three-dimensional vision according to claim 6, characterized in that, The designed model is a CAD model.

Images