Body-in-white sheet metal part hole feature visual detection method and system

By using a 3D structured light camera and point cloud processing algorithm, the problems of large errors and low efficiency of traditional measurement methods are solved, and efficient and high-precision detection of hole features in body-in-white sheet metal parts is achieved, which is suitable for the detection of sheet metal parts with complex structures.

CN118913090APending Publication Date: 2024-11-08GUANGXI UNIVERSITY OF TECHNOLOGY +1
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202410798436.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-20
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Traditional contact measurement methods suffer from large errors and low efficiency, while non-contact measurement using vision cameras has the problem of incomplete feature contour acquisition, making it difficult to meet the requirements of efficient and high-precision inspection of complex structures in body-in-white sheet metal parts.

Method used

Point cloud data is acquired using a 3D structured light camera. The point cloud data is preprocessed by combining a bounding box discretization point cloud downsampling algorithm and a boundary neighborhood feature point set extraction method based on the areal index. The data is then registered using a point cloud registration algorithm with local boundary neighborhood feature constraints. The RANSAC algorithm is used to fit the pore feature parameters, and the boundary points are extracted using the centroid-constrained alpha shapes algorithm. This achieves high-precision detection of pore features.

Benefits of technology

It enables efficient and accurate detection of hole features in body-in-white sheet metal parts, avoids human intervention, improves measurement efficiency and accuracy, and is suitable for the inspection of sheet metal parts with complex structures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118913090A_ABST
    Figure CN118913090A_ABST
Patent Text Reader

Abstract

The invention discloses a body-in-white sheet metal part hole feature visual detection method and system, and the method comprises the steps: photographing a body-in-white sheet metal part through a 3D structure light camera, and obtaining a point cloud data set of the photographed body-in-white sheet metal part; preprocessing a point cloud data set by adopting a point cloud downsampling algorithm based on bounding box discretization and a boundary neighborhood feature point set extraction method based on planar indexes; performing point cloud registration on the preprocessed point cloud set data by adopting a point cloud registration algorithm based on local boundary neighborhood feature constraint to obtain a complete point cloud template of the sheet metal workpiece; the features to be detected in the point cloud module are positioned, boundary points are segmented and extracted, and hole feature parameter fitting is carried out on the extracted boundary points. Compared with an existing measurement method, the method is more accurate, the whole measurement process does not need human intervention, and the measurement efficiency is higher.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of visual inspection and measurement, and in particular to a method and system for visual inspection of hole features of a body-in-white sheet metal part. Background Art

[0002] The Chinese automobile manufacturing industry in the 21st century is in a special period of rapid and diversified development. With the fierce competition in the automobile market, new requirements have been put forward for the iteration speed and product quality of new models. The body in white is the core element of vehicle manufacturing, and its manufacturing cost accounts for about 50% of the manufacturing cost of the whole vehicle. It is mainly composed of sheet metal parts, which are welded through multiple assembly processes. The manufacturing process of body in white sheet metal parts is usually stamping of thin plates. Affected by factors such as manufacturing deviation of stamping parts, deformation of welding parts, inaccurate positioning of tooling and welding operation errors, dimensional deviations will inevitably occur. In the modern automobile manufacturing industry, detection is the "eye" of mechanical manufacturing. Efficient and accurate online detection of body in white sheet metal parts has become a key link to ensure product quality and improve production efficiency.

[0003] Hole features are important and common features to be measured in sheet metal parts. Their main functions are positioning or assembly. They usually include round holes or special-shaped hole features, which are widely used in various white bodies, flanges, etc. for positioning or connecting parts. These assembly features usually have high manufacturing accuracy, so their assembly feature parameter deviations can reflect the dimensional errors generated during the white body welding process, and then evaluate the quality of the workpiece. Therefore, the online measurement of hole features of white body sheet metal parts is an important part of dimensional detection. Furthermore, whether it is a hole or a non-hole, it covers multiple dimensions such as size, position and posture. The measurement of round hole features mainly focuses on its own size, precise position in the part coordinate system and its posture. Non-hole features, on the other hand, mainly focus on the accuracy of their shape and position. Regardless of the type of feature, two-dimensional and three-dimensional measurements are indispensable.

[0004] Based on the structural characteristics of body-in-white sheet metal parts, and taking into account the detection method, degree of automation, and means of data acquisition, the measurement technologies currently used in the automotive manufacturing industry are mainly divided into two categories: contact measurement and non-contact measurement.

[0005] The contact measurement method works by detecting features one by one, and the main forms include manual measurement and three-coordinate machine measurement. However, the contact measurement method is limited by the mechanical movement speed of the probe and has the defect of low efficiency. It also requires direct contact with the workpiece and is not suitable for parts with complex structures or soft materials that are easily scratched. Correspondingly, its price is relatively high, and the application personnel's ability requirements are also relatively high. It is difficult to meet the high efficiency and high precision requirements of the online measurement system of the body-in-white sheet metal parts with complex structures and low rigidity.

[0006] With the rapid development of machine vision, non-contact measurement has begun to be applied to industrial inspection, forming a new modern visual measurement technology. The advantages of non-contact visual measurement are its non-contact and rapidity. Compared with traditional contact measurement methods, visual measurement does not require direct contact with the object to be measured, avoiding errors and damage caused by contact. At the same time, through high-speed cameras and image processing algorithms, object image information can be quickly obtained, greatly improving measurement efficiency.

[0007] Using structured light cameras, we can quickly obtain a large number of dense spatial 3D point clouds on the surface of the measured parts, and perform high-precision fitting and extraction of the characteristic parameters to be measured. At the same time, with industrial robots, we can obtain the required surface point cloud data of the workpiece from multiple perspectives. It is more suitable for body-in-white sheet metal parts, where the measurement elements of the measured features are complex and the structure is complex, and the on-site rapid measurement of weak rigid parts can be achieved, so as to achieve the high efficiency and high precision requirements of the online measurement system of the characteristic parameters to be measured of body-in-white sheet metal parts. Summary of the invention

[0008] In view of the above-mentioned problems, the present invention is proposed.

[0009] Therefore, the technical problems solved by the present invention are: the traditional contact measurement has large errors, low efficiency, poor repeatability, and the non-contact measurement of the visual camera has the problem of incomplete feature contour collection.

[0010] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for visual detection of hole features of body-in-white sheet metal parts, comprising: photographing the body-in-white sheet metal parts with a 3D structured light camera to obtain a point cloud data set of the photographed body-in-white sheet metal parts; preprocessing the point cloud data set by using a point cloud downsampling algorithm based on bounding box discretization and a boundary neighborhood feature point set extraction method based on a surface index; performing point cloud registration on the preprocessed point cloud set data by using a point cloud registration algorithm based on local boundary neighborhood feature constraints to obtain a complete point cloud template of the sheet metal workpiece; locating the features to be measured in the point cloud module and segmenting and extracting boundary points, and fitting the hole feature parameters of the extracted boundary points.

[0011] As a preferred solution of the visual detection method for hole features of body-in-white sheet metal parts described in the present invention, the point cloud downsampling algorithm based on bounding box discretization includes: constructing a point cloud data covariance matrix; using singular value decomposition of the covariance matrix to obtain the eigenvalues ​​and eigenvectors of the covariance matrix; using the eigenvector as the coordinate axis direction and the point cloud centroid as the coordinate axis origin, establishing a local coordinate system of the point cloud data set, and constructing a point cloud bounding box; retaining the internal point cloud data set through the region of interest, performing statistical analysis on the neighborhood of each point according to the statistical filtering algorithm, eliminating discrete point cloud data on the workpiece surface, and then downsampling the workpiece surface point cloud data after elimination to obtain a downsampled point cloud set.

[0012] As a preferred scheme of the visual detection method of hole features of body-in-white sheet metal parts described in the present invention, the method for extracting boundary neighborhood feature point sets based on surface index includes establishing a kd tree for downsampled point cloud data; establishing a covariance matrix and solving the eigenvalues ​​and corresponding eigenvectors of the point cloud data; using the eigenvalues ​​of the point cloud data to calculate the surface index of the point cloud data to extract the boundary points of the point cloud data; extracting the overlapping boundary points by using the Euclidean distance between the original point cloud and the boundary neighborhood point set extracted after downsampling to obtain the boundary neighborhood point set of the original point cloud.

[0013] As a preferred solution of the visual detection method for hole features of body-in-white sheet metal parts described in the present invention, the point cloud alignment based on local boundary neighborhood feature constraints includes: taking the union of the original point cloud boundary neighborhood point set and the downsampled point cloud set as the input for point cloud coarse alignment; completing the coordinate system one of the preprocessed point cloud data using the robot posture and hand-eye matrix; modifying the input of the ICP algorithm, taking the original point cloud boundary neighborhood point set and the downsampled point cloud set, and simply extracting the overlapping area point cloud set as the input point cloud based on the Euclidean distance between the points of the two point clouds; modifying the objective function so that the objective function converges completely on the plane and the constant curvature surface, thereby enhancing the algorithm's pertinence and convergence.

[0014] As a preferred embodiment of the visual detection method for hole features of body-in-white sheet metal parts of the present invention, the modified objective function includes: due to the problem of mutual sliding of point cloud data, using the original boundary point cloud set Λ with high density and containing positioning features bd To limit sliding, the modified objective function is expressed as:

[0015]

[0016] Where E represents the objective function; R represents the rotation matrix; t represents the translation matrix; represents the source point cloud; represents the target point cloud; Represents the normal vector corresponding to the target point cloud; express The corresponding normal vector; Indicates P λ Round up; N represents the number of corresponding point pairs between two point clouds.

[0017] As a preferred solution of the visual detection method for hole features of body-in-white sheet metal parts described in the present invention, the method includes: locating the features to be measured in the point cloud module and segmenting and extracting boundary points, using a boundary neighborhood feature point set extraction method based on a surface index to calculate the eigenvalues ​​and eigenvectors of the covariance matrix for local point cloud data, analyzing the geometric features and states of its neighborhood based on the relationship between the eigenvalues ​​and eigenvectors of the covariance matrix, and extracting hole boundary neighborhood features; using Euclidean clustering feature boundary neighborhood points to segment into multiple subsets, and using the overall spatial geometric state of each sub-cluster to achieve feature positioning; reducing the dimension of the acquired local feature point cloud and extracting boundary points, using the RANSAC algorithm to perform plane fitting on the local point cloud data, projecting the point cloud data into the local coordinate system of the fitting plane, converting the three-dimensional space point cloud data into two-dimensional plane point cloud data, and completing the dimensionality reduction of the point cloud data; after the point cloud data is reduced in dimension, using the centroid-constrained alpha shapes algorithm to extract plane boundary point data.

[0018] As a preferred solution of the visual detection method for hole features of body-in-white sheet metal parts described in the present invention, the hole feature parameter fitting includes fitting a circle to the hole feature parameters using a RANSAC algorithm.

[0019] In the second aspect, the present invention also provides a visual detection system for hole features of body-in-white sheet metal parts, including: an acquisition module, which uses a 3D structured light camera to shoot body-in-white sheet metal parts from multiple angles to obtain multiple point cloud module data sets after shooting; a data calculation module, which pre-processes the collected point cloud data sets, uses the PCA algorithm to achieve coarse alignment, and then uses the improved ICP algorithm to perform fine alignment to obtain a complete point cloud template of the sheet metal workpiece; a fitting module, which uses the RANSAC algorithm to fit a circle to the hole feature parameters to achieve hole feature fitting.

[0020] In a third aspect, the present invention further provides a computing device, comprising: a memory and a processor;

[0021] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the visual detection method of hole features of the body-in-white sheet metal part are implemented.

[0022] In a fourth aspect, the present invention further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for visually detecting hole features of body-in-white sheet metal parts.

[0023] Beneficial effects of the present invention: The present invention proposes a method and system for visual detection of hole features of sheet metal parts of a body in white. By taking photos of the sheet metal parts to be tested and extracting point cloud data, a series of preprocessing processes are performed to highlight more target features that can be selected by the registration algorithm. An ICP algorithm with local boundary neighborhood feature constraints is further used for registration to obtain a complete sheet metal part contour. In terms of feature parameter extraction, the rapid segmentation and extraction of hole features are achieved by calculating the geometric parameters of the local feature boundary point set and combining Euclidean clustering. In terms of boundary point extraction, an alpha shapes algorithm with centroid constraints is proposed for targeted extraction of internal boundary points of holes. In terms of feature parameter fitting, the RANSAC algorithm is used to fit an ellipse to extract the feature parameters of a circular hole. Compared with existing measurement methods, this method is more accurate, the entire measurement process does not require human intervention, and the measurement efficiency is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0025] Figure 1 An overall flow chart of a method for visually detecting hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the composition of a measurement system for a method for visually detecting hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a reduced voxel grid visualization of a method for visually detecting hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0028] Figure 4 A schematic diagram of an R radius search for a visual detection method for hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of plane point set segmentation of a method for visually detecting hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0030] Figure 6 A schematic diagram of boundary neighborhood point set segmentation of a method for visual detection of hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0031] Figure 7A schematic diagram of boundary wall removal in a method for visually detecting hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0032] Figure 8 A schematic diagram of an alpha shapes algorithm for a visual detection method of hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0033] Fig. 9 A schematic diagram of an improved alpha shapes algorithm for a visual detection method of hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0034] Fig.10 A schematic diagram of the operation of an improved alpha shapes algorithm for a visual detection method of hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0035] Fig.11 A schematic diagram of a waist hole ε·ε2=0 of a body-in-white sheet metal part hole feature visual detection method provided by an embodiment of the present invention;

[0036] Fig.12 A schematic diagram of a square hole ε·ε2≠0 in a method for visually detecting hole features of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0037] Fig.13 A schematic diagram of calculating waist hole characteristic parameters of a visual detection method for hole characteristics of a body-in-white sheet metal part provided by an embodiment of the present invention;

[0038] Fig.14 A schematic diagram of calculating square hole feature parameters of a body-in-white sheet metal part hole feature visual detection method provided by an embodiment of the present invention;

[0039] Fig.15 A comparison of load balancing rates of a body-in-white sheet metal hole feature visual detection method under different numbers of tasks provided by the second embodiment of the present invention;

[0040] Fig.16 A schematic diagram of convergence results of local point cloud registration of sheet metal parts in a method for visual detection of hole features of sheet metal parts of a body-in-white provided in a second embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0042] Example 1

[0043] Reference Figure 1-Figure 14 , as an embodiment of the present invention, provides a method for visually detecting hole features of a body-in-white sheet metal part, comprising:

[0044] S1: Use a 3D structured light camera to shoot the body-in-white sheet metal parts and obtain a point cloud dataset of the body-in-white sheet metal parts after shooting.

[0045] All the public data sets used in the experiment are from the Stanford University Point Cloud Database. The measurement experiment environment is configured as follows: Fanuc M-10A industrial robot, Elson AT-S1000-07C-S2 structured light camera, and workpiece test bench. The compilation language is C++, the program editor is VS2019, the hardware platform is a desktop computer with a 3.40GHz Intel(R)Xeon(r)Gold 5318Y CPU, NVIDIA GeForce GTX 4090GPU and 128GB RAM, and some algorithms are derived from the open source data processing code library PCL and OpenCV.

[0046] like Figure 2 As shown, the measurement system consists of an industrial six-axis robot, a computer, a visual sensor and a workpiece support. The workpiece is fixed on the support. The visual sensor at the end of the robot serves as a perception unit to obtain point cloud data of the workpiece surface from multiple perspectives with overlapping areas. The computer serves as a central control unit to process the collected data, obtain the size or position of the feature to be measured on the workpiece and compare it with the theoretical position, and finally issue a test report.

[0047] S2: The point cloud dataset is preprocessed by using a point cloud downsampling algorithm based on bounding box discretization and a boundary neighborhood feature point set extraction method based on surface index.

[0048] The point cloud downsampling algorithm based on bounding box discretization should be explained as follows:

[0049] First, the covariance matrix of the point cloud data is constructed, and the eigenvalues ​​and corresponding eigenvectors of the covariance matrix are calculated to construct the point cloud bounding box and discretize it into several identical rectangular grids. The centroid of the point cloud in the grid or the point cloud closest to the centroid represents all the point cloud data in the voxel grid to achieve the purpose of downsampling.

[0050] Covariance matrix M cov , calculate the covariance matrix M through SVD cov The eigenvalues ​​and eigenvectors of are used to describe the characteristics of the point set in the field, as shown in the following formula: cov Decomposed into three smaller and simpler sub-matrices to describe its important features.

[0051]

[0052] Among them, U and V are both unitary matrices, U is an m×m matrix, and the orthogonal vectors in matrix U are left singular vectors; V is an m×m matrix, and the orthogonal vectors in matrix V are right singular vectors. Σ is an m×n matrix, and all elements except the diagonal elements are 0. The elements on the diagonal are singular values, that is, the eigenvalues ​​are arranged from large to small.

[0053] Find M from the eigenvalues ​​λ1, λ2, and λ3 cov The corresponding eigenvectors ε1, ε2, ε3 are used as the directions corresponding to the natural axes X, Y, and Z of the bounding box. Calculate the position coordinates P of each vertex i The inner product of the point cloud data is used to obtain the maximum and minimum positions of the vertex along the three directions, and a point cloud directed bounding box (OBB) with a side length of l×w×h is constructed based on this. min (x min ,y min ,z min ) is the origin O, and the eigenvectors corresponding to the largest and second largest eigenvalues ​​are the directions of the coordinate axes. Figure 3 In the X and Y axes, the point cloud bounding box is discretized into Row*Col d×d×h small voxel grids with a step size of d.

[0054] The internal point cloud data set is retained through the region of interest, including some discrete point cloud data. According to the statistical filtering algorithm, the neighborhood of each point is statistically analyzed to remove the discrete point cloud data on the workpiece surface, so as to achieve the effect of removing obvious outliers. After the point cloud data is denoised, the point cloud data still contains a large amount of redundant information. In order to improve the speed of point cloud registration, the point cloud data on the workpiece surface after elimination is downsampled, that is, the point cloud data is completely divided using several small cubes, and all the point cloud data in the voxel grid is replaced by the point closest to the center point of the voxel in the original point cloud data. At this time, the data after the point cloud data is downsampled is not only evenly distributed on the original point cloud data, but also derived from the initial point cloud, which ensures the authenticity of the point cloud after downsampling.

[0055] The boundary neighborhood feature point set extraction method of the surface index establishes the covariance matrix and solves the eigenvalues ​​and corresponding eigenvectors of the point cloud data. It requires iterative traversal of the point cloud data, which is a large amount of calculation. For this reason, the point cloud data KD tree is first established to speed up the neighborhood search. In order to ensure the consistency of feature extraction and reduce the influence of cluttered point cloud data on feature extraction, this paper chooses to find the distance p from the target point i The nearby distance threshold is less than the search threshold t r The point cloud data is R radius search to establish the covariance matrix. Figure 3 As shown, The direction represents the point set M in the neighborhood of point p cov The length represents the corresponding eigenvalue. At this time, the R radius search can ensure that the point cloud related features are calculated within a fixed range of point p, while the k nearest neighbor search requires a fixed number of search neighbor points. When there are noise points or holes, the search range cannot be guaranteed. At this time It is not credible.

[0056] In particular, the point cloud neighborhood search threshold is closely related to the downsampling threshold. For this type of sheet point cloud data, the value relationship can be expressed by the formula t r ≈6d estimate.

[0057] Based on the above situation, the present invention establishes a covariance matrix M for the point cloud data and the point set in the domain after downsampling. cov After calculating the eigenvalue, the surface index of the point cloud data is calculated using the eigenvalue to extract the boundary points of the point cloud data, thereby increasing the attention paid to such features in the point cloud registration process. The surface index can be obtained by the following formula:

[0058]

[0059] Get the surface index set of all points Combine its size with the corresponding point to get the visualization result. P Sort the data in from small to large and take the i-th largest value t p is the critical point, if Then its corresponding point is determined to be the boundary neighborhood point set Λ bd The elements in , i can be estimated by the following formula:

[0060]

[0061] Among them, t r is the neighborhood search threshold; d is the downsampled voxel width; |Λ bd | is the size of the boundary neighborhood point set, which can be calculated by the following formula:

[0062] |Λ bd |=Ω+2(Row+Col-1)

[0063] If |vox(x r y c )|>0, then let vox(x r y c )=0, otherwise let vox(x r y c )=1, all vox(x r y c ) is arranged into a sparse matrix of Row*Col, and each row of the matrix data is traversed to calculate the number of element value steps, which is Ω in the formula.

[0064] It is not difficult to see that this method can extract boundary feature points stably and accurately based on the surface features of the downsampled point cloud data. However, because boundary feature points contain more features and the amount of downsampled data is small, it will affect the accuracy of point cloud registration. In this paper, the original point cloud Λ and the boundary neighborhood point set extracted after downsampling are used to extract the boundary feature points. The Euclidean distance is used to extract the overlapping boundary points, and the critical value is Get the original point cloud boundary neighborhood point set Λ bd .

[0065] Therefore, by calculating the surface index of local point cloud data and giving a reasonable critical value, the surface point cloud data corresponding to the more important positioning features or mating surfaces in sheet metal parts can be quickly and stably extracted. Using it for point cloud registration will make the registration algorithm more targeted.

[0066] S3: The preprocessed point cloud set data is registered using a point cloud registration algorithm based on local boundary neighborhood feature constraints to obtain a complete point cloud template of the sheet metal workpiece.

[0067] Take the original point cloud boundary point set Λ bd And the downsampled point cloud set Λ down The union of # As the target or source point cloud, but Λ bd It not only includes the real boundary of the workpiece but also the boundary of the camera field of view. Moreover, after unifying the coordinate system of the multi-view point cloud data, the overlapping area between the target and source point clouds is still limited. To this end, the present invention uses the Euclidean distance between the points of the two point clouds to simply extract the point cloud set Λ of the overlapping area # As input point cloud.

[0068] Simplified representation of the objective function of the point-to-plane ICP algorithm is as follows (P S -P T )·n T As shown, where P S is the source point cloud, P T is the target point cloud, n T is the normal vector corresponding to the target point cloud. If the sliding of point cloud data is not considered, the value of the objective function will converge to 0 only when the point cloud data is strictly distributed on the plane. However, the planar point cloud data obtained by the structured light camera is only approximately distributed on a plane. Analysis of its local point cloud data shows that it is distributed on several tiny surfaces. At this time, the objective function of the optimal correspondence relationship does not converge to 0.

[0069] The surface of local point cloud data distribution is simplified to a constant curvature surface, and the source point cloud P is added on its basis. S The corresponding normal vector n S, then modify the objective function to be as follows (P S -P T )·(n S +n T ), at this time, the point cloud data on a local constant curvature surface can still converge the objective function to 0. This enhances the tolerance of the algorithm to point cloud data. The special normal vector estimation radius is also R, and the model used is a plane model.

[0070] The problem of poor algorithm convergence caused by the local point cloud data not being strictly planar is solved by modifying the objective function, but the problem of point cloud data sliding against each other still exists. Because the relative relationship of features in the point cloud data obtained from multiple perspectives on the same sheet metal part does not change, the edge cutting and hole features can be easily obtained through the boundary features and the corresponding relationship can be established between point cloud data from different perspectives. For this purpose, the original boundary point cloud set Λ with high density and positioning features can be used. bd Limit its sliding.

[0071] In order to facilitate the use of feature points and their corresponding values, modify Λ # The content is shown in the following formula, where: Indicates that the point cloud data Λ bd ,Λ * Merge, if If empty, the default is 0.

[0072]

[0073] The modified objective function is:

[0074]

[0075] Among them, E represents the objective function; R represents the rotation matrix; t represents the translation matrix; represents the source point cloud; represents the target point cloud; Represents the normal vector corresponding to the target point cloud; express The corresponding normal vector; N represents the number of corresponding point pairs in the two point clouds; Indicates P λ Round up, that is, if but but

[0076] The registration algorithm in this paper is based on the input point cloud type, and the objective function can be divided into two parts: against The point set improves the convergence performance of the registration algorithm, so that it can not only converge the objective function to 0 on the standard plane, but also has the same performance on the constant curvature surface. against The boundary point set does not modify the objective function of the point-to-surface ICP algorithm, but only uses the original boundary point set of the workpiece surface to extract inherent features to limit the free sliding of its point cloud. In this way, the improved registration algorithm only enhances the adaptability of sheet point cloud data similar to the surface of sheet metal parts. If the point cloud data does not contain boundary point features, the objective function can be degenerated into the traditional point-to-surface ICP or point-to-point ICP registration algorithm, and it will not lose its function.

[0077] The second part of the objective function can be solved directly using the traditional point-to-point ICP method, and the first part can be solved using the standard linear least squares method following the point-to-surface ICP method.

[0078] S4: Locate and segment the features to be measured in the point cloud module to extract boundary points, and fit the hole feature parameters to the extracted boundary points.

[0079] The purpose of feature location segmentation is to quickly find the approximate location of features in scattered point cloud data and quickly and accurately segment the boundary feature area point cloud for boundary point extraction. The present invention calculates the surface index of point cloud data and its neighborhood point cloud data and sets a threshold to extract the boundary feature neighborhood point set.

[0080] The purpose of segmentation is to separate point cloud blocks belonging to different boundary features. Different boundary features in the boundary point cloud have the characteristic that the point cloud data in the same boundary feature is continuous, that is, the distance between adjacent points in the same boundary feature is small, and the distance between the closest points of different boundary features is large. Based on this distribution characteristic, the present invention uses the Euclidean distance clustering segmentation algorithm to segment the boundary features in the boundary point cloud.

[0081] After the segmentation is completed, multiple point cloud blocks with different boundary features are obtained. The following steps usually involve geometric shape analysis, size measurement, and comparison with theoretical models of the point cloud blocks. Through these analyses, the hole characteristics can be accurately identified, and a reliable basis can be provided for subsequent registration and detection. This implementation calculates the parameters including the centroid, eigenvalues ​​and eigenvectors of the covariance matrix for each clustered point cloud block data, which are used to identify the hole feature type and locate its approximate spatial position for further data dimensionality reduction, boundary point extraction, and feature parameter fitting.

[0082] Generally, the hole features of the sheet metal to be tested are used for fixing or connection, and due to the limitation of stamping manufacturing process, the surroundings of such features are all planes, so the point cloud blocks obtained by clustering should be plane features in theory. That is, the boundaries of the neighborhood points of the hole features are ideally continuous and closed and located on the local plane. The calculation result of the covariance matrix is ​​reflected in the minimum eigenvalue λ3≈0. In this implementation, the λ3 value is less than t λ The local point cloud set is considered to be a plane point cloud data. Combined with the characteristics that hole features are generally located on the local plane, retain λ3 <t λ The point cloud data set can be used to extract the point cloud data that may contain hole features. λ The value should be appropriately selected according to the accuracy of the structured light camera. λ = 0.15, the point cloud data subset retained after screening is as follows Figure 5 Plane features shown.

[0083] like Figure 6 As shown in the figure, in the process of local feature segmentation and extraction, a spherical area is constructed using its eigenvalue and centroid, and the boundary neighborhood point cloud data of the corresponding feature is extracted from the original point cloud data without downsampling. The position of the segmentation sphere is determined by the centroid of the feature boundary point. OK, its λ The size of is determined by the range of the point cloud data in the direction of the feature vector ε1 corresponding to the maximum feature λ1, that is, half of the length of the bounding box 0.5l. In particular, the present invention sets a safety factor to ensure the stability of numerical calculations, and sets r λ =0.6l.

[0084] In terms of boundary point extraction:

[0085] The hole features in the body-in-white sheet metal parts, regardless of their type, can be theoretically regarded as plane features. Given that the extracted point cloud data set is three-dimensional spatial data, the present invention first uses the RANSAC algorithm to perform plane fitting on the local point cloud data. Subsequently, the point cloud data is projected into the local coordinate system of the fitting plane, thereby converting the three-dimensional spatial point cloud data into two-dimensional plane point cloud data. After the point cloud data is reduced in dimensionality, the internal boundary points of the plane point cloud data are further extracted for subsequent feature parameter fitting.

[0086] When using the traditional RANSAC algorithm to perform point cloud fitting, it is necessary to reasonably select parameters based on the camera accuracy and the fluctuation of point cloud data, mainly including:

[0087] 1) Number of iterations N ransac , can be estimated by the following formula:

[0088]

[0089] Among them, the number of points in the game is β, then the probability of selecting an in-game point from the data set at a time is The probability that all N points are inside the game is ε N ; P is a constant, which represents the probability of obtaining at least one good sampling subset during the iteration process, and its value is generally between 0.9 and 0.99

[19] . The larger the values ​​of N and β are, the more algorithm iterations should be performed.

[0090] 2) The minimum number of samples X that the consistent set should contain should be large enough to ensure that the correct representation of the data set is found The model can ensure the fitting accuracy, but too high X will seriously slow down the convergence speed of the algorithm or even cause the algorithm to fail to converge. Its value should be consistent with the local point cloud dataset. The number of points included in this article is related

[0091] 3) Error tolerance threshold t ransac The selection of this value is closely related to the specific problem and has a significant impact on the performance of the algorithm. Its value should be related to the accuracy of the structured light sensor. In this embodiment, the value r alpha =4E cam .

[0092] After fitting the plane, in the process of reducing the dimension of the point cloud data, the influence of the boundary wall on the projected boundary points should be minimized. Figure 6 As shown, the present invention only retains the two sides of the plane t dis The point cloud data within the range is used for projection, thus effectively filtering out the influence of some boundary walls. dis The value range of t is determined according to the accuracy of the sensor and the material of the object being measured. In this study, when using a structured light camera to shoot weakly reflective metal parts, t is set to dis =0.2mm can achieve better removal effect.

[0093] After removing part of the boundary wall, the remaining local point cloud data is projected into the local coordinate system of the fitting plane to complete the dimensionality reduction of the point cloud data, and then the center of mass constrained alpha shapes algorithm proposed in the present invention is used to extract the plane boundary point data.

[0094] The alpha shapes algorithm can effectively extract the boundary points of the plane point cloud data, but the extracted boundary points include not only the internal hole feature boundary points but also the external boundary points, such as Figure 8 As shown. The present invention takes hole characteristics as the research object and only needs to extract the hole boundary points inside the point cloud data. In order to increase the algorithm speed and enhance the algorithm pertinence, this study introduces the centroid constraint to improve the alpha shapes algorithm.

[0095] During the calculation process, the centroid-constrained Alpha Shapes algorithm only calculates the internal boundary points close to the centroid, thereby effectively realizing the extraction of hole feature boundary points inside the local point cloud data. Fig. 9 This algorithm introduces centroid constraints to make the calculation more focused and efficient, avoid unnecessary processing of non-critical areas, and improve the accuracy and efficiency of feature extraction, as shown in Fig.10 As shown. Special r alpha The value should be related to the accuracy of the structured light sensor. In this implementation, r alpha =4E cam .

[0096] Furthermore, after the hole feature boundary points are extracted from the body-in-white sheet metal, the boundary points may only include circular hole boundary points, waist hole boundary points, square hole boundary points, and irregular hole boundary points. In view of the diversity of hole features and their respective unique feature parameter extraction methods, the hole features need to be classified before automatic measurement is realized. To this end, the present invention proposes a simple and efficient method to classify hole features. The classification process of the method is as follows:

[0097] 1. First, the circular hole and non-circular hole features are classified, and the covariance matrix of the boundary point subset is constructed to calculate the eigenvalues ​​and the corresponding eigenvectors. Ideally, the eigenvalues ​​corresponding to the circular hole feature are λ1=λ2, and the eigenvalues ​​corresponding to the non-circular hole feature are λ1>λ2.

[0098] 2. Then classify the non-circular hole features, such as Fig.11 , as shown in 17, for the centroid Construct the vector ε with the point p farthest from it. The eigenvector corresponding to the second largest eigenvalue of the boundary feature is ε2. If ε·ε2=0, it is a waist hole feature boundary. If ε·ε2≠0, it is a square hole feature boundary.

[0099] The present invention uses the RANSAC algorithm to calculate the characteristic parameters of the circular hole. The ellipse parameters W = [A, B, C, D, E, F] obtained by the RANSAC algorithm T , calculate the ellipse center O according to the following formula e (x e ,y e ) and the major and minor axis radii a e ,b e :

[0100]

[0101] The coordinates of the ellipse center O e (x e ,y e) is inversely transformed into three-dimensional space to obtain the coordinates of the center of the three-dimensional space circle O(x, y, z), and the diameter of the three-dimensional space circle D = 2b e , the circular hole normal n1 is equal to the normal direction of the fitting plane.

[0102] The waist hole boundary points can be considered as point cloud data near two straight lines plus two semicircles. In this embodiment, the eigenvector ε1 corresponding to the maximum eigenvalue λ1 of its covariance matrix is ​​first constrained, and the RANSAC algorithm with direction constraints is used to fit the two straight line models and remove their internal points. Then the RANSAC algorithm is used to fit the remaining point cloud to obtain the characteristic parameters of the two circles, such as Fig.13 shown.

[0103] Similarly, after obtaining the model parameters of the two circles, calculate the distance between their centers plus the radius of the two circles to obtain the length L, D = r1 + r2, and the midpoint of the two circle centers is taken as the center position O of the waist hole. e (x e ,y e ), and then inversely transform it into three-dimensional space to obtain the waist hole coordinates O(x, y, z) in three-dimensional space.

[0104] The boundary points of the square hole can be considered as point cloud data near the four straight lines. The present invention first uses the eigenvector ε1 corresponding to the maximum eigenvalue λ1 of its covariance matrix as a constraint, uses the RANSAC algorithm with direction constraints to fit the two straight line models and removes the internal points. Then, the two straight line models are fitted again to the remaining points with ε2 as the direction constraint. Fig.14 shown.

[0105] Calculate the intersection of two adjacent straight lines and use them as the four corner points of the square hole. Calculate the centroid of the four corner points and use them as the center position of the square hole O e (x e ,y e ) is inversely transformed into three-dimensional space to obtain the three-dimensional space square hole coordinates O(x, y, z), and calculate O e (x e ,y e ) to the four straight lines are L1, L2, D1, D2, the width of the square hole is D=D1+D2, and the length is L=L1+L2.

[0106] In particular, if the point cloud data is an irregular hole boundary feature, it can be easily removed by calculating the ratio of the number of points inside the model to the number of boundary points. If the ratio is less than 0.7, the boundary point is considered to belong to a regular hole.

[0107] The present embodiment also provides a visual detection system for hole features of body-in-white sheet metal parts, which is characterized by comprising: an acquisition module, which adopts a 3D structured light camera to shoot the body-in-white sheet metal parts from multiple angles to obtain multiple point cloud module data sets after shooting; a data calculation module, which pre-processes the collected point cloud data sets, uses the PCA algorithm to achieve coarse alignment, and then uses the improved ICP algorithm to perform fine alignment to obtain a complete point cloud template of the sheet metal workpiece; a fitting module, which uses the RANSAC algorithm to fit a circle to the hole feature parameters to achieve hole feature fitting.

[0108] This embodiment also provides a computing device, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the visual detection method of hole features of body-in-white sheet metal parts as proposed in the above embodiment.

[0109] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the visual detection method for hole features of a body-in-white sheet metal part as proposed in the above embodiment is implemented.

[0110] The storage medium proposed in this embodiment and the visual detection method for hole features of body-in-white sheet metal parts proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0111] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccessMemory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

[0112] Example 2

[0113] Reference Figure 15-16 , which is an embodiment of the present invention, provides a visual detection method for hole features of a body-in-white sheet metal part. In order to verify the beneficial effects of the present invention, a scientific demonstration is carried out through simulation experiments.

[0114] Experimental environment:

[0115] All the public data sets used in the experiment are from the Stanford University Point Cloud Database. The measurement experiment environment is configured as follows: Fanuc M-10A industrial robot, Elson AT-S1000-07C-S2 structured light camera, and workpiece test bench. The compilation language is C++, the program editor is VS2019, the hardware platform is a desktop computer with a 3.40GHz Intel(R)Xeon(r)Gold 5318Y CPU, NVIDIA GeForce GTX 4090GPU and 128GB RAM, and some algorithms are derived from the open source data processing code library PCL and OpenCV.

[0116] Point cloud preprocessing stability verification:

[0117] Through comparative experiments, the influence of the downsampling method in this paper on feature calculation is analyzed. The traditional voxel downsampling method and the improved voxel downsampling method are used to downsample the same point cloud data and calculate the surface index and isotropy value, and the calculation results are visualized from blue to red in the point cloud data according to the size. At the same time, in order to make the calculation results more intuitive, the values ​​are statistically analyzed and a histogram is constructed. The extreme value of the calculation result is taken as the domain of the horizontal axis of the histogram, and the horizontal axis is evenly divided into 200 small intervals. The number of calculation results in the statistical interval is used as the vertical axis value.

[0118] The traditional voxel downsampling method will cause layered stripes in the point cloud data, and the uneven point set in the nearby neighborhood will seriously affect the calculation results of the neighborhood point cloud feature values, which will lead to uneven surface index visualization results. It is also not difficult to see that the results obtained using the pre-improved voxel downsampling method have abnormal local peaks, and it is difficult to select critical values ​​for segmentation and extraction. In the end, it is impossible to accurately segment and extract the point cloud boundary feature neighborhood point cloud data. Compared with the improved statistical histogram, its calculation results are more stable and the difference is more obvious.

[0119] Point cloud data collected from different directions of workpieces similar to sheet metal parts are usually in the form of sheets, and their overall direction or collection area in the camera coordinate system may be slightly different, but the local structure of the point cloud data at the same position of the workpiece will not change, and theoretically will not affect the stability of the feature point extraction method of the present invention. However, for the sake of rigor, the present invention rotates the workpiece around the x-axis by 0°, 5°, 10°, 15°, 20°, and 25°, and extracts 100 sets of point cloud data from the model surface after each rotation, for a total of 600 sets of point cloud data.

[0120] All point clouds are downsampled using the traditional voxel downsampling method and the downsampling method based on bounding box discretization, and then the boundary feature neighborhood point set is directly extracted. The experimental results are shown in Fig.15 shown.

[0121] When the point cloud data is not rotated, the 95% confidence interval of the number of boundary neighborhood points extracted by any sampling method falls within a reasonable range. As the rotation angle increases, the surface feature stability of the point cloud data obtained by the traditional downsampling method becomes increasingly poor, resulting in an increasing number of misidentified boundary neighborhood points. When the rotation angle is greater than 10°, its 95% confidence interval has exceeded the range of the theoretical number of boundary neighborhood points. The downsampling method based on bounding box discretization proposed in the present invention can automatically determine the main axis direction of the point cloud data and change its downsampling method accordingly. When dealing with situations such as tilt of the point cloud data, it can provide better quality point cloud data for subsequent point cloud processing and improve its stability.

[0122] In particular, the average time for point cloud preprocessing and feature point extraction is 2.1s, which does not increase the time cost compared to other methods of calculating point cloud features. Since there are many methods for extracting point cloud features and their principles vary greatly, this implementation will no longer set up relevant comparative experiments.

[0123] Verification of accuracy and efficiency of point cloud registration algorithm:

[0124] The public data set Dragon and the workpiece surface point cloud data obtained from the CAD model are used to verify the accuracy of the pairwise point cloud registration of the ICP algorithm with local boundary neighborhood feature constraints proposed in this invention. The camera field of view is simulated to manually obtain point cloud data of different viewing angles and positions from the point cloud data set, and the collected data is rigidly transformed. The inverse of the transformation matrix is ​​used as the true value of the pose transformation matrix between any two point clouds. Point cloud data with an overlap rate greater than 30% are selected for registration to obtain the final registration matrix, and the error is measured by the distance between the local coordinate systems of the point clouds before and after the transformation.

[0125] The algorithms used as controls are the classic point-to-plane ICP, point-to-plane ICP, plane-to-plane ICP, and the improved ICP algorithm proposed in this paper. To ensure the fairness of the comparison results, the compared methods are all run using open source libraries, and the parameters of each method are carefully adjusted according to the author's recommended values. The convergence results of several ICP algorithms using public datasets are compared. The experimental results using workpiece simulation point cloud data are shown in Figure 2. Fig.16 shown.

[0126] Fig.16Each data point in represents the average value of more than 500 trials. Before each experiment, there is a different initial transformation matrix of the point cloud while ensuring the same error. The experimental results show that the convergence performance of different methods can be divided into three categories. First, the point-to-point ICP algorithm, its convergence performance approximates linear convergence, but the convergence speed is slow. Then the convergence performance of point-to-plane and plane-to-plane approximates superlinear convergence, and the ICP algorithm with local boundary feature constraints proposed in this paper converges faster than the other methods. In particular, the quality of the dataset is very different from the other two datasets. It contains many flat surfaces, which may lead to large errors in the point cloud registration process due to lack of constraints. Thanks to the homogeneous input point cloud and the improved objective function, the improved algorithm in this paper can still achieve a faster convergence speed in this case.

[0127] Verification of the effectiveness of feature positioning and segmentation extraction algorithms:

[0128] The feature positioning and segmentation extraction algorithm of the present invention is used to segment and extract the point cloud data after real shooting and multi-view stitching. The boundary neighborhood point cloud features are clustered in Euclidean style and the clustering results are displayed in different colors. By setting the point cloud quantity range of the Euclidean clustering subset, the external boundaries and larger internal holes of the point cloud can be effectively removed, and the features can be correctly separated.

[0129] After calculating the geometric features of the cluster subsets, the plane point cloud data near the original point cloud data features are segmented and extracted according to their centroid position and size.

[0130] According to the scheme proposed in the present invention, the hole boundaries in the point cloud data can be effectively segmented and extracted, and the boundary neighborhood point cloud data irrelevant to the features can be removed by setting the number of clusters to ensure the validity of the extracted local point cloud.

[0131] Boundary point extraction and feature parameter extraction accuracy verification:

[0132] Several different types of hole features are selected from the segmented features for boundary extraction. The traditional alphashapes algorithm will extract all boundary points, and new rules need to be formulated to remove external boundary points to avoid affecting the feature parameter fitting. The improved boundary point extraction algorithm of the present invention can effectively extract only the internal hole boundary points for any point cloud data after introducing the centroid judgment step.

[0133] In order to verify the efficiency of the boundary extraction algorithm, the present invention particularly numbered the surface features of the workpiece, simulated and extracted 200 groups of point cloud data on its surface, used the improved alpha shapes algorithm, extracted boundary points from these 200 groups of data, and used the RMSE value of the number of 200 groups of boundary points for each feature to quantify the stability of the extraction results. The stability and efficiency were verified, and the results are shown in the following table.

[0134] Table 1: Boundary point extraction results

[0135]

[0136] Experimental results show that the improved alpha shapes algorithm of the present invention introduces the centroid judgment criterion, and the number of extracted boundary points is more stable, and the standard deviation of the number of boundary points is less than 1.8. Since only the internal boundary is extracted, the boundary extraction efficiency is improved by 43% compared with the original. Furthermore, the extracted boundary points can be used directly without formulating a complex scheme to identify the internal and external boundary points, which can greatly improve the system operation efficiency.

[0137] The present invention uses simulated point cloud data to verify the boundary point extraction algorithm. A plurality of different types of hole features are randomly distributed on the simulated workpiece, and the extracted boundary points are classified. The experimental results are repeated using 500 groups of point cloud data and are completely consistent. Due to the small amount of calculation, the classification speed reaches the millisecond level. The classification results show that the hole feature method provided by the present invention can quickly and accurately classify different types of hole features.

[0138] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A visual detection method for hole features of body-in-white sheet metal parts, characterized in that: include: Use a 3D structured light camera to photograph the body-in-white sheet metal parts, and obtain a point cloud data set of the photographed body-in-white sheet metal parts; The point cloud dataset is preprocessed by using a point cloud downsampling algorithm based on bounding box discretization and a boundary neighborhood feature point set extraction method based on surface index. The point cloud registration algorithm based on local boundary neighborhood feature constraints is used to perform point cloud registration on the pre-processed point cloud set data to obtain a complete point cloud template of the sheet metal workpiece; The features to be measured in the point cloud module are located and segmented to extract boundary points, and the hole characteristic parameters are fitted to the extracted boundary points.

2. The visual detection method for hole features of body-in-white sheet metal parts according to claim 1, characterized in that: The point cloud downsampling algorithm based on bounding box discretization includes constructing a point cloud data covariance matrix; Use singular value decomposition to obtain the eigenvalues ​​and eigenvectors of the covariance matrix; Using the eigenvector as the direction of the coordinate axis and the centroid of the point cloud as the origin of the coordinate axis, the local coordinate system of the point cloud dataset is established and the point cloud bounding box is constructed; The internal point cloud data set is retained by the region of interest, and statistical analysis is performed on the neighborhood of each point according to the statistical filtering algorithm to eliminate the discrete point cloud data on the workpiece surface. The point cloud data on the workpiece surface after elimination is then downsampled to obtain the downsampled point cloud set.

3. The visual detection method for hole features of body-in-white sheet metal parts according to claim 2, characterized in that: The method for extracting a boundary neighborhood feature point set based on the surface index includes establishing a kd tree for downsampled point cloud data; Establish the covariance matrix and solve the eigenvalues ​​and corresponding eigenvectors of the point cloud data; Utilize the feature value of point cloud data to calculate the surface index of point cloud data and extract the boundary points of point cloud data; The Euclidean distance between the original point cloud and the boundary neighborhood point set extracted after downsampling is used to extract the overlapping boundary points to obtain the boundary neighborhood point set of the original point cloud.

4. The visual detection method for hole features of body-in-white sheet metal parts according to claim 3, characterized in that: The point cloud registration based on local boundary neighborhood feature constraints includes taking the union of the original point cloud boundary neighborhood point set and the downsampled point cloud set as input for point cloud coarse registration; The pre-processed point cloud data is used to complete the coordinate system 1 using the robot posture and hand-eye matrix; Modify the input of the ICP algorithm, take the original point cloud boundary neighborhood point set and the downsampled point cloud set, and simply extract the overlapping area point cloud set as the input point cloud based on the Euclidean distance between the points of the two point clouds; Modify the objective function so that it converges completely on the plane and the surface of constant curvature, and enhance the pertinence and convergence of the algorithm.

5. The visual detection method for hole features of body-in-white sheet metal parts according to claim 4, characterized in that: The modified objective function includes: since the point cloud data has the problem of mutual sliding, using the original boundary point cloud set Λ with high density and containing positioning features bd To limit sliding, the modified objective function is expressed as: Where E represents the objective function; R represents the rotation matrix; t represents the translation matrix; represents the source point cloud; represents the target point cloud; Represents the normal vector corresponding to the target point cloud; express The corresponding normal vector; Indicates P λ Round up; N represents the number of corresponding point pairs between two point clouds.

6. The visual detection method for hole characteristics of body-in-white sheet metal parts according to claim 5, characterized in that: The method of locating the features to be measured in the point cloud module and segmenting and extracting boundary points includes: using a boundary neighborhood feature point set extraction method based on a surface index to calculate the eigenvalues ​​and eigenvectors of the covariance matrix for local point cloud data, and analyzing the geometric features and states of its neighborhood according to the relationship between the eigenvalues ​​and eigenvectors of the covariance matrix to extract hole boundary neighborhood features; The boundary neighborhood points of Euclidean clustering features are divided into multiple subsets, and the overall spatial geometric state of each sub-cluster is used to realize feature positioning; The acquired local feature point cloud is reduced in dimension and boundary points are extracted. The local point cloud data is plane-fitted using the RANSAC algorithm. The point cloud data is projected into the local coordinate system of the fitting plane, and the three-dimensional space point cloud data is converted into two-dimensional plane point cloud data to complete the point cloud data dimensionality reduction. After the point cloud data is reduced in dimensionality, the centroid-constrained alpha shapes algorithm is used to extract the plane boundary point data.

7. The method for visually inspecting hole features of a body-in-white sheet metal part according to claim 6, characterized in that: The hole characteristic parameter fitting includes fitting a circle to the hole characteristic parameters using a RANSAC algorithm.

8. A visual inspection system for hole features of a body-in-white sheet metal part using the method according to any one of claims 1 to 7, characterized in that: include, The acquisition module uses a 3D structured light camera to shoot the body-in-white sheet metal parts from multiple perspectives to obtain multiple point cloud module data sets after shooting; The data calculation module pre-processes the collected point cloud data set, uses the PCA algorithm to achieve coarse registration, and then uses the improved ICP algorithm to perform fine registration to obtain a complete point cloud template of the sheet metal workpiece; The fitting module uses the RANSAC algorithm to fit the hole characteristic parameters to a circle, thereby achieving hole characteristic fitting.

9. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Citation Information

Cited By

  • Pose estimation method and device and electronic equipment

    CN119152040A

  • Hardware size error detection method and system based on point cloud processing

    CN120339369A

  • Round hole feature picking method and device based on sheet metal part scanning point cloud and medium

    CN121147288A

  • Sheet metal part scanning point cloud-based round hole feature picking method, device and medium

    CN121147288B

  • Circular hole extraction method and system based on secondary region-of-interest division

    CN122135017A