A method and system for identifying colloids based on point cloud algorithms

Abstract

The present invention discloses a method and system for identifying colloids based on point cloud algorithms, which relates to the field of machine vision technology. The method includes: generating a calibration matrix based on the point cloud data to calibrate the detection head; generating a hand-eye matrix based on the calibration matrix to calibrate the robotic arm; controlling the calibrated detection head based on the calibrated robotic arm to obtain workpiece point cloud data of a vehicle workpiece, and performing planar stitching processing on the workpiece point cloud data to obtain planar point cloud data; extracting features from the planar point cloud data based on histogram analysis to obtain point cloud feature data; performing colloid region identification processing based on the point cloud feature data; and calculating the three-dimensional data of the colloid based on the colloid region identification result to obtain the target three-dimensional data of the colloid. The present invention can avoid a too large deviation between the obtained colloid region identification result and the actual situation, and make the colloid identification achieve a more ideal effect.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method and system for identifying colloids based on a point cloud algorithm. Background Art

[0002] Gluing of body workpieces is a very important link in the automobile manufacturing process, which is related to the levels of vibration reduction, noise reduction and lightweight of the automobile. Therefore, after gluing, it is necessary to identify the colloids on the vehicle workpieces for colloid detection. In current colloid identification, relevant scanning equipment is required to obtain point cloud data of the vehicle workpieces. Before obtaining the point cloud data, it is necessary to calibrate the scanning equipment to ensure that accurate and comprehensive point cloud data can be obtained. At present, the calibration of scanning equipment mostly relies on manual implementation, but this method is too dependent on the professional qualities of relevant personnel and cannot guarantee the accuracy of equipment calibration, resulting in insufficient accuracy and comprehensiveness of the obtained point cloud data, affecting the reliability of colloid identification. As an important link in target recognition, point cloud feature extraction affects the accuracy of target recognition. Currently, point cloud feature extraction is usually achieved through normal change calculation, but the feature data extracted by this method is relatively limited, and more comprehensive point cloud feature data cannot be extracted. At the same time, its accuracy for feature extraction is also insufficient, resulting in a large deviation between the final colloid region recognition result and the actual situation, and the colloid identification fails to achieve the expected effect. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art. The present invention provides a method and system for identifying colloids based on a point cloud algorithm, which can avoid excessive deviation between the obtained colloid region recognition result and the actual situation, and make the colloid identification achieve a more ideal effect.

[0004] To solve the above technical problems, the present invention provides a method for identifying colloids based on a point cloud algorithm, and the method includes:

[0005] Obtaining point cloud data of a calibration board based on a detection head, generating a calibration matrix based on the point cloud data, and performing calibration processing on the detection head based on the calibration matrix to obtain a calibrated detection head;

[0006] Generating a hand-eye matrix based on the calibration matrix, and performing calibration processing on a robotic arm based on the hand-eye matrix to obtain a calibrated robotic arm;

[0007] Controlling the calibrated detection head based on the calibrated robotic arm to obtain workpiece point cloud data of a vehicle workpiece, and performing planar stitching processing on the workpiece point cloud data to obtain planar point cloud data;

[0008] Performing feature extraction on the planar point cloud data based on histogram analysis to obtain point cloud feature data;

[0009] Perform colloidal region recognition processing based on the point cloud feature data to obtain a colloidal region recognition result;

[0010] Perform colloidal three-dimensional data calculation based on the colloidal region recognition result to obtain target colloidal three-dimensional data.

[0011] Optionally, the method of generating a calibration matrix based on the point cloud data and performing calibration processing on the detection head based on the calibration matrix to obtain a calibrated detection head includes:

[0012] Obtain the reference coordinates of the calibration plate, and determine an angle correction matrix based on the reference coordinates and the point cloud data using a transformation matrix;

[0013] Obtain the pose data of the detection head, perform position deviation fitting based on the pose data to obtain position deviation data, generate a compensation matrix based on the position deviation data, and generate a calibration matrix based on the compensation matrix and the angle correction matrix;

[0014] Obtain the transformation relationship between the glue application coordinate system and the detection head coordinate system based on the calibration matrix, and perform calibration processing on the detection head based on the transformation relationship to obtain a calibrated detection head.

[0015] Optionally, the method of generating a hand-eye matrix based on the calibration matrix and performing calibration processing on the robotic arm based on the hand-eye matrix to obtain a calibrated robotic arm includes:

[0016] Construct a rotation and translation matrix based on the point cloud data using the iterative closest point algorithm to obtain the pose data of the robotic arm under different spatial variations;

[0017] Generate a hand-eye matrix based on the rotation and translation matrix and the pose data using the calibration matrix;

[0018] Obtain the robotic arm transformation matrix, and optimize the hand-eye matrix based on the robotic arm transformation matrix using a cost function to obtain an optimized hand-eye matrix;

[0019] Perform calibration processing on the robotic arm based on the optimized hand-eye matrix to obtain a calibrated robotic arm.

[0020] Optionally, the method of optimizing the hand-eye matrix based on the robotic arm transformation matrix using a cost function to obtain an optimized hand-eye matrix includes:

[0021] Calculate the registration error based on the robotic arm transformation matrix using the hand-eye calibration equations, and construct a cost function based on the registration error;

[0022] Iteratively optimize the hand-eye matrix based on the cost function to obtain an optimized hand-eye matrix.

[0023] Optionally, the planar stitching process for the workpiece point cloud data to obtain planar point cloud data includes:

[0024] Perform a filtering process on the workpiece point cloud data to obtain the filtered workpiece point cloud data;

[0025] Select seed points from the filtered workpiece point cloud data, and determine whether the seed points and the non-seed points around the seed points are in the same plane. If it is determined that the seed points and the non-seed points around the seed points are in the same plane, then the non-seed points are used as new seed points;

[0026] Iteratively determine whether the new seed points and the new non-seed points around the new seed points are in the same plane to obtain a number of target seed points;

[0027] Construct planar point cloud data based on all the target seed points.

[0028] Optionally, the feature extraction of the planar point cloud data based on histogram analysis to obtain point cloud feature data includes:

[0029] Extract view point feature histogram descriptors from the planar point cloud data to obtain view point feature histogram descriptors;

[0030] Calculate the extreme points of Gaussian curvature based on the planar point cloud data to obtain the extreme points of Gaussian curvature;

[0031] Determine the point cloud feature points of the planar point cloud data based on the curvature direction using the extreme points of Gaussian curvature;

[0032] Layer the neighborhood of the point cloud feature points to obtain a layered neighborhood, and perform histogram mapping of the point cloud feature points based on the layered neighborhood to obtain a target feature histogram. Generate point cloud feature data based on the target feature histogram and the view point feature histogram descriptors.

[0033] Optionally, the extraction of view point feature histogram descriptors from the planar point cloud data to obtain view point feature histogram descriptors includes:

[0034] Calculate the centroid of the planar point cloud data and the normal vector of each data point, and calculate the angular feature based on the centroid and the normal vector of each data point;

[0035] Perform histogram descriptor combination based on the angular feature, centroid, and normal vector of each data point to obtain view point feature histogram descriptors.

[0036] Optionally, the colloidal region recognition process based on the point cloud feature data to obtain the colloidal region recognition result includes:

[0037] Input the point cloud feature data into the colloid recognition model. The colloid recognition model performs colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result.

[0038] Optionally, calculating the three-dimensional data of the colloid based on the colloid region recognition result to obtain the target three-dimensional data of the colloid, including:

[0039] Determine the center line of the colloid region based on the colloid region recognition result using the skeleton algorithm;

[0040] Divide the center line into point number arrays, obtain several point number arrays, and perform singularity exclusion processing on each point number array to obtain several point number arrays after singularity exclusion processing;

[0041] Determine the three-dimensional point cloud data based on several point number arrays after singularity exclusion processing, and obtain the height data of the colloid based on the laser triangulation method;

[0042] Generate the target three-dimensional data of the colloid based on the three-dimensional point cloud data and the height data.

[0043] In addition, the present invention also provides a system for identifying colloids based on point cloud algorithms. The system includes:

[0044] Detection head calibration module: used to obtain the point cloud data of the calibration plate based on the detection head, generate a calibration matrix based on the point cloud data, and perform calibration processing on the detection head based on the calibration matrix to obtain a calibrated detection head;

[0045] Robotic arm calibration module: used to generate a hand-eye matrix based on the calibration matrix, and perform calibration processing on the robotic arm based on the hand-eye matrix to obtain a calibrated robotic arm;

[0046] Point cloud planar stitching module: used to control the calibrated detection head based on the calibrated robotic arm to obtain the workpiece point cloud data of the vehicle workpiece, and perform planar stitching processing on the workpiece point cloud data to obtain planar point cloud data;

[0047] Feature extraction module: used to extract features from the planar point cloud data based on histogram analysis to obtain point cloud feature data;

[0048] Colloid region recognition module: used to perform colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result;

[0049] Colloid three-dimensional data calculation module: used to calculate the three-dimensional data of the colloid based on the colloid region recognition result to obtain the target three-dimensional data of the colloid.

[0050] In the embodiment of the present invention, the detection head is calibrated based on the calibration matrix generated by the compensation matrix and the angle correction matrix, and the robotic arm is calibrated based on the hand-eye matrix generated by the calibration matrix, which can ensure the accuracy of the calibration of the detection head and the robotic arm, and ensure the accuracy and comprehensiveness of the vehicle workpiece point cloud data obtained. Performing planar stitching processing on the workpiece point cloud data can eliminate the gaps between the point cloud data and form a more complete point cloud data to improve the utilization rate of the point cloud data. Extracting features from the planar point cloud data based on histogram analysis can extract more comprehensive point cloud feature data and improve the accuracy of point cloud data feature extraction. Performing colloidal area recognition processing through the point cloud feature data can avoid excessive deviation between the obtained colloidal area recognition result and the actual situation, so as to output more reliable three-dimensional colloidal data and achieve a more ideal effect of colloidal recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0052] Figure 1 is a schematic flowchart of a method for identifying colloids based on a point cloud algorithm in an embodiment of the present invention;

[0053] Figure 2 is a schematic flowchart of a method for identifying colloids based on a point cloud algorithm in another embodiment of the present invention;

[0054] Figure 3 is a schematic structural composition diagram of a system for identifying colloids based on a point cloud algorithm in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0056] Embodiment 1

[0057] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for identifying colloids based on a point cloud algorithm in an embodiment of the present invention. The method includes:

[0058] S11: Obtain the point cloud data of the calibration board based on the detection head, generate a calibration matrix based on the point cloud data, and perform calibration processing on the detection head based on the calibration matrix to obtain the calibrated detection head;

[0059] In the specific implementation process of the present invention, the generating a calibration matrix based on the point cloud data and performing calibration processing on the detection head based on the calibration matrix to obtain the calibrated detection head includes: obtaining the reference coordinates of the calibration board, and determining the angle correction matrix using the transformation matrix based on the reference coordinates and the point cloud data; obtaining the pose data of the detection head, performing position deviation fitting based on the pose data to obtain position deviation data, and generating a compensation matrix based on the position deviation data, generating a calibration matrix based on the compensation matrix and the angle correction matrix; obtaining the transformation relationship between the glue application coordinate system and the detection head coordinate system based on the calibration matrix, and performing calibration processing on the detection head based on the transformation relationship to obtain the calibrated detection head.

[0060] Specifically, obtain the point cloud data of the calibration board based on the detection head. The point cloud data of the calibration board includes the point cloud data of at least four corner points, and the detection head is a component of a 3D laser profile scanning module. Obtain the reference coordinates of the calibration board, and determine the angle correction matrix using the transformation matrix based on the reference coordinates and the point cloud data. Determine the transformation matrix from the point cloud to the world coordinate system according to the point cloud data and the reference coordinates of the calibration board corner points, determine the deviation value between the actual angle and the preset angle of the detection head according to the transformation matrix, and determine the angle correction matrix according to the deviation value. Obtain the pose data of the detection head, perform position deviation fitting based on the pose data, perform position deviation fitting according to the pose data and the theoretical pose data to obtain position deviation data, and generate a compensation matrix based on the position deviation data, generate a calibration matrix based on the compensation matrix and the angle correction matrix. Obtain the transformation relationship between the glue application coordinate system and the detection head coordinate system based on the calibration matrix, and obtain the transformation relationship between the glue application coordinate system and the detection head coordinate system using the indirect adjustment function according to the calibration matrix. Perform calibration processing on the detection head based on the transformation relationship, obtain the reprojection target point coordinates of the calibration board in the glue application coordinate system according to the transformation relationship, obtain the reprojection error in the glue application coordinate system according to the reprojection target point coordinates, and calibrate the detection head according to the reprojection error and the transformation relationship to obtain the calibrated detection head.

[0061] S12: Generate a hand-eye matrix based on the calibration matrix, and perform calibration processing on the robotic arm based on the hand-eye matrix to obtain the calibrated robotic arm;

[0062] In the specific implementation process of the present invention, generating a hand-eye matrix based on the calibration matrix and performing calibration processing on the robotic arm based on the hand-eye matrix to obtain a calibrated robotic arm includes: constructing a rotation and translation matrix using the iterative closest point algorithm based on the point cloud data to obtain the pose data of the robotic arm under different spatial variations; generating a hand-eye matrix using the calibration matrix based on the rotation and translation matrix and the pose data; obtaining a robotic arm transformation matrix, and optimizing the hand-eye matrix using a cost function based on the robotic arm transformation matrix to obtain an optimized hand-eye matrix; and performing calibration processing on the robotic arm based on the optimized hand-eye matrix to obtain a calibrated robotic arm.

[0063] Further, optimizing the hand-eye matrix using a cost function based on the robotic arm transformation matrix to obtain an optimized hand-eye matrix includes: calculating a registration error using a hand-eye calibration equation set based on the robotic arm transformation matrix, and constructing a cost function based on the registration error; and iteratively optimizing the hand-eye matrix based on the cost function to obtain an optimized hand-eye matrix.

[0064] Specifically, an iterative closest point algorithm is used to construct a rotation and translation matrix based on the point cloud data. The iterative closest point algorithm is a point cloud registration algorithm that performs preliminary rough registration on the point cloud data to obtain an initial transformation matrix. The point cloud data is sampled to obtain a set of matching points and nearest neighbor points. The rotation matrix and translation matrix are calculated based on the set of matching points and nearest neighbor points. The rotation matrix and translation matrix are subjected to rotation and translation transformations to obtain a corresponding set of points. The initial transformation matrix calculates the transformation matrix according to the corresponding set of points using a perspective projection transformation vector, and calculates an error function. The above process is repeated until a preset number of iterations is reached to minimize the error, obtaining a rotation and translation matrix, and obtaining the pose data of the robotic arm under different spatial changes. Based on the rotation and translation matrix and the pose data, a hand-eye matrix is generated using the calibration matrix. The rotation and translation matrix and the pose data are used to calculate a pose matrix. According to the pose matrix and the calibration matrix, a matrix solution is performed using an equation system under a preset robotic arm motion state in combination with the least squares method to obtain a hand-eye matrix. Based on the robotic arm transformation matrix, the registration error is calculated using a hand-eye calibration equation system. The first transformation matrix of the calibration plate relative to the detection head coordinate system and the second transformation matrix of the end of the robotic arm relative to the robotic arm base coordinate system are obtained. A hand-eye calibration equation system is constructed based on the first transformation matrix and the second transformation matrix. The registration error is calculated using a quaternion solution method in combination with the hand-eye calibration equation system through the robotic arm transformation matrix, and a cost function is constructed based on the registration error. The hand-eye matrix is iteratively optimized based on the cost function. The hand-eye transformation matrix is converted to the Lie algebra space, and the cost function is transformed in the Lie algebra space to obtain a transformed cost function. The hand-eye matrix is iteratively optimized using the least squares estimation algorithm in combination with the transformed cost function to obtain an optimized hand-eye matrix, taking into account the influence of the robotic arm positioning accuracy on the solution of the transformation matrix, thereby improving the optimization accuracy of the hand-eye matrix. The robotic arm is calibrated based on the optimized hand-eye matrix to obtain a calibrated robotic arm.

[0065] S13: Based on the calibrated robotic arm, control the calibrated detection head to obtain the workpiece point cloud data of the vehicle workpiece, and perform planar stitching processing on the workpiece point cloud data to obtain planar point cloud data;

[0066] In the specific implementation process of the present invention, the performing planar stitching processing on the workpiece point cloud data to obtain planar point cloud data includes: performing filtering processing on the workpiece point cloud data to obtain filtered workpiece point cloud data; selecting seed points in the filtered workpiece point cloud data, and determining whether the seed points and the non-seed points around the seed points are in the same plane. If it is determined that the seed points and the non-seed points around the seed points are in the same plane, the non-seed points are used as new seed points; iteratively determine whether the new seed points and the new non-seed points around the new seed points are in the same plane to obtain a number of target seed points; construct planar point cloud data based on all the target seed points.

[0067] Specifically, based on the calibrated robotic arm, the calibrated detection head is controlled to obtain the workpiece point cloud data of the vehicle workpiece. The calibrated robotic arm moves the calibrated detection head to the target position to collect the point cloud data of the vehicle workpiece, and the workpiece point cloud data is obtained. Filter the workpiece point cloud data, perform block processing on the workpiece point cloud data to obtain several workpiece point cloud data blocks, obtain the adjacent point cloud blocks of each workpiece point cloud data block, determine the projection plane of the adjacent point cloud blocks, determine the adjacent reconstruction points of the boundary points in the workpiece point cloud data block according to the projection plane, and perform point cloud filtering processing according to the adjacent reconstruction points of the boundary points to obtain the filtered workpiece point cloud data. Select seed points from the filtered workpiece point cloud data, randomly select a data point in the filtered workpiece point cloud data as a seed point, and determine whether the seed point and the non-seed points around the seed point are in the same plane. Among them, the normal vector of the seed point is perpendicular to the ground normal vector. If it is determined that the seed point and the non-seed points around the seed point are in the same plane, then the non-seed points are used as new seed points. Iteratively determine whether the new seed points and the new non-seed points around the new seed points are in the same plane. At the same time, during the iterative determination process, all the seed points are counted by the region growing method, that is, several target seed points are obtained. Construct planar point cloud data based on all the target seed points, construct planar point cloud blocks according to all the counted seed points, and merge all the constructed planar point cloud blocks to obtain planar point cloud data. Performing planar stitching processing on the workpiece point cloud data can eliminate the gaps between the point cloud data and form a more complete point cloud data to improve the utilization rate of the point cloud data.

[0068] S14: Extract features from the planar point cloud data based on histogram analysis to obtain point cloud feature data;

[0069] In the specific implementation process of the present invention, the extracting features from the planar point cloud data based on histogram analysis to obtain point cloud feature data includes: extracting view point feature histogram descriptors from the planar point cloud data to obtain view point feature histogram descriptors; calculating the extreme points of Gaussian curvature based on the planar point cloud data to obtain the extreme points of Gaussian curvature; determining the point cloud feature points of the planar point cloud data based on the extreme points of Gaussian curvature using the curvature direction; stratifying the neighborhood of the point cloud feature points to obtain a stratified neighborhood, and performing histogram mapping of the point cloud feature points based on the stratified neighborhood to obtain a target feature histogram, and generating point cloud feature data based on the target feature histogram and the view point feature histogram descriptors.

[0070] Further, extracting a viewpoint feature histogram descriptor from the planar point cloud data to obtain a viewpoint feature histogram descriptor includes: calculating the centroid of the planar point cloud data and the normal vector of each data point, and calculating an angular feature based on the centroid and the normal vector of each data point; performing histogram descriptor combination based on the angular feature, the centroid, and the normal vector of each data point to obtain a viewpoint feature histogram descriptor.

[0071] Specifically, calculating the centroid of the planar point cloud data and the normal vector of each data point, calculating an angular feature based on the centroid and the normal vector of each data point, for each data point, calculating the angle between the normal vector of the data point and the line connecting the centroid, calculating the angles between the normal vectors of each data point, and forming an angular feature according to the angle between the normal vector of the data point and the line connecting the centroid and the angles between the normal vectors of each data point. Performing histogram descriptor combination based on the angular feature, the centroid, and the normal vector of each data point, calculating a number of viewpoint feature histogram representations through the angular feature, the centroid, and the normal vector of each data point, and connecting the number of viewpoint feature histograms in a preset order to obtain a viewpoint feature histogram descriptor. Calculating the Gaussian curvature extreme points based on the planar point cloud data, performing block rasterization on the planar point cloud data to obtain the planar point cloud data of each block raster, and obtaining the Gaussian curvature extreme points by finding the Gaussian curvature extreme points in the planar point cloud data of each block raster. Determining the point cloud feature points of the planar point cloud data based on the Gaussian curvature extreme points using the curvature direction, locating the extreme points in the other principal directions in the neighborhood of the Gaussian curvature extreme points along the direction of the principal curvature, and performing feature point retrieval along the principal direction to obtain the point cloud feature points. Layingering the neighborhood of the point cloud feature points to obtain a layered neighborhood, and performing histogram mapping of the point cloud feature points based on the layered neighborhood, calculating the depth from the point cloud feature points to the projection plane in the layered neighborhood, calculating the inclination of each layered neighborhood relative to the point cloud feature points according to the depth, mapping the inclination to a preset histogram to obtain a target feature histogram, and generating point cloud feature data based on the target feature histogram and the viewpoint feature histogram descriptor.

[0072] S15: Performing colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result;

[0073] In the specific implementation process of the present invention, performing colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result includes: inputting the point cloud feature data into a colloid recognition model, and the colloid recognition model performing colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result.

[0074] Specifically, input the point cloud feature data into the colloid recognition model. The colloid recognition model performs colloid region recognition processing based on the point cloud feature data. The colloid recognition model is a convergent model obtained by inputting a sample data set into a deep neural network for training. The deep neural network includes a conventional convolutional layer, a dilated convolutional layer, a feature fusion layer, and an upsampling layer. The conventional convolutional layer is used to generate low-level features according to the point cloud feature data. The dilated convolutional layer is used to generate several high-level abstract features according to the point cloud feature data. The feature fusion layer is used to fuse several high-level abstract features to obtain high-level fusion features. The upsampling layer is used to perform upsampling processing after integrating the high-level fusion features and the low-level features to obtain the colloid region in the vehicle workpiece, that is, to obtain the colloid region recognition result.

[0075] S16: Calculate the three-dimensional data of the colloid based on the colloid region recognition result to obtain the target three-dimensional data of the colloid.

[0076] In the specific implementation process of the present invention, the calculating the three-dimensional data of the colloid based on the colloid region recognition result to obtain the target three-dimensional data of the colloid includes: determining the center line of the colloid region based on the colloid region recognition result by using the skeleton algorithm; dividing the center line into point number arrays to obtain several point number arrays, and performing singular point exclusion processing on each point number array to obtain several point number arrays after singular point exclusion processing; determining three-dimensional point cloud data based on several point number arrays after singular point exclusion processing, and obtaining the height data of the colloid based on the laser triangulation method; generating the target three-dimensional data of the colloid based on the three-dimensional point cloud data and the height data.

[0077] Specifically, determine the target point cloud image of the colloid area in the vehicle workpiece according to the colloid area recognition result, and use the skeleton algorithm to determine the center line of the colloid area in the target point cloud image. Divide the center line into point number arrays to obtain several point number arrays. Perform singular point exclusion processing on each point number array, initialize the result array, calculate the median of each point number array, traverse each point position in the point number array, calculate the first distance between each point position and the median, calculate the second distance between each point position and its adjacent point positions, calculate the target distance according to the first distance and the second distance. If the target distance is greater than or equal to the preset threshold, then this point position is a singular point, and the determined singular points are excluded to obtain several point number arrays after singular point exclusion processing. Determine the three-dimensional point cloud data based on several point number arrays after singular point exclusion processing, perform curve fitting on several point number arrays after singular point exclusion processing using the least squares approximation method to obtain the optimized center line, determine the detection points of the optimized center line, determine the corresponding detection lines according to the detection points, calculate the intersection points of the detection lines and the colloid area, determine the three-dimensional point cloud data of the colloid according to the intersection points of the detection lines and the colloid area, such as the width and length of the colloid, and obtain the height data of the colloid based on the laser triangulation method, that is, obtain the height data of the colloid area through the principle of laser triangulation measurement. Generate the target colloid three-dimensional data based on the three-dimensional point cloud data and the height data.

[0078] In the embodiment of the present invention, calibrate the detection head based on the calibration matrix generated by the compensation matrix and the angle correction matrix, and calibrate the robotic arm based on the hand-eye matrix generated by the calibration matrix, which can ensure the accuracy of the calibration of the detection head and the robotic arm, and ensure the accuracy and comprehensiveness of the vehicle workpiece point cloud data obtained. Perform planar stitching processing on the workpiece point cloud data, which can eliminate the gaps between the point cloud data and form a more complete point cloud data to improve the utilization rate of the point cloud data. Extract the feature data of the planar point cloud data based on the histogram analysis, which can extract more comprehensive point cloud feature data and improve the accuracy of the extraction of the point cloud data features. Perform colloid area recognition processing through the point cloud feature data, which can avoid the deviation between the obtained colloid area recognition result and the actual situation being too large, so as to be able to output more reliable colloid three-dimensional data and make the colloid recognition reach a more ideal effect.

[0079] Embodiment 2

[0080] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a method for identifying a colloid based on a point cloud algorithm in another embodiment of the present invention. The method includes:

[0081] S201: Obtain the point cloud data of the calibration board based on the detection head, generate a calibration matrix based on the point cloud data, and calibrate the detection head based on the calibration matrix to obtain the detection head after calibration processing;

[0082] S202: Generate a hand-eye matrix based on the calibration matrix, and calibrate the robotic arm based on the hand-eye matrix to obtain the calibrated robotic arm;

[0083] S203: Control the calibrated detection head based on the calibrated robotic arm to obtain the workpiece point cloud data of the vehicle workpiece, and perform planar stitching processing on the workpiece point cloud data to obtain planar point cloud data;

[0084] S204: Extract viewpoint feature histogram descriptors from the planar point cloud data to obtain viewpoint feature histogram descriptors;

[0085] In the specific implementation process of the present invention, calculate the centroid of the planar point cloud data and the normal vector of each data point, calculate the angular feature based on the centroid and the normal vector of each data point, for each data point, calculate the angle between the normal vector of the data point and the line connecting the centroid, calculate the angle between the normal vectors of each data point, and form an angular feature according to the angle between the normal vector of the data point and the line connecting the centroid and the angle between the normal vectors of each data point. Combine histogram descriptors based on the angular feature, centroid, and normal vector of each data point, calculate several viewpoint feature histograms through the angular feature, centroid, and normal vector of each data point, and connect the several viewpoint feature histograms in a preset order to obtain viewpoint feature histogram descriptors.

[0086] S205: Calculate the extreme points of Gaussian curvature based on the planar point cloud data to obtain the extreme points of Gaussian curvature;

[0087] In the specific implementation process of the present invention, calculate the extreme points of Gaussian curvature based on the planar point cloud data, perform block grid division on the planar point cloud data to obtain the planar point cloud data of each block grid, and obtain the extreme points of Gaussian curvature by finding the extreme points of Gaussian curvature in the planar point cloud data of each block grid.

[0088] S206: Determine the point cloud feature points of the planar point cloud data based on the extreme points of Gaussian curvature using the curvature direction;

[0089] In the specific implementation process of the present invention, determine the point cloud feature points of the planar point cloud data based on the extreme points of Gaussian curvature using the curvature direction, locate the extreme points in the other principal directions in the neighborhood of the extreme points of Gaussian curvature along the direction of the principal curvature, and perform feature point retrieval along the principal direction to obtain point cloud feature points.

[0090] S207: Layer the neighborhood of the point cloud feature points to obtain a layered neighborhood, and perform histogram mapping of the point cloud feature points based on the layered neighborhood to obtain a target feature histogram, and generate point cloud feature data based on the target feature histogram and the viewpoint feature histogram descriptors;

[0091] In the specific implementation process of the present invention, the neighborhood of the point cloud feature points is stratified to obtain a stratified neighborhood, and histogram mapping of the point cloud feature points is performed based on the stratified neighborhood. The depth from the point cloud feature points to the projection plane in the stratified neighborhood is calculated, and the inclination of each stratified neighborhood relative to the point cloud feature points is calculated according to this depth. The inclination is mapped into a preset histogram to obtain a target feature histogram, and point cloud feature data is generated based on the target feature histogram and the view point feature histogram descriptor.

[0092] S208: Perform colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result;

[0093] S209: Perform colloid three-dimensional data calculation based on the colloid region recognition result to obtain target colloid three-dimensional data.

[0094] In the embodiment of the present invention, the detection head is calibrated based on the calibration matrix generated by the compensation matrix and the angle correction matrix, and the robotic arm is calibrated based on the hand-eye matrix generated by the calibration matrix, which can ensure the accuracy of the calibration of the detection head and the robotic arm, and ensure the accuracy and comprehensiveness of the vehicle workpiece point cloud data obtained. Performing planar stitching processing on the workpiece point cloud data can eliminate the gaps between the point cloud data and form a more complete point cloud data to improve the utilization rate of the point cloud data. Feature extraction is performed on the planar point cloud data based on histogram analysis, which can extract more comprehensive point cloud feature data and improve the accuracy of point cloud data feature extraction. Performing colloid region recognition processing through the point cloud feature data can avoid too large a deviation between the obtained colloid region recognition result and the actual situation, so as to be able to output more reliable colloid three-dimensional data and make the colloid recognition reach a more ideal effect.

[0095] Embodiment III

[0096] Please refer to Figure 3 , Figure 3 which is a schematic structural composition diagram of the colloid recognition system based on the point cloud algorithm in the embodiment of the present invention. The system includes:

[0097] Detection head calibration module 31: used to obtain the point cloud data of the calibration board based on the detection head, generate a calibration matrix based on the point cloud data, and perform calibration processing on the detection head based on the calibration matrix to obtain the calibrated detection head;

[0098] Robotic arm calibration module 32: used to generate a hand-eye matrix based on the calibration matrix, and perform calibration processing on the robotic arm based on the hand-eye matrix to obtain the calibrated robotic arm;

[0099] Point cloud planar stitching module 33: It is used to obtain workpiece point cloud data of a vehicle workpiece by means of a calibrated detection head based on the calibration-processed robotic arm, and perform planar stitching processing on the workpiece point cloud data to obtain planar point cloud data;

[0100] Feature extraction module 34: It is used to extract features from the planar point cloud data based on histogram analysis to obtain point cloud feature data;

[0101] Colloid area recognition module 35: It is used to perform colloid area recognition processing based on the point cloud feature data to obtain a colloid area recognition result;

[0102] Colloid three-dimensional data calculation module 36: It is used to calculate colloid three-dimensional data based on the colloid area recognition result to obtain target colloid three-dimensional data.

[0103] In the specific implementation process of the present invention, the specific implementation manners of the system items can refer to the implementation manners of the above method items, which will not be elaborated here.

[0104] In the embodiments of the present invention, calibrating the detection head based on the calibration matrix generated by the compensation matrix and the angle correction matrix, and calibrating the robotic arm based on the hand-eye matrix generated by the calibration matrix can ensure the accuracy of the calibration of the detection head and the robotic arm, and ensure the accuracy and comprehensiveness of the obtained vehicle workpiece point cloud data. Performing planar stitching processing on the workpiece point cloud data can eliminate the gaps between the point cloud data and form a more complete point cloud data to improve the utilization rate of the point cloud data. Extracting features from the planar point cloud data based on histogram analysis can extract more comprehensive point cloud feature data and improve the accuracy of point cloud data feature extraction. Performing colloid area recognition processing through the point cloud feature data can avoid too large a deviation between the obtained colloid area recognition result and the actual situation, so as to be able to output more reliable colloid three-dimensional data and make the colloid recognition reach a more ideal effect.

[0105] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and this program can be stored in a computer-readable storage medium. The storage medium can include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0106] In addition, the above has introduced in detail a method and system for identifying colloids based on a point cloud algorithm provided by the embodiments of the present invention. Specific examples should have been used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for identifying colloids based on a point cloud algorithm, characterized in that: The method comprises: Acquiring point cloud data of a calibration plate based on a detection head, generating a calibration matrix based on the point cloud data, and performing calibration processing on the detection head based on the calibration matrix to obtain a calibrated detection head; Generate a hand-eye matrix based on the calibration matrix, and perform calibration processing on the robotic arm based on the hand-eye matrix to obtain a calibrated robotic arm; Based on the calibrated robotic arm, the calibrated detection head is controlled to obtain workpiece point cloud data of the vehicle workpiece, and the workpiece point cloud data is subjected to planar splicing processing to obtain planar point cloud data; Extracting features from the planar point cloud data based on histogram analysis to obtain point cloud feature data; Performing colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result; Calculating colloid three-dimensional data based on the colloid region recognition result to obtain target colloid three-dimensional data; The feature extraction of the planar point cloud data based on histogram analysis to obtain point cloud feature data includes: extracting a viewpoint feature histogram descriptor from the planar point cloud data to obtain a viewpoint feature histogram descriptor; calculating Gaussian curvature extreme points based on the planar point cloud data to obtain Gaussian curvature extreme points; determining point cloud feature points of the planar point cloud data using curvature directions based on the Gaussian curvature extreme points; stratifying the neighborhood of the point cloud feature points to obtain a stratified neighborhood, and performing histogram mapping of the point cloud feature points based on the stratified neighborhood to obtain a target feature histogram, and generating point cloud feature data based on the target feature histogram and the viewpoint feature histogram descriptor; The method of extracting a viewpoint feature histogram descriptor from the planar point cloud data to obtain a viewpoint feature histogram descriptor includes: calculating the centroid of the planar point cloud data and the normal vector of each data point, and calculating an angle feature based on the centroid and the normal vector of each data point; and combining histogram descriptors based on the angle feature, the centroid and the normal vector of each data point to obtain a viewpoint feature histogram descriptor.

2. The method for identifying colloids based on point cloud algorithm according to claim 1, characterized in that: The generating a calibration matrix based on the point cloud data, and performing calibration processing on the detection head based on the calibration matrix to obtain the calibrated detection head includes: Obtaining the reference coordinates of the calibration plate, and determining an angle correction matrix using a transformation matrix based on the reference coordinates and point cloud data; Acquire the posture data of the detection head, perform position deviation fitting based on the posture data to obtain position deviation data, generate a compensation matrix based on the position deviation data, and generate a calibration matrix based on the compensation matrix and the angle correction matrix; The conversion relationship between the gluing coordinate system and the detection head coordinate system is acquired based on the calibration matrix, and the detection head is calibrated based on the conversion relationship to obtain the calibrated detection head.

3. The method for identifying colloids based on point cloud algorithm according to claim 1, characterized in that: The step of generating a hand-eye matrix based on the calibration matrix, and calibrating the robotic arm based on the hand-eye matrix to obtain the calibrated robotic arm includes: Based on the point cloud data, a rotation and translation matrix is ​​constructed using an iterative closest point algorithm to obtain position and posture data of the robotic arm in different spaces; Generate a hand-eye matrix using the calibration matrix based on the rotation and translation matrix and the position and posture data; Acquire a robot arm transformation matrix, and optimize the hand-eye matrix using a cost function based on the robot arm transformation matrix to obtain an optimized hand-eye matrix; The robotic arm is calibrated based on the optimized hand-eye matrix to obtain the calibrated robotic arm.

4. The method for identifying colloids based on point cloud algorithm according to claim 3, characterized in that: The step of optimizing the hand-eye matrix using a cost function based on the robot arm transformation matrix to obtain an optimized hand-eye matrix includes: Calculating the registration error using the hand-eye calibration equations based on the robot arm transformation matrix, and constructing a cost function based on the registration error; The hand-eye matrix is ​​iteratively optimized based on the cost function to obtain an optimized hand-eye matrix.

5. The method for identifying colloids based on point cloud algorithm according to claim 1, characterized in that: The step of performing planar splicing processing on the workpiece point cloud data to obtain planar point cloud data includes: Performing filtering processing on the workpiece point cloud data to obtain workpiece point cloud data after filtering processing; Selecting a seed point from the workpiece point cloud data after filtering, and determining whether the seed point and the non-seed points around the seed point are located in the same plane; if it is determined that the seed point and the non-seed points around the seed point are located in the same plane, using the non-seed point as a new seed point; Iteratively determine whether the new seed point and new non-seed points around the new seed point are located in the same plane to obtain a number of target seed points; Construct planar point cloud data based on all target seed points.

6. The method for identifying colloids based on point cloud algorithm according to claim 1, characterized in that: The performing colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result includes: The point cloud feature data is input into a colloid recognition model, and the colloid recognition model performs colloid region recognition processing based on the point cloud feature data to obtain a colloid region recognition result.

7. The method for identifying colloids based on point cloud algorithm according to claim 1, characterized in that: The step of calculating the colloid three-dimensional data based on the colloid region recognition result to obtain the target colloid three-dimensional data includes: Determine the center line of the colloid region using a skeleton algorithm based on the colloid region recognition result; Dividing the center line into point arrays to obtain a plurality of point arrays, and performing singular point elimination processing on each point array to obtain a plurality of point arrays after the singular point elimination processing; Determine the three-dimensional point cloud data based on the point array after several singular points are eliminated, and obtain the height data of the colloid based on the laser triangulation method; The target colloid three-dimensional data is generated based on the three-dimensional point cloud data and the height data.

8. A system for identifying colloids based on point cloud algorithm, characterized in that: The system comprises: A detection head calibration module: used for acquiring point cloud data of a calibration plate based on a detection head, generating a calibration matrix based on the point cloud data, and performing calibration processing on the detection head based on the calibration matrix to obtain a calibrated detection head; A robotic arm calibration module: used to generate a hand-eye matrix based on the calibration matrix, and perform calibration processing on the robotic arm based on the hand-eye matrix to obtain the calibrated robotic arm; Point cloud planar splicing module: used to control the calibrated detection head based on the calibrated robotic arm to obtain workpiece point cloud data of the vehicle workpiece, and perform planar splicing processing on the workpiece point cloud data to obtain planar point cloud data; Feature extraction module: used for extracting features from the planar point cloud data based on histogram analysis to obtain point cloud feature data; Colloid region identification module: used for performing colloid region identification processing based on the point cloud feature data to obtain a colloid region identification result; Colloid three-dimensional data calculation module: used to calculate the colloid three-dimensional data based on the colloid region recognition result to obtain the target colloid three-dimensional data; The feature extraction of the planar point cloud data based on histogram analysis to obtain point cloud feature data includes: extracting a viewpoint feature histogram descriptor from the planar point cloud data to obtain a viewpoint feature histogram descriptor; calculating Gaussian curvature extreme points based on the planar point cloud data to obtain Gaussian curvature extreme points; determining point cloud feature points of the planar point cloud data using curvature directions based on the Gaussian curvature extreme points; stratifying the neighborhood of the point cloud feature points to obtain a stratified neighborhood, and performing histogram mapping of the point cloud feature points based on the stratified neighborhood to obtain a target feature histogram, and generating point cloud feature data based on the target feature histogram and the viewpoint feature histogram descriptor; The method of extracting a viewpoint feature histogram descriptor from the planar point cloud data to obtain a viewpoint feature histogram descriptor includes: calculating the centroid of the planar point cloud data and the normal vector of each data point, and calculating an angle feature based on the centroid and the normal vector of each data point; and combining histogram descriptors based on the angle feature, the centroid and the normal vector of each data point to obtain a viewpoint feature histogram descriptor.

Images