A three-dimensional point cloud segmentation method for production line part identification

Abstract

The application discloses a three-dimensional point cloud segmentation method for production line part identification, comprising the following steps: for a certain to-be-identified production line, a VFH sample library containing all part point clouds of the production line scene is constructed in advance; two-dimensional image data of the production line is collected, target detection based on deep learning is performed on the two-dimensional image data to obtain a region of interest, three-dimensional point clouds are restored and pretreated; the pretreated production line point clouds are input into a hyper voxel clustering algorithm combined with edge information to generate hyper voxels; the hyper voxels are input into a region growing segmentation algorithm based on point cloud color and convexity features to segment production line part point clouds; and the segmented production line part point clouds are matched with the production line part point clouds in the VFH sample library to realize production line part point cloud identification. The method can generate hyper voxels with better-attached object boundaries, the region growing algorithm has better effect, and a technical route for production line part identification is provided.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional point cloud data processing technology, specifically relating to a three-dimensional point cloud segmentation method for production line part identification. Background Technology

[0002] In recent years, with the continuous development of technology, technologies such as autonomous driving, robotics, and intelligent manufacturing have emerged one after another. All of these technologies rely heavily on machine vision, and 3D point cloud processing, as an important part of the machine vision field, has received increasing attention. Currently, some traditional production workshops need to automatically identify and locate parts to improve production efficiency, thereby achieving purposes such as sorting and assembly. 3D point cloud data contains rich 3D spatial information, and 3D point cloud segmentation can obtain real objects from point cloud information. Therefore, this type of technology plays an important role in part recognition on production lines.

[0003] In the field of point cloud segmentation, many methods have been proposed: edge-based methods are efficient but have poor robustness; attribute-based methods are robust but not efficient enough; model-based methods require the point cloud to conform to a geometric shape with a mathematical expression; region-growing methods have good applicability, good segmentation results, and are also efficient, but are sensitive to seed point selection and growth strategies; graph-based methods have good segmentation results but require a large amount of computation; deep learning-based methods are applicable to complex scenes but require a large amount of data and computation. Typically, over-segmentation can be performed before point cloud segmentation to generate supervoxels, reducing the amount of data, removing noise points, and supervoxels contain richer information than individual point clouds. Methods for generating supervoxels mainly include voxel-based and point-based methods: (Papon J, Abramov A, Schoeler M, et al. Voxel cloudconnectivity segmentation-supervoxels for point clouds[C] / / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:2027-2034.) was the first to propose using 3D point clouds to generate supervoxels, which can obtain spatially connected supervoxels. This method is voxel-based and has high algorithm efficiency; however, it does not fully consider object boundary information, making it difficult for supervoxel boundaries to adhere well to object boundaries. Point-based methods have better segmentation results, but often require a large amount of computation. All of these problems negatively impact the accuracy and efficiency of production line part recognition. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing supervoxel generation methods and point cloud segmentation methods by providing a 3D point cloud segmentation method for production line part recognition. This method first inputs the point cloud into a supervoxel clustering algorithm that incorporates edge information to generate supervoxels whose boundaries better adhere to the object boundaries. This voxel-based algorithm is highly efficient. Then, the supervoxels are input into a region growing segmentation algorithm that combines point cloud color and concavity / convexity features. This algorithm improves the growth strategy based on the region growing concept, which offers both good segmentation results and efficiency, and combines two types of point cloud features to make the segmentation more accurate and versatile. This improves the subsequent production line part recognition effect and enhances the engineering application value of point cloud segmentation.

[0005] The present invention is achieved by at least one of the following technical solutions.

[0006] A 3D point cloud segmentation method for production line part identification includes the following steps:

[0007] S1. For a specific production line to be identified, construct a VFH sample library containing point clouds of all parts in the production line scene;

[0008] S2. Collect two-dimensional image data of the production line, perform deep learning-based target detection to obtain the region of interest, reconstruct the three-dimensional point cloud, and perform preprocessing.

[0009] S3. Input the preprocessed production line point cloud into the supervoxel clustering algorithm that combines edge information to generate supervoxels;

[0010] S4. Input the super voxels into the region growth segmentation algorithm based on the combination of point cloud color and concavity / convexity features to segment out the point cloud of production line parts.

[0011] S5. Calculate the VFH value of the segmented production line part point cloud, and match it with the production line part point cloud in the VFH sample library using KD-tree to achieve production line part point cloud recognition.

[0012] Further, step S1 includes the following steps:

[0013] S11. Select two-dimensional depth maps and RGB maps of the production line to be identified from different angles, generate a point cloud of the production line and downsample it according to the intrinsic parameters of the depth camera;

[0014] S12. Segment all part point clouds contained in the production line point cloud, calculate the VFH value of each part point cloud, and construct a sample library containing part point cloud VFH values ​​and category labels using KD-tree.

[0015] Furthermore, the two-dimensional depth map and RGB image of the production line to be identified are selected from the following angles: the point where the depth camera is perpendicular to the production line, the 60° angle, the 75° angle, the 105° angle, and the 120° angle.

[0016] Further, step S2 includes the following steps:

[0017] S21. Use a depth camera to acquire a two-dimensional RGB image and a depth image of the production line, and perform deep learning-based target detection on the two-dimensional RGB image data to obtain the region of interest.

[0018] S22. Combining the 2D RGB image and the depth image, based on the intrinsic parameters of the depth camera, the region of interest is restored to obtain a 3D point cloud and then downsampled.

[0019] Furthermore, step S3 includes the following steps:

[0020] S31. Convert the pre-processed 3D point cloud of the production line into voxels;

[0021] S32. Perform object boundary voxel detection.

[0022] S33. Remove the detected object boundary voxels from the voxel adjacency structure to prevent super voxels from clustering into other types of objects through the object boundary voxels, and better maintain the super voxel boundary attached to the object boundary.

[0023] S34. Based on the set super-voxel resolution R seed Seed voxels are selected. Since a supervoxel contains multiple voxels, R... seed >>R voxel ;

[0024] S35. Starting with each selected seed voxel, cluster them according to the feature distance to generate super voxels. After all super voxels are clustered, recalculate and update the features of each super voxel. Repeat the super voxel clustering and feature calculation and update work until the preset number of iterations is reached or the features of each super voxel are stable, and the final super voxel is obtained.

[0025] Furthermore, the specific steps of voxelization are as follows: Establish the minimum bounding box of the preprocessed 3D point cloud of the production line, and determine the voxel resolution R based on the set value. voxel The bounding box is recursively divided into eight equal parts using an octree to obtain small cubes. This process involves voxelizing the input 3D point cloud of the production line and obtaining the voxel adjacency structure. Each small cube represents a node of the octree.

[0026] Furthermore, the specific steps for object boundary voxel detection are as follows:

[0027] (1) Obtain the center point of each voxel. For each center point, form an approximate plane with its k neighborhood points such that the sum of the squares of the distances from each point in the neighborhood to the center point is minimized, i.e.:

[0028]

[0029] in Let be the normal vector of the approximate plane, d be the distance from the centroid of the approximate plane to the origin, and p be the normal vector of the approximate plane. i For the points that form an approximate plane, For p i Distance to the origin; Represents the approximate plane to be fitted;

[0030] (2) To fit the approximate plane using principal component analysis, first calculate the centroid p of all points in the neighborhood:

[0031]

[0032] (3) Calculate the decentered covariance matrix M of each point in the approximate plane:

[0033]

[0034] (4) Perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​λ1, λ2, and λ3, where λ1 < λ2 < λ3. The eigenvalues ​​represent the degree of dispersion of points in the neighborhood in the direction of the corresponding eigenvector. The smaller the eigenvalue, the lower the degree of dispersion of the points in the direction of the corresponding eigenvector. Therefore, the eigenvector corresponding to λ1 is the normal vector of the approximate plane, which is also the normal vector of the center point to be found.

[0035] (5) After obtaining the normal vector of each center point, calculate the angle between the normal vector of each center point and the normal vector of the neighboring points within the set radius. If the angle exceeds the set threshold, it is determined to be a boundary point, that is, the voxel represented by the center point is a boundary voxel.

[0036] Further, step S4 includes the following steps:

[0037] S41. Use the input hypervoxels to obtain the representative points of each hypervoxel, and arrange them in ascending order of curvature to form a seed point queue.

[0038] S42. Take a seed point from the first position in the queue, search for its k nearest neighbors, and calculate the feature distance between the seed point and each nearest neighbor. When the distance is less than a set threshold, the seed point includes its corresponding nearest neighbor in its cluster to achieve region growth. Simultaneously, determine if the nearest neighbor can be used as a seed point to inherit the region growth work of the cluster. If no nearest neighbor inherits, the region growth of that cluster ends. Repeat this step until the seed point queue is empty. The feature distance used is combined with the point cloud color and concavity / convexity feature D, i.e.

[0039]

[0040] Where λ and μ are the weights of the point cloud color distance and concavity / convexity distance, and D col D represents the color distance of the point cloud. conThis represents the convexity / concavity distance of the point cloud. The value is 0 when the relationship between two points is convex, and 1 when the relationship is concave. The specific rule for determining convexity / concavity is that the vector from point p1 to point p2 in the point cloud... normal vector to point p1 The angle α1 formed is greater than the normal vector to point p2. If the angle formed is α2, then the relationship between the two points is convex; otherwise, it is concave.

[0041] S43. Determine whether the color feature distance between each cluster and its nearest neighbor cluster is less than the set threshold. If so, merge the clusters to generate a larger segmentation block.

[0042] S44. Determine whether the number of point clouds contained in each segmented block is less than the minimum number of point clouds in the set segmented block. If so, and the segmented block has a neighboring segmented block, then assign the segmented block to the neighboring segmented block.

[0043] S45. Remove the segmentation results from the segmentation blocks whose point cloud count is less than the minimum point cloud count set for the segmentation block.

[0044] Furthermore, by judging the feature similarity between the nearest neighbor and the seed point, if it is less than the set threshold, the region growth work of the cluster can be inherited.

[0045] Further, step S5 includes: based on the VFH values ​​of the segmented production line part point cloud, using a KD-tree to search for the k closest samples in the sample library, and calculating the Euclidean distance of the VFH features between them, sorting the distances from smallest to largest, comparing the minimum distance with a set threshold, if it is less than the threshold, then the part point cloud belongs to the same class as the sample corresponding to the minimum distance; if it is greater than the threshold, then it does not belong to the same class as the sample, so as to identify the production line part.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] (1) The supervoxel clustering algorithm that combines edge information can generate supervoxels whose boundaries are better attached to the object boundary, thereby improving the effect of subsequent point cloud segmentation. Moreover, the algorithm is based on voxels and has high efficiency. At the same time, performing supervoxel clustering first can reduce the number of points and improve the efficiency of subsequent point cloud processing.

[0048] (2) The region growth segmentation algorithm based on the combination of point cloud color and concavity features adopts a growth strategy that combines two point cloud features, making the segmentation more accurate. Furthermore, by adjusting the feature weights, it can be applied to different scenarios and is more versatile.

[0049] (3) A technical approach for production line part identification is proposed: establish a VFH sample library, collect and process production line point cloud data, use the proposed point cloud segmentation algorithm to calculate the VFH value of the segmented production line part point cloud, match it with the production line part point cloud in the sample library, and the part point cloud identification can be realized, which is convenient for subsequent assembly, sorting, etc. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly described below. The drawings form part of this application, but are only non-limiting examples embodying the inventive concept and are not intended to make any limitations.

[0051] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention;

[0052] Figure 2 This is a schematic diagram of the application scenario in this embodiment;

[0053] Figure 3 This is a schematic diagram of point cloud voxelization in this embodiment;

[0054] Figure 4 This is a schematic diagram of boundary voxel culling in this embodiment;

[0055] Figure 5 This is a flowchart of the concavity / convexity determination rule used in this embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] This embodiment provides a 3D point cloud segmentation method for production line part identification, such as... Figure 1 As shown, this invention will be further explained using the 3D point cloud of a forklift tire in a production line as the research object. Because factories need to increase output and assembly workshops need to improve production efficiency, automation is used to replace manual labor. Therefore, the 3D point cloud segmentation method for part recognition in the production line provided by this invention can be applied to promote the automation level of the production line. Taking a forklift tire in the production line as an example, the production line layout scenario is as follows... Figure 2 As shown, it is now necessary to identify each part in the tire to prepare for subsequent assembly, sorting, and inspection to ensure complete assembly, thereby improving the automation level and production efficiency of the production line. This embodiment includes the following steps:

[0058] S1. For a specific production line to be identified, a VFH sample library containing point clouds of all tire parts of the production line is constructed in advance. This includes the following steps:

[0059] S11. Select two-dimensional depth maps and RGB maps of the production line tires to be identified from different angles, generate point clouds of the production line tires and downsample them according to the intrinsic parameters of the depth camera;

[0060] As one embodiment, the following angles are selected: perpendicular to the production line, at a 60° angle, at a 75° angle, at a 105° angle, and at a 120° angle;

[0061] S12. Segment the tire point cloud of the production line to include all the part point clouds, calculate the VFH value of each part point cloud, and use KD-tree to construct a sample library containing the VFH values ​​and category labels of the tire part point clouds.

[0062] S2. Acquire two-dimensional image data from the production line, perform deep learning-based target detection to obtain the tire region of interest, reconstruct the tire's three-dimensional point cloud, and perform preprocessing, specifically including the following steps:

[0063] S21. Use a depth camera to acquire a two-dimensional RGB image and a depth image of the production line, and perform deep learning-based target detection on the two-dimensional RGB image data to obtain the tire region of interest.

[0064] S22. Combining the 2D RGB image and the depth image, based on the intrinsic parameters of the depth camera, the tire region is restored to obtain a 3D point cloud and then preprocessed by downsampling.

[0065] S3. Input the preprocessed production line tire point cloud into a supervoxel clustering algorithm that incorporates edge information to generate supervoxels. This includes the following steps:

[0066] S31. Convert the pre-processed production line tires into three-dimensional point cloud voxels;

[0067] As one embodiment, the specific steps of voxelization are as follows: Establish the minimum bounding box of the pre-processed 3D point cloud of the production line tires, and determine the voxel resolution R based on the set value. voxel By recursively dividing the bounding box into eight equal parts using an octree, small cube blocks are obtained, such as... Figure 3 As shown, the input 3D point cloud of the production line tire is voxelized and the voxel adjacency structure is obtained. Each small cube represents a node of an octree.

[0068] S32. Perform object boundary voxel detection. The specific steps for object boundary voxel detection are as follows:

[0069] (1) Obtain the center point of each voxel. For each center point, form an approximate plane with its k neighborhood points, such that the sum of the squares of the distances from each point in the neighborhood to the center point is minimized.

[0070]

[0071] in Let be the normal vector of the approximate plane, d be the distance from the centroid of the approximate plane to the origin, and p be the normal vector of the approximate plane. i For the points that form an approximate plane, For p i Distance to the origin; Represents the approximate plane to be fitted;

[0072] (2) Fit the above plane using principal component analysis. First, calculate the centroid p of all points in the neighborhood.

[0073]

[0074] (3) Calculate the decentered covariance matrix M of each point in the approximate plane.

[0075]

[0076] (4) Perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​λ1, λ2, and λ3 (where λ1 < λ2 < λ3). The eigenvalues ​​represent the degree of dispersion of points in the neighborhood in the direction of the corresponding eigenvector. The smaller the eigenvalue, the lower the degree of dispersion of the points in the direction of the corresponding eigenvector. Therefore, the eigenvector corresponding to λ1 is the normal vector of the approximate plane, which is also the normal vector of the center point to be found.

[0077] (5) After obtaining the normal vector of each center point, calculate the angle between the normal vector of each center point and the normal vector of the neighboring points within the set radius. If the angle exceeds the set threshold, it is determined to be a boundary point, that is, the voxel represented by the center point is a boundary voxel.

[0078] S33. Remove the detected object boundary voxels from the voxel adjacency structure, such as Figure 4 As shown, the left side is the 26-adjacency relationship diagram of the voxel located in the middle of the cube. The 26 voxels form coplanar adjacency, edge adjacency and point adjacency with the middle voxel. The right side is the boundary voxel that has been removed from the voxel adjacency structure to prevent super voxels from clustering into other types of objects through the boundary voxels of the object, and to better maintain the super voxel boundary attached to the object boundary.

[0079] S34. Based on the set super-voxel resolution R seed Seed voxels are selected. Since a supervoxel contains multiple voxels, R... seed >>R voxel ;

[0080] S35. Starting with each selected seed voxel, cluster them according to the feature distance to generate super voxels. After all super voxels are clustered, recalculate and update the features of each super voxel. Repeat the super voxel clustering and feature calculation and update work until the preset number of iterations is reached or the features of each super voxel are stable, and the final super voxel is obtained.

[0081] S4. Input the supervoxels into a region growing segmentation algorithm based on the combination of point cloud color and concavity / convexity features to segment the point cloud of the production line tire parts. This includes the following steps:

[0082] S41. Use the input hypervoxels to obtain the representative points of each hypervoxel, and arrange them in ascending order of curvature to form a seed point queue.

[0083] S42. Take a seed point from the first position of the queue, search for its k nearest neighbors, and calculate the feature distance between the seed point and each nearest neighbor. When the distance is less than a set threshold, the seed point includes its corresponding nearest neighbor in its cluster to achieve region growth. Simultaneously, determine if the nearest neighbor can be used as a seed point to inherit the region growth work of the cluster. If no nearest neighbor inherits, the region growth of that cluster ends. Repeat this step until the seed point queue is empty. The feature distance used is combined with the point cloud color and concavity / convexity features, i.e.

[0084]

[0085] Where λ and μ are the weights of the point cloud color distance and concavity / convexity distance, and D col D represents the color distance of the point cloud. con This represents the convexity / concavity distance of the point cloud. The value is 0 when the relationship between two points is convex, and 1 when the relationship is concave. The specific rule for determining convexity / concavity is that the vector from point p1 to point p2 in the point cloud... normal vector to point p1 The angle α1 formed is greater than the normal vector to point p2. If the angle formed is α2, then the relationship between the two points is convex; otherwise, it is concave. Figure 5 As shown;

[0086] S43. Determine whether the color feature distance between each cluster and its nearest neighbor cluster is less than the set threshold. If so, merge the clusters to generate a larger segmentation block.

[0087] S44. Determine whether the number of point clouds contained in each segmented block is less than the minimum number of point clouds in the set segmented block. If so, and the segmented block has a neighboring segmented block, then assign the segmented block to the neighboring segmented block.

[0088] S45. Remove the segmentation results from the segmentation blocks whose point cloud count is less than the minimum point cloud count set for the segmentation block.

[0089] S5. Calculate the VFH value (Viewpoint Feature Histogram, a 3D feature descriptor) of the segmented production line tire part point cloud. Match the VFH value with the production line tire part point cloud in the VFH sample database using a KD-tree to achieve production line tire part point cloud recognition. Specifically, based on the segmented production line tire part point cloud VFH value, use a KD-tree to search for the k closest samples in the sample database and calculate the Euclidean distance of their VFH features. Sort the distances from smallest to largest. Compare the minimum distance with a set threshold. If the distance is less than the threshold, the part point cloud belongs to the same class as the sample corresponding to the minimum distance; if the distance is greater than the threshold, it belongs to a different class than the sample.

[0090] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A three-dimensional point cloud segmentation method for production line part identification, characterized in that, Includes the following steps: S1. For a specific production line to be identified, construct a VFH sample library containing point clouds of all parts in the production line scene, including the following steps: S11. Select two-dimensional depth maps and RGB maps of the production line to be identified from different angles, generate a point cloud of the production line and downsample it according to the intrinsic parameters of the depth camera; S12. Segment all part point clouds contained in the production line point cloud, calculate the VFH value of each part point cloud, and construct a sample library containing part point cloud VFH values ​​and category labels using KD-tree. S2. Collect two-dimensional image data of the production line, perform deep learning-based target detection to obtain the region of interest, reconstruct the three-dimensional point cloud, and perform preprocessing. S3. Input the preprocessed production line point cloud into a supervoxel clustering algorithm that incorporates edge information to generate supervoxels, including the following steps: S31. Voxelize the preprocessed 3D point cloud of the production line: Establish the minimum bounding box of the preprocessed 3D point cloud of the production line, based on the set voxel resolution R. voxel The bounding box is recursively divided into eight equal parts using an octree to obtain small cubes, which is to voxelize the input 3D point cloud of the production line and obtain the voxel adjacency structure. Each small cube represents a node of the octree. S32. Perform object boundary voxel detection: (1) Obtain the center point of each voxel. For each center point, its... k The neighborhood points form an approximate plane such that the sum of the squares of the distances from each point in the neighborhood to the plane is minimized, i.e.: in The normal vector of the approximate plane. This is the approximate distance from the centroid of the plane to the origin. For the points that form an approximate plane, for Distance to the origin; Represents the approximate plane to be fitted; (2) To fit the above approximate plane using principal component analysis, first calculate the centroid of all points in the neighborhood. : ； (3) Calculate the decentered covariance matrix of each point in the approximate plane. : ； (4) Perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues. , , ,in The eigenvalue represents the degree of dispersion of points in the neighborhood along the direction of the corresponding eigenvector. The smaller the eigenvalue, the lower the degree of dispersion of the points along the direction of the corresponding eigenvector. The corresponding eigenvector is the normal vector of the approximate plane, which is also the normal vector of the center point to be determined. (5) After obtaining the normal vector of each center point, calculate the angle between the normal vector of each center point and the normal vector of the neighboring points within the set radius. If the angle exceeds the set threshold, it is determined to be a boundary point, that is, the voxel represented by the center point is a boundary voxel. S33. Remove the detected object boundary voxels from the voxel adjacency structure to prevent super voxels from clustering into other types of objects through the object boundary voxels, and better maintain the super voxel boundary attached to the object boundary. S34. Based on the set super-voxel resolution R seed Seed voxels are selected. Since a supervoxel contains multiple voxels, R... seed >R voxel ; S35. Starting with each selected seed voxel, cluster them according to the feature distance to generate super voxels. After all super voxels are clustered, recalculate and update the features of each super voxel. Repeat the super voxel clustering and feature calculation and update work until the preset number of iterations is reached to obtain the final super voxel. S4. Input the super voxels into the region growth segmentation algorithm based on the combination of point cloud color and concavity / convexity features to segment out the point cloud of production line parts. S5. Calculate the VFH value of the segmented production line part point cloud, and match it with the production line part point cloud in the VFH sample library using a KD-tree to achieve production line part point cloud recognition. Specifically, this includes: based on the VFH value of the segmented production line part point cloud, using a KD-tree to search for the closest match in the sample library. k The system collects samples and calculates the Euclidean distance of their VFH features. The Euclidean distances are sorted from smallest to largest. The minimum distance is compared with a set threshold. If the distance is less than the threshold, the part point cloud belongs to the same class as the sample with the minimum distance. If the distance is greater than the threshold, the part does not belong to the same class as the sample. This is to identify production line parts.

2. The three-dimensional point cloud segmentation method for production line part identification according to claim 1, characterized in that, Select the 2D depth map and RGB image of the production line to be identified from the following angles: perpendicular to the production line, at a 60° angle, at a 75° angle, at a 105° angle, and at a 120° angle.

3. The three-dimensional point cloud segmentation method for production line part identification according to claim 1, characterized in that, Step S2 includes the following steps: S21. Use a depth camera to acquire a two-dimensional RGB image and a depth image of the production line, and perform deep learning-based target detection on the two-dimensional RGB image data to obtain the region of interest. S22. Combining the 2D RGB image and the depth image, based on the intrinsic parameters of the depth camera, the region of interest is restored to obtain a 3D point cloud and then downsampled.

4. The three-dimensional point cloud segmentation method for production line part identification according to claim 1, characterized in that, Step S4 includes the following steps: S41. Use the input hypervoxels to obtain the representative points of each hypervoxel, and arrange them in ascending order of curvature to form a seed point queue. S42. Take the seed point from the head of the queue and search for that point. k For each nearest neighbor, the feature distance between the seed point and its nearest neighbor is calculated. When this distance is less than a set threshold, the seed point includes its corresponding nearest neighbor in the cluster to perform region growing. Simultaneously, it is determined whether the nearest neighbor can inherit the region growing function of the cluster. If no nearest neighbor inherits, the region growing for that cluster ends. This step is repeated until the seed point queue is empty. The feature distance used is combined with the point cloud color and concavity / convexity features. D ,Right now in, , These are the weights of the point cloud color distance and the concavity / convexity distance. For point cloud color distance, This represents the convexity / concavity distance of a point cloud. The value is 0 when the relationship between two points is convex, and 1 when the relationship is concave. The specific rule for determining convexity / concavity is that when a point in the point cloud... To another point in the cloud vector With point normal vector The angle formed Greater than and point normal vector The angle formed If the two points are convex, then the relationship between them is convex; otherwise, it is concave. S43. Determine whether the color feature distance between each cluster and its nearest neighbor cluster is less than the set threshold. If so, merge the clusters to generate a larger segmentation block. S44. Determine whether the number of point clouds contained in each segmented block is less than the minimum number of point clouds in the set segmented block. If so, and the segmented block has a neighboring segmented block, then assign the segmented block to the neighboring segmented block. S45. Remove the segmentation results from the segmentation blocks whose point cloud count is less than the minimum point cloud count set for the segmentation block.

Images