An automatic detection method for oversized metal parts based on 3D point cloud data

Abstract

The present invention discloses an automatic detection method for metal oversize parts based on three-dimensional point cloud data, comprising the following steps: simplifying and denoising the registered point cloud data; performing supervoxel segmentation on the point cloud data, and constructing an adjacency graph B according to the result; performing region growth on each vertex vi, clustering the vertex vi to obtain a boundary, and constructing an adjacency graph C according to the result; using the principal component analysis method to calculate the main direction of the common boundary of two adjacent vertices, and merging and classifying the boundaries through projection; traversing C, merging clusters and updating the boundaries; iteratively merging until the number of clustering results remains unchanged and the iteration is stopped; and obtaining metal oversize parts according to the size of the metal directed bounding box after classification. Based on point cloud data reflecting the three-dimensional information of the object surface, the present invention comprehensively utilizes the segmentation method of supervoxel segmentation and the clustering method of region growth, and automatically detects metal oversize parts on the basis of large-scale and meticulous sampling, thereby improving accuracy and generalization.

Description

Technical Field

[0001] The present invention relates to the technical field of grading metal material detection by using point cloud data processing, and in particular to a method for automatically detecting oversized pieces of scrap steel based on three-dimensional point cloud data. Background Art

[0002] The present invention is mainly used for the automated detection of some metal wastes, such as scrap steel. The so-called scrap steel refers to the steel waste that does not become products in the production process of the steel plant, as well as the steel materials in the equipment and components that are scrapped after use. For the classification of scrap steel, oversized scrap steel pieces are very important judgment factors. In the scrap steel recycling process of major steel plants, most of them still use manual judgment to grade scrap steel, but are limited by the cognitive differences between different employees, the influence of subjective state factors such as employee energy, so the judgment results in the scrap steel recycling process will also have deviations and fluctuations.

[0003] With the development of technology, the current scrap steel classification process also has a trend towards automation and intelligence. Some steel mills use scrap steel intelligent grading systems based on deep learning. They use deep learning or machine learning to extract scrap steel samples, train discrimination algorithms, and directly predict discrimination results after deployment. However, this type of method has poor generalization ability for different types of scrap steel, low detection accuracy for oversized scrap steel pieces, and difficulty in accurately determining the classification level of scrap steel, resulting in low scrap steel recycling efficiency.

[0004] There is currently no algorithm for automated detection of oversized scrap steel pieces that is highly generalizable, more efficient, and has an identification accuracy that is more in line with actual production needs. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides an automatic detection method for metal oversized parts based on three-dimensional point cloud data, in order to solve the technical problems that the scrap metal identification scheme in the prior art has low recognition accuracy for oversized scrap steel parts, and it is difficult to accurately determine the classification level of scrap steel, resulting in low scrap steel recycling efficiency.

[0006] The present invention proposes an automatic detection method for metal oversized parts based on three-dimensional point cloud data, comprising the following steps:

[0007] S101, inputting the overall scrap steel point cloud data P after rough registration and icp fine registration;

[0008] S102, simplifying and denoising the point cloud data P;

[0009] S103, performing supervoxel segmentation on the point cloud data P;

[0010] S104, constructing an adjacency graph B according to the result of supervoxel segmentation;

[0011] S105, performing region growth on each vertex vi in ​​the adjacency graph B to obtain a cluster boundary;

[0012] S106, constructing an adjacency graph C according to the clustering result;

[0013] S107, using the attributes of the edges in the adjacency graph C as conditions, using the principal component analysis method to merge and classify the boundaries;

[0014] S108, traversing the adjacency graph C, merging all clusters that meet the conditions, and updating the boundaries and shared boundaries after merging;

[0015] S109, iterating and merging until the number of final clustering results remains unchanged and then stopping the iteration;

[0016] S110, obtaining a directed bounding box for the classified metal parts, and obtaining the oversized metal parts according to the size of the directed bounding box.

[0017] Furthermore, the specific content of the above-mentioned region growing method is as follows: starting from a vertex in the adjacency graph B, mark it as visited, and classify the vertices in turn: if the attribute of the edge formed by a vertex vi and its neighboring vertex vj where the outgoing edge is located is true, then the vj vertex is saved in a cluster set_class, and vj is put into the seed queue; if the attribute is false, it is put into the boundary set_bou of the cluster, and vi is deleted from the seed queue, and the execution is executed until the seed queue is empty; finally, the boundary of the cluster is set_bou_class=set_class-set_bou.

[0018] Furthermore, the method for merging and classifying boundaries includes the following steps: determining the attributes of the edges in the adjacency graph C as the second merging condition, for two adjacent vertices (two clusters a, b), their two common boundaries are known, and the principal component analysis method is used to calculate the main directions v1 and v2 of their common boundaries respectively, a is projected onto the straight line in the direction of v1, and the line segment length is l1, and b is projected onto the straight line in the direction of v2, and the line segment length is l2. The two boundaries are also projected onto the straight lines where their main directions are located to obtain line segments l3 and l4. Determine whether l3 / l2 and l4 / l1 are both greater than the preset threshold. If so, the attribute of the edge is assigned to true, otherwise false; all clusters that meet the conditions are merged. Compared with the existing technology, the beneficial effects of the present invention are embodied in:

[0019] The present invention detects oversized scrap metal parts based on point cloud data. Point cloud is a discrete point set distributed in N-dimensional space, which is a discrete sampling of object surface information. Compared with RGB data used in many methods, point cloud data can better reflect the spatial shape information of the object. At present, point cloud data is widely used in the construction of three-dimensional scenes and digital simulation of various workpieces, but no one has used point cloud data in the grading of scrap steel. There are great advantages and prospects for the automatic detection and measurement of oversized scrap steel parts using point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The figure is a flow chart of a method for automatically detecting oversized scrap steel based on three-dimensional point cloud data.

[0021] Figure 2 This is a supervoxel segmentation and clustering display of an automatic detection method for oversized scrap steel based on three-dimensional point cloud data.

[0022] Figure 3 This is a boundary clustering display diagram of an automatic detection method for oversized scrap steel based on three-dimensional point cloud data. DETAILED DESCRIPTION

[0023] The specific implementation of the present invention is described below in conjunction with the accompanying drawings and embodiments:

[0024] like Figure 1 FIG. 1 is a flowchart of a method for automatically detecting oversized scrap steel parts based on three-dimensional point cloud data proposed by the present invention. The method comprises the following steps:

[0025] Step S101, inputting the overall scrap steel point cloud data P after rough registration and icp fine registration;

[0026] Step S102, simplifying and denoising the point cloud data P;

[0027] Step S103, performing supervoxel segmentation on the point cloud data P to obtain supervoxel segmentation results M and A, where M is a container for storing supervoxel labels and supervoxel objects, and A is a container for storing supervoxel adjacency relations, and the segmentation result is only affected by the spatial distance and the normal vector, and the color does not play a role;

[0028] Step S104, construct a supervoxel adjacency graph B according to the results of M and A, where the vertex is the label of each supervoxel, and the edge has an attribute, which is the similarity of adjacent supervoxels. The similarity is of bool type, and it is necessary to determine whether the angle between the normal vectors of two supervoxels is less than a given threshold. If it is greater than the threshold, it is false, that is, they are not similar, and if it is less than the threshold, it is true, that is, they are similar;

[0029] Step S105, perform region growth on each vertex vi in ​​B, starting with a vertex, and mark it as visited. If the attribute of the edge formed by a vertex vi and its neighboring vertex vj is true, save the vj vertex in a cluster set_class and put vj in the seed queue. If the attribute is false, put it in the cluster boundary set_bou, and delete vi from the seed queue. Execute until the seed queue is empty. The final boundary of the cluster is: set_bou_class = set_class-set_bou. Figure 2 As shown, the boundary result of region growth on the vertices of the adjacency graph after supervoxel segmentation can be obtained;

[0030] Step S106, construct an adjacency graph C for the clustered results. The vertices are the labels of each cluster after clustering. For a vertex and its neighboring vertices, the adjacent boundary supervoxel sets between them can be obtained, that is, the subsets b1 and b2 of the supervoxel sets belonging to the boundaries of the respective clustering results. The two should have an adjacency relationship.

[0031] Step S107, determine the attributes of the edges in C as the second merging condition. For two adjacent vertices (two clusters a and b), their two common boundaries are known. Use the principal component analysis method to calculate the main directions v1 and v2 of their common boundaries respectively. Project a onto the straight line in the direction of v1, and the line segment length is l1. Project b onto the straight line in the direction of v2, and the line segment length is l2. The two boundaries are also projected onto the straight lines where their main directions are located to obtain line segments l3 and l4. Determine whether l3 / l2 and l4 / l1 are both greater than the preset threshold. If so, assign the attribute of the edge to true, otherwise false;

[0032] Step S108, traverse C, merge all clusters that meet the conditions, and update the boundaries and shared boundaries after merging;

[0033] Step S109, iterate and merge until the number of final clustering results remains unchanged and then stop iterating. The final clustering result is as follows: Figure 3 As shown, different single metal parts are shown in different colors;

[0034] Step S110, obtaining a directed bounding box for the classified metal parts, and obtaining the oversized metal parts according to the size of the directed bounding box.

[0035] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for automatically detecting oversized metal parts based on three-dimensional point cloud data, characterized in that: The following steps are involved: S101, inputting the overall scrap steel point cloud data P after rough registration and icp fine registration; S102, simplifying and denoising the point cloud data P; S103, performing supervoxel segmentation on the point cloud data P; S104, constructing an adjacency graph B according to the result of supervoxel segmentation; S105, performing region growth on each vertex vi in ​​the adjacency graph B to obtain a cluster boundary; S106, constructing an adjacency graph C according to the clustering result; S107, using the attributes of the edges in the adjacency graph C as conditions, using the principal component analysis method to merge and classify the boundaries; S108, traversing the adjacency graph C, merging all clusters that meet the conditions, and updating the boundaries and shared boundaries after merging; S109, iterating and merging until the number of final clustering results remains unchanged and then stopping the iteration; S110, obtaining a directed bounding box for the classified metal parts, and obtaining the oversized metal parts according to the size of the directed bounding box.

2. The method for automatically detecting oversized metal parts based on three-dimensional point cloud data according to claim 1, characterized in that: The specific content of the region growing method is as follows: starting from a vertex in the adjacency graph B, mark it as visited, and classify the vertices in turn: if the attribute of the edge formed by a vertex vi and its neighboring vertex vj where the outgoing edge is located is true, then the vj vertex is saved in a cluster set_class, and vj is put into the seed queue; if the attribute is false, it is put into the boundary set_bou of the cluster, and vi is deleted from the seed queue, and the execution is executed until the seed queue is empty; finally, the boundary of the cluster is set_bou_class=set_class-set_bou.

3. The automatic detection method of metal oversize parts based on three-dimensional point cloud data according to claim 1, characterized in that: The method for merging and classifying boundaries includes the following steps: determining the attributes of the edges in the adjacency graph C as the second merging condition, for two adjacent vertices, i.e., two clusters a and b, their two common boundaries are known, and the principal component analysis method is used to respectively calculate the main directions v1 and v2 of their common boundaries, a is projected onto a straight line in the direction of v1, and the line segment length is l1, b is projected onto a straight line in the direction of v2, and the line segment length is l2, and the two boundaries are also projected onto the straight lines where their respective main directions are located to obtain line segments l3 and l4, and judging whether l3 / l2 and l4 / l1 are both greater than a preset threshold value, if so, the attribute of the edge is assigned to true, otherwise it is false; all clusters that meet the conditions are merged.

4. The method for automatically detecting oversized metal parts based on three-dimensional point cloud data according to claim 1, characterized in that: The metal is scrap steel metal.

Images