Part surface segmentation method and device based on depth point cloud data, medium

Abstract

The present disclosure relates to a part surface segmentation method and device based on depth point cloud data, medium, the part surface segmentation method comprising: obtaining a depth map of a part s , dividing a neighborhood of the point p s to obtain a plurality of micro elements, determining a characteristic parameter of the point p s based on the plurality of micro elements and a normal vector of the neighborhood of the point p s , constructing a characteristic space of the part according to the normal vector and the characteristic parameter of the neighborhood of each point p s , mapping points in each characteristic dimension of the characteristic space to a histogram respectively, dividing each point in the characteristic space based on the histogram in each characteristic dimension to obtain a clustering result of each point, and mapping the clustering result to a surface space of the part to obtain a segmentation result of the part. The part surface segmentation method and device provided by the present disclosure can achieve high-precision clustering in the characteristic space and obtain an ideal part surface segmentation result.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision processing technology, and in particular to a method, apparatus, and medium for surface segmentation of parts based on depth point cloud data. Background Technology

[0002] Since the late 20th century, point cloud technology has gradually attracted attention. With the rapid improvement of computing performance in the 21st century, point cloud technology has been studied more and more extensively, forming a systematic operation scheme with a series of processing methods such as noise reduction, segmentation, registration, and fitting as the core. It has also been integrated into a variety of integrated point cloud processing tools, represented by the Point Cloud Library (PCL), with a complete development environment. It has been widely studied and applied in fields such as autonomous driving and remote sensing mapping.

[0003] With the advent of Industry 5.0, the people-oriented and sustainable development path is inseparable from the vigorous development of intelligent manufacturing. Computer vision, as the core of industrial inspection, plays an extremely important role, and vision-related technologies have been increasingly researched and developed. As the requirements for measurement accuracy in the industrial manufacturing field become increasingly stringent, the precision inspection of some specialized equipment is also receiving significant attention. With the rise of 3D technology, compared to traditional 2D vision, 3D images can better record surface information of objects, making high-precision inspection of part surfaces possible. 3D surface inspection can also simultaneously provide spatial shape detection and pose estimation, thus 3D data is widely used in defect detection, accuracy assessment, and other related fields.

[0004] Measuring the structure of three-dimensional objects in space is a challenging research area in computer vision. There are numerous formats and methods for storing spatial information, and these data formats are often converted into more general-purpose three-dimensional point cloud data for computation and processing.

[0005] Traditional point cloud segmentation (PCS) algorithms rigorously divide 3D point cloud data into multiple non-overlapping regions by manually calibrating geometric and statistical features to distinguish specific structures or objects within the point cloud scene. However, traditional geometric feature extraction methods often suffer from low accuracy and require extensive parameter settings and adjustments to achieve satisfactory segmentation results. These methods typically exhibit poor accuracy in segmenting parts and cannot effectively recognize combinations of various planes and curved surfaces. Existing point cloud segmentation methods often employ deep learning. While deep learning can adapt to segmentation needs in many scenarios, it requires massive amounts of data for training, making it unsuitable for point cloud segmentation in engineering applications with limited data. These issues result in existing point cloud segmentation methods being ill-suited for segmenting parts with complex surface structures. Summary of the Invention

[0006] In view of this, this disclosure proposes a method, apparatus, and medium for part surface segmentation based on deep point cloud data, which can achieve high-precision clustering in the feature space and obtain ideal part surface segmentation results.

[0007] According to one aspect of this disclosure, a method for surface segmentation of a part based on deep point cloud data is provided, comprising:

[0008] Obtain a depth map of the part, the depth map being used to indicate the point cloud data of the part;

[0009] Traverse each point p in the depth map s For the point p s The neighborhood of the point p is divided into multiple micro-elements. Based on the multiple micro-elements and the point p, s The normal vector of the neighborhood determines the point p. s The characteristic parameters, based on each point p s The feature space of the part is constructed using the normal vectors and feature parameters of the neighborhood, wherein the feature parameters are used to indicate the geometric features of the part;

[0010] Points in each feature dimension of the feature space are mapped to histograms. Based on the histograms of each feature dimension, the points in the feature space are divided to obtain the clustering results of each point. The clustering results are mapped to the surface space of the part to obtain the segmentation results of the part. The clustering results are used to indicate that points clustered in the same category have similarity.

[0011] In this way, feature parameters are determined based on the geometric features of the part, and feature space is constructed using the normal vectors of the neighborhood of each point in the depth map and feature parameters. This allows the 3D point cloud data of complex parts to be projected into a high-dimensional feature space. Then, the feature space is divided using histograms under different feature dimensions, which can better identify planes and curved surfaces, achieve high-precision clustering, and obtain ideal surface segmentation results for complex parts.

[0012] In one possible implementation, the point p s The neighborhood of a given point includes a first neighborhood and a second neighborhood, wherein the first neighborhood includes k1 points p. i Let i = 1, 2, ..., k1, and the second neighborhood includes k2 points p. j j = 1, 2...k2, k1 ≠ k2; the point p s The neighborhood is divided into multiple micro-elements, including: dividing the first neighborhood into a first micro-element group and a second micro-element group, and dividing the second neighborhood into a third micro-element group, wherein the first micro-element group, the second micro-element group, and the third micro-element group each include a different number of micro-elements.

[0013] In this way, by using a variety of different neighborhood partitioning methods, surface micro-elements can be obtained in a wide range and in different neighborhoods, which provides a foundation for the subsequent construction of a high-dimensional feature space.

[0014] In one possible implementation, the method is based on the plurality of infinitesimal elements and the point p. s The normal vector of the neighborhood determines the point p. s The feature parameters include: determining a first set based on the first infinitesimal element, determining a first parameter based on the first set and the normal vector of the first neighborhood, and determining a second parameter based on the point of the first neighborhood and the normal vector of the first neighborhood. The first set includes the normal vectors of each infinitesimal element of the first infinitesimal element. The first parameter indicates the relationship between the normal vector of the infinitesimal element of the first infinitesimal element and the normal vector of the first neighborhood. The second parameter indicates the relationship between the infinitesimal element of the first infinitesimal element and the point p. sThe relationship is as follows: A second set is determined based on the second infinitesimal element; a third parameter and a fourth parameter are determined based on the normal vectors of the second set and the first neighborhood, wherein the second set includes the normal vectors of each infinitesimal element of the second infinitesimal element, and the third and fourth parameters are used to indicate the relationship between the normal vectors of each infinitesimal element of the second infinitesimal element; a third set is determined based on the third infinitesimal element; a fifth parameter is determined based on the normal vectors of the third set and the second neighborhood, wherein the third set includes the normal vectors of each infinitesimal element of the third infinitesimal element, and the fifth parameter is used to indicate the relationship between the normal vectors of each infinitesimal element of the third infinitesimal element; the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter are used as the feature parameters.

[0015] In this way, by using the normal vectors of surface micro-elements in different neighborhoods and the normal vectors of different neighborhoods to determine the feature parameters, each point on the surface of the part can be mapped to a feature space constructed based on different feature parameters, which helps to achieve high-precision surface segmentation.

[0016] In one possible implementation, determining the third and fourth parameters based on the normal vectors of the second set and the first neighborhood includes: constructing a first covariance matrix based on the normal vectors of each element of the second infinitesimal and the normal vectors of the first neighborhood; decomposing the first covariance matrix to obtain a first principal eigenvalue and a first principal eigenvector; using the first principal eigenvalue as the third parameter and the first principal eigenvector as the fourth parameter; determining the fifth parameter based on the third set and the normal vectors of the second neighborhood includes: constructing a second covariance matrix based on the normal vectors of each element of the third infinitesimal and the normal vectors of the second neighborhood; decomposing the second covariance matrix to obtain a second principal eigenvalue; and using the second principal eigenvalue as the fifth parameter.

[0017] In this way, by constructing a covariance matrix based on the normal vectors of each infinitesimal element and the normal vectors of the neighborhood, and using the principal eigenvalues ​​and principal eigenvectors of the covariance matrix as feature parameters, the relationship between each infinitesimal element in each neighborhood can be reflected, which helps to determine a high-dimensional feature space that is conducive to clustering.

[0018] In one possible implementation, the normal vector of the first neighborhood, the first parameter, and the fourth parameter each include features of three feature dimensions, while the second parameter, the third parameter, and the fifth parameter each include features of one feature dimension; the step of determining the features based on each point p... s The feature space of the part is constructed using the normal vectors and feature parameters of the first neighborhood, including: using the features of each feature dimension of the first neighborhood's normal vector, the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter as the feature space of the point p.s Features of the 12 feature dimensions of the feature space.

[0019] Thus, for complex parts composed of complex surfaces, by discretizing each point on the surface of the part from a low-dimensional (i.e., three-dimensional) space to a high-dimensional (i.e., 12-dimensional) feature space, the influence of extreme values ​​caused by the surface processing error can be eliminated, which helps to achieve higher-precision clustering in the high-dimensional feature space.

[0020] In one possible implementation, the horizontal axis of the histogram is a first length, which includes multiple intervals of equal length, and the vertical axis of the histogram is used to indicate the number of points whose values ​​fall within the intervals. Mapping points in each feature dimension of the feature space to the histogram includes: for each feature dimension, determining the correspondence between the point values ​​and the first length; constructing the histogram based on the correspondence and the number of points within each interval. Dividing the points in the feature space based on the histograms of each feature dimension to obtain clustering results includes: determining peaks and corresponding trough regions based on the histograms of each feature dimension; dividing the histogram based on the peaks and corresponding trough regions to obtain one or more grids; and dividing the feature space based on the grids of each feature dimension to obtain the clustering results.

[0021] In this way, by mapping the point values ​​under each feature dimension of the feature space to the histogram, it is beneficial to perform clustering based on the histogram to obtain more stable segmentation results. Based on the mapping of the feature space, the peak and valley regions of the feature histogram are divided into grids, which helps to perform clustering based on the divided grids to obtain high-precision segmentation results.

[0022] In one possible implementation, after constructing the feature space of the part, the method further includes: normalizing the point values ​​under each feature dimension of the feature space.

[0023] In this way, by normalizing the data in the high-dimensional feature space, a relatively fast computation speed can be guaranteed.

[0024] According to another aspect of this disclosure, a part surface segmentation apparatus based on depth point cloud data is provided, comprising:

[0025] An acquisition module is configured to acquire a depth map of a part, the depth map being used to indicate point cloud data of the part;

[0026] The processing module is configured to traverse each point p of the depth map. s For the point p sThe neighborhood of the point p is divided into multiple micro-elements. Based on the multiple micro-elements and the point p, s The normal vector of the neighborhood determines the point p. s The characteristic parameters, based on each point p s The feature space of the part is constructed using the normal vectors and feature parameters of the neighborhood, wherein the feature parameters are used to indicate the geometric features of the part;

[0027] The segmentation module is configured to map points in each feature dimension of the feature space to histograms, divide the points in the feature space based on the histograms in each feature dimension to obtain clustering results for each point, and map the clustering results to the surface space of the part to obtain the segmentation results of the part. The clustering results are used to indicate that points clustered in the same category have similarity.

[0028] In this way, feature parameters are determined based on the geometric features of the part, and feature space is constructed using the normal vectors of the neighborhood of each point in the depth map and feature parameters. This allows the 3D point cloud data of complex parts to be projected into a high-dimensional feature space. Then, the feature space is divided using histograms under different feature dimensions, which can better identify planes and curved surfaces, achieve high-precision clustering, and obtain ideal surface segmentation results for complex parts.

[0029] In one possible implementation, the point p s The neighborhood of a given point includes a first neighborhood and a second neighborhood, wherein the first neighborhood includes k1 points p. i Let i = 1, 2, ..., k1, and the second neighborhood includes k2 points p. j j = 1, 2...k2, k1 ≠ k2; the point p s The neighborhood is divided into multiple micro-elements, including: dividing the first neighborhood into a first micro-element group and a second micro-element group, and dividing the second neighborhood into a third micro-element group, wherein the first micro-element group, the second micro-element group, and the third micro-element group each include a different number of micro-elements.

[0030] In this way, by using a variety of different neighborhood partitioning methods, surface micro-elements can be obtained in a wide range and in different neighborhoods, which provides a foundation for the subsequent construction of a high-dimensional feature space.

[0031] In one possible implementation, the method is based on the plurality of infinitesimal elements and the point p. s The normal vector of the neighborhood determines the point p. sThe feature parameters include: determining a first set based on the first infinitesimal element, determining a first parameter based on the first set and the normal vector of the first neighborhood, and determining a second parameter based on the point of the first neighborhood and the normal vector of the first neighborhood. The first set includes the normal vectors of each infinitesimal element of the first infinitesimal element. The first parameter indicates the relationship between the normal vector of the infinitesimal element of the first infinitesimal element and the normal vector of the first neighborhood. The second parameter indicates the relationship between the infinitesimal element of the first infinitesimal element and the point p. s The relationship is as follows: A second set is determined based on the second infinitesimal element; a third parameter and a fourth parameter are determined based on the normal vectors of the second set and the first neighborhood, wherein the second set includes the normal vectors of each infinitesimal element of the second infinitesimal element, and the third and fourth parameters are used to indicate the relationship between the normal vectors of each infinitesimal element of the second infinitesimal element; a third set is determined based on the third infinitesimal element; a fifth parameter is determined based on the normal vectors of the third set and the second neighborhood, wherein the third set includes the normal vectors of each infinitesimal element of the third infinitesimal element, and the fifth parameter is used to indicate the relationship between the normal vectors of each infinitesimal element of the third infinitesimal element; the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter are used as the feature parameters.

[0032] In this way, by using the normal vectors of surface micro-elements in different neighborhoods and the normal vectors of different neighborhoods to determine the feature parameters, each point on the surface of the part can be mapped to a feature space constructed based on different feature parameters, which helps to achieve high-precision surface segmentation.

[0033] In one possible implementation, determining the third and fourth parameters based on the normal vectors of the second set and the first neighborhood includes: constructing a first covariance matrix based on the normal vectors of each element of the second infinitesimal and the normal vectors of the first neighborhood; decomposing the first covariance matrix to obtain a first principal eigenvalue and a first principal eigenvector; using the first principal eigenvalue as the third parameter and the first principal eigenvector as the fourth parameter; determining the fifth parameter based on the third set and the normal vectors of the second neighborhood includes: constructing a second covariance matrix based on the normal vectors of each element of the third infinitesimal and the normal vectors of the second neighborhood; decomposing the second covariance matrix to obtain a second principal eigenvalue; and using the second principal eigenvalue as the fifth parameter.

[0034] In this way, by constructing a covariance matrix based on the normal vectors of each infinitesimal element and the normal vectors of the neighborhood, and using the principal eigenvalues ​​and principal eigenvectors of the covariance matrix as feature parameters, the relationship between each infinitesimal element in each neighborhood can be reflected, which helps to determine a high-dimensional feature space that is conducive to clustering.

[0035] In one possible implementation, the normal vector of the first neighborhood, the first parameter, and the fourth parameter each include features of three feature dimensions, while the second parameter, the third parameter, and the fifth parameter each include features of one feature dimension; the step of determining the features based on each point p... s The feature space of the part is constructed using the normal vectors and feature parameters of the first neighborhood, including: using the features of each feature dimension of the first neighborhood's normal vector, the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter as the feature space of the point p. s Features of the 12 feature dimensions of the feature space.

[0036] Thus, for complex parts composed of complex surfaces, by discretizing each point on the surface of the part from a low-dimensional (i.e., three-dimensional) space to a high-dimensional (i.e., 12-dimensional) feature space, the influence of extreme values ​​caused by the surface processing error can be eliminated, which helps to achieve higher-precision clustering in the high-dimensional feature space.

[0037] In one possible implementation, the horizontal axis of the histogram is a first length, which includes multiple intervals of equal length, and the vertical axis of the histogram is used to indicate the number of points whose values ​​fall within the intervals. Mapping points in each feature dimension of the feature space to the histogram includes: for each feature dimension, determining the correspondence between the point values ​​and the first length; constructing the histogram based on the correspondence and the number of points within each interval. Dividing the points in the feature space based on the histograms of each feature dimension to obtain clustering results includes: determining peaks and corresponding trough regions based on the histograms of each feature dimension; dividing the histogram based on the peaks and corresponding trough regions to obtain one or more grids; and dividing the feature space based on the grids of each feature dimension to obtain the clustering results.

[0038] In this way, by mapping the point values ​​under each feature dimension of the feature space to the histogram, it is beneficial to perform clustering based on the histogram to obtain more stable segmentation results. Based on the mapping of the feature space, the peak and valley regions of the feature histogram are divided into grids, which helps to perform clustering based on the divided grids to obtain high-precision segmentation results.

[0039] In one possible implementation, after constructing the feature space of the part, the device further includes a normalization module configured to normalize the point values ​​under each feature dimension of the feature space.

[0040] In this way, by normalizing the data in the high-dimensional feature space, a relatively fast computation speed can be guaranteed.

[0041] According to another aspect of this disclosure, a part surface segmentation apparatus based on deep point cloud data is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described part surface segmentation method when executing the instructions stored in the memory.

[0042] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the above-described part surface segmentation method.

[0043] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0044] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0045] Figure 1 A flowchart is shown for a part surface segmentation method provided according to an embodiment of the present disclosure.

[0046] Figure 2 A schematic diagram of neighborhood division provided according to an embodiment of the present disclosure is shown.

[0047] Figure 3 A schematic diagram of neighborhood division provided according to an embodiment of the present disclosure is shown.

[0048] Figure 4 A schematic diagram showing a mapping histogram under the target feature dimension provided according to an embodiment of the present disclosure is shown.

[0049] Figure 5 This diagram illustrates the segmentation results under the target feature dimension provided according to an embodiment of the present disclosure.

[0050] Figure 6 This diagram illustrates a clustering result determined based on two feature dimensions according to an embodiment of the present disclosure.

[0051] Figure 7 A flowchart is shown for a part surface segmentation method provided according to an embodiment of the present disclosure.

[0052] Figure 8 A schematic diagram showing the segmentation results provided according to an embodiment of the present disclosure is shown.

[0053] Figure 9A block diagram of a part surface segmentation apparatus provided according to an embodiment of the present disclosure is shown.

[0054] Figure 10 A block diagram of an apparatus for performing a part surface segmentation method according to an embodiment of the present disclosure is shown. Detailed Implementation

[0055] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0056] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0057] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0058] To facilitate understanding of the technical solutions provided by the embodiments of this disclosure by those skilled in the art, the technical environment for implementing the technical solutions will be described below.

[0059] With the widespread research and application of point cloud technology in fields such as autonomous driving and remote sensing mapping, point cloud technology has become increasingly prevalent. The rise of 3D technology, compared to traditional 2D vision, allows for better recording of surface information of objects, making high-precision surface inspection of parts possible. Measuring the structure of 3D objects in space is a challenging research direction in computer vision. There are numerous formats and methods for storing spatial information, and these data formats are often converted into more general-purpose 3D point cloud data for computation and processing.

[0060] Currently, in the design of surface topography segmentation methods for complex parts, point cloud-based surface segmentation techniques are mainly used to obtain detailed surface dimensions. Traditional point cloud segmentation algorithms strictly divide 3D point cloud data into multiple non-overlapping regions by manually calibrating geometric and statistical features to distinguish specific structures or objects in the point cloud scene. However, traditional geometric feature extraction methods often have low accuracy and require extensive parameter settings and adjustments to achieve good segmentation results. These methods typically have poor segmentation accuracy for parts and cannot effectively recognize combinations of various planes and curved surfaces. Existing point cloud segmentation methods often employ deep learning. While deep learning can adapt to segmentation needs in many scenarios, it requires massive amounts of data for training. This makes it impossible to achieve point cloud segmentation using deep learning in some engineering applications where data is limited. These issues mean that existing point cloud segmentation methods are not well-suited for segmenting parts with complex surface structures.

[0061] This disclosure provides a method for surface segmentation of parts based on depth point cloud data. It involves acquiring a depth map of the 3D point cloud data indicating the part, dividing each point in the depth map into multiple micro-elements, determining the feature parameters of each point based on the neighborhood normal vectors and these micro-elements, and constructing a feature space based on the neighborhood normal vectors and feature parameters. Essentially, the 3D point cloud data is projected into a high-dimensional feature space. Then, points in each feature dimension of the feature space are mapped to histograms, and the feature space is divided based on the histograms of each feature dimension to obtain clustering results. Finally, the clustering results are mapped back to the original surface space of the part to obtain the point cloud segmentation result of the part. By constructing a feature space using the designed geometric features of the part and dividing the feature space using histograms of different feature dimensions, this method can effectively identify planes and curved surfaces, achieve high-precision clustering, and obtain ideal surface segmentation results for the part, thereby effectively improving the segmentation accuracy and effect of complex part surfaces.

[0062] Figure 1 A flowchart illustrating a method for segmenting a part surface according to an embodiment of this disclosure is shown. Figure 1 As shown, the method for dividing the surface of a part may include:

[0063] S101. Obtain the depth map of the part.

[0064] The depth map can be used to indicate the point cloud data of the part.

[0065] S102, Traverse each point p in the depth map s For point p s The neighborhood is divided into multiple micro-elements, and based on these multiple micro-elements and point p... sThe normal vector of the neighborhood determines the point p. s The characteristic parameters, based on each point p s The feature space of the part is constructed using the normal vectors and feature parameters of the neighborhood.

[0066] The feature parameters can be used to indicate the geometric features of the part.

[0067] S103. Map the points in each feature dimension of the feature space to the histogram, divide the points in the feature space based on the histograms of each feature dimension to obtain the clustering results of each point, and map the clustering results to the surface space of the part to obtain the point cloud segmentation results of the part.

[0068] The clustering results can be used to indicate that points in the same category of TV series are similar.

[0069] In one possible implementation, the part surface segmentation method provided in this disclosure embodiment can be applied to complex parts, where a complex part refers to a part with a complex geometric surface, that is, a part with multiple planes and multiple curved surfaces.

[0070] In step S101, the depth map of the part can be used to indicate the point cloud data of the part. It is understood that the points included in the point cloud data are three-dimensional points; therefore, in this embodiment of the disclosure, the point cloud data can also be referred to as three-dimensional point cloud data. Taking a complex part as an example, in one possible implementation, the three-dimensional point cloud data of the complex part can be stored in a depth map format. That is, the acquired depth map can be used to indicate the three-dimensional point cloud data of the complex part, and subsequent segmentation steps can be processed based on the depth map.

[0071] Through steps S101 to S103, feature parameters can be determined based on the geometric features of the part, and feature space can be constructed using the normal vectors of the neighborhood of each point in the depth map and feature parameters. This allows the 3D point cloud data of complex parts to be projected into a high-dimensional feature space. Then, the feature space is divided using histograms under different feature dimensions, which can better identify planes and curved surfaces, achieve high-precision clustering, and obtain ideal surface segmentation results for complex parts.

[0072] In step S102, each point p in the depth map can be traversed. s Then, the traversed points are divided into different neighborhoods to obtain multiple micro-elements. In other words, for any point in the depth map, it can be divided into multiple neighborhoods, and each neighborhood can include a different number of points.

[0073] In one possible implementation, the point p in the depth map can be... sTwo neighborhood partitioning methods are used to obtain a first neighborhood and a second neighborhood. The first neighborhood can include k1 points p. i (x i ,y i ,z i (i = 1, 2, ..., k1), the second neighborhood can include k2 points p. j (x j ,y j ,z j ), j = 1, 2, ..., k2, where k1 ≠ k2. Therefore, the pair of points p in step S102... s Dividing the neighborhood into multiple micro-elements can include: dividing the first neighborhood to obtain a first group of micro-elements and a second group of micro-elements, and dividing the second neighborhood to obtain a third group of micro-elements, wherein the first group of micro-elements, the second group of micro-elements, and the third group of micro-elements each include a different number of micro-elements. In this way, by using multiple different neighborhood partitioning methods, surface micro-elements can be obtained over a large range and in different neighborhoods, providing a foundation for the subsequent construction of a high-dimensional feature space.

[0074] In one example, k1 can be 8, and the first neighborhood (i.e., the 8-neighborhood) can include 8 points p. i (x i ,y i ,z i ), i = 1, 2, ..., 8. When k1 = 8, for point p in the depth map... s , can be applied to point p s The 8-neighborhood is divided into two triangular micro-element groups, namely the first micro-element group and the second micro-element group. Figure 2 A schematic diagram of neighborhood division provided according to an embodiment of the present disclosure is shown. Figure 3 This diagram illustrates a neighborhood partitioning method according to an embodiment of the present disclosure. In this example, it can be implemented according to... Figure 2 The neighborhood partitioning method shown determines the first infinitesimal element, and simultaneously determines the points p1, p2, p3, and p4 of the first neighborhood used under this partitioning method, as follows. Figure 2 As shown, the first infinitesimal element group can include 4 triangular infinitesimal elements (i.e., Figure 2 △p in s p1p2、△p s p3p2、△p s p3p4、△p s p1p4); can be followed Figure 3 The neighborhood partitioning method shown determines the second infinitesimal, such as... Figure 2 As shown, the first set of infinitesimals can include 8 triangular infinitesimals (i.e., Figure 3 △p in s p1p6、△p s p6p2、△p s p2p7、△ps p7p3、△p s p3p8、△p s p8p4、△p s p4p5、△p s p5p1).

[0075] It should be noted that, although... Figure 2 , Figure 3 The neighborhood partitioning method shown is an example illustrating how to determine micro-elements, but those skilled in the art will understand that this disclosure is not limited thereto. In fact, users can flexibly set the number of points included in the neighborhood, the number of micro-elements, and the number of micro-elements according to their personal preferences and / or actual application scenarios, as long as the determined feature parameters can reflect the geometric features of complex parts as much as possible.

[0076] Additionally, when k1 = 8, for point p in the depth map... s It is also possible to determine point p. s The normal vector n of the 8-neighborhood s And unify the normal vector n globally. s The direction, where the normal vector n of the first neighborhood. s Features can include three feature dimensions.

[0077] In yet another example, k2 can be 24, and the second neighborhood can include 24 points p. j (x j ,y j ,z j ), j = 1, 2, ..., 24. In this case, for point p in the depth map s , can be applied to point p s The third infinitesimal group is obtained by partitioning the 24-neighborhood, and the third infinitesimal group may include 24 infinitesimal elements. These 24 infinitesimal elements can be obtained by solving the equation through the normal vector of the point, which will not be elaborated here.

[0078] Additionally, when k2 = 24, for point p in the depth map... s It is also possible to determine point p. s The normal vector n′ of the 24-neighborhood s And unify the normal vector n′ globally. s The direction.

[0079] In one possible implementation, step S102 is based on multiple infinitesimal elements and point p. s The normal vector of the neighborhood determines the point p. sThe feature parameters may include: determining a first set based on the first infinitesimal element, determining a first parameter based on the first set and the normal vector of the first neighborhood, and determining a second parameter based on the point of the first neighborhood and the normal vector of the first neighborhood. The first set includes the normal vectors of each infinitesimal element of the first infinitesimal element; the first parameter indicates the relationship between the normal vector of the infinitesimal element of the first infinitesimal element and the normal vector of the first neighborhood; and the second parameter indicates the relationship between the infinitesimal element of the first infinitesimal element and the point p. s The relationship is as follows: A second set is determined based on the second infinitesimal element set; a third and fourth parameter are determined based on the normal vectors of the second set and the first neighborhood. The second set includes the normal vectors of each infinitesimal element in the second infinitesimal element set, and the third and fourth parameters indicate the relationship between the normal vectors of the infinitesimal elements in the second infinitesimal element set. A third set is determined based on the third infinitesimal element set; a fifth parameter is determined based on the normal vectors of the third set and the second neighborhood. The third set includes the normal vectors of each infinitesimal element in the third infinitesimal element set, and the fifth parameter indicates the relationship between the normal vectors of the infinitesimal elements in the third infinitesimal element set. The first, second, third, fourth, and fifth parameters are used as feature parameters. In this way, by utilizing the normal vectors of surface infinitesimal elements in different neighborhoods and the normal vectors of different neighborhoods to determine feature parameters, each point on the part surface is mapped to a feature space constructed based on different feature parameters, which helps to achieve high-precision surface segmentation.

[0080] In one example, let's further illustrate the case where k1 can be 8: When determining the 4 infinitesimals of the first infinitesimal group (i.e.... Figure 2 △p in s p1p2、△p s p3p2、△p s p3p4、△p s Given p1p4), the first set {n1, n2, n3, n4} can be determined, where n1, n2, n3, and n4 can represent the normal vectors of the four infinitesimal elements mentioned above. With the first set determined, the first parameter n can be further determined by combining the normal vectors of the first neighborhood. α = [n1,n2,n3,n4]·n s Where n1, n2, n3, and n4 represent the normal vectors of the four infinitesimal elements mentioned above, respectively. s Point p s The normal vector of the 8-neighborhood, with the first parameter n α This can be used to indicate the normal vector of the above four infinitesimal elements and n. s The relationship, the first parameter n α Features can include three dimensions. Given the points p1, p2, p3, and p4 of the first neighborhood used when obtaining the first infinitesimal, the second parameter d can be determined based on the points and normal vectors of the first neighborhood. α=min([(p1-p s (p2-p) s (p3-p) s (p4-p) s )] T ·n s ), where n s Point p s The normal vector of the 8-neighborhood, p1-p s p2-p s p3-p s p4-p s They represent point p respectively s The vector pointing to point p1, point p s The vector pointing to point p2, point p s The vector pointing to point p3, point p s The vector d pointing to point p4 α This indicates that the four vectors are in n s The minimum value of the projection on, the second parameter d α It can be used to indicate the relationship between the above four infinitesimal elements and point p. s The relationship, the second parameter d α It can include features in one dimension.

[0081] In one possible implementation, determining the third and fourth parameters based on the normal vectors of the second set and the first neighborhood can further include: constructing a first covariance matrix based on the normal vectors of each element of the second set and the normal vectors of the first neighborhood; decomposing the first covariance matrix to obtain a first principal eigenvalue and a first principal eigenvector; using the first principal eigenvalue as the third parameter and the first principal eigenvector as the fourth parameter. Thus, by constructing the covariance matrix based on the normal vectors of each element and the normal vectors of the neighborhood, and using the principal eigenvalues ​​and principal eigenvectors of the covariance matrix as feature parameters, the relationships between the elements within each neighborhood can be reflected, which helps to determine a high-dimensional feature space conducive to clustering.

[0082] In one example, let's further illustrate the case where k1 can be 8: When determining the 8 infinitesimals of the second infinitesimal group (i.e.... Figure 3 △p in s p1p6、△p s p6p2、△p s p2p7、△p s p7p3、△p s p3p8、△p s p8p4、△p s p4p5、△p s Given p5p1), the second set {n5, n6, n7, n8, n9, n} can be determined. 10 n11 n 12}, where n5, n6, n7, n8, n9, n 10 n 11 n 12 This can be represented by the normal vectors of the aforementioned 8 infinitesimal elements. Given the second set, the first covariance matrix, in column form as shown in equation (i), can be further determined by combining the normal vectors of the first neighborhood:

[0083]

[0084] In the formula, n i Let n represent the normal vector of each of the above 8 infinitesimal elements. s Point p s The normal vector of the 8-neighborhood.

[0085] By performing matrix decomposition on the first covariance matrix, we can obtain the first principal eigenvalue and the first principal eigenvector. The first principal eigenvalue can be used as the third parameter d. β And the first principal eigenvector can be used as the fourth parameter n. β The third parameter d β and the fourth parameter number n β The third parameter d can be used to indicate the relationship between the normal vectors of the individual elements of the second infinitesimal group. β It can include a feature with one dimension, and a fourth parameter n. β Features can include three feature dimensions.

[0086] In one possible implementation, determining the fifth parameter based on the normal vectors of the third set and the second neighborhood can further include: constructing a second covariance matrix based on the normal vectors of each element of the third set and the normal vectors of the second neighborhood; decomposing the second covariance matrix to obtain the second principal eigenvalues; and using the second principal eigenvalues ​​as the fifth parameter. Thus, by constructing the covariance matrix based on the normal vectors of each element and the normal vectors of the neighborhood, and using the principal eigenvalues ​​and principal eigenvectors of the covariance matrix as feature parameters, the relationships between elements within each neighborhood can be reflected, which helps to determine a high-dimensional feature space conducive to clustering.

[0087] In one example, let's further illustrate the case where k2 can be 24: Given the 24 infinitesimals of the third infinitesimal, we can determine the third set {n1, n2, ..., n}. 24}, where n1, n2...n 24 This can be represented by the normal vectors of each of the aforementioned 24 infinitesimal elements. Given the third set, the second covariance matrix, in column form as shown in equation (ii), can be further determined by combining the normal vectors of the second neighborhood:

[0088]

[0089] In the formula, n i Let n′ represent the normal vector of each of the above 24 infinitesimal elements. s Point p s The normal vector of the 24-neighborhood. It should be noted that the same notation exists in this example and other examples, such as n5, n6, etc., but those skilled in the art should understand that the same notation can represent different meanings in different examples, and should not be used to limit the protection scope of the embodiments of this disclosure.

[0090] The second principal eigenvalue can be obtained by matrix decomposition of the second covariance matrix. This second principal eigenvalue can be used as the fifth parameter d. γ The fifth parameter d γ The fifth parameter d can be used to indicate the relationship between the normal vectors of the individual elements of the third infinitesimal. γ It can include features in one dimension.

[0091] In one possible implementation, when the normal vector, first parameter, and fourth parameter of the first neighborhood each include features of three feature dimensions, and the second parameter, third parameter, and fifth parameter each include features of one feature dimension, the step S102 based on point p... s The normal vector and feature parameters of the neighborhood are used to construct the point p. s The feature space can include: features of each feature dimension of the first neighborhood's normal vector, first parameter, second parameter, third parameter, fourth parameter, and fifth parameter as points p. s Features of the 12 feature dimensions of the feature space.

[0092] These feature parameters can reflect the geometric features of complex parts, and a feature space S = [n] can be constructed based on these geometric features. s ,d α ,d β ,d γ ,n α ,n β ], where n s Let d represent the normal vector of the first neighborhood. α Indicates the second parameter, d β Indicates the third parameter, d γ This represents the fifth parameter, n. α This represents the first parameter, n. β This indicates the fourth parameter.

[0093] Thus, for complex parts composed of complex surfaces, by discretizing each point on the surface of the part from a low-dimensional (i.e., three-dimensional) space to a high-dimensional (i.e., 12-dimensional) feature space, the influence of extreme values ​​caused by the surface processing error can be eliminated, which helps to achieve higher-precision clustering in the high-dimensional feature space.

[0094] In one possible implementation, after constructing the feature space of the part, the part surface segmentation method may further include: normalizing the point values ​​in each feature dimension of the feature space. This normalization of the data in the high-dimensional feature space ensures faster computation speed.

[0095] In one example, the feature space can be normalized according to the following formula (iii), limiting the range of values ​​in each dimension of the feature space to between 0 and 1:

[0096]

[0097] In the formula, S i Let i represent the eigenvalues ​​in the i-th feature dimension of the feature space S, where i = 1, 2, ..., 12, min(S i ) represents the minimum value of the eigenvalues ​​in the i-th feature dimension, max(S) i ) represents the maximum value of the feature value under the i-th feature dimension.

[0098] In step S103, grid clustering can be used to cluster points in the feature space, thereby achieving point cloud segmentation of complex parts.

[0099] In one possible implementation, step S103, mapping points in each feature dimension of the feature space to a histogram, may include: for each feature dimension, determining the correspondence between point values ​​and a first length, and constructing a histogram based on the correspondence and the number of points within each interval. The first length indicates the horizontal axis length of the histogram and may include multiple intervals of equal length. The vertical axis of the histogram indicates the number of points whose values ​​fall within an interval. This mapping of point values ​​in each feature dimension of the feature space to a histogram facilitates subsequent clustering based on the histogram, resulting in more stable segmentation results.

[0100] Figure 4 This diagram illustrates a mapping histogram process according to an embodiment of the present disclosure. In one example, one of the 12 feature dimensions described above is used as the target feature dimension for illustration: [Example follows] Figure 4 As shown, the first length in this example is 100. The first length can include 100 intervals, each with a length of 1. After determining the correspondence between the point values ​​under the target feature dimension and these 100 intervals, the number of points in each interval can be further determined, thereby obtaining a histogram under the target feature dimension. The horizontal axis of the histogram has a length of 100, and the vertical axis of the histogram is used to indicate the number of points whose point values ​​are in each interval.

[0101] In one possible implementation, step S103, which involves dividing the feature space based on histograms of each feature dimension to obtain clustering results, may include: determining peaks and corresponding trough regions based on the histograms of each feature dimension; dividing the histogram based on the peaks and corresponding trough regions to obtain one or more grids; and dividing the feature space based on the grids of each feature dimension to obtain clustering results. The clustering results can be used to indicate the similarity of points clustered within the same category. Thus, by dividing the feature space based on the peak and trough regions of the feature histogram and then performing grid division, it facilitates subsequent clustering based on the divided grids to obtain high-precision segmentation results.

[0102] Figure 5 This diagram illustrates the partitioning results under the target feature dimension provided in the embodiments of this disclosure. In one example, one of the aforementioned 12 feature dimensions is used as the target feature dimension for illustration: [Example follows] Figure 5 As shown, the number of points in each length interval can be determined based on the histogram under the target feature dimension (i.e., Figure 5 (The curve in the histogram), then each peak and its corresponding trough can be divided into a region according to the number of peaks on the histogram (for example, Figure 5 ④ in the diagram includes a peak and two troughs on either side, thus allowing us to obtain... Figure 5 The five grids shown in the image (① to ⑤) represent the grid division results.

[0103] Figure 6 This diagram illustrates a clustering result determined based on two feature dimensions according to an embodiment of the present disclosure. In one example, two of the aforementioned 12 feature dimensions (i.e., the first feature dimension and the second feature dimension) are used as examples: the partitioning results under the first feature dimension and the second feature dimension are obtained respectively. In this example, the partitioning result under the first feature dimension is along... Figure 6 The two grids distributed along the vertical axis, the partitioning result under the second feature dimension is along... Figure 6 The feature space is divided using three grids distributed along the horizontal axis, and these two feature dimensions / five grids are used to obtain the overall feature space. Figure 6 The three clustering results are shown. This can be extended to 12 feature dimensions: obtaining similar results across each feature dimension. Figure 5 The partitioning results (a total of 12 results) are similar to those of the previous ones. Figure 6 The feature space is divided using the partitioning results under each feature dimension, resulting in several grids in the feature space. Points with similar features are clustered in the same grid, and points in the same grid can be regarded as a clustering result, thus realizing the partitioning of the feature space.

[0104] Given the clustering results for each point in the feature space, these results can be mapped to the surface space of the part to obtain the segmentation results. Thus, for geometric parts composed of complex surfaces, even in the absence of extensive measurement data, high-precision identification and segmentation are achieved in a high-dimensional feature space through the design and arrangement of geometric features and the feature space. Compared to existing technologies, this approach provides better identification of planes and curved surfaces. Furthermore, by designing differentiated features for different surfaces, segmentation of combined geometric surfaces can be achieved while maintaining fast computation speed. This discretization mapping from low to high dimensions can eliminate the influence of extreme values ​​caused by surface processing errors, and combining histogram peak values ​​for surface clustering yields more stable segmentation results.

[0105] Figure 7 A flowchart is shown for a part surface segmentation method provided according to an embodiment of the present disclosure. Figure 8 A schematic diagram illustrating segmentation results provided according to embodiments of the present disclosure is shown. In one example, for the depth map of the part surface (i.e. Figure 8 (The first image from left to right in the middle a), such as Figure 7 As shown, the depth map is divided into two types of infinitesimal elements: α (the first infinitesimal element mentioned above) and β (the second infinitesimal element mentioned above). Then, feature spaces (S mentioned above) are obtained on α and β respectively. The feature space can include 12 feature dimensions. The segmentation result is obtained by grid partitioning based on feature peaks. For visualization of one of the feature dimensions of the feature space, see [link to visualization]. Figure 8 The second image from the left in image a shows how the surface of the part is segmented based on this feature dimension. Figure 8 The third image from left to right in image a shows the segmentation result. Figure 8 The same logic applies to b and c as to a, so it will not be repeated here.

[0106] Figure 9 A block diagram of a part surface segmentation apparatus provided according to an embodiment of the present disclosure is shown. Figure 9 As shown, the part surface dividing device 900 may include:

[0107] Acquisition module 901 is configured to acquire a depth map of a part, the depth map being used to indicate point cloud data of the part;

[0108] Processing module 902, configured to traverse each point p of the depth map. s For the point p s The neighborhood of the point p is divided into multiple micro-elements. Based on the multiple micro-elements and the point p, s The normal vector of the neighborhood determines the point p. s The characteristic parameters, based on each point p sThe feature space of the part is constructed using the normal vectors and feature parameters of the neighborhood, wherein the feature parameters are used to indicate the geometric features of the part;

[0109] The segmentation module 903 is configured to map points in each feature dimension of the feature space to histograms, divide the points in the feature space based on the histograms in each feature dimension to obtain clustering results for each point, and map the clustering results to the surface space of the part to obtain the segmentation results of the part. The clustering results are used to indicate that points clustered in the same category have similarity.

[0110] In this way, feature parameters are determined based on the geometric features of the part, and feature space is constructed using the normal vectors of the neighborhood of each point in the depth map and feature parameters. This allows the 3D point cloud data of complex parts to be projected into a high-dimensional feature space. Then, the feature space is divided using histograms under different feature dimensions, which can better identify planes and curved surfaces, achieve high-precision clustering, and obtain ideal surface segmentation results for complex parts.

[0111] In one possible implementation, the point p s The neighborhood of a given point includes a first neighborhood and a second neighborhood, wherein the first neighborhood includes k1 points p. i Let i = 1, 2, ..., k1, and the second neighborhood includes k2 points p. j j = 1, 2...k2, k1 ≠ k2; the point p s The neighborhood is divided into multiple micro-elements, including: dividing the first neighborhood into a first micro-element group and a second micro-element group, and dividing the second neighborhood into a third micro-element group, wherein the first micro-element group, the second micro-element group, and the third micro-element group each include a different number of micro-elements.

[0112] In this way, by using a variety of different neighborhood partitioning methods, surface micro-elements can be obtained in a wide range and in different neighborhoods, which provides a foundation for the subsequent construction of a high-dimensional feature space.

[0113] In one possible implementation, the method is based on the plurality of infinitesimal elements and the point p. s The normal vector of the neighborhood determines the point p. sThe feature parameters include: determining a first set based on the first infinitesimal element, determining a first parameter based on the first set and the normal vector of the first neighborhood, and determining a second parameter based on the point of the first neighborhood and the normal vector of the first neighborhood. The first set includes the normal vectors of each infinitesimal element of the first infinitesimal element. The first parameter indicates the relationship between the normal vector of the infinitesimal element of the first infinitesimal element and the normal vector of the first neighborhood. The second parameter indicates the relationship between the infinitesimal element of the first infinitesimal element and the point p. s The relationship is as follows: A second set is determined based on the second infinitesimal element; a third parameter and a fourth parameter are determined based on the normal vectors of the second set and the first neighborhood, wherein the second set includes the normal vectors of each infinitesimal element of the second infinitesimal element, and the third and fourth parameters are used to indicate the relationship between the normal vectors of each infinitesimal element of the second infinitesimal element; a third set is determined based on the third infinitesimal element; a fifth parameter is determined based on the normal vectors of the third set and the second neighborhood, wherein the third set includes the normal vectors of each infinitesimal element of the third infinitesimal element, and the fifth parameter is used to indicate the relationship between the normal vectors of each infinitesimal element of the third infinitesimal element; the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter are used as the feature parameters.

[0114] In this way, by using the normal vectors of surface micro-elements in different neighborhoods and the normal vectors of different neighborhoods to determine the feature parameters, each point on the surface of the part can be mapped to a feature space constructed based on different feature parameters, which helps to achieve high-precision surface segmentation.

[0115] In one possible implementation, determining the third and fourth parameters based on the normal vectors of the second set and the first neighborhood includes: constructing a first covariance matrix based on the normal vectors of each element of the second infinitesimal and the normal vectors of the first neighborhood; decomposing the first covariance matrix to obtain a first principal eigenvalue and a first principal eigenvector; using the first principal eigenvalue as the third parameter and the first principal eigenvector as the fourth parameter; determining the fifth parameter based on the third set and the normal vectors of the second neighborhood includes: constructing a second covariance matrix based on the normal vectors of each element of the third infinitesimal and the normal vectors of the second neighborhood; decomposing the second covariance matrix to obtain a second principal eigenvalue; and using the second principal eigenvalue as the fifth parameter.

[0116] In this way, by constructing a covariance matrix based on the normal vectors of each infinitesimal element and the normal vectors of the neighborhood, and using the principal eigenvalues ​​and principal eigenvectors of the covariance matrix as feature parameters, the relationship between each infinitesimal element in each neighborhood can be reflected, which helps to determine a high-dimensional feature space that is conducive to clustering.

[0117] In one possible implementation, the normal vector of the first neighborhood, the first parameter, and the fourth parameter each include features of three feature dimensions, while the second parameter, the third parameter, and the fifth parameter each include features of one feature dimension; the step of determining the features based on each point p... s The feature space of the part is constructed using the normal vectors and feature parameters of the first neighborhood, including: using the features of each feature dimension of the first neighborhood's normal vector, the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter as the feature space of the point p. s Features of the 12 feature dimensions of the feature space.

[0118] Thus, for complex parts composed of complex surfaces, by discretizing each point on the surface of the part from a low-dimensional (i.e., three-dimensional) space to a high-dimensional (i.e., 12-dimensional) feature space, the influence of extreme values ​​caused by the surface processing error can be eliminated, which helps to achieve higher-precision clustering in the high-dimensional feature space.

[0119] In one possible implementation, the horizontal axis of the histogram is a first length, which includes multiple intervals of equal length, and the vertical axis of the histogram is used to indicate the number of points whose values ​​fall within the intervals. Mapping points in each feature dimension of the feature space to the histogram includes: for each feature dimension, determining the correspondence between the point values ​​and the first length; constructing the histogram based on the correspondence and the number of points within each interval. Dividing the points in the feature space based on the histograms of each feature dimension to obtain clustering results includes: determining peaks and corresponding trough regions based on the histograms of each feature dimension; dividing the histogram based on the peaks and corresponding trough regions to obtain one or more grids; and dividing the feature space based on the grids of each feature dimension to obtain the clustering results.

[0120] In this way, by mapping the point values ​​under each feature dimension of the feature space to the histogram, it is beneficial to perform clustering based on the histogram to obtain more stable segmentation results. Based on the mapping of the feature space, the peak and valley regions of the feature histogram are divided into grids, which helps to perform clustering based on the divided grids to obtain high-precision segmentation results.

[0121] In one possible implementation, after constructing the feature space of the part, the device further includes a normalization module configured to normalize the point values ​​under each feature dimension of the feature space.

[0122] In this way, by normalizing the data in the high-dimensional feature space, a relatively fast computation speed can be guaranteed.

[0123] In some embodiments, the functions or modules of the part surface segmentation device provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the part surface segmentation method embodiments above, which will not be repeated here for the sake of brevity.

[0124] This disclosure also proposes a part surface segmentation device based on deep point cloud data, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described part surface segmentation method when executing the instructions stored in the memory.

[0125] In some embodiments, the functions or modules of the part surface segmentation device provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the part surface segmentation method embodiments above, which will not be repeated here for the sake of brevity.

[0126] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the aforementioned part surface segmentation method. The computer-readable storage medium can be volatile or non-volatile.

[0127] In some embodiments, the functions or modules of the computer-readable storage medium provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the part surface segmentation method embodiments above, which will not be repeated here for the sake of brevity.

[0128] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described part surface segmentation method.

[0129] In some embodiments, the functions or modules of the computer-readable storage medium provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the part surface segmentation method embodiments above, which will not be repeated here for the sake of brevity.

[0130] Figure 10 This is a block diagram illustrating an apparatus 1900 for... according to an exemplary embodiment. For example, apparatus 1900 may be provided as a server or terminal device. See also... Figure 10The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0131] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0132] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0133] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0134] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0135] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0136] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0137] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0138] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0139] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0141] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for surface segmentation of parts based on depth point cloud data, characterized in that, include: Obtain a depth map of the part, the depth map being used to indicate the point cloud data of the part; The depth map is traversed, and the neighborhood of each point is divided into multiple micro-elements. The feature parameters of each point are determined based on the multiple micro-elements and the normal vector of the neighborhood of each point. The feature space of the part is constructed according to the normal vector of the neighborhood of each point and the feature parameters, wherein the feature parameters are used to indicate the geometric features of the part. Points in each feature dimension of the feature space are mapped to histograms. Based on the histograms of each feature dimension, the points in the feature space are divided to obtain the clustering results of each point. The clustering results are mapped to the surface space of the part to obtain the segmentation results of the part. The clustering results are used to indicate that points clustered in the same category have similarity. The neighborhood of a point includes a first neighborhood and a second neighborhood, wherein the first neighborhood includes k1 points, i=1, 2...k1, and the second neighborhood includes k2 points, j=1, 2...k2, where k1≠k2; The process of dividing the neighborhood of the point into multiple micro-elements includes: dividing the first neighborhood into a first micro-element group and a second micro-element group, and dividing the second neighborhood into a third micro-element group, wherein the first micro-element group, the second micro-element group, and the third micro-element group each include a different number of micro-elements. Determining the feature parameters of a point based on the normal vectors of the plurality of micro-elements and the neighborhood of the point includes: A first set is determined based on the first infinitesimal element, a first parameter is determined based on the first set and the normal vector of the first neighborhood, and a second parameter is determined based on the point of the first neighborhood and the normal vector of the first neighborhood. The first set includes the normal vectors of each infinitesimal element of the first infinitesimal element, the first parameter is used to indicate the relationship between the normal vector of the infinitesimal element of the first infinitesimal element and the normal vector of the first neighborhood, and the second parameter is used to indicate the relationship between the infinitesimal element of the first infinitesimal element and the point. A second set is determined based on the second infinitesimal element, and a third parameter and a fourth parameter are determined based on the second set and the normal vector of the first neighborhood. The second set includes the normal vectors of each infinitesimal element of the second infinitesimal element, and the third parameter and the fourth parameter are respectively used to indicate the relationship between the normal vectors of each infinitesimal element of the second infinitesimal element. A third set is determined based on the third infinitesimal element, and a fifth parameter is determined based on the third set and the normal vector of the second neighborhood. The third set includes the normal vectors of each infinitesimal element of the third infinitesimal element, and the fifth parameter is used to indicate the relationship between the normal vectors of each infinitesimal element of the third infinitesimal element. The first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter are used as the feature parameters.

2. The method according to claim 1, characterized in that, The step of determining the third and fourth parameters based on the normal vectors of the second set and the first neighborhood includes: A first covariance matrix is ​​constructed based on the normal vectors of each element of the second infinitesimal group and the normal vector of the first neighborhood. The first covariance matrix is ​​decomposed to obtain the first principal eigenvalue and the first principal eigenvector. The first principal eigenvalue is used as the third parameter, and the first principal eigenvector is used as the fourth parameter. The step of determining the fifth parameter based on the normal vectors of the third set and the second neighborhood includes: constructing a second covariance matrix based on the normal vectors of each element of the third set and the normal vectors of the second neighborhood; decomposing the second covariance matrix to obtain a second principal eigenvalue; and using the second principal eigenvalue as the fifth parameter.

3. The method according to claim 1, characterized in that, The normal vector of the first neighborhood, the first parameter, and the fourth parameter each include features of three feature dimensions, while the second parameter, the third parameter, and the fifth parameter each include features of one feature dimension. The construction of the feature space of the part based on the normal vectors and feature parameters of the neighborhood of each point includes: The features of each feature dimension of the first neighborhood normal vector, the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter are used as the features of the 12 feature dimensions of the feature space of the point.

4. The method according to claim 1, characterized in that, The horizontal axis of the histogram has a first length, which includes multiple intervals of equal length. The vertical axis of the histogram is used to indicate the number of points whose values ​​fall within the intervals. The step of mapping the points under each feature dimension of the feature space to a histogram includes: for each feature dimension, determining the correspondence between the point value under the feature dimension and a first length, and constructing the histogram based on the correspondence and the number of points in each interval; The step of dividing the feature space based on histograms under each feature dimension to obtain the clustering result of each point includes: determining peaks and corresponding valley regions based on histograms under each feature dimension; dividing the histogram based on the peaks and corresponding valley regions to obtain one or more grids; and dividing the feature space based on the grids under each feature dimension to obtain the clustering result.

5. The method according to claim 1 or 4, characterized in that, After constructing the feature space of the part, the method further includes: The point values ​​in each feature dimension of the feature space are normalized.

6. A part surface segmentation device based on depth point cloud data, characterized in that, include: An acquisition module is configured to acquire a depth map of a part, the depth map being used to indicate point cloud data of the part; The processing module is configured to traverse each point of the depth map, divide the neighborhood of each point into multiple micro-elements, determine the feature parameters of each point based on the multiple micro-elements and the normal vector of the neighborhood of each point, and construct the feature space of the part according to the normal vector of the neighborhood of each point and the feature parameters, wherein the feature parameters are used to indicate the geometric features of the part. The neighborhood of a point includes a first neighborhood and a second neighborhood, wherein the first neighborhood includes k1 points, i=1, 2...k1, and the second neighborhood includes k2 points, j=1, 2...k2, where k1≠k2; The process of dividing the neighborhood of the point into multiple micro-elements includes: dividing the first neighborhood into a first micro-element group and a second micro-element group, and dividing the second neighborhood into a third micro-element group, wherein the first micro-element group, the second micro-element group, and the third micro-element group each include a different number of micro-elements. Determining the feature parameters of a point based on the normal vectors of the plurality of micro-elements and the neighborhood of the point includes: A first set is determined based on the first infinitesimal element, a first parameter is determined based on the first set and the normal vector of the first neighborhood, and a second parameter is determined based on the point of the first neighborhood and the normal vector of the first neighborhood. The first set includes the normal vectors of each infinitesimal element of the first infinitesimal element, the first parameter is used to indicate the relationship between the normal vector of the infinitesimal element of the first infinitesimal element and the normal vector of the first neighborhood, and the second parameter is used to indicate the relationship between the infinitesimal element of the first infinitesimal element and the point. A second set is determined based on the second infinitesimal element, and a third parameter and a fourth parameter are determined based on the second set and the normal vector of the first neighborhood. The second set includes the normal vectors of each infinitesimal element of the second infinitesimal element, and the third parameter and the fourth parameter are respectively used to indicate the relationship between the normal vectors of each infinitesimal element of the second infinitesimal element. A third set is determined based on the third infinitesimal element, and a fifth parameter is determined based on the third set and the normal vector of the second neighborhood. The third set includes the normal vectors of each infinitesimal element of the third infinitesimal element, and the fifth parameter is used to indicate the relationship between the normal vectors of each infinitesimal element of the third infinitesimal element. The first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter are used as the feature parameters; The segmentation module is configured to map points in each feature dimension of the feature space to histograms, divide the points in the feature space based on the histograms in each feature dimension to obtain clustering results for each point, and map the clustering results to the surface space of the part to obtain the segmentation results of the part. The clustering results are used to indicate that points clustered in the same category have similarity.

7. The apparatus according to claim 6, characterized in that, The step of determining the third and fourth parameters based on the normal vectors of the second set and the first neighborhood includes: A first covariance matrix is ​​constructed based on the normal vectors of each element of the second infinitesimal group and the normal vector of the first neighborhood. The first covariance matrix is ​​decomposed to obtain the first principal eigenvalue and the first principal eigenvector. The first principal eigenvalue is used as the third parameter, and the first principal eigenvector is used as the fourth parameter. The step of determining the fifth parameter based on the normal vectors of the third set and the second neighborhood includes: constructing a second covariance matrix based on the normal vectors of each element of the third set and the normal vectors of the second neighborhood; decomposing the second covariance matrix to obtain a second principal eigenvalue; and using the second principal eigenvalue as the fifth parameter.

8. The apparatus according to claim 6, characterized in that, The normal vector of the first neighborhood, the first parameter, and the fourth parameter each include features of three feature dimensions, while the second parameter, the third parameter, and the fifth parameter each include features of one feature dimension. The construction of the feature space of the part based on the normal vectors and feature parameters of the neighborhood of each point includes: The features of each feature dimension of the first neighborhood normal vector, the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter are used as the features of the 12 feature dimensions of the feature space of the point.

9. The apparatus according to claim 6, characterized in that, The horizontal axis of the histogram has a first length, which includes multiple intervals of equal length. The vertical axis of the histogram is used to indicate the number of points whose values ​​fall within the intervals. The step of mapping the points under each feature dimension of the feature space to a histogram includes: for each feature dimension, determining the correspondence between the point value under the feature dimension and a first length, and constructing the histogram based on the correspondence and the number of points in each interval; The step of dividing the feature space based on histograms under each feature dimension to obtain the clustering result of each point includes: determining peaks and corresponding valley regions based on histograms under each feature dimension; dividing the histogram based on the peaks and corresponding valley regions to obtain one or more grids; and dividing the feature space based on the grids under each feature dimension to obtain the clustering result.

10. The apparatus according to claim 6 or 9, characterized in that, After constructing the feature space of the part, the device further includes: A normalization module is configured to normalize the point values ​​in each feature dimension of the feature space.

11. A part surface segmentation device based on depth point cloud data, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the part surface segmentation method according to any one of claims 1 to 5 when executing instructions stored in the memory.

12. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the part surface segmentation method according to any one of claims 1 to 5.

Images