Three-dimensional point cloud segmentation method based on graph isomorphism network

Abstract

The disclosure provides a three-dimensional point cloud segmentation method based on graph isomorphism network, and relates to the technical field of computers. First, the first three-dimensional point cloud data to be segmented is acquired, then a first topological graph corresponding to the first three-dimensional point cloud data is constructed, and finally the first topological graph is input into a pre-trained graph isomorphism segmentation network to obtain the segmentation result of the first three-dimensional point cloud data. Thus, by constructing the topological graph of the three-dimensional point cloud data to be segmented, the topological graph is processed by using the graph isomorphism segmentation network, and the segmentation result of the three-dimensional point cloud data is obtained, so that the local detail features in the three-dimensional point cloud data are effectively captured, the segmentation result is more sensitive to local details, and the accuracy and effect of three-dimensional point cloud segmentation are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a three-dimensional point cloud segmentation method based on graph isomorphism network. BACKGROUND

[0002] With the development of deep learning technology, three-dimensional point cloud segmentation methods based on neural networks gradually rise, and especially graph neural networks are gradually applied to the analysis and processing of point clouds due to their advantages in processing non-Euclidean data. However, the existing graph neural network methods cannot effectively capture the local details of point cloud data in point cloud segmentation, resulting in low accuracy of the segmentation results. SUMMARY

[0003] The present disclosure provides a three-dimensional point cloud segmentation method based on graph isomorphism network.

[0004] According to an aspect of the present disclosure, a three-dimensional point cloud segmentation method based on graph isomorphism network is provided, comprising:

[0005] obtaining first three-dimensional point cloud data to be segmented;

[0006] constructing a first topological graph corresponding to the first three-dimensional point cloud data;

[0007] inputting the first topological graph into a pre-trained graph isomorphism segmentation network to obtain a segmentation result of the first three-dimensional point cloud data.

[0008] The present disclosure provides a three-dimensional point cloud segmentation method based on graph isomorphism network. First, the first three-dimensional point cloud data to be segmented is obtained, then the first topological graph corresponding to the first three-dimensional point cloud data is constructed, and finally the first topological graph is input into the pre-trained graph isomorphism segmentation network to obtain the segmentation result of the first three-dimensional point cloud data. Thus, by constructing the topological graph of the three-dimensional point cloud data to be segmented, the topological graph is processed by using the graph isomorphism segmentation network to obtain the segmentation result of the three-dimensional point cloud data, so that the local detail features in the three-dimensional point cloud data can be effectively captured, the segmentation result is more sensitive to local details, and the accuracy and effect of the three-dimensional point cloud segmentation are improved. BRIEF DESCRIPTION OF DRAWINGS

[0009] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure together with the specification.

[0010] Figure 1 A flowchart of a three-dimensional point cloud segmentation method based on graph isomorphism network provided by the present disclosure is shown.

[0011] Figure 2A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network according to an embodiment of the present disclosure is shown in FIG. 1.

[0012] Figure 3 A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network according to an embodiment of the present disclosure is shown in FIG. 1.

[0013] Figure 4 A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network according to an embodiment of the present disclosure is shown in FIG. 1.

[0014] Figure 5 An implementation detail diagram of the graph isomorphism segmentation network in the present disclosure is shown in FIG. 1.

[0015] Figure 6 A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network according to an embodiment of the present disclosure is shown in FIG. 1.

[0016] Figure 7 A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network according to an embodiment of the present disclosure is shown in FIG. 1.

[0017] The specific embodiments of the present disclosure have been shown in the above-described drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present disclosure by any means, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0018] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same or similar components are denoted by the same reference numerals throughout the drawings and the following description, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0019] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0020] It should be noted that the acquisition, transmission, storage, use, processing, etc. of data in the technical solutions of the present disclosure all comply with the relevant provisions of national laws and regulations.

[0021] It should be noted that in the embodiments of the present disclosure, some software, components, models and the like in the prior art may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility in the implementation of the technical solutions of the present disclosure, but does not mean that the applicant has or will necessarily use the scheme.

[0022] The method for three-dimensional point cloud segmentation based on graph isomorphism network according to the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0023] Figure 1 A flowchart of a method for three-dimensional point cloud segmentation based on graph isomorphism network according to the embodiments of the present disclosure is shown.

[0024] As shown in Figure 1 , the method comprises:

[0025] Step 101, obtaining first three-dimensional point cloud data to be segmented.

[0026] The first three-dimensional point cloud data can include each point and its corresponding three-dimensional spatial coordinates.

[0027] It should be noted that the application scenario to which the first three-dimensional point cloud data belongs can be determined according to actual needs. For example, the first three-dimensional point cloud data can be obstacle point cloud data in an autonomous driving scenario, or can also be path and roadblock point cloud data in a robot navigation scenario, or can also be building point cloud data in a city modeling scenario, and the like, which are not limited by the present disclosure.

[0028] Optionally, when obtaining the first three-dimensional point cloud data to be segmented, any of the following can be included:

[0029] Receiving first three-dimensional point cloud data to be segmented scanned by a laser radar; in the case of receiving a three-dimensional point cloud data segmentation instruction, obtaining the first three-dimensional point cloud data to be segmented contained in the instruction.

[0030] The specific structure and type of the laser radar can be determined as needed. For example, it can be a vehicle-mounted laser radar, an industrial laser radar, a surveying and mapping laser radar, and the like, which are not limited by the present disclosure.

[0031] Step 102, constructing a first topological graph corresponding to the first three-dimensional point cloud data.

[0032] In the present disclosure, after obtaining the first three-dimensional point cloud data to be segmented, in order to make the point cloud segmentation processing more effectively capture the local details of the point cloud data, a first topological graph corresponding to the first three-dimensional point cloud data can be constructed before segmenting the first three-dimensional point cloud data.

[0033] It should be noted that the method of constructing the first topology graph corresponding to the first three-dimensional point cloud data can be pre-set or determined as needed. For example, the first topology graph can be constructed by a k-nearest neighbor clustering-based method, and the present disclosure does not limit this.

[0034] In step 103, the first topology graph is input into a pre-trained graph isomorphism segmentation network to obtain a segmentation result of the first three-dimensional point cloud data.

[0035] It should be noted that the specific architecture of the graph isomorphism segmentation network can be determined as needed. For example, the graph isomorphism segmentation network can include a feature extraction module and a feature enhancement module, and the present disclosure does not limit this.

[0036] The segmentation result can include a probability that each point in the first three-dimensional point cloud data belongs to each category, which can be a two-dimensional matrix composed of points and categories, and the numerical value in the matrix represents the probability.

[0037] The category can be a category obtained by processing and segmenting the nodes in the first topology graph by the graph isomorphism segmentation network.

[0038] In the present disclosure, after the first topology graph corresponding to the first three-dimensional point cloud data is constructed, the first topology graph is input into a pre-trained graph isomorphism segmentation network to obtain a segmentation result of the first three-dimensional point cloud data output by the graph isomorphism segmentation network. Thus, the topology graph of the three-dimensional point cloud data is segmented by the graph isomorphism network, which effectively captures the local detail features of the three-dimensional point cloud data, thereby improving the accuracy of three-dimensional point cloud segmentation.

[0039] In the present embodiment, the first three-dimensional point cloud data to be segmented is first obtained, then the first topology graph corresponding to the first three-dimensional point cloud data is constructed, and finally the first topology graph is input into a pre-trained graph isomorphism segmentation network to obtain a segmentation result of the first three-dimensional point cloud data. Thus, by constructing the topology graph of the three-dimensional point cloud data to be segmented, the topology graph is processed by the graph isomorphism segmentation network to obtain a segmentation result of the three-dimensional point cloud data, thereby effectively capturing the local detail features in the three-dimensional point cloud data, making the segmentation result more sensitive to local details, and thereby improving the accuracy and effect of three-dimensional point cloud segmentation.

[0040] Figure 2 A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network according to an embodiment of the present disclosure is provided.

[0041] As shown in Figure 2 The method comprises the following steps:

[0042] In step 201, the first three-dimensional point cloud data to be segmented is obtained.

[0043] The specific implementation form of step 201 can refer to the detailed description in other embodiments of the present disclosure, and will not be repeated here.

[0044] In step 202, the coordinates of each three-dimensional point in the first three-dimensional point cloud data are determined.

[0045] In the present disclosure, after obtaining the first three-dimensional point cloud data to be segmented, in order to accurately and reliably construct the first topology graph of the first three-dimensional point cloud data, the coordinates of each three-dimensional point in the first three-dimensional point cloud data can be determined first.

[0046] In step 203, based on the coordinates of each three-dimensional point, the distance values respectively corresponding to the three-dimensional point and other three-dimensional points in the first three-dimensional point cloud data are determined.

[0047] It should be noted that the distance values respectively corresponding to each three-dimensional point and other three-dimensional points in the first three-dimensional point cloud data can be calculated by a preset method. For example, the Euclidean distance can be used to determine the distance values between each three-dimensional point and other three-dimensional points, which is not limited in the present disclosure.

[0048] In step 204, based on the distance values respectively corresponding to each three-dimensional point and other three-dimensional points, a preset number of nearest neighbor three-dimensional points corresponding to each three-dimensional point are determined.

[0049] The preset number can be set as needed. For example, the preset number can be 5, that is, each three-dimensional point can have 5 nearest neighbor three-dimensional points, which is not limited in the present disclosure.

[0050] The nearest neighbor three-dimensional point is the other three-dimensional point that is closest in space to a certain three-dimensional point, that is, the nearest neighbor three-dimensional point of the three-dimensional point.

[0051] In step 205, based on each three-dimensional point and its corresponding preset number of nearest neighbor three-dimensional points, a first topology graph corresponding to the first three-dimensional point cloud data is constructed.

[0052] In the present disclosure, after determining each three-dimensional point in the first three-dimensional point cloud data and its corresponding preset number of nearest neighbor three-dimensional points, a first topology graph corresponding to the first three-dimensional point cloud data can be constructed based on each three-dimensional point and its nearest neighbor three-dimensional points, thereby providing conditions for accurately and reliably segmenting the first three-dimensional point cloud data.

[0053] In step 206, the first topology graph is input into a pre-trained graph isomorphism segmentation network to obtain a segmentation result of the first three-dimensional point cloud data.

[0054] The specific implementation form of step 206 can refer to the detailed description in other embodiments of the present disclosure, and will not be repeated here.

[0055] In the embodiments of the present disclosure, first, the first three-dimensional point cloud data to be segmented is acquired, and the coordinates of each three-dimensional point in the first three-dimensional point cloud data are determined. Then, based on the coordinates of each three-dimensional point, the distance values respectively corresponding to the three-dimensional point and other three-dimensional points in the first three-dimensional point cloud data are determined. Based on the distance values respectively corresponding to each three-dimensional point and other three-dimensional points, the preset number of nearest neighbor three-dimensional points corresponding to each three-dimensional point are determined. Then, based on each three-dimensional point and the preset number of nearest neighbor three-dimensional points corresponding thereto, the first topology graph corresponding to the first three-dimensional point cloud data is constructed. Finally, the first topology graph is input into the pre-trained graph isomorphism segmentation network to obtain the segmentation result of the first three-dimensional point cloud data. In this way, by determining the preset number of nearest neighbor three-dimensional points of each three-dimensional point in the three-dimensional point cloud data to be segmented, the topology graph is constructed, and the graph isomorphism segmentation network is used to process the topology graph to obtain the segmentation result of the three-dimensional point cloud data. Thus, by combining the topology structure of the three-dimensional point cloud data to segment the three-dimensional point cloud data, the segmentation result can better reflect the local features of the point cloud data, and the accuracy and reliability of the three-dimensional point cloud segmentation are improved.

[0056] Figure 3 A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network provided by the embodiments of the present disclosure.

[0057] As shown in Figure 3 , the method comprises:

[0058] Step 301, acquiring first three-dimensional point cloud data to be segmented.

[0059] Step 302, determining the coordinates of each three-dimensional point in the first three-dimensional point cloud data.

[0060] Step 303, based on the coordinates of each three-dimensional point, determining the distance values respectively corresponding to the three-dimensional point and other three-dimensional points in the first three-dimensional point cloud data.

[0061] Step 304, based on the distance values respectively corresponding to each three-dimensional point and other three-dimensional points, determining the preset number of nearest neighbor three-dimensional points corresponding to each three-dimensional point.

[0062] The specific implementation forms of steps 301 to 304 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0063] Step 305, based on each three-dimensional point and the preset number of nearest neighbor three-dimensional points corresponding thereto, clustering the first three-dimensional point cloud data, and dividing the first three-dimensional point cloud data into multiple clusters.

[0064] In the present disclosure, after determining the preset number of nearest neighbor three-dimensional points corresponding to each three-dimensional point, the first three-dimensional point cloud data can be clustered by a k-nearest neighbor clustering method, and the first three-dimensional point cloud data is divided into a plurality of clusters. The value of k is the same as the preset number of nearest neighbor three-dimensional points of the three-dimensional points.

[0065] Step 306, determining the cluster center point of each cluster.

[0066] In the present disclosure, after the first three-dimensional point cloud data is divided into a plurality of clusters, the three-dimensional point with the smallest average distance in each cluster can be taken as the cluster center point of each cluster by using formula (1) as shown below. Formula (1) is shown below, and formula (1) is only an example, and the present disclosure is not limited thereto:

[0067] , (1)

[0068] wherein, is the cluster center; and is not the same point as is the number of three-dimensional points in the cluster.

[0069] Step 307, establishing a connection edge between each two three-dimensional points in each cluster, and establishing a connection edge between each two cluster center points, to obtain a first topology graph corresponding to the first three-dimensional point cloud data.

[0070] In the present disclosure, after determining the cluster center point of each cluster, a complete graph within the cluster can be constructed by establishing a connection edge between each two three-dimensional points in each cluster, and a single connection edge between each two cluster center points can be established to reflect the relationship between different clusters. Since different clusters can represent different categories, the connection edge between the cluster center points between the clusters can reflect the relationship between different categories, so that the first topology graph corresponding to the first three-dimensional point cloud data not only includes the local structural features of the point cloud data, but also includes the global context information of the point cloud data, thereby providing an effective data structure for the subsequent graph isomorphism segmentation network, and improving the segmentation effect.

[0071] Step 308, inputting the first topology graph into the pre-trained graph isomorphism segmentation network to obtain a segmentation result of the first three-dimensional point cloud data.

[0072] The specific implementation form of step 308 can refer to the detailed description in other embodiments of the present disclosure, and will not be described in detail here.

[0073] ​In the embodiments of the present disclosure, first, the first three-dimensional point cloud data to be segmented is acquired, and the coordinates of each three-dimensional point in the first three-dimensional point cloud data are determined. Then, based on the coordinates of each three-dimensional point, the distance values respectively corresponding to the three-dimensional point and other three-dimensional points in the first three-dimensional point cloud data are determined, and based on the distance values respectively corresponding to each three-dimensional point and other three-dimensional points, the preset number of nearest neighbor three-dimensional points corresponding to each three-dimensional point are determined. Then, based on each three-dimensional point and the preset number of nearest neighbor three-dimensional points corresponding thereto, the first three-dimensional point cloud data is clustered, the first three-dimensional point cloud data is divided into multiple clusters, and the cluster center points of each cluster are determined. Finally, the connection edges are established between each two three-dimensional points in each cluster, and the connection edges are established between each two cluster center points, the first topology graph corresponding to the first three-dimensional point cloud data is obtained, and the first topology graph is input into the graph isomorphism segmentation network pre-trained to obtain the segmentation result of the first three-dimensional point cloud data. Thus, after the preset number of nearest neighbor three-dimensional points corresponding to each three-dimensional point in the three-dimensional point cloud data to be segmented is determined, based on each three-dimensional point and the corresponding nearest neighbor three-dimensional points, the three-dimensional point cloud data is clustered, the point cloud data is divided into multiple clusters, and the cluster center points of each cluster are determined. The connection edges are established between each two points in each cluster, and the connection edges are established between each two cluster center points, the topology graph corresponding to the point cloud data is constructed, the graph isomorphism segmentation network is used to process the topology graph, and the segmentation result of the point cloud data is obtained. Therefore, the segmentation result can effectively reflect the local features of the point cloud data, and the quality and accuracy of the segmentation result are improved.

[0074] Figure 4 A flowchart of a three-dimensional point cloud segmentation method based on a graph isomorphism network provided by the embodiments of the present disclosure.

[0075] As Figure 4 shown, the method comprises:

[0076] Step 401, acquiring first three-dimensional point cloud data to be segmented.

[0077] Step 402, constructing a first topology graph corresponding to the first three-dimensional point cloud data.

[0078] The specific implementation forms of steps 401 to 402 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0079] Step 403, using a graph isomorphism segmentation network to extract features of the first topology graph, and acquiring high-dimensional features of the first topology graph.

[0080] The high-dimensional features can include local features and global features of the first topological graph. For example, the local features can include coordinates, normal directions, curvatures, and local densities, and the global features can include global structural features reflecting the overall shape and topological relationship of the point cloud, context features of the point cloud, i.e., spatial dependency between points, and semantic features, i.e., segmentation labels, and the like, without limitation.

[0081] Optionally, when the feature extraction is performed on the first topological graph by using the graph isomorphism segmentation network to obtain the high-dimensional features of the first topological graph, the graph isomorphism convolution and the maximum pooling down-sampling can be used in the graph isomorphism segmentation network to perform the feature extraction on the first topological graph, so as to obtain the high-dimensional features of the first topological graph.

[0082] At step 404, the feature enhancement is performed on the high-dimensional features by using the graph isomorphism segmentation network to obtain enhanced features of the first topological graph.

[0083] The enhanced features can include local feature details, global context features, and topological features of the first topological graph, and the like. For example, the enhanced features can include coordinates of points, edges of point cloud data, textures, and probabilities of each point belonging to each category, and the like, without limitation.

[0084] It should be noted that the enhanced features can be a two-dimensional matrix for representing probabilities of points in the point cloud data belonging to each category.

[0085] Optionally, when the feature enhancement is performed on the high-dimensional features by using the graph isomorphism segmentation network to obtain the enhanced features of the first topological graph, the up-sampling and the graph isomorphism convolution can be used in the graph isomorphism segmentation network to perform the feature enhancement on the high-dimensional features, so as to obtain the enhanced features of the first topological graph.

[0086] The specific implementation of the graph isomorphism segmentation network in the three-dimensional point cloud segmentation method based on the graph isomorphism network will be described below. Figure 5 The specific implementation of the graph isomorphism segmentation network in the three-dimensional point cloud segmentation method based on the graph isomorphism network will be described below. Figure 5 The specific implementation of the graph isomorphism segmentation network in the three-dimensional point cloud segmentation method based on the graph isomorphism network will be described below.

[0087] Figure 5 The graph isomorphism segmentation network is described below by taking two layers as an example.

[0088] It should be noted that for each node in the topological graph, the graph isomorphism convolution updates the feature representation of the node by aggregating features of other neighbor nodes (and some combination of its own features). Specifically, the update rule of the graph isomorphism convolution can be shown in the following formula (2):

[0089] , (2)

[0090] wherein, is a node In the first layer, the feature representation of the node is a set of neighbor nodes of the node . is a multi-layer perception, used for feature transformation of the first layer, is a learnable parameter.

[0091] It should be noted that in the present disclosure, the same is used in different layers, i.e., it does not change with the number of layers .

[0092] As shown in FIG. 1, when the graph isomorphism segmentation network is used to extract and enhance the features of the topology graph, first, the feature extraction part: Figure 5 ①: The input data is point cloud data, and the size of the point cloud data is

[0093] After two-layer ( ) graph isomorphism convolution, the extracted features are output, and the size of the extracted features is .

[0094] wherein, n is the number of points in the point cloud data, wherein, 3 represents that each point is a three-dimensional vector (i.e., the three-dimensional space coordinates of the point).

[0095] ②: The features extracted in ① are input, and max pooling is used for down-sampling, and the extracted features are output, and the size of the extracted features is .

[0096] ③: The features extracted in ② are input, and after two-layer ( ) graph isomorphism convolution, the extracted features are output, and the size of the extracted features is .

[0097] ④: The features extracted in ③ are input, and max pooling is used for down-sampling, and the extracted features are output, and the size of the extracted features is .

[0098] ⑤: The features extracted in ④ are input, and max pooling is used for down-sampling, and the extracted features are output, and the size of the extracted features is .

[0099] Thus, the features output in ⑤ are the high-dimensional features of the topology graph.

[0100] Then, the high-dimensional features are enhanced:

[0101] ​​⑥: input the features extracted from ⑤ and perform upsampling, and concatenate the features extracted from ④ in the channel dimension, output the enhanced features, and the size is .

[0102] ⑦: input the enhanced features from ⑥, pass through two layers of graph isomorphism convolution, and output the enhanced features, and the size is . .

[0103] ⑧: input the enhanced features from ⑦ and perform upsampling, and concatenate the features extracted from ③ in the channel dimension, output the enhanced features, and the size is .

[0104] ⑨: input the enhanced features from ⑧, pass through two layers of graph isomorphism convolution, and output the enhanced features, and the size is . .

[0105] ⑩: input the enhanced features from ⑨ and perform upsampling, and concatenate the features extracted from ① in the channel dimension, output the enhanced features, and the size is .

[0106] ⑪: input the enhanced features from ⑩, pass through two layers of graph isomorphism convolution, and output the enhanced features, and the size is . .

[0107] Wherein, m is the number of segmentation categories.

[0108] Therefore, the features output by ⑪ are the enhanced features of the topological graph, which can also be confirmed as the segmentation result of the three-dimensional point cloud data.

[0109] Through the graph isomorphism segmentation network proposed in the disclosure, the feature extraction and feature enhancement are performed on the topological graph corresponding to the point cloud data, so that the segmentation result more effectively captures the local detail features of the point cloud, and the accuracy of the three-dimensional point cloud segmentation is improved.

[0110] Step 405, determine the enhanced features as the segmentation result corresponding to the first three-dimensional point cloud data.

[0111] In the disclosure, after obtaining the enhanced features of the first topological graph, the enhanced features can be determined as the segmentation result corresponding to the first three-dimensional point cloud data, so as to improve the sensitivity of the segmentation result to the local details of the point cloud data, and ensure the accuracy of the three-dimensional point cloud data segmentation.

[0112] In the embodiments of the present disclosure, firstly, the first three-dimensional point cloud data to be segmented is acquired, and a first topological graph corresponding to the first three-dimensional point cloud data is constructed, then the feature extraction is performed on the first topological graph by using the graph isomorphism segmentation network, the high-dimensional features of the first topological graph are acquired, then the feature enhancement is performed on the high-dimensional features by using the graph isomorphism segmentation network, the enhanced features of the first topological graph are acquired, and finally the enhanced features are determined as the segmentation result corresponding to the first three-dimensional point cloud data. Therefore, after the topological graph corresponding to the three-dimensional point cloud data to be segmented is constructed, the feature extraction and the feature enhancement are performed on the topological graph by using the graph isomorphism segmentation network, the enhanced features of the topological graph are obtained, and the enhanced features are determined as the segmentation result of the three-dimensional point cloud data, so that the segmentation result not only contains the local details of the point cloud, but also contains the global context features of the point cloud, and the accuracy and the accuracy of the segmentation result are improved.

[0113] The following will be described in combination with Figure 6 The flow of the three-dimensional point cloud segmentation method based on the graph isomorphism network in the present disclosure is exemplified, Figure 6 The flowchart of the three-dimensional point cloud segmentation method based on the graph isomorphism network in the present disclosure is shown.

[0114] As shown in Figure 6 , firstly, as shown in part (a) of Figure 6 , the three-dimensional point cloud data to be segmented is acquired, then as shown in part (b) of Figure 6 , the topological graph of the three-dimensional point cloud data is constructed based on k-nearest neighbor clustering, then as shown in part (c) of Figure 6 , the topological graph is input into the graph isomorphism segmentation network, and then the point cloud segmentation result output by the graph isomorphism segmentation network is obtained, as shown in part (d) of Figure 6 . Therefore, the local detail features in the three-dimensional point cloud data can be effectively captured, the segmentation result is more sensitive to local details, and the accuracy and the effect of the three-dimensional point cloud segmentation are improved.

[0115] Figure 7 The flowchart of the method for obtaining the graph isomorphism segmentation network in the three-dimensional point cloud segmentation method based on the graph isomorphism network provided in the embodiments of the present disclosure is shown.

[0116] As shown in Figure 7 , the method for obtaining the graph isomorphism segmentation network comprises the following steps:

[0117] In step 701, a training data set is acquired, wherein the training data set comprises second three-dimensional point cloud data and corresponding real segmentation results.

[0118] The training data set can be a pre-set three-dimensional point cloud training data set, and the present disclosure does not limit this.

[0119] In this disclosure, the second 3D point cloud data is the 3D point cloud data used to train the graph isomorphic segmentation network. ,in, The number of points in the second-dimensional point cloud data. For the first point cloud A point is a three-dimensional vector, as shown in formula (3) below:

[0120] , (3)

[0121] in, In three-dimensional space axis coordinate values, In three-dimensional space axis coordinate values, In three-dimensional space Axis coordinate values. Therefore, a complete point cloud dataset is... The tensor.

[0122] The actual segmentation result can be the segmentation result of manually annotated second-dimensional point cloud data, and this disclosure does not limit it.

[0123] Step 702: Construct the second topology map corresponding to the second three-dimensional point cloud data.

[0124] The specific implementation of constructing the second topology map corresponding to the second three-dimensional point cloud data is similar to the specific implementation of constructing the first topology map corresponding to the first three-dimensional point cloud data in the above embodiments of this disclosure. The second topology map can be constructed by referring to the specific implementation of constructing the first topology map, and will not be described in detail here.

[0125] Step 703: Input the second topology graph into the initial graph isomorphic segmentation network and obtain the predicted segmentation result output by the initial graph isomorphic segmentation network.

[0126] The initial graph isomorphic segmentation network can be pre-set, and this disclosure does not limit it.

[0127] The predicted segmentation result can include the probability that each point in the point cloud data belongs to each category, as predicted by the initial graph isomorphic segmentation network. It can be of size [size missing]. tensor, The number of categories to be divided.

[0128] Step 704: Determine the loss value between the predicted segmentation result and the actual segmentation result.

[0129] In the present disclosure, after obtaining the predicted segmentation result, in order to improve the effect of the graph isomorphism segmentation network, the initial graph isomorphism segmentation network can be modified by calculating the loss value between the predicted segmentation result and the real segmentation result. Since the segmentation problem is a multi-classification problem, the present disclosure can determine the loss value between the predicted segmentation result and the real segmentation result through a multiclass cross-entropy loss function (MCEL), denoted as . The calculation of the loss value is shown in the following formula (4):

[0130] , (4)

[0131] wherein, is the predicted segmentation result; is the real segmentation result.

[0132] Step 705, in the case where the loss value is greater than or equal to the threshold value, the parameter gradient of the initial graph isomorphism segmentation network is calculated based on the loss value through the back propagation algorithm.

[0133] wherein, the threshold value can be pre-set, which can be used to determine the loss value critical value when the graph isomorphism segmentation network is well trained. For example, the threshold value can be 0.01, which is not limited by the present disclosure.

[0134] In the present disclosure, in the case where the loss value is greater than or equal to the threshold value, it can be determined that the deviation between the predicted segmentation result and the real segmentation result is large, and the segmentation network effect is poor, and the network has not been well trained. At this time, the parameter gradient of the initial graph isomorphism segmentation network can be calculated based on the loss value through the back propagation algorithm.

[0135] Step 706, based on the parameter gradient, the parameters of the initial graph isomorphism segmentation network are updated.

[0136] In the present disclosure, after determining the parameter gradient of the initial graph isomorphism segmentation network, the parameters of the initial graph isomorphism segmentation network can be updated based on the parameter gradient by using the stochastic gradient descent method (SGD), so as to optimize the network parameters and provide conditions for subsequent network training.

[0137] Step 707, return to perform the operation of obtaining the training data set until the loss value is less than the threshold value, and obtain the trained graph isomorphism segmentation network.

[0138] In the present disclosure, after updating the parameters of the initial graph isomorphism segmentation network, the operation of obtaining the de-training data set can be returned to be executed until the loss value is less than the threshold value, it can be determined that the deviation between the prediction segmentation result of the segmentation network and the real segmentation result is small at this time, the network performance is good, it can be determined that the segmentation network has been trained well, and the trained graph isomorphism segmentation network is obtained.

[0139] Optionally, after obtaining the trained graph isomorphism segmentation network, the performance and robustness of the trained graph isomorphism segmentation network can also be tested, especially the accuracy of local details and segmentation boundaries. The specific testing method can be determined as needed, and the present disclosure does not limit it.

[0140] In the embodiments of the present disclosure, the training data set is first obtained, then the second topological graph corresponding to the second three-dimensional point cloud data is constructed, and the second topological graph is input into the initial graph isomorphism segmentation network to obtain the prediction segmentation result output by the initial graph isomorphism segmentation network. Then the loss value between the prediction segmentation result and the real segmentation result is determined, and in the case that the loss value is greater than or equal to the threshold value, the parameter gradient of the initial graph isomorphism segmentation network is calculated based on the loss value by using the back propagation algorithm. Finally, the parameters of the initial graph isomorphism segmentation network are updated based on the parameter gradient, and the operation of obtaining the training data set is returned to be executed until the loss value is less than the threshold value, and the trained graph isomorphism segmentation network is obtained. Therefore, by obtaining the point cloud data and the corresponding real segmentation result from the training data set, the topological graph of the point cloud data is constructed and input into the initial graph isomorphism segmentation network, the loss value between the prediction result output by the initial graph isomorphism segmentation network and the real result is calculated, and the parameter gradient of the initial graph isomorphism segmentation network is determined based on the loss value. The parameters of the initial segmentation network are adjusted using the parameter gradient, and the operation of obtaining the training data set is returned until the loss value is less than the threshold value, and the trained graph isomorphism segmentation network is obtained, thereby improving the performance and effect of the graph isomorphism segmentation network, and providing conditions for improving the accuracy of three-dimensional point cloud segmentation based on the graph isomorphism network.

[0141] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the aspects of the present disclosure disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure that come within the scope of the three claims and their equivalents. It is submitted that the specification and examples are illustrative only and not limiting of the true scope and spirit of the present disclosure.

[0142] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0143] It should be noted that in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise stated.

[0144] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions or other processes. The various embodiments of the present disclosure can include additional or fewer steps or processes, and the order of the steps or processes can be changed, including according to the function involved, without departing from the scope of the embodiments of the present disclosure, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0145] It should be understood that various parts of the present disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0146] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, which includes one or a combination of steps of the method embodiments when executed.

[0147] In addition, each functional unit in each embodiment of the present disclosure can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of software functional module. The integrated module, if realized in the form of software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0148] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0149] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the specification, the illustrative expressions of the above terms do not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0150] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present disclosure, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present disclosure.

Claims

1. A method for three-dimensional point cloud segmentation based on graph isomorphism network, characterized in that, The method comprises: obtaining first three-dimensional point cloud data to be segmented; determining the coordinates of each three-dimensional point in the first three-dimensional point cloud data; based on the coordinates of each three-dimensional point, determining the distance values respectively corresponding to the three-dimensional point and other three-dimensional points in the first three-dimensional point cloud data; based on the distance values respectively corresponding to each three-dimensional point and the other three-dimensional points, determining a preset number of nearest neighbor three-dimensional points corresponding to each three-dimensional point; based on each three-dimensional point and its corresponding preset number of nearest neighbor three-dimensional points, clustering the first three-dimensional point cloud data based on k-neighbor clustering, and dividing the first three-dimensional point cloud data into multiple clusters; determining the cluster center points of each cluster; establishing a connection edge between each two three-dimensional points in each cluster, and establishing a connection edge between each two cluster center points, to obtain a first topological graph corresponding to the first three-dimensional point cloud data; in a graph isomorphism segmentation network, using graph isomorphism convolution and maximum pooling downsampling to extract features of the first topological graph, and obtaining high-dimensional features of the first topological graph; using the graph isomorphism segmentation network to enhance the features of the high-dimensional features, and obtaining enhanced features of the first topological graph; determining the enhanced features as the segmentation result corresponding to the first three-dimensional point cloud data.

2. The method of claim 1, wherein, The method of obtaining the first three-dimensional point cloud data to be segmented comprises any of the following: receiving first three-dimensional point cloud data to be segmented scanned by a laser radar; in a case where a three-dimensional point cloud data segmentation instruction is received, obtaining first three-dimensional point cloud data to be segmented included in the instruction.

3. The method of claim 1, wherein, The method of using the graph isomorphism segmentation network to enhance the features of the high-dimensional features, and obtaining enhanced features of the first topological graph, comprises: in the graph isomorphism segmentation network, using upsampling and graph isomorphism convolution to enhance the features of the high-dimensional features, and obtaining enhanced features of the first topological graph.

4. The method of claim 1, wherein, The method for obtaining the graph isomorphism segmentation network comprises: obtaining a training data set, wherein the training data set comprises second three-dimensional point cloud data and corresponding real segmentation results; constructing a second topological graph corresponding to the second three-dimensional point cloud data; inputting the second topological graph into an initial graph isomorphism segmentation network to obtain a predicted segmentation result output by the initial graph isomorphism segmentation network; determining a loss value between the predicted segmentation result and the real segmentation result; in a case where the loss value is greater than or equal to a threshold value, calculating a parameter gradient of the initial graph isomorphism segmentation network by a back propagation algorithm based on the loss value; updating the parameters of the initial graph isomorphism segmentation network based on the parameter gradient; returning to perform the operation of obtaining the training data set until the loss value is less than the threshold value, and obtaining a trained graph isomorphism segmentation network.

Images