Point cloud segmentation method and system based on double channels

Abstract

The invention discloses a point cloud segmentation method and system based on two channels, and relates to the technical field of computer vision and artificial intelligence, and the method comprises the steps: collecting point cloud data; creating a feature channel according to the three-dimensional point cloud data; and performing feature fusion and classification according to the feature channel. According to the point cloud segmentation method based on the two channels, different types of features are processed in different channels, high calculation efficiency is kept while global feature dependence is captured, and the method is suitable for processing and segmentation tasks of large-scale point cloud data. Two different network architectures are used, the structure and parameters of the model are flexibly adjusted to adapt to different application requirements, and a more efficient and reliable solution is provided for a point cloud segmentation task in a complex three-dimensional scene. The method has better effects in the aspects of segmentation precision, segmentation result accuracy and universality.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and artificial intelligence, and in particular to a dual-channel based point cloud segmentation method and system. Background Art

[0002] With the development of 3D sensor technology, such as LiDAR (laser radar) and RGB-D cameras, 3D point cloud data has become an important source of information for object recognition and environmental perception. Point cloud data processing has important application value in the fields of autonomous driving, robot navigation, 3D reconstruction, etc. However, due to the sparsity, irregularity and high-dimensional characteristics of point cloud data, traditional 2D image processing methods are difficult to directly apply to point cloud data, and it is necessary to develop special processing methods for 3D point clouds.

[0003] Existing point cloud segmentation methods mainly include segmentation methods based on traditional machine learning, segmentation methods based on deep learning, and multi-scale feature extraction methods. However, these methods still have some limitations when dealing with complex scenes.

[0004] Traditional point cloud segmentation methods based on machine learning, such as classification methods based on random forests or support vector machines, mainly rely on manually designed geometric features. These methods perform well when dealing with simple scenes with obvious geometric features, but when faced with high-dimensional point cloud data in complex environments, it is difficult to accurately capture and express the subtle differences in point cloud data, resulting in insufficient segmentation accuracy. In addition, manually designed features often depend on specific application scenarios and lack universality.

[0005] Point cloud processing methods based on deep learning mainly include methods based on convolutional neural networks (CNN), methods based on graph convolutional networks (GCN), and methods based on attention mechanisms. Among them, PointNet and its improved version PointNet++ proposed methods to process point cloud data by extracting point features and aggregating local neighborhood features. However, these methods still have shortcomings in dealing with global feature dependencies. In addition, the PointTransformer method based on the global attention mechanism can capture the global dependencies in the point cloud, but due to its high computational complexity, it limits its application in large-scale point cloud datasets.

[0006] Existing single-channel methods can usually only make a compromise between local features and global features when processing features of different scales, and it is difficult to take both into account at the same time.

[0007] The above methods usually only consider one feature of point cloud data, such as geometric shape or spatial distribution, and ignore other potentially useful information in point cloud data (such as color, reflection intensity, etc.). This single feature extraction strategy causes existing methods to often fail to fully utilize the multiple information in point clouds when facing complex three-dimensional scenes, resulting in insufficient accuracy and robustness of segmentation results. Summary of the invention

[0008] In view of the above-mentioned problems, the present invention is proposed.

[0009] Therefore, the technical problem solved by the present invention is that the existing point cloud segmentation methods have insufficient segmentation accuracy, lack of universality, still have deficiencies in processing global feature dependencies, and are unable to handle complex scenes.

[0010] To solve the above technical problems, the present invention provides the following technical solutions: a dual-channel point cloud segmentation method, comprising collecting point cloud data; creating a feature channel according to the three-dimensional point cloud data; and performing feature fusion and classification according to the feature channel.

[0011] As a preferred solution of the dual-channel point cloud segmentation method described in the present invention, wherein: the collecting point cloud data includes acquiring three-dimensional point cloud data;

[0012] The three-dimensional point cloud data includes three-dimensional coordinates, normal vectors, and curvature.

[0013] As a preferred solution of the dual-channel point cloud segmentation method described in the present invention, wherein: the creation of a feature channel according to the three-dimensional point cloud data includes constructing a geometric feature channel and a local feature channel;

[0014] The geometric feature channel uses a multi-layer perceptron to extract the geometric features of point cloud data;

[0015] The local feature channel uses a Transformer-based multi-head self-attention mechanism to extract local features, capture the spatial dependencies between points, enhance the spatial information with position information, and output a local feature vector.

[0016] As a preferred solution of the dual-channel point cloud segmentation method described in the present invention, wherein: the feature channel created according to the three-dimensional point cloud data includes a geometric feature channel including more than one hidden layer, and a nonlinear activation function is used after each hidden layer;

[0017] The geometric feature channels include:

[0018] Normalize the input geometry to ensure that features are learned at the same scale;

[0019] Input the geometric information of each point into the multi-layer perceptron, where the geometric information includes three-dimensional coordinates, normal vector and curvature;

[0020] The geometric features of point cloud data are extracted through a multi-layer perceptron, and a high-dimensional geometric feature vector of each point is output.

[0021] As a preferred solution of the dual-channel point cloud segmentation method described in the present invention, wherein: the creation of a feature channel according to the three-dimensional point cloud data includes a Transformer in a local feature channel adopting a multi-head self-attention mechanism, and the number of the multi-head self-attention mechanisms is 2 to 8;

[0022] The local feature channels include:

[0023] Find the k nearest neighbors of each point in space;

[0024] Output the relative coordinates of each neighborhood point relative to the center point, and generate local features through pooling operations;

[0025] The local features are combined with the position encoding and input into the Transformer-based multi-head self-attention mechanism to extract the local context features of each point and generate a local feature vector;

[0026] The point cloud data determines the neighborhood points through k-nearest neighbor search, and the neighborhood features are aggregated through pooling operations in the local feature channel to form a local feature representation;

[0027] The multi-head self-attention mechanism includes:

[0028] Perform linear transformation on the input local features to generate query, key and value matrices;

[0029] Output the similarity between the query and the key through the dot product, and normalize the similarity using the softmax function;

[0030] Perform weighted summation of the value matrix according to the normalized similarity to generate multi-head self-attention output features;

[0031] The multi-head self-attention output features are processed using residual connections and layer normalization and further processed by a feed-forward neural network.

[0032] As a preferred solution of the dual-channel point cloud segmentation method described in the present invention, wherein: the feature fusion and classification according to the feature channel includes fusing the geometric feature channel and the local feature channel to output the geometric features and local features of the point cloud data, and adaptively adjusting the weight coefficient to generate an efficient joint feature representation;

[0033] The multi-layer perceptron in the attention mechanism fusion step includes one or more hidden layers, each of which is followed by an activation function;

[0034] The attention mechanism includes:

[0035] Concatenate the geometric features and local features in the channel dimension to generate a concatenated feature vector;

[0036] Input the concatenated feature vector into a multi-layer perceptron and output the attention weights of geometric features and local features;

[0037] Use the output attention weights to perform weighted adjustment on the geometric features and local features to generate a weighted feature vector;

[0038] The weighted geometric features and local features are added together to generate a fused feature vector.

[0039] As a preferred solution of the dual-channel point cloud segmentation method described in the present invention, wherein: the feature fusion and classification according to the feature channel includes a convolution layer and a fully connected layer, which are used to further extract high-order features, and map the high-order features to the classification probability of each point in the point cloud through the Softmax function, so as to realize the semantic segmentation of the point cloud;

[0040] The classification probability of each output point is output through the Softmax function, and the category corresponding to the maximum probability value is used as the semantic classification result;

[0041] Data augmentation, including rotation, scaling, and translation operations.

[0042] Another object of the present invention is to provide a dual-channel point cloud segmentation system, which can solve the problem of low computational efficiency of current point cloud segmentation methods by processing different types of features in different channels respectively.

[0043] As a preferred solution of the dual-channel point cloud segmentation system described in the present invention, it includes: an initialization module, a feature channel construction module, a feature fusion module; a classification module; the initialization module is used to collect three-dimensional point cloud data; the feature channel construction module is used to construct geometric feature channels and local feature channels; the feature fusion module is used to output the geometric features and local features of point cloud data; the classification module is used to extract high-order features.

[0044] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a step of a point cloud segmentation method based on a dual channel.

[0045] A computer-readable storage medium stores a computer program, which implements the steps of a dual-channel point cloud segmentation method when executed by a processor.

[0046] Beneficial effects of the present invention: The dual-channel point cloud segmentation method provided by the present invention processes different types of features in different channels respectively, while capturing global feature dependencies and maintaining high computational efficiency, and is suitable for large-scale point cloud data processing and segmentation tasks. Using two different network architectures, MLP and Transformer, the structure and parameters of the model can be flexibly adjusted to adapt to different application requirements, providing a more efficient and reliable solution for point cloud segmentation tasks in complex three-dimensional scenes. The introduction of this method is expected to promote the further development of point cloud segmentation technology and provide more extensive support for related application fields. The present invention has achieved better results in segmentation accuracy, segmentation result accuracy and universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0048] Figure 1 An overall flow chart of a dual-channel point cloud segmentation method provided for the first embodiment of the present invention.

[0049] Figure 2 A geometric feature channel flow chart of a dual-channel point cloud segmentation method provided for the first embodiment of the present invention.

[0050] Figure 3 A local feature channel flow chart of a dual-channel point cloud segmentation method provided for the first embodiment of the present invention.

[0051] Figure 4 A diagram of a Transformer-based multi-head self-attention mechanism for a dual-channel point cloud segmentation method provided in the first embodiment of the present invention.

[0052] Figure 5 A feature fusion graph based on a self-attention mechanism for a dual-channel point cloud segmentation method provided in the first embodiment of the present invention.

[0053] Figure 6 An overall flow chart of a dual-channel point cloud segmentation system provided for the third embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0055] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides a point cloud segmentation method based on dual channels. The method includes the following steps S1 to S5:

[0056] In S1: three-dimensional point cloud data is obtained, wherein the three-dimensional point cloud data includes but is not limited to three-dimensional coordinates, normal vectors, curvature, etc.

[0057] In S2: A geometric feature channel that uses a multi-layer perceptron (MLP) to extract geometric features of point cloud data.

[0058] In S3: A local feature channel uses a Transformer-based multi-head self-attention mechanism to extract local features, capture the spatial dependencies between points, and enhance the spatial information by combining the position information to generate a local feature vector.

[0059] In S4: A feature fusion module is used to fuse the geometric features and local features of the point cloud data output by the geometric feature channel and the local feature channel, and adaptively adjust the weight coefficients to generate an efficient joint feature representation.

[0060] In S5: a classification module, the classification module includes several convolutional layers and fully connected layers, which are used to further extract high-order features, and map the high-order features to the classification probability of each point in the point cloud through the Softmax function to achieve semantic segmentation of the point cloud.

[0061] The acquired three-dimensional point cloud data, where each point contains information including but not limited to three-dimensional coordinates (x i ,y i ,z i ), normal vector (n xi ,n yi ,n zi ), curvature c i , density d i , input the acquired point cloud data into the geometric feature channel and the local feature channel respectively;

[0062] like Figure 2As shown, in this embodiment, before using a multi-layer perceptron to extract the geometric features of the point cloud data in the geometric feature channel, the point cloud data is preprocessed, and the specific steps are as follows:

[0063] (1) The three-dimensional coordinates (x i ,y i ,z i ), normal vector (n xi ,n yi ,n zi ), curvature c i , density d i Splice to form a feature vector P containing all geometric information i geo ;

[0064] P i geo =[x i ,y i ,z i ,n xi ,n yi ,n zi ,c i ,d i ]

[0065] (2) Furthermore, in the geometric eigenvector P i geo Before inputting into the geometric feature extraction module, P i geo Standardization is performed using the following method, where μ and σ are the mean and standard deviation of the geometric features, respectively;

[0066]

[0067] (3) The normalized geometric feature vector P i geo_norm Input into the multi-layer perceptron;

[0068] Further, obtain the normalized geometric feature vector P i geo_norm Finally, a multi-layer perceptron is used to extract the geometric features of the point cloud data. The multi-layer perceptron contains multiple fully connected layers. The output of each layer is the linear transformation output of the previous layer obtained by nonlinear transformation of the activation function. It is used to perform nonlinear transformation on the features of the points point by point and extract higher-dimensional geometric features. The steps include:

[0069] (1) After the first fully connected layer, P i geo_normMapped into a higher dimensional feature space, where W1 is the weight matrix, b1 is the bias vector, and σ() is the ReLU activation function;

[0070]

[0071] (2) After multiple hidden layers, more complex geometric features are extracted. Each hidden layer consists of a fully connected layer and an activation function, where h k represents the output of the kth layer;

[0072] h k =σ(W k ·h k-1 +b k )

[0073] (3) Feature F output from the last layer g (P i ) is the geometric feature descriptor of the point, where L is the number of layers of the multilayer perceptron, and d g is the dimension of the output feature;

[0074]

[0075] In this embodiment, the role of the local feature channel is to capture the local context information of each point in the point cloud. Different from the geometric feature channel which mainly focuses on the geometric properties of the point, the local feature channel focuses on the relationship between the point and its neighboring points, such as Figure 3 As shown in the figure, the core module of the local feature channel is the multi-head self-attention mechanism based on Transformer. Prior to this, it is necessary to encode the three-dimensional coordinates of the point cloud data and construct the neighborhood features of each point. The specific steps are as follows:

[0076] (1) Use sine and cosine functions to encode three-dimensional coordinates, so that the Transformer can perceive the location information of each point;

[0077] PE([x i ,y i ,z i ])=[sin(x i ),cos(x i ),sin(y i ),cos(y i ),sin(z i ),cos(z i )];

[0078] (2) For each point P i , using the k nearest neighbor search method to construct its neighborhood features {P j} j∈k-NN(Pi), and calculate each neighborhood point relative to the center point P i The relative coordinate ΔP ij , these relative coordinates are used to describe the local structure P i loc_knn ;

[0079] ΔP ij =P j -P i =(x j -x i ,y j -y i ,z j -z i );

[0080] The Pool here can be an operation such as average pooling or maximum pooling for aggregating neighborhood features;

[0081] (3) Further, the local feature vector of each point is summed with the position code and normalized to obtain the normalized local feature vector P i loc_norm , ensuring that different features have equal weight and influence during model training;

[0082] P i loc =P i loc_knn +PE([x i ,y i ,z i ]);

[0083]

[0084] Furthermore, if Figure 4 As shown, the local feature vector P after normalization i loc_norm , through the Transformer's multi-head self-attention mechanism, the complex relationship between each point and its neighboring points is captured. The specific steps are as follows:

[0085] (1) Perform a linear transformation on the input local features to obtain the query, key, and value matrices, where W Q , W K , W V is a learnable weight matrix;

[0086]

[0087]

[0088]

[0089] (2) Calculate the similarity between the query and the key of each neighboring point through the dot product and normalize it using the Softmax function;

[0090]

[0091] (3) Furthermore, after each attention head is normalized, the outputs of multiple self-attention heads are concatenated, where each head i represents the output of a self-attention head, W O is the output weight matrix;

[0092] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0093] (4) After each layer of attention mechanism, residual connections and layer normalization operations are used to improve training stability, and the training is processed through a two-layer feedforward neural network;

[0094] P i loc_out =LayerNorm(P i loc +MutilHead(Q,K,V)

[0095] P i loc_final =LayerNorm(P i loc_out +FFN(P i loc ))

[0096] (5) After being processed by multiple layers of Transformer, the local feature channel will output the local feature vector of each point, expressed as:

[0097]

[0098] The geometric features F of the point cloud data are output in the geometric feature channel and the local feature channel respectively. g (P i ) and local features F l (P i )After that, the features need to be fused;

[0099] like Figure 5As shown, in this embodiment, the attention mechanism is used to fuse these two features. Before this, the geometric features and local features need to be spliced ​​in the channel dimension for subsequent calculation of the attention weight, where the geometric feature F g (P i ) has a dimension of d g , local feature F l (P i ) has a dimension of d l , then the concatenated feature F concat (P i ) has a dimension of d g +d l , where || represents the concatenation operation;

[0100]

[0101] Furthermore, after obtaining the concatenated feature F concat (P i ), a shared multi-layer perceptron (MLP) is used to calculate the attention weights. The specific steps are as follows:

[0102] (1) The concatenated features F are transformed through a fully connected layer concat (P i ) is mapped to an intermediate feature representation, where W mid is the weight matrix, b mid is the bias term, σ() is the nonlinear activation function (such as ReLU);

[0103] h mid =σ(W mid ·F concat (P i )+b mid )

[0104] (2) The intermediate feature h mid Mapped onto two scalars, σ′ g and σ′ l Corresponding to the weights of geometric features and local features, respectively, where ω g and ω l is the weight vector, b g and b l is the bias term;

[0105] α′ g =ω g ·h mid +b g ,

[0106] α′ l =ω l ·h mid +bl

[0107] (2) To ensure that the weights are between 0 and 1 and their sum is 1, the generated weights are normalized using the Softmax function: α g , α l =Softmax([α′ g , α′ l ]);

[0108] (3) Further, after calculating the attention weight α g and a l After that, they are applied to the corresponding geometric features and local features, and the weights of the features of each channel are adjusted accordingly, where:

[0109] Represents an element-by-element multiplication operation;

[0110] F′ g (P i )=α g ·F g (P i )

[0111] F′l(P i )=α l ·F l (P i )

[0112] (4) Finally, the weighted geometric features and local features are added together to obtain the fused joint feature F fused (P i ), the fused feature F fused (P i ) is a comprehensive description of the geometric attributes and local contextual relationships of each point, combining the advantages of geometric features and local features, and dynamically adjusting the weights of the two according to the specific circumstances of the input data;

[0113] F fused (P i )=F g ′(P i )+F l ′(P i )

[0114] Finally, if Figure 5 As shown, in this embodiment, the fused feature F fused (P i ) is input into the fully connected layer to further aggregate features and output the classification probability F of each point fc (P i ), where W fc is the weight matrix, b fcis the bias term;

[0115] F fc (P i )=W fc ·F fused (P i )+b fc

[0116] The final classification result is output through the Softmax function, the probability P of each point belonging to each category i , where P i =Softmax(F fc (P i ));

[0117] In this embodiment, a dual-channel approach is used to achieve the segmentation of point cloud data. Unlike a single-channel model that performs poorly in specific tasks, a multi-layer perceptron (MLP) is good at extracting global geometric features, but is not effective in capturing local details. The Transformer can capture local features and the relationship between points well due to its attention mechanism, but is inefficient when processing global features. By combining the two, a multi-layer perceptron is used to capture the macroscopic geometric structure of the point cloud, and the Transformer focuses on the local details in the point cloud data. The features extracted by the two channels are fused through the attention mechanism, and the weights of the two channels are dynamically adjusted to help the model understand the point cloud structure more comprehensively and enhance the model's perception of different types of features, thereby improving the accuracy of segmentation.

[0118] Example 2, reference Figure 2-Figure 5 , which is an embodiment of the present invention, provides a point cloud segmentation method based on dual channels. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0119] First, a dual-channel approach is used to achieve the segmentation of point cloud data. Unlike single-channel models that perform poorly in specific tasks, multi-layer perceptrons (MLPs) are good at extracting global geometric features, but are not effective in capturing local details. Transformer can capture local features and the relationship between points well due to its attention mechanism, but is inefficient when processing global features. By combining the two, a multi-layer perceptron is used to capture the macroscopic geometric structure of the point cloud, while Transformer focuses on the local details in the point cloud data. The features extracted by the two channels are fused through the attention mechanism, and the weights of the two channels are dynamically adjusted to help the model understand the point cloud structure more comprehensively and enhance the model's perception of different types of features, thereby improving the accuracy of segmentation.

[0120] In this embodiment, a dual-channel point cloud segmentation method is provided. In order to verify its beneficial effects, the same experimental equipment and experimental data are used to compare the experimental results of the method proposed in this patent with those of the existing PointNet, PointNet++, KPConv, RandLa-Net, and Transformer methods, and the evaluation criteria shown below are adopted.

[0121]

[0122]

[0123]

[0124] Among them, TP represents the number of points correctly predicted as positive; FP represents the number of points incorrectly predicted as positive; TN represents the number of points correctly predicted as negative; FN represents the number of points incorrectly predicted as negative, IoU represents the degree of overlap between the prediction result of a single category and the true label; mIoU is the average of the IoU of all categories, indicating the average segmentation performance of the model on all categories; OA refers to the average of the accuracy of all categories, indicating the average classification performance of the model on all categories.

[0125] In the embodiment, the experimental equipment is a ThinkStation P920 workstation, which is implemented using the PyTorch platform. The graphics card of the workstation is NVIDIA RTX A4000, and the CPU is Intel Xeon Platinum 8260 processor * 2. The experimental data is captured by a drone on the railway in the suburbs of eastern Chinese cities, covering approximately 5 kilometers of railway. 80% of the data is divided into a training set and the remaining 20% ​​is divided into a validation set for evaluating the performance of the model. At the same time, the fixed railway infrastructure in the data set is divided into 7 categories, namely ground, catenary, guardrail, track bed, pole, protective wall, and rail.

[0126] Table 1 Comparative experimental results

[0127]

[0128] It can be clearly seen from the experimental results of various deep learning network models in Table 1 that the model proposed in the present invention is significantly better than other mainstream models in terms of overall accuracy (OA) and mean intersection over union (mIoU), showing excellent performance. Compared with the traditional PointNet model, the overall accuracy of the model of the present invention is improved by 61.4%, compared with PointNet++, it is improved by 25.7%, and compared with KPConv, it is improved by 9%. In terms of intersection over union mIoU, the performance of the model of the present invention is also superior, which is 40.3% higher than PointNet, 27.6% higher than PointNet++, and 17% higher than KPConv.

[0129] Further analysis of the detection results of different categories revealed that the model of the present invention performed well in multiple key scenarios. Compared with the PointNet++ and KPConv models, the mIoU of the model of the present invention in the categories of Protective Wall, Track Panel, and Cable increased by 42.7%, 31.7%, and 17.4%, respectively. Especially in the categories of Cluster and Guardrail, the model of the present invention outperformed other models, increasing by 49.7% and 22.3%, respectively, and showed stronger robustness in complex scenarios.

[0130] The model of the present invention not only has a breakthrough in accuracy, but also can better balance speed and accuracy in the recognition and segmentation of various targets. Therefore, in general, the model proposed by the present invention is more suitable for target detection and classification tasks in complex scenarios, especially showing greater advantages in real-time performance and accuracy.

[0131] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0132] Example 3, reference Figure 6 , is an embodiment of the present invention, and provides a dual-channel based point cloud segmentation system, including an initialization module, a feature channel construction module, a feature fusion module; a classification module;

[0133] The initialization module is used to collect three-dimensional point cloud data, the feature channel construction module is used to construct geometric feature channels and local feature channels, the feature fusion module is used to output the geometric features and local features of point cloud data, and the classification module is used to extract high-order features.

[0134] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0136] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0137] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0138] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A point cloud segmentation method, characterized in that: include: Collect point cloud data; Creating a feature channel according to the three-dimensional point cloud data; Feature fusion and classification are performed according to the feature channels.

2. The point cloud segmentation method according to claim 1, characterized in that: The collecting of point cloud data includes obtaining three-dimensional point cloud data; The three-dimensional point cloud data includes three-dimensional coordinates, normal vectors, and curvature.

3. The point cloud segmentation method according to claim 2, characterized in that: The creating of the feature channel according to the three-dimensional point cloud data includes constructing a geometric feature channel and a local feature channel; The geometric feature channel uses a multi-layer perceptron to extract the geometric features of point cloud data; The local feature channel uses a Transformer-based multi-head self-attention mechanism to extract local features, capture the spatial dependencies between points, enhance the spatial information with position information, and output a local feature vector.

4. The point cloud segmentation method according to claim 3, characterized in that: The feature channel created according to the three-dimensional point cloud data includes a geometric feature channel including more than one hidden layer, and a nonlinear activation function is used after each hidden layer; The geometric feature channels include: Normalize the input geometry to ensure that features are learned at the same scale; Input the geometric information of each point into the multi-layer perceptron, where the geometric information includes three-dimensional coordinates, normal vector and curvature; The geometric features of point cloud data are extracted through a multi-layer perceptron, and a high-dimensional geometric feature vector of each point is output.

5. The point cloud segmentation method according to claim 4, characterized in that: The creating of the feature channel according to the three-dimensional point cloud data includes the Transformer in the local feature channel adopting a multi-head self-attention mechanism, and the number of the multi-head self-attention mechanisms is 2 to 8; The local feature channels include: Find the k nearest neighbors of each point in space; Output the relative coordinates of each neighborhood point relative to the center point, and generate local features through pooling operations; The local features are combined with the position encoding and input into the Transformer-based multi-head self-attention mechanism to extract the local context features of each point and generate a local feature vector; The point cloud data determines the neighborhood points through k-nearest neighbor search, and the neighborhood features are aggregated through pooling operations in the local feature channel to form a local feature representation; The multi-head self-attention mechanism includes: Perform linear transformation on the input local features to generate query, key and value matrices; Output the similarity between the query and the key through the dot product, and normalize the similarity using the softmax function; Perform weighted summation of the value matrix according to the normalized similarity to generate multi-head self-attention output features; The multi-head self-attention output features are processed using residual connections and layer normalization and further processed by a feed-forward neural network.

6. The point cloud segmentation method according to claim 5, characterized in that: The feature fusion and classification according to the feature channel includes fusing the geometric feature channel and the local feature channel to output the geometric features and local features of the point cloud data, and adaptively adjusting the weight coefficient to generate an efficient joint feature representation; The multi-layer perceptron in the attention mechanism fusion step includes one or more hidden layers, each of which is followed by an activation function; The attention mechanism includes: Concatenate the geometric features and local features in the channel dimension to generate a concatenated feature vector; Input the concatenated feature vector into a multi-layer perceptron and output the attention weights of geometric features and local features; Use the output attention weights to perform weighted adjustment on the geometric features and local features to generate a weighted feature vector; The weighted geometric features and local features are added together to generate a fused feature vector.

7. The point cloud segmentation method according to claim 6, characterized in that: The feature fusion and classification according to the feature channel includes a convolution layer and a fully connected layer, which are used to further extract high-order features, and map the high-order features to the classification probability of each point in the point cloud through the Softmax function, so as to realize the semantic segmentation of the point cloud; The classification probability of each output point is output through the Softmax function, and the category corresponding to the maximum probability value is used as the semantic classification result; Data augmentation, including rotation, scaling, and translation operations.

8. A system using the point cloud segmentation method according to any one of claims 1 to 7, characterized in that: Including initialization module, feature channel construction module, feature fusion module; Classification module; The initialization module is used to collect three-dimensional point cloud data; The feature channel construction module is used to construct a geometric feature channel and a local feature channel; The feature fusion module is used to output the geometric features and local features of point cloud data; The classification module is used to extract high-order features.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the point cloud segmentation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the point cloud segmentation method according to any one of claims 1 to 7 are implemented.

Images