Three-dimensional point cloud semantic segmentation method and system based on graph neural network and window attention mechanism, and storage medium

Abstract

The application relates to the field of three-dimensional point cloud semantic segmentation, and specifically discloses a three-dimensional point cloud semantic segmentation method and system based on a graph neural network and a window attention mechanism, a storage medium, and an asymmetric feature is designed, that is, coordinate features are additionally added on the basis of original point cloud features, so that the point cloud features change from rgb-xyz two groups of information to rgb-xyz-xyz three groups of information, which is beneficial to subsequent extraction of local geometric information; the combination of the graph neural network and the kernel point convolution can extract more abundant, real and effective local information; the window attention mechanism extracts local context information, and then the local context information is combined with the local geometric information extracted by the graph neural network, so that the accuracy of semantic segmentation is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud semantic segmentation, specifically to a 3D point cloud semantic segmentation method and system, and storage medium based on graph neural networks and window attention mechanisms. Background Technology

[0002] 3D data has wide applications across various fields, including autonomous driving, robotics, remote sensing, healthcare, and design. 3D data is typically represented in different formats, including depth images, point clouds, meshes, and volumetric meshes. As a commonly used format, point cloud representation preserves the original geometric information in 3D space without any discretization. Therefore, point clouds are the preferred representation for many scene understanding applications, such as autonomous driving and robotics. Furthermore, 3D point cloud semantic segmentation is fundamental to 3D scene semantic understanding and analysis, and is crucial in fields such as navigation and localization, pattern recognition, and autonomous driving.

[0003] Semantic segmentation methods for 3D point clouds are mainly divided into methods based on conventional supervised machine learning and methods based on deep learning.

[0004] Methods based on conventional supervised machine learning can be divided into two categories. One category classifies each point or cluster of points based solely on individual features, such as maximum likelihood classifiers based on Gaussian mixture models, support vector machines (SVMs), random forests, and Bayesian discriminant classifiers. The other category consists of statistical context models, such as associated and unassociated Markov networks, conditional random fields (CRFs), simplified Markov random fields (MRFs), multi-level inference programs focusing on point cloud statistics and relational information at different scales, and spatial inference machines that model the inherent medium- to long-term dependencies in the data. These algorithms primarily design features for segmentation specific to certain tasks and are suitable for semantic segmentation in simple scenarios. However, for complex scenarios, it is difficult to find suitable feature combinations.

[0005] Deep learning-based methods can be mainly divided into three categories: multi-view based, voxel-based, and point-based.

[0006] The multi-view-based method projects 3D point clouds using multiple views. Figure 2 The image is represented in 3D and then processed using a 2D CNN. The classification result is then restored to 3D. Multi-view methods effectively address the structuring problem of point cloud data, but these methods suffer from two serious drawbacks. First, they do not fully utilize the underlying geometric and structural information because the projection step inevitably introduces information loss. Second, their performance is sensitive to viewpoint selection and occlusion; the multi-view projected image must cover all spatial areas containing points. For large, complex scenes, it is difficult to select enough suitable viewpoints for multi-view projection. Therefore, few studies have applied multi-view-based deep learning architectures to 3D point cloud semantic segmentation.

[0007] Voxel-based methods first divide the point cloud into a set of voxels. Then, a 3D convolutional neural network is used for voxel segmentation. Finally, all points within a voxel are assigned the same semantic label as that voxel. However, compared to point clouds, voxel structures have lower resolution, and voxelization inherently leads to information loss. Furthermore, high resolution results in higher memory and computational costs, while low resolution leads to loss of detail. Choosing a suitable resolution in practice is not easy.

[0008] Point-based methods work directly on irregular point clouds without further transformation, reducing geometric information loss during dimensionality conversion and enabling end-to-end training. This approach has relatively high universality and is currently a hot research topic.

[0009] In recent years, attention mechanisms have been gradually introduced into the field of 3D point cloud segmentation and have made significant progress. However, many attention-based methods ignore the role of local geometric features and make insufficient use of local geometric features, which limits the accuracy of semantic segmentation. Summary of the Invention

[0010] The purpose of this invention is to overcome the problems existing in the prior art and provide a 3D point cloud semantic segmentation method, system, and storage medium based on graph neural networks and window attention mechanism. This method can deeply integrate local geometric features into the window attention mechanism, effectively combine local geometric information and contextual information, improve the accuracy of current 3D point cloud semantic segmentation algorithms, and enrich high-accuracy 3D point cloud semantic segmentation algorithms.

[0011] To achieve the above objectives, the first aspect of the present invention provides a 3D point cloud semantic segmentation method based on graph neural networks and window attention mechanism, comprising the following steps:

[0012] S1. Obtain the input point cloud;

[0013] S2. Reconstruct the input point cloud, swap any two sets of target information positions of the points, and add an additional set of information positions;

[0014] S3. Use edge convolution in a graph neural network to extract the edge relationships between points and their neighbors from the reconstructed point cloud, resulting in a point cloud containing these edge relationships. F represents the information dimension of a point, and N represents the number of points;

[0015] S4. Process the input point cloud from step S1 using kernel convolution to extract local information from the point cloud and obtain the final point cloud. Point clouds containing edge relationships and local feature point cloud The input point cloud for the attention mechanism is obtained by summing the data and then processing it through batch normalization and activation functions.

[0016] S5. Execute the window attention mechanism to obtain context information. Will and By merging them together, a comprehensive set of local features can be obtained;

[0017] S6. Downsample the comprehensive local features under the window attention mechanism, then perform window attention multiple times, and achieve decoding operation by upsampling and aggregating features, finally completing semantic segmentation.

[0018] A second aspect of the present invention provides a 3D point cloud semantic segmentation system based on graph neural networks and window attention mechanisms, comprising:

[0019] The input point cloud reconstruction module is used to reconstruct the acquired input point cloud by swapping any two sets of target information positions of the points and adding an additional set of information positions.

[0020] The edge convolution extraction module is used to extract the edge relationships between points and their neighbors using edge convolution in a graph neural network, resulting in a point cloud containing these edge relationships. F represents the information dimension of a point, and N represents the number of points;

[0021] The kernel convolution processing module is used to process the input point cloud in step S1 using kernel convolution to extract local information of the point cloud and obtain the final point cloud. Point clouds containing edge relationships and local feature point cloud The input point cloud for the attention mechanism is obtained by summing the results and then processing them through batch normalization and activation functions.

[0022] The window attention mechanism module is used to execute the window attention mechanism and obtain context information. Will and By merging them together, a comprehensive set of local features can be obtained;

[0023] The semantic segmentation module is used to downsample the comprehensive local features under the window attention mechanism, then perform window attention multiple times, and realize the decoding operation by upsampling and aggregating features, and finally complete the semantic segmentation.

[0024] A third aspect of the present invention provides a machine-readable storage medium storing instructions that cause a machine to perform the steps of the three-dimensional point cloud semantic segmentation method based on graph neural networks and window attention mechanisms described above.

[0025] A third aspect of the present invention provides a processor for running a program, wherein the program is executed to perform: a three-dimensional point cloud semantic segmentation method based on graph neural networks and window attention mechanisms.

[0026] Through the above technical solutions, the present invention has the following beneficial effects: 1) Asymmetric features are designed, that is, coordinate features are added on the basis of the original point cloud features, so that the point cloud features change from two sets of rgb-xyz information to three sets of rgb-xyz-xyz information, which is beneficial to the subsequent extraction of local geometric information; 2) The combination of graph neural network and kernel point convolution can extract richer, more realistic and effective local information; 3) The window attention mechanism extracts local context information, and then combines it with the local geometric information extracted by graph neural network, which significantly improves the accuracy of semantic segmentation. Attached Figure Description

[0027] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] Figure 1 This is a schematic diagram of the process structure of the "3D point cloud semantic segmentation network based on graph neural network and window attention" according to an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the structure of a graph neural network according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the window attention mechanism according to an embodiment of the present invention; wherein, (a) is a schematic diagram of the window attention mechanism, and (b) is a schematic diagram of the mouth attention mechanism.

[0031] Figure 4 This is a visual comparison of the segmentation results according to an embodiment of the present invention. Detailed Implementation

[0032] To further illustrate the features of the present invention, the technical solution of the present invention will be described in more detail below through specific embodiments. However, the present invention can be implemented in many places different from those described herein, and those skilled in the art can make similar improvements without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0033] Existing point cloud segmentation algorithms that directly process points do not fully utilize local geometric information, nor do they effectively combine local geometric information with contextual information, resulting in low segmentation accuracy. The first aspect of this invention provides a 3D point cloud semantic segmentation method based on graph neural networks and a window attention mechanism, such as... Figure 1 As shown, it includes the following steps:

[0034] Phase I: Point embedding layer + two consecutive window attention modules

[0035] 1.1 Point Embedding Layer.

[0036] First, obtain the input point cloud. There are N points in total, each consisting of color and coordinate information, containing a total of 6 dimensions. The i-th point can be represented as p. i =(x i ,y i ,z i ,r i ,g i ,b i Then, the point cloud is input into a point embedding layer consisting of edge convolutions and kernel point convolutions.

[0037] For EdgeConvolution (EdgeConv) based on graph neural networks, the input is the reconstructed input point cloud, which is obtained by swapping the positions of the coordinate and color information of the point cloud, and adding an additional set of coordinate information, p. i =(x i ,y i ,z i ,r i ,g i ,b i Adjusted to p i ′=(r i ,g i ,b i ,x i ,y i ,z i ,x i ,y i ,z i Each point contains two sets of coordinate information, and the adjusted point cloud can be represented as follows: like Figure 2 As shown, points are used as vertices of the graph. The relationship between a point and its neighboring points is used as the edge of the graph. Construct a graph representing the local point cloud structure It is dynamically constructed using the nearest neighbors in the feature space, with the number of nearest neighbors set to 10 for example.

[0038] Further, extract the image Midpoint p i ′ and adjacent point p j The boundary relationship between ' and ' is defined by the following formula:

[0039] e ij =h Θ (p i ′,p′j ) = h Θ (p i ′||p′ j -p i ′) (1)

[0040] Where h Θ It is a neural network with a set of learnable parameters Θ, used to map pairs of 9-dimensional point cloud features to a specified F dimension. The edge function selection combines global shape structure and local neighborhood information into an asymmetric edge function, where the global shape structure is determined by the center coordinate p. i 'Capture, local neighborhood information is obtained by p' j '-p' capture, where "||" represents horizontal splicing, combining global shape structure and local neighborhood information to form a point pair p. i ′、p j The marginal relationship of ′.

[0041]

[0042] Furthermore, the features of each point are updated by aggregating all edge relationships of each point using max aggregation, resulting in new point features p. i-edge The calculation process is as shown in formula (2), and the final result is a point cloud containing edge relationships. F represents the information dimension of a point, and N represents the number of points.

[0043] For kernel-based convolution (KPConv), the input is the original input point cloud, i.e. p i =(x i ,y i ,z i ,r i ,g i ,b i The original input point cloud is processed using kernel point convolution (KPConv) to extract local information and obtain the final point cloud. The formula for KPConv is as follows:

[0044]

[0045] Where, x i and x is The point in, f i It is point x i Corresponding features is the radius neighborhood centered at x, where It is the selected radius. The key part is the neighboring position y of the center point x. i =x i The kernel function g with -x as input is defined as follows:

[0046]

[0047] The domain of g is the sphere. It is the core point (kernel points), y i =x i -x represents the neighboring locations of the center point x. It is to remove features from dimension D in Mapping to D out The correlation weight matrix. h is... and y i The correlation between them, when The closer to y i When the correlation is higher, this correlation should be higher; this paper uses linear correlation:

[0048]

[0049] The influence distance of the core point is σ = 1.0.

[0050] At the end of the point embedding layer, the point cloud contains edge relationships. and local feature point cloud The sum of the two is then processed through batch normalization and activation functions to obtain the input point cloud for the attention mechanism. at the same time, Jump to the attention module of all subsequent windows.

[0051] 1.2 Window Attention Module

[0052] Use multiple window attention modules to process point clouds. For example... Figure 3 As shown, the point cloud is divided into different windows, and the multi-head self-attention mechanism is restricted to computation within a local window. Points are scattered across different windows. Each query point only needs to consider its neighbors within the same window, rather than focusing on all points as in global self-attention. Multi-head self-attention executes independently within each window. Different windows may contain different numbers of points, and k... t This is represented as the number of points within the t-th window. Formally, assume N h It is the number of heads, N d N is the dimension of each head. c =N h ×N d Feature dimension, for the input point in the t-th window The multi-head self-attention formula in the t-th window is:

[0053] q = Linear q (x),k=Linear k (x),v=Linear v(x), (6)

[0054] attn i,j,h =q i,h ·k j,h ,

[0055] attn i,.,h =softmax(attn) i,.,h ),

[0056]

[0057] in The query, key, value, and · representation vector q are obtained from x through three linear layers. i,h and k j,h The dot product between them. It is an attention map. These are aggregated features, which are then projected onto the output features.

[0058] Furthermore, context-based adaptive relative position encoding (cRPE) is incorporated into the attention mechanism. In cRPE, for a point in the t-th window... We represent the xyz coordinates as Therefore, the relative xyz coordinates between query and key The formula is:

[0059] r i,j,m =p i,m -p j,m ,1≤i,j≤k t ,m∈{1,2,3}. (7)

[0060] To map relative coordinates to corresponding position codes, we maintain three learnable lookup tables. These correspond to the x-axis, y-axis, and z-axis, respectively. Since relative coordinates are continuous floating-point numbers, we will use r... i,j,m The range, i.e. (-s win ,s win ) is quantized into L discrete parts, and the relative coordinates r i,j,m Mapped to the following table:

[0061]

[0062] Where s win It refers to the window size. It is a quantization of size. This indicates rounding down.

[0063] We search these tables to retrieve the embeddings corresponding to the indexes and sum them to obtain the positional encoding. (9)

[0065] e i,j =t x [idx i,j,1 ]+t y [idx i,j,2 ]+t z [idx i,j,3 ]

[0066] in This represents the idx-th item in table t. It is a positional encoding.

[0067] In practice, the tables used for queries, keys, and values ​​are not shared. Therefore, we distinguish them by adding superscripts, where... This represents the x-axis table for the query. Similarly, the positional codes for query, key, and value are represented by e. q ,e k ,e v express.

[0068] Then, the positional encoding is multiplied by the query and key to obtain the positional offset. We then added this to the attention graph. Furthermore, we added value features with corresponding positional encodings, followed by weighting and aggregation. Finally, we updated the multi-head self-attention formula (6) in the window to a version with context-relative positional encoding (cRPE).

[0069]

[0070] like Figure 3 As shown, local geometric features Local features Context information When combined, the comprehensive local features are obtained, as shown in the following formula.

[0071]

[0072] Then, the local feature data (feats) are input into the displacement. The second window's attention module will process it.

[0073] Phase II: Downsampling + Two consecutive window attention modules

[0074] The features output from Stage I are downsampled, and then two consecutive window attention modules are used to further extract features. The second window is shifted.

[0075] Phase III: Downsampling + Six consecutive window attention modules

[0076] The features output from Stage II are downsampled, and then six consecutive window attention modules are used to further extract features, with each subsequent window shifted based on the previous window.

[0077] Phase IV: Downsampling + Two consecutive window attention modules

[0078] The features output from Stage III are downsampled, and then two consecutive window attention modules are used to further extract features. The second window is shifted.

[0079] Finally, the decoding operation is achieved by upsampling and aggregating the features from each stage, thus realizing semantic segmentation.

[0080] This invention combines graph neural networks and kernel convolutions to form an embedding layer, enabling the acquisition of more comprehensive, richer, and more realistic local information. Furthermore, a window attention mechanism is used to extract local contextual information. After each extraction of contextual information, the window is shifted by half its width. This window shifting operation continuously acquires contextual information from different regions, resulting in comprehensive contextual information. Simultaneously, the local geometric information extracted by the graph neural network is combined with the contextual information of each window and participates in the next window attention mechanism, achieving a deep fusion of local geometric features and the attention mechanism. This operation has proven highly effective, significantly improving segmentation accuracy.

[0081] Based on the same inventive concept, a second aspect of the present invention provides a 3D point cloud semantic segmentation system based on graph neural networks and window attention mechanisms, including...

[0082] The input point cloud reconstruction module is used to reconstruct the acquired input point cloud by swapping any two sets of target information positions of the points and adding an additional set of information positions.

[0083] The edge convolution extraction module is used to extract the edge relationships between points and their neighbors using edge convolution in a graph neural network, resulting in a point cloud containing these edge relationships. F represents the information dimension of a point, and N represents the number of points;

[0084] The kernel convolution processing module is used to process the input point cloud in step S1 using kernel convolution to extract local information of the point cloud and obtain the final point cloud. Point clouds containing edge relationships and local feature point cloud The input point cloud for the attention mechanism is obtained by summing the results and then processing them through batch normalization and activation functions.

[0085] The window attention mechanism module is used to execute the window attention mechanism and obtain context information. Will and By merging them together, a comprehensive set of local features can be obtained;

[0086] The semantic segmentation module is used to downsample the comprehensive local features under the window attention mechanism, then perform window attention multiple times, and realize the decoding operation by upsampling and aggregating features, and finally complete the semantic segmentation.

[0087] A third aspect of the present invention provides a machine-readable storage medium storing instructions that cause a machine to perform the steps of the three-dimensional point cloud semantic segmentation method based on graph neural networks and window attention mechanisms described above.

[0088] A fourth aspect of the present invention provides a processor for running a program, wherein the program is executed to perform: a three-dimensional point cloud semantic segmentation method based on graph neural networks and window attention mechanism.

[0089] Test Cases: Semantic segmentation results were obtained using the embodiments of this invention, as shown in Table 1 for S3DIS and as shown in Table 2 for ScanNetv2. As can be seen from Tables 1 and 2, on S3DIS, our performance significantly outperforms other models. The mean intersection-over-union ratio (mIoU) of our invention is 0.8% higher than the best model PTv2 in 2022, the mean accuracy (mAcc) is 1% higher than PTv2, and the overall accuracy (OA) reaches 91.8%, also surpassing PTv2. On ScanNetv2, our method's Val mIoU surpasses other methods, including voxel-based methods, and exceeds the voxel-based MinkowskiNet by 1.0% mIoU. The mean intersection-over-union ratio (Test mIoU) on the test set also reaches state-of-the-art levels. Figure 4 As shown, Input is the input point cloud, Ground Truth is the ground truth label of the input point cloud, Baseline is the semantic segmentation result of the baseline model, and Ours is the semantic segmentation result of this invention. Each color represents an object; for example, green represents the floor, and dark blue represents the wall. The comparison between the model of this invention and the baseline model demonstrates the superiority of this invention. The method of this invention can correctly identify the object highlighted in the yellow box, thanks to the deep integration of graph neural networks and window attention mechanisms.

[0090] Table 1 S3DIS semantic segmentation results

[0091]

[0092] Table 2 shows the results of ScanNetv2 semantic segmentation.

[0093]

[0094]

[0095] In summary, this invention has the following beneficial effects: 1) It designs asymmetric features, that is, it adds coordinate features on the basis of the original point cloud features, so that the point cloud features change from two sets of information (rgb-xyz) to three sets of information (rgb-xyz-xyz), which is beneficial for subsequent extraction of local geometric information; 2) The combination of graph neural network and kernel convolution can extract richer, more realistic and effective local information; 3) The window attention mechanism extracts local contextual information, and then combines it with the local geometric information extracted by graph neural network, which significantly improves the accuracy of semantic segmentation.

[0096] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings; however, the present invention is not limited thereto. Within the scope of the inventive concept, various simple modifications can be made to the technical solutions of the present invention, including combinations of various specific technical features in any suitable manner. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately. However, these simple modifications and combinations should also be considered as the content disclosed in the present invention and are all within the protection scope of the present invention.

Claims

1. A 3D point cloud semantic segmentation method based on graph neural networks and window attention mechanism, characterized in that, Includes the following steps: S1. Obtain the input point cloud; S2. Reconstruct the input point cloud, swap any two sets of target information positions of the points, and add an additional set of information positions; S3. Use edge convolution in a graph neural network to extract the edge relationships between points and their neighbors from the reconstructed point cloud, resulting in a point cloud containing these edge relationships. F represents the information dimension of a point, and N represents the number of points; S4. Process the input point cloud from step S1 using kernel convolution to extract local information of the point cloud and obtain the final point cloud. Point clouds containing edge relationships and local feature point cloud The input point cloud for the attention mechanism is obtained by summing the results and then processing them through batch normalization and activation functions. ; S5. Execute the window attention mechanism to obtain context information. ,Will and By merging them together, a comprehensive set of local features can be obtained; S6. Downsample the comprehensive local features under the window attention mechanism, then perform window attention multiple times, and achieve decoding operation by upsampling and aggregating features, finally completing semantic segmentation.

2. The method according to claim 1, characterized in that, In step S2, the positions of the coordinate and color information of the points are swapped, and an additional set of coordinate information is added. At this time, the i-th point is represented as... It contains a total of 9 dimensions of information, and the entire point cloud is represented as .

3. The method according to claim 2, characterized in that, In step S3, the step of extracting the edge relationships between points and their neighboring points using edge convolution in a graph neural network to obtain local geometric features specifically involves: S31. Using points as vertices of a graph The relationship between a point and its adjacent points is used as the edge of the graph. Construct a graph representing the local point cloud structure. , It is dynamically constructed using the nearest neighbors in the feature space, where N represents the number of points; S32, Extraction Image midpoint , and adjacent points The boundary relationship between them is defined by the following formula: ， in It has a set of learnable parameters A neural network is used to map pairs of 9-dimensional point cloud features to a specified F dimension. The edge function selection combines global shape structure and local neighborhood information into an asymmetric edge function, where the global shape structure is determined by the center coordinates. Capture, local neighborhood information is Capture, "||" indicates horizontal splicing, which combines global shape structure and local neighborhood information to form a point pair. Edge relationships; S33. Update the features of each point by aggregating all edge relationships of each point using max aggregation to obtain new point features. Ultimately, a point cloud containing edge relationships is obtained. New point features The calculation process and formula are as follows: 。 4. The method according to claim 2, characterized in that, In step S5, the execution window attention mechanism is used to obtain context information. Specifically, the process includes the following: S51. Divide the point cloud into different windows and restrict the multi-head self-attention mechanism to computation within the local window range. For the input points in the t-th window... The multi-head self-attention formula for the t-th window is: ， ， ， ， ， in The query, key, value, and · are obtained from x through three linear layers, representing a vector. and dot product between It is an attention map. These are aggregated features, which are then projected onto the output features. k t N represents the number of points within the t-th window. h It is the number of heads, N d It is the dimension of each head. Feature dimension; S52. Context-based adaptive relative position encoding (cRPE) is added to the attention mechanism. In cRPE, for a point in the t-th window... Representing the xyz coordinates as , The relative xyz coordinates between query and key The formula is: ； S53, will The range, that is Quantize into L discrete parts and convert relative coordinates Mapped to the following table: ， Where is s win It refers to the window size. It is a quantification of size. Indicates rounding down; S54. Three learnable lookup tables for lookup maintenance. , respectively corresponding to The x, y, and z axes are used to retrieve the embeddings corresponding to the index, and these are summed to obtain the position code. ， in This represents the idx-th item in table t. It is a positional encoding; S55. Use superscripts to distinguish between tables used for queries, keys, and values. The x-axis table represents the query, and the position codes for the query, key, and value are respectively represented by... This means that the positional encoding is then multiplied by the query and key to obtain the positional offset. The value features with corresponding position codes are added to the attention graph; then weighted and aggregated; finally, the multi-head self-attention formula in the t-th window is updated to the context-relative position code version formula. ， ， ， 。 5. The method according to claim 4, characterized in that, The , and The formula for obtaining comprehensive local feature data (feats) by combining them is as follows: 。 6. The method according to claim 4, characterized in that, Before proceeding to step S6, first shift the window position. Using the output features of the previous window as input, window attention is performed in the new window.

7. The method according to any one of claims 1-6, characterized in that, In step S4, the formula for kernel convolution is as follows: ， Where, x i and x is The point in the middle, It is point x i Corresponding features , is the radius neighborhood centered at x, where It is the selected radius, with the neighboring positions of the center point x. The kernel function g, which is used as input, is defined as follows: ， The domain of g is the sphere. , That's the core point. It is the location of the neighbor of the center point x. It is to change the dimension of features Mapped to The correlation weight matrix, h is and y i The correlation between them.

8. A 3D point cloud semantic segmentation system based on graph neural networks and window attention mechanism, characterized in that, include: The input point cloud reconstruction module is used to reconstruct the acquired input point cloud by swapping any two sets of target information positions of the points and adding an additional set of information positions. The edge convolution extraction module is used to extract the edge relationships between points and their neighbors using edge convolution in a graph neural network, resulting in a point cloud containing these edge relationships. F represents the information dimension of a point, and N represents the number of points; The kernel convolution processing module is used to process the input point cloud in step S1 using kernel convolution to extract local information of the point cloud and obtain the final point cloud. Point clouds containing edge relationships and local feature point cloud The input point cloud for the attention mechanism is obtained by summing the results and then processing them through batch normalization and activation functions. ; The window attention mechanism module is used to execute the window attention mechanism and obtain context information. ,Will and By merging them together, a comprehensive set of local features can be obtained; The semantic segmentation module is used to downsample the comprehensive local features under the window attention mechanism, then perform window attention multiple times, and realize the decoding operation by upsampling and aggregating features, and finally complete the semantic segmentation.

9. A machine-readable storage medium storing instructions that cause a machine to perform the steps of any one of the methods of claims 1-7 of this application.

10. A processor, characterized in that, Used to run a program, wherein the program is run to execute: the 3D point cloud semantic segmentation method based on graph neural networks and window attention mechanism as described in any one of claims 1-7.

Images