A PointNeXt point cloud segmentation method and system incorporating linear attention mechanism

Abstract

This invention proposes a PointNeXt point cloud segmentation method and system that integrates a linear attention mechanism, relating to the field of semantic segmentation technology. The method includes: acquiring target 3D point cloud data and inputting it into an improved PointNeXt network; using a progressive linear attention module to extract local features and model the global context of the target 3D point cloud data to obtain corresponding local and global point cloud features; performing channel-dimensional concatenation and nonlinear fusion to obtain a first intermediate feature; downsampling the first intermediate feature to obtain a downsampled feature; enhancing and dynamically weighting the downsampled feature using the channel attention and spatial attention mechanisms of an enhanced dual-attention multilayer perception module to obtain a target fused feature; and using the upsampled target fused feature to perform semantic segmentation of the target 3D point cloud data to determine the semantic label corresponding to each point in the target 3D point cloud data.

Description

Technical Field

[0001] This invention relates to the field of semantic segmentation technology, and in particular to a PointNeXt point cloud segmentation method and system that integrates a linear attention mechanism. Background Technology

[0002] Point cloud segmentation is a core task in 3D vision, aiming to assign semantic labels to each point. PointNeXt, as an efficient point cloud processing network, has shown application potential in multiple 3D vision fields due to its modular design and ability to handle the disorder of point clouds, such as autonomous driving and intelligent transportation, robot grasping and manipulation, medical image analysis, and virtual reality and game development. PointNeXt often uses multilayer perceptrons (MLP), hierarchical feature extraction modules (SetAbstraction), and inverse residual multilayer perceptrons (InvResMLP) for feature extraction, achieving hierarchical feature extraction from local to global. However, it has limitations in global context modeling and local feature enhancement, often ignoring the differences in the importance of features at different scales and dimensions in the point cloud, resulting in limited segmentation accuracy in complex scenes. Summary of the Invention

[0003] In view of this, the present invention proposes a PointNeXt point cloud segmentation method and system that integrates a linear attention mechanism.

[0004] The technical solution of this invention is implemented as follows: The first aspect of this invention provides a PointNeXt point cloud segmentation method that integrates a linear attention mechanism, implemented through an improved PointNeXt network; the PointNeXt network includes a progressive linear attention module, a hierarchical feature extraction module, and an enhanced dual-attention multilayer perception module; the method includes:

[0005] Acquire target 3D point cloud data and input the target 3D point cloud data into an improved PointNeXt network. Use the progressive linear attention module to extract local features and model the global context of the target 3D point cloud data to obtain the corresponding point cloud local features and point cloud global features. Then, perform channel dimension splicing and nonlinear fusion on the point cloud local features and the point cloud global features to obtain the first intermediate feature.

[0006] The first intermediate feature is downsampled using the hierarchical feature extraction module to obtain downsampled features; and the downsampled features are enhanced and dynamically weighted using the channel attention mechanism and spatial attention mechanism of the enhanced dual attention multilayer perception module to obtain target fusion features.

[0007] The target 3D point cloud data is semantically segmented using the target fusion features after upsampling to determine the semantic label corresponding to each point in the target 3D point cloud data.

[0008] Based on the above technical solutions, preferably, the progressive linear attention module includes a first multilayer perceptron, a linear attention submodule, and a second multilayer perceptron; the progressive linear attention module is used to extract local features and model the global context of the target 3D point cloud data to obtain corresponding point cloud local features and point cloud global features, and the point cloud local features and point cloud global features are then concatenated by channel dimensions and nonlinearly fused to obtain a first intermediate feature, including:

[0009] The local features of the point cloud are obtained by encoding local features of the target three-dimensional point cloud data through the first multilayer perceptron.

[0010] The local features of the point cloud are input into the linear attention submodule for feature extraction. The computational complexity of standard self-attention is reduced by combining the kernel function approximation method to obtain the global features of the point cloud.

[0011] After concatenating the local and global features of the point cloud along the channel dimension, the first intermediate feature is obtained by nonlinear fusion through the second multilayer perceptron.

[0012] Based on the above technical solutions, preferably, the step of inputting the local features of the point cloud into the linear attention submodule for feature extraction, and combining the kernel function approximation method to reduce the computational complexity of standard self-attention, to obtain the global features of the point cloud, includes:

[0013] The query matrix Q and key matrix K corresponding to the local features of the point cloud are nonlinearly transformed using a feature mapping function to obtain the mapped features;

[0014] The mapped features are dynamically normalized and weighted to obtain aggregated features;

[0015] By using a global gating mechanism to dynamically balance the local details and global context of the aggregated features, the global features of the point cloud are obtained.

[0016] Based on the above technical solutions, preferably, the step of downsampling the first intermediate feature using the hierarchical feature extraction module to obtain the downsampled feature includes:

[0017] Multiple key points are selected from the point cloud corresponding to the first intermediate feature as the center of the corresponding local region. By defining the local neighborhood, the point cloud is clustered into multiple point cloud combinations.

[0018] Local features are extracted from each point cloud combination using a multilayer perceptron to generate a local feature vector for each point cloud combination, thus obtaining downsampled features.

[0019] Based on the above technical solutions, preferably, the enhancement and dynamic weight fusion of the downsampled features through the channel attention mechanism and spatial attention mechanism of the enhanced dual-attention multilayer perception module to obtain the target fusion features includes:

[0020] The downsampled features are locally aggregated to obtain locally aggregated features;

[0021] The local aggregated features are enhanced using a channel attention mechanism to obtain a first enhanced feature; the first enhanced feature is then spatially enhanced using at least two parallel dilated convolution branches to obtain a second enhanced feature.

[0022] The first enhanced feature and the second enhanced feature are added element by element, and after global average pooling, they are input into a small weight network. Dynamic fusion weights are generated through the Softmax function, and the target fusion feature is generated after weighted fusion.

[0023] Based on the above technical solutions, preferably, the enhancement of the local aggregated features using a channel attention mechanism to obtain the first enhanced feature includes:

[0024] The local aggregated features are subjected to global average pooling to obtain the statistics of each channel, and the channel weights of each channel are generated by one-dimensional convolution.

[0025] The statistics of each channel are weighted and aggregated based on the channel weights to obtain the first enhanced feature.

[0026] Based on the above technical solutions, preferably, the step of using at least two parallel dilated convolution branches to perform spatial attention enhancement on the first enhanced feature to obtain the second enhanced feature includes:

[0027] The first enhanced feature is filtered by multiple parallel convolutional branches with different dilation rates to capture spatial context features of different ranges, and the spatial weight of each parallel convolutional branch is obtained by the Sigmoid function.

[0028] The spatial context features of each parallel convolutional branch are weighted and aggregated based on the spatial weights to obtain the second enhanced feature.

[0029] More preferably, a second aspect of the present invention provides a PointNeXt point cloud segmentation system incorporating a linear attention mechanism, comprising: a feature extraction module, a feature fusion module, and a semantic segmentation module; wherein,

[0030] The feature extraction module is configured to acquire target 3D point cloud data, input the target 3D point cloud data into an improved PointNeXt network, use the progressive linear attention module of the PointNeXt network to perform local feature extraction and global context modeling on the target 3D point cloud data to obtain corresponding point cloud local features and point cloud global features, and perform channel dimension splicing and nonlinear fusion on the point cloud local features and the point cloud global features to obtain the first intermediate feature;

[0031] The feature fusion module is configured to downsample the first intermediate feature using the hierarchical feature extraction module of the PointNeXt network to obtain downsampled features; and to enhance and dynamically weight the downsampled features using the channel attention mechanism and spatial attention mechanism of the enhanced dual attention multilayer perception module in the PointNeXt network to obtain target fusion features.

[0032] The semantic segmentation module is configured to perform semantic segmentation on the target 3D point cloud data using the target fusion features after upsampling, and determine the semantic label corresponding to each point in the target 3D point cloud data.

[0033] More preferably, a third aspect of the present invention provides an electronic device, including a processor and a memory; the memory has a computer program stored thereon, wherein the computer program, when executed by the processor, implements the PointNeXt point cloud segmentation method with fused linear attention mechanism as described in the first aspect.

[0034] More preferably, a fourth aspect of the present invention provides a non-transitory computer storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the PointNeXt point cloud segmentation method with fused linear attention mechanism as described in the first aspect.

[0035] The PointNeXt point cloud segmentation method and system of the present invention, which integrates a linear attention mechanism, has the following advantages over the prior art:

[0036] 1. By introducing a progressive linear attention module, global context modeling is performed on the target 3D point cloud data, capturing local geometric features of the point cloud from multiple angles. The attention mechanism is used to explicitly model long-distance dependencies between points in the point cloud, capturing global structural information. In the enhanced dual-attention multilayer perception module, channel attention and spatial attention mechanisms are introduced. The channel attention mechanism dynamically adjusts the weights of features from different channels, while the spatial attention mechanism balances the weights of features at different spatial locations. This fully considers the differences in the importance of features at different scales and dimensions in the point cloud, dynamically adjusting feature weights, thereby improving the model's segmentation accuracy and robustness in complex scenes.

[0037] 2. By utilizing the hierarchical attention mechanism of the progressive linear attention input module, point cloud dependencies are gradually established from local to global. Under the premise of ensuring linear complexity, kernel function approximation and global enhancement mechanism are used to achieve near-standard self-attention expressive ability, overcoming the common problem of accuracy loss in point cloud tasks caused by linear attention mechanisms. In complex scenarios, it can more accurately understand the overall layout of point clouds and reduce confusion between different semantic categories.

[0038] 3. A channel attention mechanism is employed to enhance local aggregated features, automatically learning the importance weights of features from different channels. This allows the model to focus more on features crucial to the segmentation task. Parallel dilated convolutions are used to simultaneously capture contextual information from different ranges within the point cloud, effectively perceiving everything from local fine structures to larger-scale scene layouts. Based on this, dynamic fusion is performed to ensure that valuable feature information is fully utilized, making the generated target fused features more representative and discriminative, thereby improving segmentation performance and generalization ability in various complex scenarios. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a PointNeXt point cloud segmentation method incorporating a linear attention mechanism, provided in an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the structure of the improved PointNeXt network provided in an embodiment of the present invention;

[0042] Figure 3A schematic diagram of the structure of a PointNeXt point cloud segmentation system that incorporates a linear attention mechanism, provided in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0045] In some embodiments, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating a PointNeXt point cloud segmentation method incorporating a linear attention mechanism, provided by an embodiment of the present invention. The PointNeXt point cloud segmentation method incorporating a linear attention mechanism is implemented using an improved PointNeXt network. The PointNeXt network includes a progressive linear attention module, a hierarchical feature extraction module, and an enhanced dual-attention multilayer perception module. The method includes:

[0046] S110: Acquire the target 3D point cloud data and input the target 3D point cloud data into the improved PointNeXt network. Use the progressive linear attention module to extract local features and model the global context of the target 3D point cloud data to obtain the corresponding point cloud local features and point cloud global features. Then, perform channel dimension splicing and nonlinear fusion on the point cloud local features and point cloud global features to obtain the first intermediate feature.

[0047] In this embodiment, a progressive linear attention module replaces the MLP layer in the original PointNeXt network. The progressive linear attention module employs a linear complexity attention mechanism to establish long-range dependencies in the point cloud. An enhanced dual-attention multilayer perception module, fusing channel and spatial attention, replaces the InvResMLP layer in the original PointNeXt network. This enhanced dual-attention multilayer perception module adaptively balances the contributions of channel and spatial attention through a dynamic weight fusion mechanism.

[0048] In some embodiments, the progressive linear attention module includes a first multilayer perceptron, a linear attention submodule, and a second multilayer perceptron; the progressive linear attention module is used to extract local features and model the global context of the target 3D point cloud data to obtain corresponding local and global point cloud features, and the local and global point cloud features are then concatenated along the channel dimension and fused nonlinearly to obtain a first intermediate feature, including:

[0049] The local features of the point cloud are obtained by encoding the local features of the target 3D point cloud data through the first multilayer perceptron.

[0050] The local features of the point cloud are input into the linear attention submodule for feature extraction. The kernel function approximation method is combined to reduce the computational complexity of standard self-attention and obtain the global features of the point cloud.

[0051] After concatenating the local and global features of the point cloud along the channel dimension, the first intermediate feature is obtained by nonlinear fusion through a second multilayer perceptron.

[0052] In this embodiment, a first multilayer perceptron encodes local features of the target 3D point cloud data, capturing the geometric relationships and attribute differences between points and their neighbors to obtain local point cloud features. Then, a linear attention mechanism replaces standard self-attention, and a kernel function approximation reduces computational complexity. The complexity of standard self-attention is... Linear attention can reduce its complexity to a minimum using kernel methods. Local and global features are concatenated along the channel dimension to form a higher-dimensional feature vector. A second multilayer perceptron then performs a nonlinear transformation on the concatenated features to learn the interaction between local and global features, thus obtaining the first intermediate feature. Here, the linear attention mechanism can employ architectures such as Performer or Linear Transformer, achieving efficient self-attention computation through kernel function approximation.

[0053] In some embodiments, local features of the point cloud are input into a linear attention submodule for feature extraction. A kernel function approximation method is used to reduce the computational complexity of standard self-attention, resulting in global features of the point cloud, including:

[0054] The query matrix Q and key matrix K corresponding to the local features of the point cloud are nonlinearly transformed using a feature mapping function to obtain the mapped features;

[0055] Dynamically normalize and weightedly aggregate the mapped features to obtain aggregated features;

[0056] By using a global gating mechanism to dynamically balance the local details and global context of aggregated features, global features of the point cloud are obtained.

[0057] The standard self-attention is calculated as follows:

[0058] ;

[0059] Where Q, K, and V are the query, key, and value matrices, respectively, with dimensions of [missing information]. N is the number of points, and d is the feature dimension. and Complexity-related.

[0060] Linear attention reduces computational complexity through kernel function approximation; specifically, it uses feature mapping. Mapping Q and K to a new space allows attention computation to be written as:

[0061] ;

[0062] here, Exponential functions or linear mappings are typically used. We can use linear mappings in the following ways:

[0063] ;

[0064] The output of linear attention can be represented as:

[0065] ;

[0066] in: , It is the sigmoid function, and MLP is a multilayer perceptron. An MLP module for extracting global contextual features. It is a global context enhancement feature. It is a globally gated variable. It is a linear attention output.

[0067] The process of obtaining global features of a point cloud can be specifically achieved by extracting local features of the point cloud through a local MLP, then using linear attention to capture the global context, and finally fusing the local and global features.

[0068] Let the input features be Then the local features are: The global features are: .

[0069] in, Characterize linear attention in point clouds;

[0070] The fusion features are:

[0071] .

[0072] S120: The first intermediate feature is downsampled using the hierarchical feature extraction module to obtain the downsampled feature; and the downsampled feature is enhanced and dynamically weighted by the channel attention mechanism and spatial attention mechanism of the enhanced dual attention multilayer perception module to obtain the target fusion feature.

[0073] In some embodiments, the hierarchical feature extraction module is used to downsample the first intermediate feature to obtain the downsampled feature, including:

[0074] Multiple key points are selected from the point cloud corresponding to the first intermediate feature as the center of the corresponding local region. By defining the local neighborhood, the point cloud is clustered into multiple point cloud combinations.

[0075] A multilayer perceptron is used to extract local features from each point cloud combination, generating a local feature vector for each point cloud combination, thus obtaining downsampled features.

[0076] In this embodiment, for each keypoint, a fixed radius value can be set, and all points less than or equal to the radius value from the keypoint are classified as points within the local neighborhood of the keypoint, forming a point cloud combination. Alternatively, the K nearest points to the keypoint can be selected as points within its local neighborhood, forming a point cloud combination. K is a pre-set parameter. All selected keypoints are traversed, and according to the neighborhood definition method, the points in the point cloud are assigned to the corresponding point cloud combinations containing the keypoints. Each point cloud combination represents a local region, including the central keypoint and its surrounding neighborhood points. The feature matrix of each point cloud combination is sequentially input into a pre-designed multilayer perceptron. After multilayer forward propagation calculations, a local feature vector for each point cloud combination is generated, yielding downsampled features.

[0077] In some embodiments, the downsampled features are enhanced and dynamically weighted using the channel attention and spatial attention mechanisms of the enhanced dual-attention multilayer perception module to obtain target fusion features, including:

[0078] Local aggregation is performed on the downsampled features to obtain locally aggregated features;

[0079] The local aggregated features are enhanced using a channel attention mechanism to obtain the first enhanced feature; the first enhanced feature is then spatially enhanced using at least two parallel dilated convolution branches to obtain the second enhanced feature.

[0080] The first and second enhanced features are added element by element, then global average pooling is applied and the result is input into a small weighted network. The Softmax function is used to generate dynamic fusion weights, which are then weighted and fused to generate the target fusion feature.

[0081] The channel attention mechanism inputs compressed information into a small network consisting of fully connected layers and activation functions. First, a fully connected layer reduces the dimensionality, then a ReLU activation function is applied, and finally another fully connected layer restores the dimensionality to the number of channels. A sigmoid activation function is then used to obtain the weights for each channel. The spatial attention mechanism performs dilated convolutions on the first enhanced feature at each dilated convolution branch. Spatial attention is calculated after the dilated convolution output of each branch. This can be achieved by performing max pooling and average pooling on the features at each spatial location along the channel dimension, concatenating the two, and then passing the convolutions through a convolutional layer and a sigmoid activation function to obtain the spatial attention weights.

[0082] In some embodiments, a channel attention mechanism is used to enhance the local aggregated features to obtain a first enhanced feature, including:

[0083] Global average pooling is performed on the local aggregated features to obtain the statistics for each channel, and one-dimensional convolution is used to generate the channel weights for each channel.

[0084] The statistics of each channel are weighted and aggregated based on the channel weights to obtain the first enhanced feature.

[0085] In this embodiment, operations such as KNN graph convolution, BallQuery, or max pooling are used to aggregate neighborhood information. Let the input point be P and the feature be F, then the local aggregated feature can be represented as:

[0086] .

[0087] Efficient Channel Attention (ECA) is used in combination with Global Average Pooling (GAP) and one-dimensional convolution to generate channel weights.

[0088] GAP calculates statistics for each channel:

[0089] ;

[0090] Capturing inter-channel relationships using one-dimensional convolution, kernel size Adaptive selection based on the number of channels C:

[0091] ;

[0092] in, b are hyperparameters. This indicates taking the nearest odd number, for example: .

[0093] The channel weights are obtained using the Sigmoid activation function:

[0094] ;

[0095] The first enhanced feature can be represented as:

[0096] .

[0097] In some embodiments, spatial attention enhancement is applied to the first enhanced feature using at least two parallel dilated convolution branches to obtain the second enhanced feature, including:

[0098] The first enhanced feature is filtered by multiple parallel convolutional branches with different dilation rates to capture spatial context features of different ranges, and the spatial weight of each parallel convolutional branch is obtained by the Sigmoid function.

[0099] The spatial context features of each parallel convolutional branch are weighted and aggregated based on spatial weights to obtain the second enhanced feature.

[0100] In this embodiment, multiple parallel convolutional branches are used, each with a different dilation rate. For each branch i, the dilation rate is... The kernel size can generally be set to 3:

[0101] ;

[0102] The outputs of multiple branches are summed or averaged, and then the spatial weights are obtained using the sigmoid function.

[0103] ;

[0104] The second enhancement feature can be represented as:

[0105] .

[0106] The first and second enhanced features are added element-wise, and then two weights are generated by passing them through a small weight network (typically including GAP and MLP). and These are used for the output of channel attention and spatial attention, respectively.

[0107] Let the channel attention output be Spatial attention output is The fusion weights are calculated as follows:

[0108] ;

[0109] The target fusion feature can be represented as:

[0110] ;

[0111] Through a pointwise convolution (1x1 convolution) for feature transformation:

[0112] ;

[0113] The features obtained after pointwise convolution are compared with the input features. Perform residual connections to obtain the final output of the enhanced dual-attention multilayer perception module:

[0114] .

[0115] S130: Use the target fusion features after upsampling to perform semantic segmentation on the target 3D point cloud data and determine the semantic label corresponding to each point in the target 3D point cloud data.

[0116] In this embodiment, the downsampled target fusion features can be restored to the original point cloud resolution through the feature propagation module, so as to assign a semantic label to each point. Upsampling restores the spatial resolution of the 3D point cloud data. At the same time, feature propagation fuses multi-scale information. The combination of the two can take into account both global semantics and local details, thereby ensuring the accuracy of semantic segmentation.

[0117] In an optional embodiment, please refer to Figure 2 , Figure 2 This is a schematic diagram of the improved PointNeXt network provided in this embodiment of the invention. The progressive attention module, as the input port module, captures important relationships between different points in the point cloud through an attention mechanism, providing a more discriminative feature representation for subsequent processing. Here, the input dimension [N, F] represents N points, each with F-dimensional features. The hierarchical feature extraction module performs hierarchical processing on the input point cloud features, gradually reducing the number of points while increasing the feature dimension of each point, thus reducing the amount of data while extracting higher-level and more representative features. The enhanced dual-attention multilayer perception module is combined with the hierarchical feature extraction module. First, the enhanced dual-attention multilayer perception module captures the complex relationships and important features between points, and then the hierarchical feature extraction module performs feature aggregation and abstraction on the local point set to extract feature information of the local region. The feature propagation module is used to restore the downsampled features to the resolution of the original point cloud. Through upsampling operations, such as interpolation and deconvolution, the number of points is gradually increased. At the same time, it can combine shallow features for feature fusion to ensure that the restored features not only have high-level semantic information but also retain low-level geometric details, ultimately outputting a feature representation with the same dimensions as the original point cloud.

[0118] In some embodiments, please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a PointNeXt point cloud segmentation system incorporating a linear attention mechanism, provided in an embodiment of the present invention. The present invention provides a PointNeXt point cloud segmentation system 300 incorporating a linear attention mechanism, comprising: a feature extraction module 310, a feature fusion module 320, and a semantic segmentation module 330; wherein,

[0119] The feature extraction module 310 is configured to acquire target 3D point cloud data and input the target 3D point cloud data into an improved PointNeXt network. The progressive linear attention module of the PointNeXt network is used to extract local features and model the global context of the target 3D point cloud data to obtain the corresponding point cloud local features and point cloud global features. The point cloud local features and point cloud global features are then spliced ​​together by channel dimension and fused nonlinearly to obtain the first intermediate feature.

[0120] The feature fusion module 320 is configured to downsample the first intermediate feature using the hierarchical feature extraction module of the PointNeXt network to obtain the downsampled feature; and to enhance and dynamically weight the downsampled feature through the channel attention mechanism and spatial attention mechanism of the enhanced dual attention multilayer perception module in the PointNeXt network to obtain the target fusion feature.

[0121] The semantic segmentation module 330 is configured to perform semantic segmentation on the target 3D point cloud data using the target fusion features after upsampling, and determine the semantic label corresponding to each point in the target 3D point cloud data.

[0122] In some embodiments, the progressive linear attention module includes a first multilayer perceptron, a linear attention submodule, and a second multilayer perceptron; the feature extraction module 310 is specifically configured as follows:

[0123] The local features of the point cloud are obtained by encoding the local features of the target 3D point cloud data through the first multilayer perceptron.

[0124] The local features of the point cloud are input into the linear attention submodule for feature extraction. The kernel function approximation method is combined to reduce the computational complexity of standard self-attention and obtain the global features of the point cloud.

[0125] After concatenating the local and global features of the point cloud along the channel dimension, the first intermediate feature is obtained by nonlinear fusion through a second multilayer perceptron.

[0126] In some embodiments, the feature extraction module 310 is specifically configured as follows:

[0127] The query matrix Q and key matrix K corresponding to the local features of the point cloud are nonlinearly transformed using a feature mapping function to obtain the mapped features;

[0128] Dynamically normalize and weightedly aggregate the mapped features to obtain aggregated features;

[0129] By using a global gating mechanism to dynamically balance the local details and global context of aggregated features, global features of the point cloud are obtained.

[0130] In some embodiments, the feature fusion module 320 is specifically configured as follows:

[0131] Multiple key points are selected from the point cloud corresponding to the first intermediate feature as the center of the corresponding local region. By defining the local neighborhood, the point cloud is clustered into multiple point cloud combinations.

[0132] A multilayer perceptron is used to extract local features from each point cloud combination, generating a local feature vector for each point cloud combination, thus obtaining downsampled features.

[0133] In some embodiments, the feature fusion module 320 is specifically configured as follows:

[0134] Local aggregation is performed on the downsampled features to obtain locally aggregated features;

[0135] The local aggregated features are enhanced using a channel attention mechanism to obtain the first enhanced feature; the first enhanced feature is then spatially enhanced using at least two parallel dilated convolution branches to obtain the second enhanced feature.

[0136] The first and second enhanced features are added element by element, then global average pooling is applied and the result is input into a small weighted network. The Softmax function is used to generate dynamic fusion weights, which are then weighted and fused to generate the target fusion feature.

[0137] In some embodiments, the feature fusion module 320 is specifically configured as follows:

[0138] Global average pooling is performed on the local aggregated features to obtain the statistics for each channel, and one-dimensional convolution is used to generate the channel weights for each channel.

[0139] The statistics of each channel are weighted and aggregated based on the channel weights to obtain the first enhanced feature.

[0140] In some embodiments, the feature fusion module 320 is specifically configured as follows:

[0141] The first enhanced feature is filtered by multiple parallel convolutional branches with different dilation rates to capture spatial context features of different ranges, and the spatial weight of each parallel convolutional branch is obtained by the sigmoid function.

[0142] The spatial context features of each parallel convolutional branch are weighted and aggregated based on spatial weights to obtain the second enhanced feature.

[0143] It should be noted that the PointNeXt point cloud segmentation system with fused linear attention mechanism provided in this application embodiment and the PointNeXt point cloud segmentation method with fused linear attention mechanism provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned PointNeXt point cloud segmentation method with fused linear attention mechanism, and the repeated parts will not be described again.

[0144] In some embodiments, please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 400 provided in this embodiment includes a processor 410 and a memory 420; the memory 420 stores a computer program, wherein the computer program, when executed by the processor, implements the aforementioned PointNeXt point cloud segmentation method incorporating a linear attention mechanism.

[0145] Specifically, processor 410 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 410 may also include onboard memory for caching purposes. Processor 410 may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0146] Memory 420 may be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory 420 may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory 420 include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and may also be random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0147] This application also provides a non-transitory computer-readable medium storing a computer program that, when executed by a processor, implements the PointNeXt point cloud segmentation method with a fused linear attention mechanism described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0148] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0149] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A PointNeXt point cloud segmentation method fused with linear attention mechanism, characterized in that, This is achieved through an improved PointNeXt network; the PointNeXt network includes a progressive linear attention module, a hierarchical feature extraction module, and an enhanced dual-attention multilayer perception module; the method includes: The process involves acquiring target 3D point cloud data and inputting it into an improved PointNeXt network. A progressive linear attention module is used to extract local features and model the global context of the target 3D point cloud data, yielding corresponding local and global point cloud features. These local and global features are then concatenated along the channel dimension and nonlinearly fused to obtain a first intermediate feature. The progressive linear attention module comprises a first multilayer perceptron, a linear attention submodule, and a second multilayer perceptron. The process of using the progressive linear attention module to extract local features and model the global context of the target 3D point cloud data is described above. Modeling is performed to obtain corresponding local and global point cloud features. The local and global point cloud features are then concatenated along the channel dimension and nonlinearly fused to obtain a first intermediate feature. This process includes: encoding local features of the target 3D point cloud data using a first multilayer perceptron to obtain local point cloud features; inputting the local point cloud features into the linear attention submodule for feature extraction, and using a kernel function approximation method to reduce the computational complexity of standard self-attention to obtain global point cloud features; concatenating the local and global point cloud features along the channel dimension, and then nonlinearly fusing them using a second multilayer perceptron to obtain the first intermediate feature. The first intermediate feature is downsampled using the hierarchical feature extraction module to obtain downsampled features; and the downsampled features are enhanced and dynamically weighted using the channel attention mechanism and spatial attention mechanism of the enhanced dual attention multilayer perception module to obtain target fusion features. The target 3D point cloud data is semantically segmented using the target fusion features after upsampling to determine the semantic label corresponding to each point in the target 3D point cloud data.

2. The PointNeXt point cloud segmentation method with fused linear attention mechanism as described in claim 1, characterized in that, The step of inputting the local features of the point cloud into the linear attention submodule for feature extraction, and combining the kernel function approximation method to reduce the computational complexity of standard self-attention, yields the global features of the point cloud, including: The query matrix Q and key matrix K corresponding to the local features of the point cloud are nonlinearly transformed using a feature mapping function to obtain the mapped features; The mapped features are dynamically normalized and weighted to obtain aggregated features; By using a global gating mechanism to dynamically balance the local details and global context of the aggregated features, the global features of the point cloud are obtained.

3. The PointNeXt point cloud segmentation method incorporating a linear attention mechanism as described in claim 1, characterized in that, The step of downsampling the first intermediate feature using the hierarchical feature extraction module to obtain downsampled features includes: Multiple key points are selected from the point cloud corresponding to the first intermediate feature as the center of the corresponding local region. By defining the local neighborhood, the point cloud is clustered into multiple point cloud combinations. Local features are extracted from each point cloud combination using a multilayer perceptron to generate a local feature vector for each point cloud combination, thus obtaining downsampled features.

4. The PointNeXt point cloud segmentation method with fused linear attention mechanism as described in claim 1, characterized in that, The enhanced dual-attention multilayer perception module enhances and dynamically weights the downsampled features to obtain target fusion features, including: The downsampled features are locally aggregated to obtain locally aggregated features; The local aggregated features are enhanced using a channel attention mechanism to obtain a first enhanced feature; the first enhanced feature is then spatially enhanced using at least two parallel dilated convolution branches to obtain a second enhanced feature. The first enhanced feature and the second enhanced feature are added element by element, and after global average pooling, they are input into a small weight network. Dynamic fusion weights are generated through the Softmax function, and the target fusion feature is generated after weighted fusion.

5. The PointNeXt point cloud segmentation method with fused linear attention mechanism as described in claim 4, characterized in that, The method of using channel attention mechanism to enhance the local aggregated features to obtain the first enhanced feature includes: The local aggregated features are subjected to global average pooling to obtain the statistics of each channel, and the channel weights of each channel are generated by one-dimensional convolution. The statistics of each channel are weighted and aggregated based on the channel weights to obtain the first enhanced feature.

6. The PointNeXt point cloud segmentation method with fused linear attention mechanism as described in claim 4, characterized in that, The second enhanced feature is obtained by spatial attention enhancement of the first enhanced feature using at least two parallel dilated convolution branches, including: The first enhanced feature is filtered by multiple parallel convolutional branches with different dilation rates to capture spatial context features of different ranges, and the spatial weight of each parallel convolutional branch is obtained by the Sigmoid function. The spatial context features of each parallel convolutional branch are weighted and aggregated based on the spatial weights to obtain the second enhanced feature.

7. A PointNeXt point cloud segmentation system incorporating a linear attention mechanism, characterized in that, include: The module consists of a feature extraction module, a feature fusion module, and a semantic segmentation module; among which, The feature extraction module is configured to acquire target 3D point cloud data, input the target 3D point cloud data into an improved PointNeXt network, and use the progressive linear attention module of the PointNeXt network to perform local feature extraction and global context modeling on the target 3D point cloud data to obtain corresponding point cloud local features and point cloud global features. The point cloud local features and point cloud global features are then concatenated along the channel dimension and nonlinearly fused to obtain a first intermediate feature. The progressive linear attention module includes a first multilayer perceptron, a linear attention submodule, and a second multilayer perceptron. Specifically, the feature extraction module is configured to: encode local features of the target 3D point cloud data using the first multilayer perceptron to obtain point cloud local features; input the point cloud local features into the linear attention submodule for feature extraction, and combine kernel function approximation methods to reduce the computational complexity of standard self-attention to obtain point cloud global features; concatenate the point cloud local features and point cloud global features along the channel dimension, and then perform nonlinear fusion using the second multilayer perceptron to obtain the first intermediate feature. The feature fusion module is configured to downsample the first intermediate feature using the hierarchical feature extraction module of the PointNeXt network to obtain downsampled features; and to enhance and dynamically weight the downsampled features using the channel attention mechanism and spatial attention mechanism of the enhanced dual attention multilayer perception module in the PointNeXt network to obtain target fusion features. The semantic segmentation module is configured to perform semantic segmentation on the target 3D point cloud data using the target fusion features after upsampling, and determine the semantic label corresponding to each point in the target 3D point cloud data.

8. An electronic device comprising a processor and a memory; said memory having a storage for a computer program, wherein, When the computer program is executed by the processor, it implements the PointNeXt point cloud segmentation method with fused linear attention mechanism as described in any one of claims 1 to 6.

9. A non-transitory computer storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the PointNeXt point cloud segmentation method with a fused linear attention mechanism as described in any one of claims 1 to 6.

Images