3D point cloud classification segmentation method based on dynamic edge convolution and residual error double attention

Abstract

The invention relates to the technical field of image processing, in particular to a 3D point cloud classification segmentation method based on dynamic edge convolution and residual double attention, which comprises the following steps: acquiring an image to be processed; a 3D-AGCN network is constructed; feature alignment is carried out by using a Transform encoder, feature groups are obtained through a plurality of EADEC, RDA and AMFE modules, the feature groups are connected in series, and then convolution compression is carried out to obtain shared features; the classification branch performs global maximum pooling and average pooling on the shared features, and outputs classification scores; and the segmentation branches repeat the shared features and the one-hot category vectors to each point and then splice the shared features and the one-hot category vectors, and a segmentation score is output. The method solves the problem that an existing method still needs to be improved in the aspects of effectively fusing local details and global contexts and optimizing model performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a 3D point cloud classification and segmentation method based on dynamic edge convolution and residual double attention. BACKGROUND

[0002] Three-dimensional (3D) point clouds have become the main representation of object geometry and spatial structure, widely used in robotics, 3D object recognition and virtual reality, etc. In recent years, with the rapid development of 3D sensing technology, the amount of point cloud data has exploded, providing rich and detailed geometric information for representing real-world environments. A core challenge in 3D computer vision is to perform semantic segmentation and classification on point clouds, aiming to accurately assign semantic labels to each point or identify and classify complete objects from raw 3D data.

[0003] Traditional deep learning techniques such as convolutional neural networks (CNNs) are good at handling regular data structures, but their effectiveness is limited when dealing with unordered point clouds. Early methods convert point clouds into regular grids (such as voxel grids or two-dimensional projections) to adapt to traditional CNN architectures. Although these methods are effective on small-scale data, the conversion process leads to loss of spatial information and increased computational complexity, limiting their scalability and accuracy on larger and more complex point clouds.

[0004] To solve this problem, point-based methods directly operate on raw point cloud data, skipping the step of converting to regular grids. These methods preserve fine-grained spatial details and improve segmentation performance. In addition, graph-based methods utilize point-to-point relationships to capture larger-scale contextual information, enhancing segmentation results. However, these methods still have limitations in handling large variations in point density and cross-class generalization capabilities.

[0005] Recent research shows that Transformer models have achieved great success in natural language processing and two-dimensional image analysis, and can also be effectively applied to point cloud data. Transformers are good at modeling long-range dependencies and capturing global context, which is particularly important for understanding complex structures. For example, Point Cloud Transformer (PCT) and Point Transformer (PT) use self-attention mechanisms to simulate long-range point-to-point relationships, improving segmentation and classification performance. Although Transformers have made significant progress, their ability to capture fine-grained local features, especially in small-scale point clouds, remains a challenge.

[0006] In summary, significant progress has been made in point cloud segmentation and classification for smaller datasets using both point basis methods and Transformer methods. These methods have shown potential in overcoming the challenges of point cloud irregularities and density variations, but there is still room for improvement in how to effectively integrate local details with global context and optimize model performance. Summary of the Invention

[0007] To address the shortcomings of existing methods, this invention solves the problem that existing methods still need improvement in effectively integrating local details with global context and optimizing model performance.

[0008] The technical solution adopted in this invention is: a 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention, comprising the following steps: Step 1: Acquire the image to be processed; In a preferred embodiment of the present invention, the image to be processed is a 3D point cloud image.

[0009] Step 2: Construct a 3D-AGCN network; use the Transform encoder to align features, and after passing through several EADEC, RDA and AMFE modules, obtain feature groups. Concatenate the feature groups and perform convolutional compression to obtain shared features; the classification branch performs global max pooling and average pooling on the shared features and outputs the classification score; the segmentation branch repeats the shared features and one-hot class vectors at each point and then concatenates them to output the segmentation score.

[0010] As a preferred embodiment of the present invention, the 3D-AGCN network includes: The image to be processed is sequentially input into the Transform encoder, the first EADEC module, and the first RDA module; one output of the first RDA module is sequentially connected to the second EADEC module, the second RDA module, and the first AMFE module; one output of the first AMFE module is sequentially connected to the third EADEC module, the third RDA module, and the second AMFE module; one output of the second AMFE module is sequentially connected to the fourth EADEC module and the fourth RDA module; the output of the fourth RDA module is concatted with the other outputs of the first AMFE module, the second AMFE module, and the first RDA module; the output of the concat operation is concatenated with the first Conv convolution, and one output of the first Conv convolution is fed into the SegHead MLP module after Cat Vector operation and Repeat-N operation, along with the outputs of the second RDA module, the third RDA module, and the fourth RDA module.

[0011] In a preferred embodiment of the present invention, the other end of the output of the first Conv convolution is input into GAP and GMP respectively, and then Concat operation is performed before passing through the MLP module.

[0012] In a preferred embodiment of the present invention, the AMFE module includes: The input feature map is passed through three branches of DwConv with kernels of 1 × 1, 3 × 3, and 5 × 5, and then through a 1x1 PWConv. The outputs of the three branches of PWConv are then summed point by point with weights.

[0013] In a preferred embodiment of the present invention, the EADEC module includes: Let X = [ x 1 ,...,x N ] ∈ R C × N Given a point cloud as input, calculate the kernel-weighted local density based on the point cloud coordinates and features. ; Will Mapping to adaptive neighborhood size ; gamma Control the slope; µ To achieve adaptive normalization for the batch average density; and Set the lower and upper limits for the neighborhood size respectively; It is an adaptive neighborhood size Sigmoid mapping; Building dynamic neighborhoods Then calculate the high-dimensional side features of the difference-attention convolution. , weight ; Construct a three-branch MLP, corresponding to the output , To share the MLP weight matrix, the output of the three-branch MLP is performed with a Batch Normalization (BN) operation and then concatenated with X to obtain: , It is a weight matrix.

[0014] In a preferred embodiment of the present invention, the RDA module includes: Input feature map After three branches, the output of the first branch is: , sigma It is the Sigmoid activation function. The channel convolution kernel is denoted by GAP(X), which represents global average pooling. Output of the second branch W s Spatial convolution kernel; The third branch is The combined output of the three branches is X. attn= X⊙ ⊙ ,⊙ represents element-wise multiplication; The fusion output of the three branches and Add the residuals together.

[0015] As a preferred embodiment of the present invention, a 3D point cloud classification and segmentation system based on dynamic edge convolution and residual dual attention includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention.

[0016] As a preferred embodiment of the present invention, a computer-readable medium storing computer program code implements a 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention when executed by a processor.

[0017] The beneficial effects of this invention are: 1. By using key modules such as residual dual attention mechanism, adaptive multi-scale feature extraction module and adaptive dynamic edge convolution, the model's feature representation ability and generalization performance for complex point cloud data are effectively improved. 2. The residual dual attention mechanism combines channel and spatial attention and introduces residual connections to avoid information suppression and gradient vanishing. 3. The adaptive multi-scale feature extraction module uses multi-scale convolution and adaptive weighting to flexibly capture features at different scales; the adaptive dynamic edge convolution improves the traditional method by introducing adaptive neighborhood size and differential convolution in conjunction with the multi-scale strategy to accurately capture the geometric relationship between points; 4. The method of this invention achieves significant improvements in both 3D point cloud classification and segmentation tasks. In classification tasks, dynamic edge convolution and multi-scale feature fusion enable the model to perform well in terms of balancing and complementing global and local features, thereby improving classification accuracy. In segmentation tasks, attention mechanism and adaptive neighborhood selection enhance the ability to recognize detailed structures and complex topologies, achieving higher average intersection-union ratio (mIoU) and classification accuracy. 5. 3D point cloud data faces challenges such as noise interference, uneven density, and variable shape and scale. The network proposed in this invention provides a practical solution by using mechanisms such as residual attention, dynamic edge convolution, and multi-scale fusion. The combination of adaptive neighborhood and attention can cope with changes in point cloud density and highlight key geometric details. Multi-scale convolution ensures that the network can take into account both local and global feature representation capabilities, laying the foundation for further research on point cloud-related 3D vision tasks. Attached Figure Description

[0018] Figure 1 This is a flowchart of the 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention of the present invention; Figure 2 This is the AMFE module of the present invention; Figure 3 This is the EADEC module of the present invention; Figure 4 This is the RDA module of the present invention; Figure 5 This is a schematic diagram of the DGCNN model; Figure 6 This is a comparison of results from different methods on ModelNet40; Figure 7 This is a comparison of the accuracy of the present invention with that of the DGCNN model; Figure 8 This is a comparison between the present invention and the DGCNN model Gray; Figure 9 This invention compares the recall of the DGCNN model with that of the present invention; Figure 10 These are the effect images of different point cloud processing methods of the present invention; Figure 11 These are images showing the results of DGCNN processing different point clouds; Figure 12 These are PointNet renderings of different point cloud processing results; Figure 13 These are the results of segmenting different point clouds using the method of this invention; Figure 14 These are images showing the results of DGCNN segmenting different point clouds; Figure 15 These are PointNet segmentation results for different point clouds. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments. The drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0020] The 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention includes the following steps: Dynamic Graph CNN (DGCNN) is characterized by "dynamic graph building". + Edge convolution ( EdgeConv The core concept is "reconstruction of the network at each layer based on the current features". k -NN graph, to achieve a closed loop of "feature update → neighborhood reconstruction"; Figure 5 This demonstrates the complete data stream and tensor size; let the first... l The layer input feature is H (l) ∈R B × C × NFirst, search the neighborhood in the feature space. Then, for the center point x i and its neighboring points x j Constructing differential splicing edge features, the process is represented by formula (1): (1) The first term explicitly characterizes the local geometric gradient, while the second term preserves the central context; all edge features are processed through a shared multilayer perceptron. (·) The nonlinear mapping is obtained as shown in formula (2): (2) Then, element-wise max pooling is performed within the neighborhood, as shown in formula (3): (3) This allows for the acquisition of new node representations and the commencement of the next round of dynamic graph construction; as the number of layers increases, the network gradually aggregates discriminative geometry and global semantics.

[0021] DGCNN laid the technological foundation for the 3D-AGCN proposed in this invention: dynamic mapping provides a pluggable neighborhood adaptation interface, paving the way for the introduction of density-aware neighborhoods in the EADEC module; the differential coding concept inspired the dual attention design of the RDA module in both spatial and channel dimensions; and the multi-layer feature aggregation framework enabled the AMFE module to naturally integrate multi-scale contexts; understanding the working principle of DGCNN is therefore a prerequisite for explaining the three upgrades of 3D-AGCN.

[0022] Step 1: Acquire the image to be processed; The image to be processed is a 3D point cloud image; the image is segmented and classified. Step 2: Construct a 3D-AGCN network; Figure 1 The complete data flow of 3D-AGCN on classification and segmentation tasks is presented; the middle horizontal channel is a shared skeleton; the top branch outputs the classification score, and the bottom branch outputs the segmentation score. The network repeatedly stacks EADEC (Edge-Aware DynamicEdgeConv), RDA (Residual Dual Attention), and AMFE (Adaptive Multi-Scale Feature Extraction) in four stages to progressively strengthen local geometry, global semantics, and scale context.

[0023] Staged feature extraction input points TransformAfter encoder alignment, the feature set {F1} is obtained by sequentially passing through four EADEC → RDA → AMFE modules. , F2 , F3 , F4}; they are concatenated in the channel dimension as F concat ∈ R B × 704 × N Then, it is compressed into 1024-dimensional shared features through 1×1 convolution; the shared features of the classification branches are then subjected to global max pooling and average pooling respectively, and concatenated to form g ∈ R. B × 2048 Two fully connected layers with batch normalization and Dropout (512→256→ C Output classification output scores ∈ R B × C The splitting branch concatenates the 1024-dimensional shared features and one-hot class vectors at each point to form H. seg ∈ R B × 1088 × N At the same time, put F2 , F3 , F4 is injected via cross-layer jumper. Segmentation Head (MLP(256) , 256 , 128 ,P Output segmentation output scores ∈ R B × P × N If further downsampling is required, FPS pooling can be inserted and interpolation can be used to recover dense predictions; EADEC retains edge gradients through density-adaptive neighborhood; RDA suppresses noise and highlights key regions with channel and spatial attention; AMFE alleviates sampling unevenness and scale mixing by using three-scale convolution.

[0024] Figure 1 In the example, EADEC64→64, k neighbors are adaptively selected for each point, and three MLP branches are parallel (64→32). After concat, 96 is obtained. Then, the residual mapping is added to keep 96. The purpose is to enrich the local edge relationships without changing the overall bandwidth.

[0025] For example, RDA64 utilizes channel attention (GAP→1×1Conv→σ), spatial attention (DW 3×3→σ), point-by-point recalibration + residual → maintain 96, with the aim of emphasizing meaningful channel / spatial responses.

[0026] For example, AMFE-128 (key broadening point) uses depthwise separable convolutions at three scales (1×1, 3×3, and 5×5) to perform 96→96 for each branch. Then, 1×1 Conv is used to reweight the features. After adding the features at multiple scales, 1×1 Conv is used to perform 96→128. The goal is to complete the channel expansion 96→128 without significantly increasing the number of parameters.

[0027] like Figure 2 For the Adaptive Multi-Scale Feature Extraction (AMFE) module, the four-layer EdgeConv produces features {F1}. , F2 , F3 , In DGCNN, F4 is typically directly concatenated and globally pooled; this invention inserts at the concatenation point. AMFE --Parallel 1 / 3 / 5 separable convolutions with pointwise weighted fusion- Before entering global pooling or segmentation upsampling, multi-scale semantics are uniformly integrated to alleviate scale mixing. In 3D point cloud data processing, effectively capturing feature information at different scales is key to improving model performance. Due to the complexity of point cloud data, single-scale feature extraction often cannot fully reflect its local details and global structure. Therefore, how to combine multi-scale feature extraction with adaptive weighting mechanisms to improve the representation capability of point cloud data has become an urgent problem to be solved. To this end, this invention proposes AMFE. The AMFE module combines multi-scale convolution operations with an adaptive feature adjustment mechanism, which can flexibly extract features from different scales and dynamically adjust their weights according to the contribution of each scale, thereby improving the network's expressive ability when processing complex point cloud data.

[0028] The AMFE module mainly consists of three core parts: a multi-scale convolution module, an adaptive feature adjustment module, and a feature fusion operation.

[0029] First, multiple convolutional kernels of different scales (such as 1 × 1 × 1, 3 × 3 × 3, 5 × 5 × 5) are used in parallel to extract features from the point cloud; in this way, the network can simultaneously focus on features at different scales, enhancing its ability to learn multi-scale information; Equation (4) represents the extraction of features from the point cloud using convolutional kernels W of different scales. si ( i = 1 , 2 , 3) Perform a convolution operation on the input feature X, using the following formula: (4) Among them, W si These correspond to convolution kernels of 1 × 1 × 1, 3 × 3 × 3, and 5 × 5 × 5, respectively. This indicates a convolution operation.

[0030] Next, to adaptively weight the features at each scale, the AMFE module uses a one-dimensional convolutional layer to weight the features at each scale; specifically, it utilizes a convolutional kernel W. ai For each scale of feature X i Weighting is applied to dynamically adjust the importance of features at different scales; Formula (5) gives the calculation formula for the adaptive weighting process: (5) Among them, W ai ∈ R C × C For adaptive weighted convolution kernels, C This represents the number of channels.

[0031] Finally, by fusing all weighted multi-scale features, a comprehensive feature representation containing both local and global information is obtained; Equation (6) represents the calculation process of multi-scale feature fusion: (6) Through this adaptive multi-scale feature extraction and fusion mechanism, the model can effectively capture multi-scale features in point cloud data. By automatically selecting an appropriate receptive field for each point, the adaptive multi-scale module directly solves the problem of uneven point density. In dense regions, smaller-scale features (capturing fine-grained geometric information) are given higher weights; while in sparse regions or to capture coarse shape structures, larger-scale features are emphasized. This ensures that important structural information can be captured at an appropriate scale regardless of the sampling density. Furthermore, the multi-scale method handles complex topologies by observing point cloud geometry from multiple perspectives—it can simultaneously capture small protrusions and overall shape contours; the result is a rich multi-scale feature representation. F MS It is comprehensive and context-aware, laying the foundation for more effective graph convolution in the next stage.

[0032] like Figure 4 For the Edge-aware Adaptive Dynamic EdgeConv (EADEC) module, the traditional DGCNN (EdgeConv) uses a fixed edge convolution module. k Neural networks (NNs) treat dense and sparse regions equally, making it difficult to account for density variations and complex geometry in point clouds. Therefore, this invention modifies the original DGCNN in Stages 0 and 2. 1 / 3. Upgraded to EADEC, it introduces a density-geometric collaborative mechanism at the three levels of "graph construction → feature encoding → aggregation".

[0033] Density-driven dynamic neighborhood: Let X = [ x 1 ,...,x N ] ∈ R C × N Input point cloud; use kernel-weighted local density, the formula is: (7) in, rho i Using Gaussian kernel to measure points x i The attenuation sum of the Euclidean distances between points in the neighborhood can reduce the impact of noise points on density estimation. r 0 controls the neighborhood radius. sigma d Determines the core width.

[0034] Mapping to an adaptive neighborhood size, the formula is: (8) Wherein, the Sigmoid mapping guarantees k i Monotonically increases with density; gamma Control the slope; µ To achieve adaptive normalization for the batch average density; and Set lower and upper limits for the neighborhood size: when the local density is extremely low, At least for To ensure robustness of features; while in dense areas, No more than This avoids the increased detail smoothing and computational overhead caused by excessively large neighborhoods; It is an adaptive neighborhood size The Sigmoid mapping.

[0035] Then, a dynamic neighborhood is constructed, using the following formula: (9) Formula (9) first uses a Top-K search based on Euclidean distance to find the center point. Choose the closest One neighbor, get the neighborhood. ; Formula (10) then works within the same neighborhood: for each First, construct local features. Then combine it with the learnable vector Perform the inner product and take the exponent, then finally... Soft-Max normalization is performed within the range to obtain the attention weights. Therefore, Equation (9) is responsible for selecting which neighbors to include, while Equation (10) assigns weights and extracts features among the selected neighbors. The two sequentially complete the process from neighborhood determination to weighted aggregation.

[0036] Choose each point independently k i Find the nearest neighbor to avoid overfitting in dense areas and undersampling in sparse areas; Differential-Attention Convolution High-Dimensional Side Features Its weight for: (10) in, and The first and the The original feature vector of each point; dimension is , This indicates the number of original feature channels. In common cases, it only contains three-dimensional coordinates, but it can also include extended attributes such as color and normal vectors. The Euclidean distance between two points is used to explicitly encode geometric similarity; concatenation yields... It retains absolute position information while enhancing relative geometric difference, which is then used as input for subsequent attention calculations; The weight vectors are learnable, and the importance of each component is adaptively adjusted through backpropagation during training; after Soft-Max normalization... Describe the center point To the neighbors Attention intensity, satisfying This allows for the automatic emphasis on key neighbors that contribute more to the local structure during feature aggregation; Indicates adaptive neighborhood size A defined set of neighbor indices; No. m Branch output: (11) in, For the first Shared MLP weight matrix for three branches: Each branch has independent parameters, which are shared across all points and correspond to different kernel widths or receptive domains, thus enabling the extraction of multi-scale difference features within the same neighborhood; Max pooling can automatically select the most significant directional features during aggregation, while improving rotation invariance.

[0037] Three groups share MLP ( W m Multi-scale difference features are captured with different kernel widths, and then the neighborhood is max-pooled to enhance rotation invariance.

[0038] Multi-scale fusion and splicing of three residual branches, batch normalization and direct connection of residuals: (12) in, This represents the edge feature representation obtained through multi-scale feature fusion and batch normalization operations; These represent the point cloud features calculated at different scales; This represents the number of neighboring nodes, i.e., the number of point clouds within the neighborhood. It is a weight matrix used for linear mapping of features to ensure effective transfer of features between different layers; The input point cloud feature matrix contains the original features of each point; batch normalization (BN) normalizes these features, which helps to accelerate network training and improve stability.

[0039] The edge feature representations obtained through these steps can provide crucial geometric information in subsequent processing and network training, and can automatically adapt to noise and irregular connections in point cloud data, thereby enhancing the model's sensitivity and robustness under complex geometries.

[0040] Note: Batch normalized stable training; residual W r X avoids over-smoothing and preserves the original coordinate characteristics.

[0041] The EADEC module excels at characterizing complex point cloud topologies; by dynamically updating the neighbor graph at each layer, the network naturally adapts to the geometry of objects—connections between points on the same surface are strengthened as features evolve, while outlier connections are weakened or dropped; the introduced attention weights... α ij This enables the model to distinguish between key structural neighbors and secondary neighbors: even if two points are spatially close, the point that contributes more to the local geometry will receive a higher weight. This directly addresses the challenge of irregular connections in point clouds, where traditional convolutions are often disturbed and their performance degrades when there are many trivial or noisy neighbors. In addition, the addition of residual connections ensures that information between consecutive EdgeConv layers is not lost and mitigates the problems caused by deep neighborhood aggregation. In summary, the EADEC module provides a flexible and powerful local feature aggregation mechanism that maintains sensitivity and robustness to complex geometries under conditions of noise and point density variations.

[0042] like Figure 3This invention proposes a residual dual-attention module (RDA). After the four EdgeConv layers progressively ascend to deeper semantics, RDA is inserted at the outputs of Stage 1 and Stage 3 of EdgeConv. This RDA performs residual recalibration of features using channel-spatial dual attention to suppress noise and highlight key regions. In 3D point cloud data processing, effectively selecting and strengthening key features while suppressing irrelevant or redundant information is crucial for improving model performance. Traditional attention mechanisms, such as channel attention and spatial attention, while improving feature representation quality to some extent, often suffer from overemphasizing local features or neglecting important global features when balancing global and local features. Therefore, this invention proposes an innovative residual dual-attention module that combines channel attention and spatial attention mechanisms and introduces residual connections. This allows the network to better select and strengthen key information, improving the accuracy and robustness of feature representation.

[0043] This module consists of two main components: a channel attention mechanism and a spatial attention mechanism, which weight features in the channel dimension and the spatial dimension, respectively. The channel attention mechanism dynamically adjusts the importance of each channel by perceiving global information, thereby enhancing attention to key channels. The spatial attention mechanism enhances the network's sensitivity to local regions by focusing on local information in the spatial dimension, especially for fine-grained geometric features in point cloud data, and can effectively mine local structures.

[0044] The channel attention mechanism first performs global average pooling on the input features to extract global information, and then calculates the attention weights for each channel through convolution. α c This reflects the importance of each channel in the current task; then, weights are used. α c For input features X Weighting is applied to suppress features of irrelevant channels; formula (13) represents this process: (13) in, sigma W is the Sigmoid activation function. c ∈ R C × C / 16 Let GAP(X) be the channel convolution kernel, representing the convolution kernel for input feature matrix X ∈ R. B × C × N Global average pooling.

[0045] The spatial attention mechanism, on the other hand, uses spatial convolution operations to weight each spatial location of the input features, thereby obtaining the attention weight for each spatial location. α s This operation can effectively enhance the network's attention to local regions (especially geometric features); Equation (14) gives the calculation process of the spatial attention mechanism: (14) Among them, W s ∈ R C × C / 16 It is a spatial convolution kernel.

[0046] The combination of channel attention and spatial attention mechanisms enables the network to simultaneously enhance key information in both channel and spatial dimensions; specifically, by weighting each feature channel and spatial location, a weighted feature representation X is finally obtained. attn Formula (15) gives the calculation method for the weighted feature representation: X attn = X⊙ ⊙ (15) Here, ⊙ represents element-wise multiplication.

[0047] To avoid information loss during transmission, the module employs a residual connection strategy; by using the original features... X Features after weighting X attn Adding features ensures that the network can enhance features while preventing excessive suppression of the information flow of the original features, thus avoiding gradient vanishing and information loss problems; finally, the output features are... X out Calculated using the following formula (16): X out = X +X attn (16) The RDA module significantly improves the representational power of 3D-AGCN by dynamically reweighting features in spatial and channel dimensions; it maintains the integrity of the original features through residual paths, ensuring that attention does not overwrite the original geometric information; this produces a rich feature map in which key point cloud patterns are prominent, laying a solid foundation for subsequent multi-scale processing.

[0048] Point cloud classification experiment: This embodiment evaluates the effectiveness of the model of the present invention in the domains and tasks of point cloud classification and point cloud segmentation; for the point cloud classification task, the ModelNet40 dataset is used; for the point cloud segmentation task, the ShapeNet dataset is used.

[0049] All experiments were conducted on the PyTorch framework using Ubuntu 18.04 as the operating system and an NVIDIA GeForce RTX 3090 GPU (Graphics Processing Unit). The Adam optimizer was used to optimize the model parameters, with an initial learning rate of 0.01, which was then reduced to 0.001 using cosine annealing. The batch sizes for classification and segmentation experiments were set to 32 and 16, respectively, and Dropout was set to 0.5. To ensure fairness, all training parameters were consistent with the baseline method.

[0050] The ModelNet40 dataset contains a total of 12,311 3D point cloud models across 40 different categories, with 9,843 models assigned to the training set and 2,468 to the test set.

[0051] The ShapeNet dataset is a point cloud component segmentation dataset for fine-grained shape recognition. It contains a total of 16 object categories, 50 common component instances, and 16,881 3D point cloud models. There are 2-6 component segmentation classes under each object category. Of these, 14,006 models are used for training and 2,874 are used for testing.

[0052] To ensure maximum fairness and reproducibility of the comparison, the Adam optimizer, batch size of 32, number of sampling points of 1024, and cosine annealing learning rate scheduling were uniformly adopted, with a fixed random seed set. Other core hyperparameters (initial learning rate, number of training epochs, weight decay, etc.) strictly followed the optimal configurations published in the official README or original papers of each baseline method (if no learning rate decay strategy was provided, annealing was uniformly set to 1 × 10⁻⁶). 3 During training, only the weights with the best validation set metrics are saved, and all models are evaluated under the same testing process. This ensures that each method runs with recognized optimal hyperparameters while eliminating interference from environmental differences and manual parameter tuning.

[0053] like Figure 6 In classification tasks, mean accuracy (mAcc) and overall accuracy (OA) are used as evaluation metrics. The experimental results on the ModelNet40 dataset are shown in Table 1.

[0054] Table 1. Comparison of classification results of different methods on the ModelNet40 dataset

[0055] in This represents the current optimal value; Figure 7 , Figure 8 and Figure 9 The results are shown for 3D-AGCN and DGCNN on the ModelNet40 test set, respectively. 3D-AGCN achieved an overall OA of 93. . 8%, mAcc = 91 . Even with a 3% threshold, 74% of the 40 categories can still remain "unchanged or improved"; among them, keyboard , bowl , person The accuracy rate reached 100% for all three categories. tv_stand, flower_pot , lamp Increase the difficulty of geometrically detailed examples by +3. +6 pp, confirming that EADEC's density-adaptive neighborhood can avoid over-smoothing of sparse edges; RDA's channel-spatial attention can suppress redundancy and amplify key geometric features, while AMFE's parallel multi-scale convolution takes into account both local texture and overall shape at both coarse and fine levels; only wardrobe , xbox , sink The three categories exhibit disadvantages: the total number of samples in these categories is less than 80, and the shapes are regular and the textures are simple, making training more reliant on data diversity than on network structure improvement; at the same time, EADEC struggles to leverage its edge perception advantage on objects with highly uniform point density, and the attention module lacks exploitable channel differences; from a heatmap perspective, the red diagonal blocks are mainly concentrated in geometrically complex or detailed structures, indicating that the three modules have stronger recall and discrimination power for "difficult" categories; the blue blocks only appear in the furniture category, which has scarce samples and simple shapes, suggesting that further improvements can be made through data augmentation or category-specific fine-tuning; in summary, 3D-AGCN achieves "lossless or gain-free" results in the vast majority of categories, and the number of high-gain categories is about three times that of low-gain categories, fully demonstrating the universality and effectiveness of the proposed three modules.

[0056] Point cloud segmentation experiment: To ensure a fair comparison, the Adam optimizer, batch size of 16, farthest point sampling (FPS) of 1024, and cosine annealing learning rate scheduling (learning rate gradually decaying to 1 × 10⁻⁶) were uniformly used across all methods. 3 Furthermore, all models were trained within a uniform training budget of 200 epochs; if the baseline paper used fewer epochs, the training was extended to 200 epochs based on its original optimal hyperparameters, and performance was confirmed to be unaffected on the validation set; the remaining core hyperparameters (initial learning rate, weight decay, etc.) strictly followed the optimal values ​​reported by the official implementations of each baseline or the original paper; each ShapeNet shape was first sampled at 1024 points using FPS, and the three-dimensional coordinates (x, y, z ) and surface normal ( n x ,n y ,n z The model weights are concatenated as network inputs; during training, only the model weights with the highest mIoU on the validation set are saved and evaluated under a unified test pipeline to eliminate the influence of environmental differences and manual parameter tuning.

[0057] Evaluation metrics: In the segmentation task, the mean intersection-over-union ratio (mIoU) and the mean class intersection-over-union ratio (Cls.mIoU) for each class are used as evaluation metrics; Mean Intersection over Union (mIoU) is an important evaluation metric commonly used in semantic segmentation tasks. It measures the degree of overlap between the predicted segmentation results and the actual segmentation results. Specifically, mIoU is obtained by averaging the Intersection over Union (IoU) of all categories. For each category, the Intersection over Union (IoU) is calculated by dividing the area of ​​intersection between the predicted and actual results of that category by the area of ​​their union.

[0058] set up n ii Indicates the first i The class was correctly predicted as the first i The number of classes (i.e., true instances), ∑ k j=0 n ij Indicates the first i The actual number of samples in the class Indicates the prediction is the first i The number of samples in each class; where, k It is the total number of categories; The experiment sampled 1024 points from each shape model and used the three-dimensional coordinates of the point cloud and the surface normal as the input to the network. The relevant experimental results are shown in Tables 2 and 3. Table 2 Comparison of segmentation results of different methods on the ShapeNet dataset (Group 1)

[0059] Table 3: Comparison of segmentation results of different methods on the ShapeNet dataset (Group 2)

[0060] As shown in Tables 2 and 3, the 3D-AGCN of this invention achieves an overall mIoU of 86.1% on the ShapeNet segmentation task, which is 0.9-2.4pp higher than the other five baselines; and achieves the highest IoU in 6 out of 16 component categories (Plane , Car , Chair , Earphone , Mug and alongside DGCNN Rocket The three modules contribute differently to this result: residual dual attention (); RDA By suppressing redundant channels and highlighting key regions at the input, Plane Wings and Chair Larger plane edges, such as those on the backrest, are more clearly defined; adaptive multi-scale feature extraction ( AMFE It can automatically switch the sensory field in areas with significant density differences, such as the car body and the headphone stem, taking into account both the overall outline and details, thereby significantly improving the sensory experience. Car and Earphone IoU; while density-aware dynamic edge convolution ( EADEC Through adjustable neighborhood k i Accurate modeling of thin-walled or perforated structures, for Mug Handle and Rocket The effect is particularly noticeable on slender shapes such as the tip; the sequential closed loop of "attention filtering → multi-scale supplementation → density adaptive propagation" between modules enables the network to maintain boundary continuity and component integrity in complex topology and non-uniform point density scenarios, ultimately achieving a robust lead over most categories.

[0061] To more intuitively evaluate the segmentation performance of each method, Figure 10 , 11 12 and Figure 13 , 14 Figure 15 presents the visualization results of component segmentation for typical category point clouds. DGCNN is used as the baseline method, PointNet as the comparison method, and Ours as the network proposed in this invention. Points with incorrect predictions are indicated in black for easy observation. It can be seen that in complex boundary regions such as skateboard wheels, mug handles, and the connection between earphones and headbands, the method of this invention can more accurately separate components; while DGCNN and PointNet still show missed or incorrect classifications in these fine-grained structures. Especially in details such as rocket tail fins, pistol trigger guards, and cap visors, the method of this invention almost completely eliminates missegmentation. These results demonstrate that the network of this invention can more effectively capture key geometric features in point clouds and achieve superior component segmentation quality across multiple categories of instances.

[0062] Figure 10 , 11 12 are three methods in bag , cap , skateboard and mugComparison of point cloud segmentation results across four categories (green indicates correct prediction, orange-red indicates incorrect prediction).

[0063] Figure 13 , 14 15 represents three methods in rocket , pistol , earphone Comparison of point cloud segmentation results across three categories (green indicates correct prediction, orange-red indicates incorrect prediction).

[0064] ablation experiment

[0065] 1. Ablation experiment for classification task

[0066] Table 4 Results of Classification Network Ablation Experiment

[0067] As shown in Table 4, the overall accuracy remains in the range of 92.9-93.1% when any one of the sub-modules (RDA, AMFE, EADEC) is retained alone; the OA can be increased to 93.2-93.3% when any two modules are used together, and reaches 93.6% when all three modules are activated; this stepwise gain reveals that: RDA First, noise is reduced through channel-space attention; AMFE Then, learnable multi-scale convolutional weights are used to balance dense and sparse regions; EADEC Ultimately, density-aware dynamic graph convolution is used to finely restore the local topology; the three processes sequentially link "feature purification → scale supplementation → neighborhood self-adjustment" into a closed loop, maximizing the accumulation of discriminative information at different levels, thereby bringing performance improvements beyond the linear superposition of parameters.

[0068] 2. Ablation experiment of segmentation task

[0069] Table 5. Results of segmentation network ablation experiments

[0070] The segmentation experiments (Table 5) show a similar increasing trend: the mIoU of a single module configuration is about 85.1–85.2%, the mIoU of a two-module combination can reach up to 85.7%, and the complete three-module combination further increases it to 86.1%; unlike classification tasks, segmentation relies more on boundary continuity and local consistency.

[0071] This invention AMFE+EADEC The coupling contribution is most significant: the former introduces multi-scale semantics, ensuring complete contextual information; the latter dynamically reorganizes the neighborhood at each scale, mitigating the break caused by density gradients; and then superimposed... RDAAfter channel / space filtering, the interference of noise points on boundary determination can be further reduced, so that complex structures such as thin walls and perforations can maintain fine and smooth segmentation output. Therefore, the pipelined collaboration of the three modules ensures that the network can still robustly segment each component and maintain high IoU in scenarios with diverse shapes and uneven density.

[0072] This invention presents a 3D Attention Graph Convolutional Network (3D-AGCN) for efficient point cloud classification and partial segmentation tasks. Specifically, the modules designed in this invention aim to solve the following problems: Fixed-scale operators often struggle to capture fine-grained local details while maintaining a global scene understanding; this problem is particularly pronounced when sampling density changes drastically. The inability to balance both can lead to the loss of key local features or insufficient global context understanding, hindering fine-grained segmentation and overall scene understanding. We aim to enable networks to adapt to multiple scales by introducing a flexible feature extraction mechanism to dynamically adjust the weights of local details and global context. Local receptive fields, commonly used in many networks, can effectively capture neighboring spatial relationships, but they struggle to model long-distance semantic dependencies, leading to blurred object boundaries and semantic misclassification. This is especially critical in complex scenes where relationships between distant objects are essential for accurate segmentation. This problem stems from the limited scope of local receptive fields, which cannot cover a broader scene context. To address this, this invention expands the receptive field by jointly modeling local features and long-distance dependencies. Traditional k-nearest neighbor (k-NN) graphs struggle to adapt to the dynamically changing topology of point clouds, particularly when dense clusters and sparse sampling regions coexist. The network struggles to fully understand the global structure and topology of large-scale point clouds, resulting in performance degradation. The solution is to achieve dynamic adaptation to the point cloud topology, employing a flexible mechanism to adjust point relationships based on the evolutionary feature space in real time.

[0073] The main contributions of this invention are as follows: 1. An adaptive multi-scale feature extraction mechanism is proposed, based on a lightweight multilayer perceptron (MLP) backbone network, supplemented by a dynamic multi-scale feature extractor. This module utilizes learnable scale coefficients to enable the model to dynamically adjust between preserving fine-grained local details and capturing global context, achieving an effective fusion of detailed features and overall scene understanding.

[0074] 2. A residual dual attention module is introduced, built based on multi-scale features. This module simultaneously models inter-channel dependencies and spatial saliency, selectively enhances informative features, suppresses irrelevant noise, ensures smooth gradients in deep networks, and improves the model's attention to key point cloud features.

[0075] 3. Incorporating adaptive dynamic edge convolution improves the network's robustness to different point cloud densities. This module reconstructs the k-NN graph in an evolving feature space, dynamically adjusts the neighborhood size, and generates topology-aware convolution kernels, significantly improving the model's performance in diverse real-world scenarios.

[0076] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention, characterized in that, Includes the following steps: Step 1: Acquire the image to be processed; Step 2: Construct a 3D-AGCN network; use a Transform encoder to align features, and then pass through several EADEC, RDA and AMFE modules to obtain feature groups. Concatenate the feature groups and perform convolutional compression to obtain shared features. The classification branch performs global max pooling and average pooling on the shared features and outputs the classification score; the segmentation branch concatenates the shared features and one-hot class vectors at each point and outputs the segmentation score.

2. The 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in claim 1, characterized in that, The 3D-AGCN network includes: The image to be processed is sequentially input into the Transform encoder, the first EADEC module, and the first RDA module; one output of the first RDA module is sequentially connected to the second EADEC module, the second RDA module, and the first AMFE module; one output of the first AMFE module is sequentially connected to the third EADEC module, the third RDA module, and the second AMFE module; one output of the second AMFE module is sequentially connected to the fourth EADEC module and the fourth RDA module; the output of the fourth RDA module is concatted with the other outputs of the first AMFE module, the second AMFE module, and the first RDA module; the output of the concat operation is concatenated with the first Conv convolution, and one output of the first Conv convolution is fed into the SegHead MLP module after Cat Vector operation and Repeat-N operation, along with the outputs of the second RDA module, the third RDA module, and the fourth RDA module.

3. The 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in claim 2, characterized in that, The output of the first Conv convolution is fed into GAP and GMP respectively, then Concat is performed, and finally passed through the MLP module.

4. The 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in any one of claims 2 and 3, characterized in that, The AMFE module includes: The input feature map is passed through three branches of DwConv with kernels of 1 × 1, 3 × 3, and 5 × 5, and then through a 1x1 PWConv. The outputs of the three branches of PWConv are then summed point by point with weights.

5. The 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in any one of claims 2 and 3, characterized in that, The EADEC module includes: Let X = [ x 1 ,...,x N ] ∈ R C × N Given a point cloud as input, calculate the kernel-weighted local density based on the point cloud coordinates and features. ; Will Mapping to adaptive neighborhood size ; γ Control the slope; µ To achieve adaptive normalization for the batch average density; and Set the lower and upper limits for the neighborhood size respectively; It is an adaptive neighborhood size Sigmoid mapping; Building dynamic neighborhoods Then calculate the edge features. and weight ; Obtain the output of the three-branch MLP , To share the MLP weight matrix; After performing a BN operation and concatenating it with X, we get: , It is a weight matrix.

6. The 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in any one of claims 2 and 3, characterized in that, The RDA module includes: Input feature map After three branches, the output of the first branch is: , σ It is the Sigmoid activation function. The channel convolution kernel is denoted by GAP(X), which represents global average pooling. Output of the second branch W s Spatial convolution kernel; The third branch is The combined output of the three branches is X. attn = X⊙ ⊙ ,⊙ represents element-wise multiplication; X attn and Add the residuals together.

7. The 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in claim 1, characterized in that, The image to be processed is a 3D point cloud image.

8. A 3D point cloud classification and segmentation system based on dynamic edge convolution and residual dual attention, characterized in that, include: Memory is used to store instructions that can be executed by the processor; A processor for executing instructions to implement the 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in any one of claims 1-7.

9. A computer-readable medium storing computer program code, characterized in that, The computer program code, when executed by a processor, implements the 3D point cloud classification and segmentation method based on dynamic edge convolution and residual dual attention as described in any one of claims 1-7.

Images