Point cloud multi-scale feature segmentation method and device based on attention mechanism, equipment and medium

Abstract

The invention relates to a point cloud multi-scale feature segmentation method and device based on an attention mechanism, equipment and a medium. The method comprises the following steps: constructing a multi-scale feature group by extracting geometric and semantic features of point cloud data, and calculating local geometric complexity and semantic importance based on a real-time strategy to generate a dynamic weight coefficient; the time sequence activation sequence of the multi-scale branches is controlled according to the weight priority, and fine segmentation of the key area is achieved; and finally, strategy parameters are optimized through a closed-loop feedback mechanism, and a self-adaptive scene evolution resource allocation system is formed. According to the method, the problems that a static attention mechanism cannot adapt to scene changes and multi-scale computing resource allocation is unbalanced in the prior art are solved through a dynamic allocation strategy, the segmentation precision and the computing efficiency in a complex dynamic scene are remarkably improved, and reliable technical support is provided for real-time three-dimensional perception scenes such as automatic driving.

Description

Technical Field

[0001] This invention relates to the field of image analysis and processing computation, specifically to a point cloud multi-scale feature segmentation method, apparatus, device, and medium based on an attention mechanism. Background Technology

[0002] Multi-scale feature segmentation of point clouds, as a core technology in the field of 3D vision, plays a crucial role in areas such as autonomous driving environmental perception and industrial 3D inspection. This method extracts local geometric features and global semantic features of point cloud data at different spatial scales to achieve accurate differentiation of various objects in complex 3D scenes. Especially when processing unstructured point clouds, it is necessary to simultaneously consider fine-grained local features and macro-level scene understanding. Traditional methods typically employ a multi-scale feature pyramid structure combined with an attention mechanism to improve segmentation accuracy.

[0003] However, existing technologies have significant limitations in adapting to dynamic scenes: static attention mechanisms cannot adjust resource allocation in real time based on the geometric complexity and semantic importance of point cloud regions, resulting in insufficient feature extraction of key regions. Simultaneously, the parallel processing of multi-scale branches lacks temporal optimization strategies, making it difficult to prioritize high-value regions when computational resources are limited. This leads to excessive resource consumption in distant sparse point clouds or background regions, while segmentation accuracy decreases for nearby highly dynamic objects. This resource allocation imbalance further prevents the system from accumulating scene cognitive experience during continuous frame processing, hindering the implementation of a progressive optimization mechanism similar to human visual focus switching. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a point cloud multi-scale feature segmentation method, apparatus, device and medium based on an attention mechanism that can dynamically allocate computing resources and adaptively optimize processing timing.

[0005] The objective of this invention is achieved through the following solution:

[0006] In a first aspect, the present invention provides a point cloud multi-scale feature segmentation method based on an attention mechanism, comprising the following steps:

[0007] S1: Perform 3D coordinate feature extraction and semantic label prediction on the acquired raw point cloud data. Convert the point cloud coordinates into feature vectors with semantic information through a preset point cloud segmentation network to generate an initial point cloud feature set.

[0008] S2: Extract local geometric features and global semantic features from the initial point cloud feature set, and use hierarchical sampling to construct neighborhood structures at different scales to generate a multi-scale feature vector set containing multi-resolution features;

[0009] S3: Based on the attention allocation strategy of the current frame, perform dynamic attention weight allocation on the multi-scale feature vector group, calculate the adaptive weight coefficient of the region attention according to the local geometric complexity and semantic importance, and generate the segmentation feature region with priority labeling.

[0010] S4: Perform multi-scale branch activation and feature fusion processing on the segmented feature regions. Based on adaptive weight coefficients, control the execution order of feature extraction branches and fuse the output features of each branch to generate optimized segmentation results.

[0011] S5: Update the closed-loop strategy for the optimized segmentation results, adjust the attention allocation parameters of the multi-scale branches in combination with the segmentation error, and generate the attention allocation strategy for the next frame point cloud. The attention allocation strategy is used to indicate the feature extraction priority of different regions.

[0012] In one embodiment, S1 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0013] S11: Standardize the coordinates of the acquired raw point cloud data, transform the coordinates of each point to a preset reference coordinate system, and generate a standardized point cloud;

[0014] S12: Extract neighborhood features from the standardized point cloud, call the k-nearest neighbor algorithm to construct a local point group with spatial topology and calculate the normal vector to generate a local geometric feature set;

[0015] S13: Perform semantic label prediction on the local geometric feature set, use a pre-trained classifier to identify the object category of each point, and generate an initial point cloud feature set with semantic labels.

[0016] In one embodiment, S2 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0017] S21: Divide the initial point cloud feature set into scales, generate sampling point clouds of different resolutions through voxel meshing, and generate multi-scale point cloud subsets;

[0018] S22: Extract local features from a multi-scale point cloud subset, calculate the curvature and density distribution of the point group at each scale, and generate scale feature vectors;

[0019] S23: Perform global feature fusion on the scale feature vectors, call the graph convolutional network to aggregate cross-scale information of neighboring point features, and generate multi-scale feature vector groups.

[0020] In one embodiment, S3 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0021] S31: Based on the attention allocation strategy of the current frame, calculate the geometric complexity of the multi-scale feature vector group, calculate the curvature gradient change rate and point density distribution characteristics according to the strategy parameters, and generate a complexity distribution map.

[0022] S32: Perform semantic weight allocation on the complexity distribution map, and combine it with a predefined object category priority table to mark key regions in the semantic structure of the 3D scene to generate a semantic weight matrix;

[0023] S33: The complexity distribution map and semantic weight matrix are fused and calculated. A comprehensive priority score for the region is generated through the weighting rules defined by the strategy. Priority-labeled segmentation feature regions are generated. The segmentation feature regions are used to indicate the activation order of multi-scale branches.

[0024] In one embodiment, step S31 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0025] S311: Perform curvature gradient calculation on the multi-scale feature vector group, solve the surface curvature change rate by the normal vector difference of the k nearest neighbor point group, and generate the curvature gradient tensor.

[0026] S312: Perform norm normalization on the curvature gradient tensor, calculate the L2 norm of the curvature gradient at each point, and apply nonlinear mapping to generate normalized curvature intensity.

[0027] S313: The point cloud density distribution of the normalized curvature intensity and the multi-scale feature vector group is fused. The geometric complexity of the intensity of regional feature changes is calculated through Sigmoid activation and logarithmic transformation, generating a geometric complexity distribution map. The formula for calculating geometric complexity is:

[0028]

[0029] Where C is the geometric complexity and σ is the sigmoid activation function. Let ρ be the L2 norm of the curvature gradient, ρ be the point cloud density, and ε be the smoothing factor, where ε = 10. -5 .

[0030] In one embodiment, S4 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0031] S41: Perform branch sequence control on the segmented feature region, activate high, medium and low resolution branches in descending order of priority score, and generate branch execution sequence;

[0032] S42: Extract features from the branch execution sequence, perform geometric feature calculations for each scale branch in sequence, and generate multi-branch output features;

[0033] S43: Attention-weighted fusion of multi-branch output features is performed. Cross-scale features in the cross-scale feature space are integrated through the channel attention mechanism to generate optimized segmentation results. The optimized segmentation results are used to indicate the semantic segmentation structure of the 3D scene.

[0034] In one embodiment, step S5 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0035] S51: Measure the error of the optimized segmentation results, calculate the difference between the predicted results of the point cloud segmentation and the intersection-union ratio of the true labels, and generate a segmentation error index;

[0036] S52: Iterate the parameters of the segmentation error index, adjust the geometric complexity calculation coefficients of the geometric complexity evaluation model according to the error direction, and generate updated attention parameters;

[0037] S53: Encapsulate the updated attention parameters into a strategy, build a priority allocation rule library based on historical segmentation performance, and generate the attention allocation strategy for the next frame's point cloud.

[0038] Secondly, the present invention provides a point cloud multi-scale feature segmentation device based on an attention mechanism, which is configured with the following modules:

[0039] The initial point cloud feature extraction module is used to extract three-dimensional coordinate features and predict semantic labels from the acquired raw point cloud data. It converts the point cloud coordinates into feature vectors with semantic information through a preset point cloud segmentation network to generate an initial point cloud feature set.

[0040] The multi-scale feature vector generation module is used to extract local geometric features and global semantic features from the initial point cloud feature set. It uses hierarchical sampling to construct neighborhood structures at different scales and generate multi-scale feature vector sets containing multi-resolution features.

[0041] The dynamic attention weight allocation module is used to dynamically allocate attention weights to multi-scale feature vector groups based on the attention allocation strategy of the current frame. It calculates the adaptive weight coefficient of regional attention based on local geometric complexity and semantic importance, and generates segmented feature regions with priority labels.

[0042] The multi-scale feature fusion processing module is used to perform multi-scale branch activation and feature fusion processing on the segmented feature regions. It controls the execution order of feature extraction branches based on adaptive weight coefficients and fuses the output features of each branch to generate optimized segmentation results.

[0043] The attention allocation strategy update module is used to update the closed-loop strategy of the optimized segmentation results. It adjusts the attention allocation parameters of the multi-scale branches in combination with the segmentation error and generates the attention allocation strategy for the next frame point cloud. The attention allocation strategy is used to indicate the feature extraction priority of different regions.

[0044] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-mentioned point cloud multi-scale feature segmentation methods based on the attention mechanism.

[0045] Fourthly, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-mentioned point cloud multi-scale feature segmentation methods based on an attention mechanism.

[0046] In summary, this application provides a point cloud multi-scale feature segmentation method based on an attention mechanism. Addressing the issue that static attention mechanisms cannot adjust in real-time according to scene changes, this method utilizes a dynamic weighting mechanism that integrates local geometric complexity and semantic importance. This enables adaptive focusing on key regions, significantly improving the segmentation accuracy of complex surfaces and highly dynamic objects. To address the imbalance in multi-scale branch resource allocation, priority-based branch temporal activation control enables targeted scheduling of computational resources, prioritizing high-value regions for high-resolution processing, thus achieving an optimal balance between computational efficiency and segmentation quality. To address the lack of incremental optimization capabilities, a closed-loop policy update mechanism continuously iterates attention parameters through segmentation error feedback, constructing a scene-adaptive priority rule base and achieving dynamic evolution of the system during continuous processing. The overall solution, through the collaborative efforts of dynamic weight allocation, temporal resource scheduling, and closed-loop policy optimization, establishes an intelligent processing logic that mimics human visual focus switching, providing reliable technical support for real-time 3D perception scenarios such as autonomous driving, balancing accuracy, efficiency, and evolutionary capabilities.

[0047] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0048] Figure 1 A flowchart illustrating a point cloud multi-scale feature segmentation method based on an attention mechanism, provided for an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of the process for generating segmented feature regions with priority markers, provided in an embodiment of this application.

[0050] Figure 3 This is a schematic diagram of the structure of a point cloud multi-scale feature segmentation device based on an attention mechanism, provided for another embodiment of this application. Detailed Implementation

[0051] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0053] In one embodiment, such as Figure 1 As shown, a point cloud multi-scale feature segmentation method based on an attention mechanism is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0054] S1: Perform 3D coordinate feature extraction and semantic label prediction on the acquired raw point cloud data. Convert the point cloud coordinates into feature vectors with semantic information through a preset point cloud segmentation network to generate an initial point cloud feature set.

[0055] Specifically, the system acquires raw point cloud data through various sensor devices, such as LiDAR, 3D scanners, or depth cameras. For example, in autonomous driving scenarios, LiDAR acquires the 3D coordinates of objects around the vehicle by emitting laser pulses into the surrounding environment and receiving reflected signals. LiDAR scans at specific time intervals and angular resolutions; the time difference between each laser pulse's return is converted into distance information, thus generating 3D point cloud data containing a large number of points that record the reflection positions of the scanned object's surface. 3D scanners are widely used in industrial inspection or indoor scene reconstruction. They perform omnidirectional scanning of objects or scenes, using optical projection and imaging principles to accurately measure the geometry of the object's surface and convert the measurement results into point cloud data. Depth cameras use infrared structured light or time-of-flight technology to capture the depth information of each pixel in the scene in real time, thereby constructing point cloud data. The raw point cloud data acquired by these sensors typically records the spatial distribution of the object's surface as points, with each point containing x, y, and z coordinate values, and possibly additional information such as reflection intensity.

[0056] Specifically, the system extracts 3D coordinate features from the raw point cloud data. It iterates through each point in the raw point cloud data, extracts its 3D coordinate information, and integrates this information to form a set containing the coordinate data of all points. The system then performs semantic label prediction processing on the raw point cloud data, extracting the point cloud data with 3D coordinate features and inputting it into a pre-trained point cloud segmentation network model. This network model, based on deep learning algorithms, learns the feature patterns of different object categories in the point cloud data through training on a large amount of semantically labeled point cloud data.

[0057] During the prediction process, the network performs semantic category judgment on each point or local region of a point, assigns a semantic label to each point, and fuses the extracted 3D coordinate features with the predicted semantic labels to construct a feature vector containing coordinate and semantic information, forming an initial point cloud feature set. This feature set is a preliminary abstraction and semantic representation of the original point cloud data, providing a comprehensive feature foundation for further feature extraction and segmentation processing.

[0058] S2: Extract local geometric features and global semantic features from the initial point cloud feature set, and use hierarchical sampling to construct neighborhood structures at different scales to generate multi-scale feature vector groups containing multi-resolution features.

[0059] Specifically, during local geometric feature extraction, the system performs a neighborhood search for each point in the initial point cloud feature set. The search is based on spatial positional relationships, identifying other points within a certain range around each point, forming a local neighborhood. The system calculates the relative positional relationships of points within the local neighborhood and obtains parameters reflecting the local shape through vector operations. These parameters include the distribution pattern of points within the neighborhood and the distance relationships between points, collectively constituting the local geometric features.

[0060] During global semantic feature extraction, the system performs a holistic analysis of the initial point cloud feature set, calculating the correlation between different points. The correlation is determined based on the similarity of feature vectors. By aggregating the correlations, the system integrates feature information across the entire scope to obtain features that reflect the relationships between different parts of the overall scene, i.e., global semantic features.

[0061] During the hierarchical sampling process, the system determines different sampling ranges according to preset scale division rules, with each sampling range corresponding to a scale. At each scale, the system selects points according to the sampling rules and constructs a neighborhood structure for that scale. The neighborhood structures at different scales undergo local geometric feature extraction and global semantic feature extraction respectively to obtain feature vectors for the corresponding scales. Feature vectors from multiple scales are combined to form a multi-scale feature vector group.

[0062] S3: Based on the attention allocation strategy of the current frame, perform dynamic attention weight allocation on the multi-scale feature vector group, calculate the adaptive weight coefficient of the region attention according to the local geometric complexity and semantic importance, and generate the segmentation feature region with priority label.

[0063] Specifically, the attention allocation strategy for the current frame is a rule or model formulated based on the overall characteristics of the current point cloud scene and task requirements. This strategy comprehensively considers factors such as the geometric complexity and semantic importance of different regions in the point cloud data. Geometric complexity is evaluated by analyzing local geometric features such as point cloud density and surface curvature changes within the region, while semantic importance is determined based on the priority of object categories in a specific application scenario. The system dynamically allocates attention weights to each feature vector in the multi-scale feature vector group according to the attention allocation strategy for the current frame.

[0064] Preferably, the system can employ a deep learning-based attention mechanism model, such as self-attention or its variants. Taking self-attention as an example, the system calculates the correlation between different feature vectors to obtain an attention score matrix. Then, the system processes these scores using a normalization function, converting them into attention weight coefficients in the form of a probability distribution. These weight coefficients reflect the relative importance of different feature vectors in the current scene and can be dynamically adjusted according to geometric complexity and semantic importance, so that feature vectors in key regions receive higher weights.

[0065] The system combines the calculated adaptive weight coefficients with a multi-scale feature vector group, performing weighted processing on the feature vectors to generate priority-labeled segmentation feature regions. The priority label is represented by the magnitude of the weight coefficients; point cloud regions corresponding to feature vectors with higher weight coefficients are considered high-priority segmentation feature regions. These regions contain more critical feature information for the current task, allowing subsequent processing to prioritize in-depth analysis and computation of these regions, thereby achieving rational allocation of computing resources and improving segmentation accuracy.

[0066] S4: Perform multi-scale branch activation and feature fusion processing on the segmented feature regions. Based on the adaptive weight coefficient, control the execution order of the feature extraction branches and fuse the output features of each branch to generate optimized segmentation results.

[0067] Specifically, the system controls the execution order of feature extraction branches based on adaptive weight coefficients. Multi-scale branches are network structure branches designed for feature extraction at different scales, with each branch focusing on extracting feature information at a specific scale. The activation order is controlled by the region priority reflected by the adaptive weight coefficients, prioritizing the activation of branches corresponding to higher-priority segmentation feature regions.

[0068] For example, after activating each multi-scale branch and extracting the corresponding features, the system performs feature fusion on the extracted features. Feature fusion can be achieved through various methods, such as simple concatenation, weighted summation, or more complex deep learning-based fusion network structures. Preferably, the system can employ a graph neural network-based feature fusion method, treating the features extracted from different scale branches as nodes in a graph, and using graph convolution to interact and fuse the features between nodes, thereby integrating feature information from different scales to generate a fused feature representation.

[0069] The system can adopt a network structure based on pixel-level or point-level classification to perform segmentation category judgment on each point or voxel unit in the fused features, and generate optimized segmentation results. The optimized segmentation results, while retaining the main semantic structure of the original segmentation results, improve the recognition accuracy of object boundaries in complex scenes and the distinguishability of different category regions through multi-scale feature fusion and branch activation strategy optimization, thereby obtaining more accurate and reliable point cloud segmentation results.

[0070] S5: Update the closed-loop strategy for the optimized segmentation results, adjust the attention allocation parameters of the multi-scale branches in combination with the segmentation error, and generate the attention allocation strategy for the next frame point cloud. The attention allocation strategy is used to indicate the feature extraction priority of different regions.

[0071] Specifically, the system calculates segmentation error based on manually labeled ground truth values. The system calculates classification error and region error. Region error is determined by the intersection-over-union (IoU) ratio of the predicted and ground truth regions, reflecting the degree of overlap between the two regions. The system combines the classification and region errors with preset weights to obtain a comprehensive error, which is used to measure the accuracy of the segmentation result. The system adjusts the attention allocation parameters based on the comprehensive error, achieved through error backpropagation, which is based on a preset optimization algorithm.

[0072] For points with high overall error in high-priority regions, the system increases the weighting of their geometric complexity and semantic importance; for points with low overall error in low-priority regions, the system reduces their corresponding computational resource allocation. The adjusted parameters are stored in a parameter buffer, which manages parameters by frame index. Based on the updated parameters, the system generates an attention allocation strategy for the next frame. This strategy includes recommended processing branches, computational resource allocation ratios, and neighborhood radius adjustment values ​​for each priority region. Recommended processing branches are determined based on region priority and feature processing requirements; computational resource allocation ratios are matched to region importance; and neighborhood radius adjustment values ​​are set based on segmentation error feedback to optimize feature extraction accuracy. The system stores the attention allocation strategy in a temporal strategy queue, which stores strategies frame-by-frame for use in the S3 step of the next frame, enabling iterative optimization of the strategy during continuous frame processing.

[0073] In summary, this application provides a point cloud multi-scale feature segmentation method based on an attention mechanism. Addressing the issue that static attention mechanisms cannot adjust in real-time according to scene changes, this method utilizes a dynamic weighting mechanism that integrates local geometric complexity and semantic importance. This enables adaptive focusing on key regions, significantly improving the segmentation accuracy of complex surfaces and highly dynamic objects. To address the imbalance in multi-scale branch resource allocation, priority-based branch temporal activation control enables targeted scheduling of computational resources, prioritizing high-value regions for high-resolution processing, thus achieving an optimal balance between computational efficiency and segmentation quality. To address the lack of incremental optimization capabilities, a closed-loop policy update mechanism continuously iterates attention parameters through segmentation error feedback, constructing a scene-adaptive priority rule base and achieving dynamic evolution of the system during continuous processing. The overall solution, through the collaborative efforts of dynamic weight allocation, temporal resource scheduling, and closed-loop policy optimization, establishes an intelligent processing logic that mimics human visual focus switching, providing reliable technical support for real-time 3D perception scenarios such as autonomous driving, balancing accuracy, efficiency, and evolutionary capabilities.

[0074] In one embodiment, S1 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0075] S11: Standardize the coordinates of the acquired raw point cloud data, transform the coordinates of each point to a preset reference coordinate system, and generate a standardized point cloud.

[0076] Specifically, the system acquires raw point cloud data from a 3D scanning device. This data consists of discretely distributed points in 3D space, each containing 3D coordinate information. Some points also have additional attributes such as reflection intensity and color. The preset reference coordinate system is a pre-defined 3D Cartesian coordinate system. The origin and coordinate axis directions of this system are determined based on fixed reference objects in the scene. These fixed reference objects are objects whose positions do not change in the scene.

[0077] For example, the system transforms the coordinates of each point in the acquired raw point cloud data to a preset reference coordinate system. This transformation is achieved using a coordinate transformation matrix, which is generated based on the positional and angular relationship between the original point cloud coordinate system and the preset reference coordinate system. The system first calculates the difference between the origin of the original point cloud coordinate system and the origin of the preset reference coordinate system, using this as a coordinate offset. The coordinates of each point are then subtracted from this offset to complete the translation transformation. Next, a rotation matrix is ​​used to rotate the translated coordinates. This rotation matrix is ​​determined based on the angle between the coordinate axes of the two coordinate systems, ensuring that the transformed coordinate axes align with the preset reference coordinate system.

[0078] The system scales the coordinates after translation and rotation, based on the scale standard of the preset reference coordinate system, to eliminate coordinate scale differences caused by different acquisition scenarios or devices. Finally, the coordinates of all points are integrated to generate a standardized point cloud, in which each point is represented by three-dimensional coordinates under the preset reference coordinate system.

[0079] S12: Extract neighborhood features from the standardized point cloud, call the k-nearest neighbor algorithm to construct a local point group with spatial topology and calculate the normal vector to generate a local geometric feature set.

[0080] Specifically, the system calls the k-nearest neighbor algorithm to search for surrounding points that meet the spatial distance conditions, using each point in the standardized point cloud as the center, thus constructing a local point group. This local point group constitutes the spatial topology of the point, reflecting its spatial relationship with surrounding points. The system calculates the Euclidean distance between each point and other points in the local point group, sorts the points in the local point group according to the magnitude of the Euclidean distance, and selects points that meet the quantity requirements to form the neighboring point group. The system calculates the normal vector of the point based on the neighboring point group. During the calculation, the system first organizes the coordinates of each point in the neighboring point group into a coordinate matrix. The number of rows in the coordinate matrix is ​​the same as the number of points in the neighboring point group, and the number of columns is 3, corresponding to the x, y, and z coordinates respectively. The system calculates the covariance matrix using the coordinate matrix, which is constructed from the covariance values ​​of each coordinate component.

[0081] The system performs eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors. The eigenvector corresponding to the smallest eigenvalue is selected as the normal vector for that point, and its direction is perpendicular to the surface formed by the neighboring point group. The system also calculates other geometric features of the neighboring point group, including the average Euclidean distance between the point and all points in the neighboring group (mean distance); the variance of these Euclidean distances (distance variance); and the ratio of the number of points in the neighboring point group to the volume of the neighborhood space (point density). The system integrates the normal vector, average distance, distance variance, and point density to form the local geometric features of each point. The local geometric features of all points are then aggregated to generate a local geometric feature set.

[0082] S13: Perform semantic label prediction on the local geometric feature set, use a pre-trained classifier to identify the object category of each point, and generate an initial point cloud feature set with semantic labels.

[0083] Specifically, the system calls a pre-trained classifier, which is a machine learning model trained on a large amount of point cloud data with semantic labels. During training, the model learns the relationship between local geometric features and semantic labels. The input of the pre-trained classifier is the local geometric feature vector of each point in the local geometric feature set, and the output is the probability distribution of the point belonging to each category. The probability distribution consists of a set of probability values, each probability value corresponds to a preset object category, and the sum of all probability values ​​is 1.

[0084] For example, the system processes the probability distribution of each point, selecting the category with the highest probability value as the object category of that point. This object category belongs to a predefined semantic category set, which contains the category names of all possible objects in the scene. The system assigns the determined object category as a semantic label to the corresponding point, while retaining the point's probability distribution as the confidence information of the semantic label. It then fuses the local geometric features of each point in the local geometric feature set with the corresponding semantic label using feature vector concatenation. The semantic label is converted into an encoding vector using one-hot encoding, meaning the length of the encoding vector is the same as the number of categories in the semantic category set, with a value of 1 for the corresponding object category position and 0 for other positions. The encoding vector is then concatenated with the local geometric feature vector to form a new feature vector. After completing these steps, the system integrates the fused feature vectors of all points to generate an initial point cloud feature set with semantic labels. This feature set contains both geometric and semantic features of the points, providing data support for subsequent multi-scale feature processing.

[0085] In one embodiment, S2 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0086] S21: Divide the initial point cloud feature set into scales, generate sampled point clouds of different resolutions through voxel meshing, and generate multi-scale point cloud subsets.

[0087] Specifically, the system performs scale division on the initial point cloud feature set according to preset resolution levels, with different levels corresponding to different voxel sizes. Preferably, the system uses voxel meshing to process the initial point cloud feature set, dividing the 3D space into several voxel meshes. The size of each voxel mesh is determined according to the resolution of the corresponding scale; the higher the resolution, the smaller the voxel mesh.

[0088] Within each voxel grid, the system selects a representative point as the sampling point for that voxel. The selection method involves calculating the average coordinates of all points within the voxel grid and using the point corresponding to this average as the sampling point. This method downsamples the initial point cloud feature set, resulting in sampled point clouds at different resolutions. These sampled point clouds at different resolutions constitute different subsets, which are then combined to generate multi-scale point cloud subsets. Each subset corresponds to a specific scale and contains the point cloud feature information at that scale.

[0089] S22: Extract local features from a multi-scale point cloud subset, calculate the curvature and density distribution of the point group at each scale, and generate scale feature vectors.

[0090] Specifically, at each scale, the system uses each point in the subset as the center, determines the neighborhood range based on the voxel size of that scale, and selects points within the neighborhood to form local point groups. The system calculates the curvature of each local point group, using the normal vectors of each point in the group and determining the curvature value through the rate of change of the angle between the normal vectors. The curvature value reflects the degree of curvature of the surface formed by the local point group. Simultaneously, the system calculates the density distribution of the local point groups by statistically analyzing the spatial distribution of the point groups within the neighborhood, including the proportion of points in different sub-regions and the uniformity parameters of the distribution.

[0091] The system integrates the curvature value and density distribution parameters of each point into a vector. The dimension of the vector is determined according to the number of features. It contains the curvature information and density distribution information of the point at the current scale. The system performs the above operation on all points in the multi-scale point cloud subset to generate a feature set corresponding to each scale. The vector in each set is the scale feature vector at that scale.

[0092] S23: Perform global feature fusion on the scale feature vectors, call the graph convolutional network to aggregate cross-scale information of neighboring point features, and generate multi-scale feature vector groups.

[0093] Specifically, the system invokes a graph convolutional network. First, it constructs a graph structure for the point cloud at each scale. The nodes of the graph are the points at that scale, and the node features are the corresponding scale feature vectors. The system calculates the adjacency relationships between nodes, determining the existence and weight of edges through spatial distance and feature similarity. The spatial distance is calculated based on three-dimensional coordinates, and the feature similarity is determined by a similarity measure of scale feature vectors.

[0094] Graph convolutional networks aggregate the features of neighboring nodes for each node using an adjacency matrix. During this aggregation process, the system establishes a cross-scale connection mechanism, enabling spatially proximate nodes at different scales to form feature transfer paths and achieve cross-scale information exchange. Preferably, the system updates node features through multi-layer graph convolution operations. Each layer combines the current node's features with the aggregated features of its neighboring nodes, gradually incorporating broader spatial correlation information into the features. After a preset number of graph convolution processes, the system integrates the updated feature vectors at each scale to form a multi-scale feature vector group. This vector group contains local features at each scale and global correlation information across scales.

[0095] In one embodiment, such as Figure 2 As shown, S3 of the point cloud multi-scale feature segmentation method based on attention mechanism provided by the present invention specifically includes the following steps:

[0096] S31: Based on the attention allocation strategy of the current frame, calculate the geometric complexity of the multi-scale feature vector group, calculate the curvature gradient change rate and point density distribution characteristics according to the strategy parameters, and generate a complexity distribution map.

[0097] Specifically, the system extracts the curvature value of each point, calculates the curvature difference between adjacent points, and forms the curvature gradient change rate. Simultaneously, the system analyzes the point density distribution characteristics, counts the number of points in the neighborhood of each point, calculates the density value, and integrates the curvature gradient change rate and point density distribution characteristics to assign a complexity value to each point. The magnitude of the complexity value reflects the geometric complexity of the region. The system maps these complexity values ​​onto the entire 3D scene to generate a complexity distribution map. Preferably, the complexity distribution map is obtained through the following steps:

[0098] S311: Perform curvature gradient calculation on the multi-scale feature vector group, solve the surface curvature change rate by the normal vector difference of the k nearest neighbor point group, and generate the curvature gradient tensor.

[0099] Specifically, the system extracts the normal vectors of each point from a multi-scale feature vector set, which originate from local geometric feature sets. For each point, the system calls the k-nearest neighbor algorithm to determine k nearest neighbor points in the multi-scale feature vector set, forming a k-nearest neighbor group, where the value of k is preset by the average density of the point cloud. The system calculates the difference between the normal vector of this point and the normal vector of each nearest neighbor point, i.e., subtracting the normal vector of this point from the normal vector of the nearest neighbor point, resulting in k difference vectors. The system calculates the rate of change of surface curvature using the following formula:

[0100]

[0101] Where, Δn i It is the difference vector of the normal vector of the i-th nearest neighbor, d i r is the spatial distance between the nearest neighbor and the center point. i Let be the rate of change of curvature in the i-th direction. The system has k 'r' values. i Tensor decomposition yields a 3×3 curvature gradient tensor. The tensor elements cover the curvature change components and cross terms in the x, y, and z directions, providing a complete representation of curvature changes in three-dimensional space.

[0102] S312: Perform norm normalization on the curvature gradient tensor, calculate the L2 norm of the curvature gradient at each point, and apply nonlinear mapping to generate normalized curvature intensity.

[0103] Specifically, for each curvature gradient tensor, the system calculates the L2 norm:

[0104]

[0105] Among them, t mn These are elements of the curvature gradient tensor. Let L2 be the norm of the curvature gradient. The system applies to all points. Perform a nonlinear mapping, the mapping formula is:

[0106]

[0107] in, and These are the minimum and maximum values ​​of the L2 norm for all points, respectively, and s is the normalized curvature intensity, ranging from [0,1], reflecting the relative strength of curvature changes.

[0108] S313: The point cloud density distribution of the normalized curvature intensity and the multi-scale feature vector group are fused. The geometric complexity of the intensity of regional feature change is calculated by Sigmoid activation and logarithmic transformation, and a geometric complexity distribution map is generated.

[0109] Specifically, the system extracts the point cloud density ρ from the multi-scale feature vector set, where ρ is the number of points per unit volume, and the unit volume size is consistent with the corresponding scale voxel. The formula for calculating the geometric complexity is:

[0110]

[0111] Where C is the geometric complexity and ρ is the sigmoid activation function. Let ρ be the L2 norm of the curvature gradient, ρ be the point cloud density, and ε be the smoothing factor, where ε = 10. -5 The system first calculates... The product of log(ρ+ε) and C is obtained. The system arranges the C values ​​of all points according to their spatial coordinates, generating a two-dimensional geometric complexity distribution map. The coordinates correspond to a three-dimensional spatial projection, and the pixel value is C, which intuitively presents the geometric complexity of the region.

[0112] S32: Perform semantic weight allocation on the complexity distribution map, and combine it with the predefined object category priority table to mark the key regions in the semantic structure of the 3D scene to generate a semantic weight matrix.

[0113] Specifically, the system loads a predefined object category priority table, which contains all possible object categories in the 3D scene and a priority parameter for each category. The priority parameter is pre-set based on the importance of the object category in the application scenario. The system parses the semantic structure of the 3D scene covered by the complexity distribution map according to the predefined object category priority table. The parsing process combines an initial point cloud feature set with semantic labels to determine the object category corresponding to each spatial location. Based on the priority parameter of the object category, the system assigns semantic weights to different regions in the 3D scene semantic structure. The assignment rule is: regions belonging to high-priority categories receive higher semantic weights, and regions belonging to low-priority categories receive lower semantic weights. The system annotates key regions in the 3D scene. Key regions are those that simultaneously satisfy both high geometric complexity and high semantic priority. The annotation process is achieved by identifying the intersection of high-value regions and high-priority category regions in the complexity distribution map.

[0114] The system converts the semantic weights of each region into a matrix form. The rows and columns of the matrix correspond to the spatial grid of the 3D scene, and each element value in the matrix corresponds to the semantic weight of the corresponding grid region. This generates a semantic weight matrix, which fully records the semantic importance distribution of each region in the 3D scene.

[0115] S33: The complexity distribution map and semantic weight matrix are fused and calculated. A comprehensive priority score for the region is generated through the weighting rules defined by the strategy. Priority-labeled segmentation feature regions are generated. The segmentation feature regions are used to indicate the activation order of multi-scale branches.

[0116] Specifically, the system loads the weighting rules defined in the attention allocation strategy. These rules specify the weight ratio between complexity values ​​and semantic weights during the fusion process, with the ratio determined based on scenario processing requirements and resource allocation goals. According to the weighting rules, the system performs a weighted summation of the complexity value at each location in the complexity distribution map and the corresponding semantic weight in the semantic weight matrix. The result is the region's comprehensive priority score for that location. A higher region comprehensive priority score indicates a higher comprehensive value for that region in terms of both geometric complexity and semantic importance.

[0117] The system organizes the comprehensive priority scores of all regions and sorts the regions in the 3D scene from highest to lowest score. Based on the sorting results, the regions in the 3D scene are prioritized and marked with different levels. The level classification criteria follow the preset threshold in the attention allocation strategy, and the threshold is set according to the granularity requirements of resource allocation. The system integrates the marked regions into segmentation feature regions, which include the spatial range, comprehensive priority score, and priority level of each region. The segmentation feature regions are directly used to indicate the activation order of subsequent multi-scale branches. That is, the multi-scale branches corresponding to regions with high comprehensive priority scores are activated first, and the branches corresponding to regions with low scores are activated later, thereby achieving targeted allocation of computing resources.

[0118] In one embodiment, S4 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0119] S41: Perform branch sequence control on the segmented feature region, activate high, medium and low resolution branches in descending order of priority score, and generate branch execution sequence.

[0120] Specifically, the system reads priority scores from the segmented feature regions. These priority scores characterize the importance of different regions in semantic segmentation. The system compares the priority scores numerically, arranges them in descending order, and determines the division into high-priority, medium-priority, and low-priority regions. High-priority regions are assigned to high-resolution branches, medium-priority regions to medium-resolution branches, and low-priority regions to low-resolution branches. Different resolution branches correspond to different feature extraction accuracies and computational ranges.

[0121] Based on the above allocation results, the system schedules computing resources and activates the high-resolution branch, medium-resolution branch, and low-resolution branch sequentially. The activation process is implemented through the system's internal task scheduling mechanism to ensure that each branch starts in sequence. The system records the activation order and corresponding processing area of ​​each branch, generating a branch execution sequence. The branch execution sequence includes information such as branch type, start time, and processing area range, which is used to guide the orderly execution of subsequent feature extraction processes.

[0122] S42: Extract features from the branch execution sequence, perform geometric feature calculations for each scale branch in sequence, and generate multi-branch output features.

[0123] Specifically, for high-priority regions, the system calculates the local geometric features of each point within that region, including the normal vector, curvature, and distance deviation from neighboring points. The normal vector is obtained through eigenvalue decomposition of the covariance matrix of neighboring points, the curvature is calculated based on the rate of change of the normal vector, and the distance deviation is the difference between the average distance between the point and its neighboring points. For medium-priority regions, the system calculates the density distribution, average curvature, and neighborhood volume of the point cluster within that region. The density distribution is obtained through point count within a unit space, and the neighborhood volume is the volume of the smallest cube containing the neighboring points. For low-priority regions, the system calculates the overall geometric distribution features within that region, including the centroid coordinates of the point cluster, the region's volume percentage, and the principal direction vector. The centroid coordinates are the average of the coordinates of all points within the region, and the principal direction vector is determined through principal component analysis of the region's point cloud. The system stores the features extracted from each branch separately, forming high-resolution, medium-resolution, and low-resolution feature sets. These feature sets together constitute the multi-branch output features, which contain geometric feature information at different scales.

[0124] S43: Attention-weighted fusion of multi-branch output features is performed. Cross-scale features in the cross-scale feature space are integrated through the channel attention mechanism to generate optimized segmentation results. The optimized segmentation results are used to indicate the semantic segmentation structure of the 3D scene.

[0125] Specifically, the system invokes a channel attention mechanism to evaluate the importance of each feature channel in the multi-branch output features. The evaluation process is based on the response strength of the feature channels, which is obtained through global statistical calculation of the feature values. The system generates channel weights based on the evaluation results. These weights reflect the contribution of each feature channel to semantic segmentation, assigning higher weights to channels with higher contributions. The multi-branch output features are then multiplied by their corresponding channel weights along the channel dimension to complete the weighted processing. The weighted features retain important feature information while suppressing secondary feature information.

[0126] For example, the system performs cross-scale integration on the weighted multi-branch output features. During integration, high-resolution features are concatenated with medium-resolution features, and medium-resolution features are concatenated with low-resolution features. The concatenation is based on spatial location correspondence to ensure spatial alignment of features at different scales. Convolutional operations are then used to unify the dimensions and fuse information in the concatenated features, generating a fused feature set. Preferably, the system performs semantic classification on the fused feature set. Classification can be achieved by using the Softmax function to calculate the probability of each point belonging to each category. The category with the highest probability is taken as the final semantic label for that point. The final semantic labels of all points are then integrated to generate an optimized segmentation result. The optimized segmentation result is represented by the correspondence between three-dimensional coordinates and semantic labels, clearly indicating the semantic segmentation structure of each part in the three-dimensional scene.

[0127] In one embodiment, step S5 of the point cloud multi-scale feature segmentation method based on an attention mechanism provided by the present invention specifically includes the following steps:

[0128] S51: Measure the error of the optimized segmentation results, calculate the difference between the predicted results of the point cloud segmentation and the intersection-union ratio of the true labels, and generate a segmentation error index.

[0129] Specifically, the system acquires optimized segmentation results and ground truth labels. The optimized segmentation results include predicted semantic labels for each point in the 3D scene, while the ground truth labels are the actual object category annotations for each point in the 3D scene. The system divides the 3D scene into continuous cubic evaluation units according to spatial coordinates, with each evaluation unit containing a preset number of point cloud data. The system calculates the intersection-union ratio (IUGR) for each evaluation unit, using the following formula:

[0130]

[0131] Where A is the set of points corresponding to the predicted semantic labels within the evaluation unit, B is the set of points corresponding to the ground truth labels within the evaluation unit, |A|∩B| represents the number of points contained in the intersection of the two set of points, and |A|∪B| represents the number of points contained in the union of the two set of points. The system sets a preset benchmark Intersection over Union (IoU). ref Calculate the difference between the intersection and union ratios:

[0132] e i =|IoU i -IoU ref |

[0133] Among them, IoU i Let e ​​be the intersection-union ratio of the i-th evaluation unit. i The system assigns weights w to the intersection-union ratio (IU) differences of the evaluated units based on their spatial coverage or semantic category importance. i Calculate the comprehensive error index:

[0134]

[0135] Where n is the total number of evaluation units, the system will evaluate the e of each evaluation unit i Integrating with the comprehensive error E, a segmentation error index is generated that includes the spatial distribution and overall level of the error. The segmentation error index is stored in array form, and the array index corresponds to the evaluation unit number.

[0136] S52: Iterate the parameters of the segmentation error index, adjust the geometric complexity calculation coefficients of the geometric complexity evaluation model according to the error direction, and generate updated attention parameters.

[0137] Specifically, the system extracts the intersection-union ratio difference e from the segmentation error index. i The direction of error is determined through statistical analysis, when e i An error greater than 0 is considered a positive error, corresponding to a prediction result deviating from the true label; when e i A value ≤ 0 is considered a negative error, corresponding to a prediction result that matches the true label. Preferably, the system calls the geometric complexity evaluation model, and the formula for calculating the geometric complexity C is:

[0138] C = α·k + β·d

[0139] Where k is the average curvature of the point cloud region, d is the density variance of the point cloud region, and α and β are the geometric complexity calculation coefficients to be adjusted. For the positive error region, if the C value of the model output is lower than a preset threshold, α and β are increased; if the C value is higher than the threshold, α and β are decreased. The formula for the adjustment coefficients is as follows:

[0140]

[0141] Where γ is the iteration step size, C pred,i C represents the geometric complexity predicted by the model. true,i The geometric complexity is calculated based on the real labels. The system iterates repeatedly until E converges, generating updated attention parameters containing adjusted α, β, and associated weights. These parameters are stored in a floating-point array.

[0142] S53: Encapsulate the updated attention parameters into a strategy, build a priority allocation rule library based on historical segmentation performance, and generate the attention allocation strategy for the next frame's point cloud.

[0143] Specifically, the system reads historical segmentation data from the storage unit, including the E values ​​of each frame, updated attention parameters, and corresponding segmentation results, and sorts them by timestamp to form a historical segmentation performance dataset. Based on this dataset, the system constructs a priority allocation rule base, the core of which is the priority scoring formula:

[0144] P=λ·C+(1-λ)·S

[0145] Where P is the region priority score, λ is the geometric complexity weight, C is the geometric complexity of calculating the attention parameters, and S is the semantic importance score, which is obtained by multiplying the predefined category weights by the predicted probabilities. The system uses the least squares method to fit the relationship between P and E in historical data to determine the optimal value of λ. The system generates a resource allocation formula based on the rule base:

[0146]

[0147] Among them, R j Let P be the percentage of computing resources allocated to the j-th region, m be the total number of regions, and P be the percentage of computing resources allocated to the j-th region. j Let P be the resource demand index for the j-th region. k Let R be the resource demand index for the k-th region; the system will use R j The threshold range of λ and the multi-scale branch activation order are integrated to generate the attention allocation strategy for the next frame. The strategy is stored in structured data form, including the λ value, the hierarchical threshold of λ, and the corresponding R values ​​for each priority. j The branch activation sequence is stored at a specified address in the policy cache, and can be accessed via an index during processing in the next frame.

[0148] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0149] Based on the same inventive concept, this application also provides an attention-based point cloud multi-scale feature segmentation device for implementing the attention-based point cloud multi-scale feature segmentation method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more attention-based point cloud multi-scale feature segmentation device embodiments provided below can be found in the limitations of the attention-based point cloud multi-scale feature segmentation method described above, and will not be repeated here.

[0150] Preferably, such as Figure 3As shown, the present invention provides a point cloud multi-scale feature segmentation device 600 based on an attention mechanism, which is configured with the following modules:

[0151] The initial point cloud feature extraction module 610 is used to perform three-dimensional coordinate feature extraction and semantic label prediction processing on the acquired raw point cloud data. The point cloud coordinates are converted into feature vectors with semantic information through a preset point cloud segmentation network to generate an initial point cloud feature set.

[0152] The multi-scale feature vector generation module 620 is used to extract local geometric features and global semantic features from the initial point cloud feature set. It uses hierarchical sampling to construct neighborhood structures at different scales and generate a multi-scale feature vector set containing multi-resolution features.

[0153] The dynamic attention weight allocation module 630 is used to dynamically allocate attention weights to the multi-scale feature vector group based on the attention allocation strategy of the current frame, calculate the adaptive weight coefficient of the region attention according to the local geometric complexity and semantic importance, and generate the segmented feature region with priority label.

[0154] The multi-scale feature fusion processing module 640 is used to perform multi-scale branch activation and feature fusion processing on the segmented feature regions. It controls the execution order of feature extraction branches based on adaptive weight coefficients and fuses the output features of each branch to generate optimized segmentation results.

[0155] The attention allocation strategy update module 650 is used to update the closed-loop strategy of the optimized segmentation result, adjust the attention allocation parameters of the multi-scale branch in combination with the segmentation error, and generate the attention allocation strategy of the next frame point cloud. The attention allocation strategy is used to indicate the feature extraction priority of different regions.

[0156] Preferably, the initial point cloud feature extraction module 610 provided in this application is configured with the following units:

[0157] The point cloud coordinate standardization unit is used to standardize the coordinates of the acquired raw point cloud data, transform the coordinates of each point to a preset reference coordinate system, and generate a standardized point cloud.

[0158] The neighborhood geometric feature extraction unit is used to extract neighborhood features from the standardized point cloud. It calls the k-nearest neighbor algorithm to construct a local point group with spatial topology and calculates the normal vector to generate a local geometric feature set.

[0159] The semantic label prediction unit is used to predict semantic labels for local geometric feature sets, identify the object category of each point using a pre-trained classifier, and generate an initial point cloud feature set with semantic labels.

[0160] Preferably, the multi-scale feature vector generation module 620 provided in this application is configured with the following units:

[0161] Multi-scale point cloud partitioning unit is used to partition the initial point cloud feature set by scale, and generate sampling point clouds of different resolutions through voxel meshing to generate multi-scale point cloud subsets.

[0162] The local scale feature extraction unit is used to extract local features from multi-scale point cloud subsets, calculate the curvature and density distribution of point groups at each scale, and generate scale feature vectors.

[0163] The cross-scale feature fusion unit is used to perform global feature fusion on scale feature vectors. It calls the graph convolutional network to aggregate cross-scale information of neighboring point features and generate multi-scale feature vector groups.

[0164] Preferably, the dynamic attention weight allocation module 630 provided in this application is configured with the following units:

[0165] The geometric complexity calculation unit is used to calculate the geometric complexity of multi-scale feature vector groups based on the attention allocation strategy of the current frame. It calculates the curvature gradient change rate and point density distribution characteristics according to the strategy parameters and generates a complexity distribution map.

[0166] The semantic weight allocation unit is used to allocate semantic weights to the complexity distribution map, and combine it with a predefined object category priority table to mark key regions in the semantic structure of the 3D scene to generate a semantic weight matrix.

[0167] The region priority scoring unit is used to fuse the complexity distribution map and the semantic weight matrix. It generates a comprehensive region priority score through the weighting rules defined by the strategy, and generates priority-labeled segmentation feature regions. The segmentation feature regions are used to indicate the activation order of multi-scale branches.

[0168] Preferably, the geometric complexity calculation unit includes a curvature gradient calculation subunit, a curvature intensity normalization subunit, and a geometric complexity fusion calculation subunit. Specifically, the curvature gradient calculation subunit performs curvature gradient calculation on the multi-scale feature vector group, solving for the surface curvature change rate by subtracting the normal vectors of the k-nearest neighbor group to generate a curvature gradient tensor; the curvature intensity normalization subunit performs norm normalization on the curvature gradient tensor, calculating the L2 norm of the curvature gradient at each point and applying a nonlinear mapping to generate a normalized curvature intensity; and the geometric complexity fusion calculation subunit fuses the normalized curvature intensity with the point cloud density distribution of the multi-scale feature vector group, calculating the geometric complexity of the region feature change intensity through sigmoid activation and logarithmic transformation to generate a geometric complexity distribution map.

[0169] Preferably, the multi-scale feature fusion processing module 640 provided in this application is configured with the following units:

[0170] The branch sequence control unit is used to control the branch sequence of the segmented feature region, activate high, medium and low resolution branches in descending order of priority score, and generate a branch execution sequence.

[0171] The multi-branch feature extraction unit is used to extract features from the branch execution sequence, execute the geometric feature calculation of each scale branch in sequence, and generate multi-branch output features.

[0172] The cross-scale feature fusion unit is used to perform attention-weighted fusion of multi-branch output features. It integrates cross-scale features in the cross-scale feature space through a channel attention mechanism to generate optimized segmentation results. The optimized segmentation results are used to indicate the semantic segmentation structure of the 3D scene.

[0173] Preferably, the attention allocation strategy update module 650 provided in this application is configured with the following units:

[0174] The segmentation error measurement unit is used to measure the error of the optimized segmentation results, calculate the difference between the intersection-union ratio of the predicted results of the point cloud segmentation and the true labels, and generate a segmentation error index.

[0175] The attention parameter iterative update unit is used to iterate the parameters of the segmentation error index, adjust the geometric complexity calculation coefficients of the geometric complexity evaluation model according to the error direction, and generate updated attention parameters.

[0176] The attention strategy encapsulation unit is used to encapsulate the updated attention parameters into a strategy, build a priority allocation rule library based on historical segmentation performance, and generate the attention allocation strategy for the next frame point cloud.

[0177] In one embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described attention-based multi-scale feature segmentation method for point clouds.

[0178] In one embodiment, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described attention-based multi-scale feature segmentation method for point clouds.

[0179] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0180] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0181] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A point cloud multi-scale feature segmentation method based on an attention mechanism, characterized in that, Includes the following steps: S1: Perform 3D coordinate feature extraction and semantic label prediction on the acquired raw point cloud data. Convert the point cloud coordinates into feature vectors with semantic information through a preset point cloud segmentation network to generate an initial point cloud feature set. S2: Extract local geometric features and global semantic features from the initial point cloud feature set, construct neighborhood structures at different scales using hierarchical sampling, and generate a multi-scale feature vector set containing multi-resolution features; S3: Based on the attention allocation strategy of the current frame, perform dynamic attention weight allocation on the multi-scale feature vector group, calculate the adaptive weight coefficient of the region attention according to the local geometric complexity and semantic importance, and generate the segmentation feature region with priority labeling. S4: Perform multi-scale branch activation and feature fusion processing on the segmented feature region, control the execution order of feature extraction branches based on adaptive weight coefficients and fuse the output features of each branch to generate an optimized segmentation result; S5: Update the closed-loop strategy for the optimized segmentation result, adjust the attention allocation parameters of the multi-scale branch in combination with the segmentation error, and generate the attention allocation strategy for the next frame point cloud. The attention allocation strategy is used to indicate the feature extraction priority of different regions.

2. The method according to claim 1, characterized in that, S1 includes: S11: Standardize the coordinates of the acquired raw point cloud data, transform the coordinates of each point to a preset reference coordinate system, and generate a standardized point cloud; S12: Extract neighborhood features from the standardized point cloud, call the k-nearest neighbor algorithm to construct a local point group with spatial topology and calculate the normal vector to generate a local geometric feature set; S13: Perform semantic label prediction on the local geometric feature set, use a pre-trained classifier to identify the object category of each point, and generate an initial point cloud feature set with semantic labels.

3. The method according to claim 1, characterized in that, S2 includes: S21: The initial point cloud feature set is divided into scales, and sampling point clouds of different resolutions are generated by voxel meshing to generate multi-scale point cloud subsets. S22: Extract local features from the multi-scale point cloud subset, calculate the curvature and density distribution of the point group at each scale, and generate scale feature vectors; S23: Perform global feature fusion on the scale feature vectors, call the graph convolutional network to aggregate cross-scale information of neighboring point features, and generate a multi-scale feature vector group.

4. The method according to claim 1, characterized in that, S3 includes: S31: Based on the attention allocation strategy of the current frame, calculate the geometric complexity of the multi-scale feature vector group, calculate the curvature gradient change rate and point density distribution characteristics according to the strategy parameters, and generate a complexity distribution map. S32: Perform semantic weight allocation on the complexity distribution map, and combine the predefined object category priority table to mark the key regions in the semantic structure of the 3D scene to generate a semantic weight matrix; S33: The complexity distribution map and the semantic weight matrix are fused and calculated. A comprehensive priority score of the region is generated by the weighting rules defined by the strategy. A segmentation feature region with priority label is generated. The segmentation feature region is used to indicate the activation order of multi-scale branches.

5. The method according to claim 1, characterized in that, S31 includes: S311: Perform curvature gradient calculation on the multi-scale feature vector group, solve the surface curvature change rate by the normal vector difference of the k nearest neighbor point group, and generate the curvature gradient tensor. S312: Perform norm normalization on the curvature gradient tensor, calculate the L2 norm of the curvature gradient at each point, and apply nonlinear mapping to generate normalized curvature intensity. S313: The point cloud density distribution of the normalized curvature intensity and the multi-scale feature vector group is fused. The geometric complexity of the regional feature change intensity is calculated by Sigmoid activation and logarithmic transformation, generating a geometric complexity distribution map. The formula for calculating the geometric complexity is: Where C is the geometric complexity and σ is the sigmoid activation function. Let ρ be the L2 norm of the curvature gradient, ρ be the point cloud density, and ε be the smoothing factor, where ε = 10. -5 .

6. The method according to claim 1, characterized in that, S4 includes: S41: Perform branch sequence control on the segmented feature region, activate high, medium and low resolution branches in descending order of priority score, and generate a branch execution sequence; S42: Extract features from the branch execution sequence, perform geometric feature calculations for each scale branch in sequence, and generate multi-branch output features; S43: Perform attention-weighted fusion on the multi-branch output features, integrate cross-scale features in the cross-scale feature space through the channel attention mechanism, and generate an optimized segmentation result. The optimized segmentation result is used to indicate the semantic segmentation structure of the three-dimensional scene.

7. The method according to any one of claims 1-6, characterized in that, S5 includes: S51: Perform error measurement on the optimized segmentation result, calculate the difference between the predicted result of the point cloud segmentation result and the intersection-union ratio of the true label, and generate a segmentation error index; S52: Iterate the parameters of the segmentation error index, adjust the geometric complexity calculation coefficients of the geometric complexity evaluation model according to the error direction, and generate updated attention parameters; S53: Encapsulate the updated attention parameters into a strategy, construct a priority allocation rule library based on historical segmentation performance, and generate an attention allocation strategy for the next frame point cloud.

8. A point cloud multi-scale feature segmentation device based on an attention mechanism, characterized in that, The device includes: The initial point cloud feature extraction module is used to extract three-dimensional coordinate features and predict semantic labels from the acquired raw point cloud data. It converts the point cloud coordinates into feature vectors with semantic information through a preset point cloud segmentation network to generate an initial point cloud feature set. The multi-scale feature vector generation module is used to extract local geometric features and global semantic features from the initial point cloud feature set, and to construct neighborhood structures at different scales by using hierarchical sampling to generate a multi-scale feature vector set containing multi-resolution features. The dynamic attention weight allocation module is used to dynamically allocate attention weights to the multi-scale feature vector group based on the attention allocation strategy of the current frame, calculate the adaptive weight coefficient of the region attention according to the local geometric complexity and semantic importance, and generate the segmentation feature region with priority label. The multi-scale feature fusion processing module is used to perform multi-scale branch activation and feature fusion processing on the segmented feature region. It controls the execution order of feature extraction branches based on adaptive weight coefficients and fuses the output features of each branch to generate an optimized segmentation result. The attention allocation strategy update module is used to update the closed-loop strategy of the optimized segmentation result, adjust the attention allocation parameters of the multi-scale branch in combination with the segmentation error, and generate the attention allocation strategy of the next frame point cloud. The attention allocation strategy is used to indicate the feature extraction priority of different regions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images