Methods, apparatus, media and equipment for dense point-level prediction of large-scale point clouds

Abstract

This application provides a method, apparatus, medium, and device for dense point-level prediction of large-scale point clouds, relating to the field of point cloud processing technology. The method includes: dividing the original large-scale point cloud into planar subsets and non-planar subsets based on planarity-related indices; applying super-point clustering dominated by radiative and geometric features to the planar and non-planar subsets respectively, and generating super-point segmentation results at at least three scales; determining the local aggregation features of super-points at each scale, and extracting global context features by combining spatial serialization and bidirectional state space scanning; mapping the global context features of multi-scale super-points back to the point level and fusing them in a residual manner; and finally connecting the point-level task head to perform dense point-level prediction of large-scale point clouds. This application can stably complete high-precision dense point-level prediction on point clouds with hundreds of millions of points.

Description

Technical Field

[0001] This application relates to the field of point cloud processing technology, and in particular to a method, apparatus, medium and equipment for dense point-level prediction of large-scale point clouds. Background Technology

[0002] With the rapid iteration of remote sensing technologies such as LiDAR (Light Detection and Ranging), the scale and information dimensions of point cloud data acquisition have significantly improved. Currently, it is possible to stably acquire point cloud data volumes reaching hundreds of millions or even larger. This type of data not only contains rich 3D geometric information (such as coordinates, normals, and curvature) but also integrates radiation information (such as RGB color, reflection intensity, and multispectral data), providing fundamental support for point-level semantic understanding in the field of computer vision. The core task of point-level semantic understanding is to achieve fine-grained classification and attribute prediction for each point. Its technological achievements have been widely applied in key areas such as high-precision map construction for autonomous driving, digital twin city modeling, terrain mapping, and AR / VR scene generation, becoming one of the core technologies driving these fields towards higher resolution and higher precision.

[0003] However, facing massive, high-dimensional, and unstructured large-scale point cloud data, achieving accurate point-level semantic parsing while ensuring computational efficiency and controllable memory remains a serious challenge. Traditional processing methods generally struggle to balance adaptability to large-scale data with the integrity of feature representation. Constrained by problems such as excessive memory consumption, soaring computational complexity, and insufficient utilization of multi-source information, they are unable to meet the real-time, high-precision processing requirements of hundreds of millions of point cloud scenarios. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, medium and equipment for dense point-level prediction of large-scale point clouds, which can stably complete high-precision dense point-level prediction on point clouds with hundreds of millions of points.

[0005] According to a first aspect of this application, a method for dense point-level prediction of large-scale point clouds is provided, comprising:

[0006] Local geometric statistical analysis is performed on the original large-scale point cloud, and the original large-scale point cloud is divided into planar subsets and non-planar subsets based on planarity-related indices.

[0007] A superpoint clustering scheme dominated by radiation features is adopted for the planar subset, and a superpoint clustering scheme dominated by geometry features is adopted for the non-planar subset. At least three superpoint segmentation results at different scales are obtained by adjusting the clustering parameters. In each superpoint segmentation result, there are radiation-feature-dominated superpoints derived from the planar subset and geometry-feature-dominated superpoints derived from the non-planar subset.

[0008] For each scale of superpoint, a local feature aggregation operation with linear complexity is performed to obtain local aggregated features containing complete details of the superpoint's interior. Based on the local aggregated features, global context features of multi-scale superpoints are extracted at the superpoint level through spatial serialization processing combined with bidirectional state space scanning.

[0009] The global context features of the multi-scale superpoints are mapped back to the point-level dimension of the original large-scale point cloud according to the membership relationship of the superpoint clusters to obtain multi-scale point-level features. The multi-scale point-level features are aggregated by residual fusion and finally connected to the point-level task head to perform dense point-level prediction of the large-scale point cloud.

[0010] According to a second aspect of this application, a dense point-level prediction device for large-scale point clouds is provided, comprising:

[0011] The analysis module is used to perform local geometric statistical analysis on the original large-scale point cloud and divide the original large-scale point cloud into planar subsets and non-planar subsets based on planarity-related indices.

[0012] The clustering module is used to employ a radiation-dominated superpoint clustering scheme for the planar subset and a geometry-dominated superpoint clustering scheme for the non-planar subset. By adjusting the clustering parameters, it obtains superpoint segmentation results at least three different scales. In each scale of the superpoint segmentation result, there are radiation-dominated superpoints derived from the planar subset and geometry-dominated superpoints derived from the non-planar subset.

[0013] The extraction module is used to perform a local feature aggregation operation with linear complexity for each superpoint to obtain local aggregated features containing complete details inside the superpoint. Based on the local aggregated features, the global context features of the multi-scale superpoint are extracted at the superpoint level by combining spatial serialization processing with bidirectional state space scanning.

[0014] The prediction module is used to map the global context features of the multi-scale superpoints back to the point-level dimension of the original large-scale point cloud according to the membership relationship of the superpoint clusters, to obtain multi-scale point-level features. The multi-scale point-level features are aggregated by residual fusion and finally connected to the point-level task head to perform dense point-level prediction of the large-scale point cloud.

[0015] According to a third aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described dense point-level prediction method for large-scale point clouds.

[0016] According to a fourth aspect of this application, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described dense point-level prediction method for large-scale point clouds.

[0017] By employing the above technical solutions, this application provides a method, apparatus, medium, and device for dense point-level prediction of large-scale point clouds. First, the original large-scale point cloud is divided into planar subsets and non-planar subsets based on a flatness index, which lays the foundation for subsequent differentiated feature utilization, avoids clustering bias caused by confusion between planar and non-planar features, and improves the targeting of feature utilization. Second, super-point clustering dominated by radiation features and geometric features is adopted for the two types of subsets respectively, and super-point segmentation results at least three scales are obtained by adjusting parameters. This can fully utilize the radiation information of the planar subsets and the geometric information of the non-planar subsets, and enrich the feature receptive field through multi-scale design, reducing the accumulation of inter-layer errors. Furthermore, for each scale of super-point... By performing local feature aggregation with linear complexity, the complete details inside the superpoint are preserved while ensuring computational efficiency and controllable memory. Then, global context features are extracted by combining spatial serialization with bidirectional state space scanning, which can effectively solve the problems of high computational complexity, loss of local details and insufficient capture of global correlations in traditional methods. Finally, the global context features of multi-scale superpoints are mapped back to the point-level dimension according to the membership of member points. Combined with residual fusion to aggregate multi-scale point-level features, feature propagation errors can be reduced. Finally, the point-level task head is connected to complete dense prediction, which can realize high-precision, real-time point-level semantic parsing in the scenario of hundreds of millions of point clouds, and meet the needs of high-precision point cloud processing in fields such as autonomous driving high-precision maps and digital twin cities.

[0018] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 A flowchart illustrating a dense point-level prediction method for large-scale point clouds provided in an embodiment of this application is shown.

[0021] Figure 2 A flowchart illustrating a dense point-level prediction method for large-scale point clouds provided in another embodiment of this application is shown.

[0022] Figure 3 This illustration shows a structural schematic diagram of a dense point-level prediction device for large-scale point clouds provided in an embodiment of this application. Detailed Implementation

[0023] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0024] With the rapid iteration of remote sensing technologies such as LiDAR (Light Detection and Ranging), the scale and information dimensions of point cloud data acquisition have significantly improved. Currently, it is possible to stably acquire point cloud data volumes reaching hundreds of millions or even larger. This type of data not only contains rich 3D geometric information (such as coordinates, normals, and curvature) but also integrates radiation information (such as RGB color, reflection intensity, and multispectral data), providing fundamental support for point-level semantic understanding in the field of computer vision. The core task of point-level semantic understanding is to achieve fine-grained classification and attribute prediction for each point. Its technological achievements have been widely applied in key areas such as high-precision map construction for autonomous driving, digital twin city modeling, terrain mapping, and AR / VR scene generation, becoming one of the core technologies driving these fields towards higher resolution and higher precision.

[0025] However, facing massive, high-dimensional, and unstructured large-scale point cloud data, achieving accurate point-level semantic parsing while ensuring computational efficiency and controllable memory remains a serious challenge. Traditional processing methods generally struggle to balance adaptability to large-scale data with the integrity of feature representation. Constrained by problems such as excessive memory consumption, soaring computational complexity, and insufficient utilization of multi-source information, they are unable to meet the real-time, high-precision processing requirements of hundreds of millions of point cloud scenarios.

[0026] To address the aforementioned technical problems, embodiments of the present invention provide a dense point-level prediction method for large-scale point clouds, such as... Figure 1 As shown, the method includes:

[0027] Step 110: Perform local geometric statistical analysis on the original large-scale point cloud, and divide the original large-scale point cloud into planar subsets and non-planar subsets based on the planarity related index.

[0028] Among them, the original large-scale point cloud refers to an unstructured set of points, usually in the hundreds of millions or larger, collected by remote sensing equipment such as lidar, containing 3D geometric structure information (such as spatial coordinates and normal vectors) and radiation information (such as RGB color and reflection intensity); local geometric statistical analysis refers to the process of analyzing the geometric attributes of the surrounding local area for each point in the point cloud, which may include calculating the k-neighbor covariance matrix of the point, decomposing the covariance matrix to obtain eigenvalues, and then deriving statistical quantities such as normal, curvature, and flatness that reflect the local geometric shape, so as to quantitatively capture the local geometric features of each point; flatness-related indicators are quantitative indicators used to determine whether the local area of ​​the point cloud is close to a plane, which mainly include flatness and local curvature. Flatness is calculated using the eigenvalues ​​of the covariance matrix; the closer the value is to 1, the closer the local area is to a plane. Local curvature is also calculated using eigenvalues; the smaller the value, the closer the local area is to a plane. Both are used to determine the planar attributes of a point. A planar subset refers to the set of points whose local geometry conforms to planar characteristics (e.g., flatness reaches a preset threshold) after being judged by flatness-related indicators. These typically correspond to planar areas such as roads, grass, and walls in a scene, and their radiation information (e.g., RGB, reflection intensity) has stronger discriminative power. A non-planar subset refers to the set of points whose local geometry does not conform to planar characteristics (e.g., flatness does not reach a preset threshold) after being judged by flatness-related indicators. These typically correspond to non-planar areas such as building corners, tree branches, and terrain undulations in a scene, and their geometric characteristics (e.g., curvature, normal consistency) have stronger discriminative power.

[0029] In this embodiment of the disclosure, for each point in the original large-scale point cloud, the geometric attributes of its surrounding local area can be analyzed. For example, the covariance matrix of the k-neighborhood (the k nearest neighboring points in space) of each point can be calculated, and the feature values ​​reflecting the local geometric shape can be obtained through matrix decomposition. Then, flatness-related indicators such as flatness (or local curvature) can be calculated based on these feature values. After that, a quantization threshold (such as a flatness threshold of 0.6) is set, and points whose local geometric shape conforms to the planar features (such as flatness not less than 0.6) are classified as planar subsets, and points whose local geometric shape does not conform to the planar features (such as flatness less than 0.6) are classified as non-planar subsets. Finally, the original large-scale point cloud is split into two branches according to the differences in geometric attributes.

[0030] By accurately capturing the local geometric morphological differences of point clouds through local geometric statistics, and relying on quantified flatness-related indicators, feature confusion between planar and non-planar point clouds can be avoided. For example, point clouds of roads (planar areas) and building corners (non-planar areas) can be prevented from mixing. This can improve the accuracy of subsequent super-point clustering, reduce the overall complexity of point cloud processing, and ensure the efficiency and accuracy of large-scale point cloud processing.

[0031] Step 120: For planar subsets, adopt a superpoint clustering scheme dominated by radiation features; for non-planar subsets, adopt a superpoint clustering scheme dominated by geometry features; and obtain superpoint segmentation results at least three different scales by adjusting the clustering parameters.

[0032] The "at least three different scales" refers to at least three core superpoint scale levels generated for point cloud processing of the same scene, with the number of member points, coverage, and semantic association capabilities of superpoints at each scale exhibiting gradient differences. In the following implementation steps of this disclosure, the technical solution of this application is illustrated using at least three different scales, including small, medium, and large scales, as examples, but this does not constitute a specific limitation. Among them, small-scale superpoints average about 10 member points per superpoint, covering a small spatial range and focusing on preserving local fine details of the point cloud (such as the edges of road markings and the subtle structure of building corners); medium-scale superpoints average about 30 member points per superpoint, with moderate coverage and spatial correlation capabilities, and can convey contextual information at the road segment or area level (such as the relationship between a road segment and adjacent sidewalks); large-scale superpoints average about 50 member points per superpoint, covering a large spatial range and providing regional global context (such as the overall relationship between roads, buildings, and greenery in urban scenes). The superpoint segmentation result refers to the structured data set formed after dividing a subset of the point cloud into several superpoints through superpoint clustering. Each superpoint is a local set composed of several original points with similar features. In the superpoint segmentation results at each scale, there are both radial feature-dominated superpoints derived from planar subsets and geometric feature-dominated superpoints derived from non-planar subsets.

[0033] In this embodiment of the disclosure, based on the attribute differences between planar subsets and non-planar subsets obtained from the initial splitting of the point cloud, superpoint clustering is performed on the planar subsets, focusing on using radiation features (such as RGB color, reflection intensity, and other information that can reflect material or surface attributes) to aggregate planar points with similar attributes (such as points on the same road or the same grassland) to form superpoints; superpoint clustering is performed on the non-planar subsets, focusing on using geometric features (such as curvature consistency, normal matching degree, point density distribution, spatial connectivity, and other information that can reflect three-dimensional structure) to aggregate non-planar points with similar structures (such as points on the same building corner or the same tree branch) to form superpoints; at the same time, by adjusting key parameters in the clustering process (such as the spectral threshold of planar subset clustering, the connectivity radius of non-planar subset clustering, etc.), the number of member points contained in the superpoints is controlled, thereby generating at least three sets of superpoints at different scales (such as small scale, medium scale, and large scale). Each set of superpoints at each scale contains superpoints dominated by radiation features derived from the planar subsets and superpoints dominated by geometric features derived from the non-planar subsets, ultimately completing multi-scale superpoint segmentation.

[0034] By employing differentiated feature-driven superpoint clustering for planar and non-planar subsets, the core discriminative features of both types of subsets can be fully explored. Radiation features can accurately distinguish material differences in planar regions (such as roads and road markings), while geometric features can effectively identify structural differences in non-planar regions (such as buildings and trees). This avoids the limitations of using a single feature to process both types of subsets. Simultaneously, by adjusting the clustering parameters, at least three different scales of superpoint segmentation results can be obtained. This not only preserves the ability of small-scale superpoints to capture local details but also enriches global contextual information through medium- and large-scale superpoints. This reduces the accumulation of inter-layer errors in subsequent feature learning processes and lays the foundation for subsequent multi-scale feature fusion and high-precision point-level prediction. Overall, this improves the targeting and effectiveness of large-scale point cloud processing, balancing the preservation of local details with the coverage of global information.

[0035] Step 130: Perform a local feature aggregation operation with linear complexity for each superpoint to obtain local aggregated features containing complete details of the superpoint's interior. Based on the local aggregated features, extract the global context features of the multi-scale superpoints by combining spatial serialization processing with bidirectional state space scanning at the superpoint level.

[0036] In this context, each point in the original large-scale point cloud is a basic unit constituting a superpoint. A superpoint is a local set formed by clustering multiple points with similar characteristics (geometric or radial features) from the original point cloud. Each scale of superpoints refers to at least three scales (small, medium, and large) generated by adjusting the superpoint granularity. Small-scale superpoints average approximately 10 points / superpoint, medium-scale approximately 30 points / superpoint, and large-scale approximately 50 points / superpoint. Local aggregation features are feature vectors obtained through linear complexity local feature aggregation. These vectors condense the common features of member points within a superpoint while preserving key detail differences (such as variations in reflection intensity in different areas within the same road superpoint), providing a precise representation of the superpoint's local features. Spatial serialization processing refers to calculating the 3D centroid of the superpoint. Hilbert curve indexing is the process of arranging superpoints at the same scale in an ordered manner. Its purpose is to maintain the local continuity of spatially adjacent superpoints in the sequence, providing a spatial correlation basis for global context extraction. Bidirectional state space scanning refers to performing forward (from the beginning to the end of the sequence) and backward (from the end to the beginning of the sequence) recursive state space scans on the local aggregated feature sequence of ordered superpoints, and concatenating or weighting the two outputs to capture the long-distance forward and backward correlations between superpoints. Global context features refer to a feature set that covers multiple scales and includes local details of superpoints at each scale and global correlation information between superpoints. The global context features at each scale correspond to the spatial and semantic correlations of superpoints at that scale, jointly supporting the subsequent multi-scale feature fusion.

[0037] In this embodiment of the disclosure, for each superpoint at each scale, the computational cost increases linearly with the number of member points within the superpoint. The features of all member points within the superpoint are aggregated. While controlling the computational cost, the detailed differences within the superpoint are fully preserved, ultimately obtaining local aggregated features that accurately represent the local features of the superpoint. Then, according to the spatial position association of the superpoints (such as sorting by the 3D spatial index of the superpoint centroid), the superpoints of the same scale are organized into an ordered sequence to ensure that spatially adjacent superpoints maintain continuity in the sequence. Then, forward and reverse state space recursive scans are performed on the local aggregated feature sequence of the ordered superpoints, and the two outputs are concatenated or weighted and fused to finally extract multi-scale superpoint global context features that cover all scales and include local details and global associations of the superpoints.

[0038] By using linear complexity for local feature aggregation, while ensuring the efficiency of processing large-scale point clouds of hundreds of millions, it can completely preserve the internal details of superpoints, solving the problem of detail loss caused by traditional max / average pooling; spatial serialization processing, through index sorting, can ensure the spatial continuity of the superpoint sequence and avoid disordered arrangement from destroying spatial correlation information; bidirectional state space scanning can fully capture the long-distance forward and backward correlations between superpoints, making up for the deficiency of insufficient global information coverage in single-direction scanning.

[0039] Step 140: Map the global context features of multi-scale superpoints back to the point-level dimension of the original large-scale point cloud according to the membership relationship of the superpoint clusters to obtain multi-scale point-level features. Use residual fusion to aggregate the multi-scale point-level features, and finally connect the point-level task head to perform dense point-level prediction of the large-scale point cloud.

[0040] In this embodiment of the disclosure, the global context features of the corresponding scale superpoint can be accurately mapped to all the original points contained in the superpoint, so that each original point obtains features of the corresponding scale, thereby forming multi-scale point-level features. Then, feature transfer loss can be reduced by residual connection, and the dimensions of features of different scales can be integrated by combining dimensional concatenation to achieve efficient aggregation of multi-scale point-level features. Finally, the aggregated features are input into the point-level task head. After the feature dimensions are adjusted by the adaptation step in the task head, the classification step performs semantic category determination on each point in the original large-scale point cloud, generates the prediction result for each point, and completes dense point-level prediction.

[0041] By reflecting features based on member point affiliation, the accuracy of mapping super-point features to point-level dimensions can be ensured, avoiding feature misalignment. When aggregating multi-scale point-level features, the residual fusion method can preserve the local details of small-scale features and the global correlation of large-scale features, while significantly reducing the accumulation of inter-layer errors in traditional fusion methods. Finally, by directly predicting each point through the point-level task head, high-precision dense point-level prediction of hundreds of millions of point clouds can be achieved while maintaining linear computational complexity and memory friendliness, meeting the needs of autonomous driving, digital twin cities and other scenarios for fine semantic understanding.

[0042] In summary, the dense point-level prediction method for large-scale point clouds provided by this invention firstly divides the original large-scale point cloud into planar subsets and non-planar subsets based on a flatness index. This lays the foundation for subsequent differentiated feature utilization, avoids clustering bias caused by confusion between planar and non-planar features, and improves the targeting of feature utilization. Secondly, for the two types of subsets, super-point clustering dominated by radiation features and geometric features is adopted respectively. By adjusting the parameters, super-point segmentation results at least three scales are obtained. This not only fully utilizes the radiation information of the planar subsets and the geometric information of the non-planar subsets, but also enriches the feature receptive field through multi-scale design, reducing the accumulation of inter-layer errors. Furthermore, a linear complexity algorithm is performed on the super-points at each scale. Local feature aggregation preserves the complete details inside the superpoint while ensuring computational efficiency and controllable memory. Then, global context features are extracted by combining spatial serialization with bidirectional state space scanning, which can effectively solve the problems of high computational complexity, loss of local details, and insufficient capture of global correlations in traditional methods. Finally, the global context features of multi-scale superpoints are mapped back to the point-level dimension according to the membership of member points. Combined with residual fusion, multi-scale point-level features are aggregated, which can reduce feature propagation errors. Finally, the point-level task head is connected to complete dense prediction, which can realize high-precision, real-time point-level semantic parsing in the scenario of hundreds of millions of point clouds, and meet the needs of high-precision point cloud processing in fields such as autonomous driving high-precision maps and digital twin cities.

[0043] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the implementation of this embodiment, this embodiment also provides another method for dense point-level prediction of large-scale point clouds, such as... Figure 2 As shown, the method includes:

[0044] Step 210: Perform local geometric statistical analysis on the original large-scale point cloud, and divide the original large-scale point cloud into planar subsets and non-planar subsets based on the planarity related index.

[0045] In specific application scenarios, step 210 of the embodiment may include the following steps:

[0046] Step 210-1: Perform dimensional normalization on the coordinate and radiation information of the original large-scale point cloud to eliminate numerical differences between data of different dimensions.

[0047] Among them, coordinate information refers to the spatial location data of each point in the point cloud, which usually exists in the form of XYZ three-dimensional coordinates and is used to characterize the specific orientation of the point in three-dimensional space; radiation information refers to the information carried by each point in the point cloud that reflects the material or properties of the object's surface, mainly including RGB color, reflection intensity, multispectral data, etc.

[0048] In this embodiment of the disclosure, the coordinate information describing spatial location and the radiation information describing surface properties in point cloud data can be processed using appropriate normalization algorithms (such as standardization or normalization to a fixed interval), respectively. This maps the values ​​of the two types of information to the same or similar order of magnitude, thereby eliminating the problem of feature weight imbalance caused by excessive differences in the original values ​​of data from different dimensions. This provides basic data of uniform order of magnitude and direct comparability for subsequent processing steps such as local geometric statistical analysis and superpoint clustering.

[0049] Step 210-2: Calculate the k-neighborhood data of each point in the normalized original large-scale point cloud, and construct the covariance matrix of each point based on the k-neighborhood data.

[0050] In the embodiments of this disclosure, for the original large-scale point cloud after dimensional normalization, for each point, its k nearest neighbor points in three-dimensional space can be found by spatial distance calculation (such as Euclidean distance). The set of these neighbor points and their carried features (such as coordinates, normals, etc.) together constitute the k-neighborhood data of the point. Then, based on the k-neighborhood data, a covariance matrix is ​​generated by calculating the covariance between features to describe the spatial distribution dispersion of points in the neighborhood and the correlation of features. This covariance matrix can quantitatively reflect the geometric shape of the local area around each point (such as whether it is biased towards a plane, curved surface or irregular distribution).

[0051] By calculating k-neighborhood data, the local spatial correlation information of each point can be accurately captured, avoiding geometric misjudgment caused by relying solely on information from a single point. The covariance matrix constructed based on k-neighborhood data can transform the discrete distribution of points within the neighborhood into a quantifiable matrix form, providing core data support for subsequent calculation of planar criteria such as flatness and local curvature through eigenvalue decomposition, ensuring the accuracy of the separation between planar subsets and non-planar subsets.

[0052] Step 210-3: Perform eigenvalue decomposition on the covariance matrix of each point to obtain eigenvalues ​​that reflect the local geometry of the surrounding area. Calculate flatness or local curvature based on the eigenvalues ​​and use flatness or local curvature as a plane criterion to determine the plane properties of each point.

[0053] Among them, planar attributes refer to specific attributes in which the geometric shape of the local region where a point is located in a point cloud conforms to the characteristics of a near-planar plane. They are the core basis for determining whether a point belongs to a planar subset.

[0054] In this embodiment of the disclosure, the covariance matrix describing the geometric distribution of a point's neighborhood can be decomposed into three eigenvalues ​​(usually ordered from largest to smallest as λ1≥λ2≥λ3) through matrix operations. The numerical differences of these eigenvalues ​​can directly reflect the geometric shape of the local area surrounding the point. Based on these eigenvalues, flatness or local curvature can be calculated using a specific formula: the flatness formula is P=(λ2... The flatness or local curvature is calculated as C = λ3 / (λ1 + λ2 + λ3), where a value closer to 1 indicates a local area closer to a plane. The smaller the value, the closer the local area is to a plane. Using the calculated flatness or local curvature as a plane criterion, the plane attribute of each point can be determined.

[0055] By transforming the local geometric information contained in the covariance matrix into intuitive eigenvalues ​​through eigenvalue decomposition, a quantitative basis for determining planar attributes can be provided, avoiding the errors of traditional subjective experience-based judgments. The two criteria of flatness and local curvature measure local geometric shapes from different dimensions, ensuring the accuracy of the judgment results and effectively distinguishing between planar areas (such as roads and walls) and non-planar areas (such as building edges and tree branches).

[0056] Step 210-4: Points whose corresponding planar attribute determination results are not less than the preset threshold are assigned to the planar subset, and the remaining points are assigned to the non-planar subset, thus completing the point cloud dual-branch splitting.

[0057] In this embodiment of the present disclosure, after calculating the planar attribute determination result (flatness or local curvature) of each point through eigenvalue decomposition, classification can be performed according to a pre-set quantization threshold (e.g., the flatness threshold is set to 0.6): if the flatness of a point is not less than 0.6 (or the local curvature is not greater than the corresponding threshold), it indicates that the local region where the point is located conforms to planar features, and it is classified into the planar subset; if the planar attribute determination result of a point is less than the preset threshold, the local region where the point is located exhibits non-planar features, and it is classified into the non-planar subset; through this threshold-based classification method, the original large-scale point cloud is finally completely split into two independent subsets, planar and non-planar, forming a dual-branch data structure that can be processed using differentiated schemes in the future.

[0058] By quantifying and classifying the results of planar attribute determination by setting a preset threshold, the objectivity and consistency of the division between planar and non-planar subsets can be ensured, avoiding classification errors caused by subjective experience judgment. Splitting the point cloud into two branches according to local geometric features allows subsequent processing (such as using radial features to dominate clustering for planar subsets and using geometric features to dominate clustering for non-planar subsets) to accurately match the attribute characteristics of each subset, avoiding clustering bias or insufficient feature utilization caused by the mixing of point clouds with different geometric attributes.

[0059] Step 220: Perform radiation feature-driven superpoint clustering on the planar subset to obtain planar superpoints. The radiation features include at least RGB color and reflection intensity.

[0060] In this embodiment of the disclosure, when performing super-point clustering on a subset of planes that has passed the planar attribute determination and whose local geometry is close to a plane, traditional geometric features such as spatial coordinates are no longer used as the core criteria. Instead, the focus is on using radiation features that reflect the differences in material and properties of the object's surface as the primary basis for clustering. These features include at least RGB color (reflecting visual color differences) and reflection intensity (reflecting the differences in the object's surface's ability to reflect laser light). By setting a similarity threshold for radiation features (such as the range of RGB color differences and the fluctuation range of reflection intensity) and necessary spatial connectivity constraints, points with similar radiation features and spatial adjacency in the planar subset are aggregated into local sets with unified semantic attributes. These sets are the planar superpoints. The points within each planar superpoint are highly similar in radiation features such as RGB color and reflection intensity, corresponding to a planar region with a specific function or material in the scene.

[0061] Step 230: Perform geometric feature-driven superpoint clustering on the non-planar subset to obtain local superpoint segments. The geometric features include at least curvature consistency, normal matching degree, point density distribution, and spatial connectivity.

[0062] Among them, curvature consistency refers to the fact that the curvature values ​​of all points within a local area are within a preset range, reflecting the uniformity of the bending law of the three-dimensional structure in that area (e.g., in a high curvature area at the corner of a building, the curvature difference between points is small, i.e., high curvature consistency); normal matching degree refers to the fact that the angle between the normal vector directions of all points within a local area is within a preset threshold, reflecting the consistency of the surface orientation of that area (e.g., the normal vectors of points on the side of a street lamp pole are all perpendicular to the pole's axis, i.e., high normal matching degree); point density distribution refers to the similarity in the spatial density of points within a local area, reflecting the compactness of the structure in that area. Features (such as the significant difference in point density between the edge of a building wall and the branch of a tree); spatial connectivity refers to the fact that points are within a preset distance range in three-dimensional space, which is the basic constraint of superpoint clustering, ensuring that the aggregated superpoints are continuous local segments in space, rather than a set of discrete points; local superpoint segments refer to the set of local points with unified three-dimensional structural attributes and semantic meanings obtained by performing geometric feature-driven superpoint clustering on non-planar subsets. Each local superpoint segment corresponds to a specific three-dimensional structural region in the scene (such as a building corner or a tree branch).

[0063] In this embodiment of the disclosure, when performing super-point clustering on non-planar subsets with local geometric shapes that are three-dimensional or irregular, geometric features that accurately reflect the differences in their three-dimensional structure can be used as the core similarity criterion, rather than relying on radial features; wherein the geometric features include at least curvature consistency, normal matching degree, point density distribution, and spatial connectivity. By setting similarity thresholds for these geometric features (such as curvature difference range, normal angle threshold), points with similar geometric features and spatial connectivity in the non-planar subsets are aggregated into local sets with unified three-dimensional structural attributes, and the resulting sets are the local super-point segments.

[0064] Step 240: By adjusting the size of the clustering parameters, the planar superpoints are made to form at least three radiation feature-dominant superpoints of different scales, and the local superpoint segments are made to form at least three geometric feature-dominant superpoints of different scales.

[0065] Among them, radiation feature-dominated superpoints refer to superpoints whose core aggregation criteria are radiation features (RGB color, reflection intensity), and the radiation features of points within the superpoints are highly similar; geometric feature-dominated superpoints refer to superpoints whose core aggregation criteria are geometric features (curvature consistency, normal matching degree, point density distribution), and the geometric features of points within the superpoints are highly similar.

[0066] In this embodiment of the disclosure, for superpoint clustering dominated by the radial features of planar subsets and superpoint clustering dominated by the geometric features of non-planar subsets, the clustering parameters affecting the superpoint size can be adjusted to control the number of member points included in the superpoint, thereby generating at least three sets of superpoints at different scales (e.g., small-scale, medium-scale, and large-scale). For planar subsets, the similarity threshold and spatial connectivity radius of the radial features are mainly adjusted; for non-planar subsets, the similarity threshold and spatial connectivity radius of the geometric features are mainly adjusted. Specifically, by adjusting the clustering parameters to small parameters (e.g., small connectivity radius, strict feature similarity threshold), the superpoints aggregate only a small number of adjacent points with highly similar features, forming small-scale superpoints; by adjusting the clustering parameters to medium parameters, the superpoints aggregate a medium number of points, forming medium-scale superpoints; and by adjusting the clustering parameters to large parameters, the superpoints aggregate more adjacent points, forming large-scale superpoints. Ultimately, planar superpoints generate at least three radiation-dominated superpoints of different scales, while local superpoint fragments of non-planar subsets generate at least three geometric-dominated superpoints of different scales, and the scale levels of the two types of superpoints match each other (e.g., both planar and non-planar superpoints contain small, medium, and large scales).

[0067] By adjusting clustering parameters to generate multi-scale superpoints, small-scale superpoints can accurately preserve local details (such as small-scale superpoints in planar subsets distinguishing fine road markings, and small-scale superpoints in non-planar subsets distinguishing fine branches of trees), while medium- and large-scale superpoints can gradually cover a wider spatial range and contextual relationships (such as large-scale superpoints covering an entire road segment or a whole building area). At the same time, the scale hierarchy matching of planar and non-planar superpoints can ensure that subsequent multi-scale feature fusion can accurately correspond to the contextual information of the same spatial range, avoiding feature deviations caused by scale misalignment.

[0068] Step 250: Perform a local feature aggregation operation with linear complexity for each superpoint to obtain local aggregated features containing complete details of the superpoint's interior.

[0069] For embodiments of this disclosure, step 250 may include the following steps:

[0070] Step 250-1: For a single superpoint at any scale, collect the input feature vectors of all member points within that superpoint.

[0071] In this embodiment of the disclosure, an independent superpoint can be selected from at least three different scales. Then, the core feature data carried by each of the original points (i.e., member points) contained in the superpoint can be extracted and summarized in vector form. The composition of the input feature vector is adapted to the difference in superpoint type: if it is a radiation feature-dominated superpoint, the input feature vector includes radiation features such as RGB color and reflection intensity; if it is a geometric feature-dominated superpoint, the input feature vector includes geometric features such as curvature, normal, and point density.

[0072] Step 250-2: Perform lightweight initial convergence on the input feature vector to obtain the initial aggregated features of the superpoint.

[0073] In this embodiment of the disclosure, for the input feature vectors of all member points within a single superpoint, a simple operation with low computational complexity and minimal information loss (such as element-wise averaging or summation, rather than complex deep network mapping) can be used for preliminary feature integration. This process does not change the core dimensions and key details of the features; it only condenses the scattered features of multiple member points within the superpoint into a feature vector that can initially represent the commonalities of the superpoint. This condensed vector is the initial aggregated feature of the superpoint, serving as the basic input for subsequent more refined feature aggregation operations. This lightweight initial aggregation operation avoids the computational burden of traditional complex aggregation methods (such as deep convolution), adapts to the efficient processing requirements of large-scale point clouds, and can quickly extract the common features of member points within the superpoint, providing a concise and complete initial input for subsequent sparse Mamba local aggregation.

[0074] Step 250-3: Randomly sort the input feature vectors of the member points within the superpoint to obtain the member point feature sequence, and append the initial aggregated feature as the tail aggregated token to the end of the member point feature sequence to construct the input sequence.

[0075] In this embodiment of the disclosure, the input feature vectors of all member points contained in a single superpoint can be arranged into an ordered member point feature sequence by randomly shuffling the order. The initial aggregated features that represent the commonalities of superpoints, obtained through lightweight initial aggregation, are added to the end of the member point feature sequence as the last feature unit. Finally, a complete input sequence of member point feature sequence + tail aggregation token is constructed. This sequence contains both the individual feature information of all member points and the common features of superpoints, which can provide structured data for the recursive processing of subsequent local aggregation of superpoints.

[0076] In this operation, random sorting, under the premise of high homogeneity of member point features within the superpoint, not only saves the computational cost of sorting by spatial or feature dimension, adapting to the high-efficiency processing requirements of large-scale point clouds, but also avoids the local feature bias that may be caused by fixed sorting. The addition of the tail aggregation token can provide summary common features for the subsequent recursive scanning of the state space model, enabling more efficient integration of the individual feature information of all member points during the recursive process, ensuring that the superpoint aggregation features cover both details and commonalities.

[0077] Step 250-4: Scan the input sequence using a linear recursive approach, and selectively retain key features through an input-dependent gating mechanism, outputting local aggregated features containing complete details inside the superpoints.

[0078] In this embodiment of the disclosure, the sequence can be scanned element by element through discrete recursive operations of the state-space model, following the order of the input sequence. Specifically, the output state of the previous element in the sequence can be used as a basis, combined with the feature vector of the current element, and then processed through a discretized state transition formula ( In the formula, , These are the state-space coefficients. This is the current state. This represents the state after processing the previous element. This is the input feature vector of the current element. , Update the state element by element (to the length of the input sequence), and simultaneously calculate the initial output for the current element. In this process, the input-dependent gating mechanism is based on the input feature vector of the current element. Generate a gating vector; the value of the gating vector determines the initial output. The degree of preservation. A value close to 1 retains the key features of the element, while a value close to 0 filters out redundant information. Finally, when scanning to the end of the sequence and the last aggregated token, its output integrates the key features of all member points and the common information of the superpoints, forming a local aggregated feature that contains complete details inside the superpoint (individual differences and overall commonalities of member points).

[0079] Step 260: Based on local aggregation features, global context features of multi-scale superpoints are extracted at the superpoint level by combining spatial serialization processing with bidirectional state space scanning.

[0080] For embodiments of this disclosure, step 260 may include the following steps:

[0081] Step 260-1: Calculate the centroid coordinates of all hyperpoints at each scale, and generate a 3D Hilbert curve index for each hyperpoint based on the centroid coordinates.

[0082] For the embodiments of this disclosure, for all superpoints at each scale, the three-dimensional spatial coordinates (XYZ) of all member points within each superpoint can be used as a basis. By calculating the average of the coordinates in the three dimensions of X, Y, and Z respectively, the centroid coordinates that can accurately represent the overall spatial position of the superpoint can be obtained. Then, the centroid coordinates of each superpoint are used as the input of the 3D Hilbert space filling curve. The curve is used to convert the three-dimensional centroid coordinates into a unique one-dimensional value, namely the 3D Hilbert curve index of each superpoint.

[0083] Step 260-2: Sort the local aggregated features of all superpoints at this scale according to the 3D Hilbert curve index to form an ordered superpoint feature sequence.

[0084] In this embodiment of the disclosure, after extracting the local aggregated features of all superpoints at a certain scale and generating a 3D Hilbert curve index for each superpoint based on its centroid coordinates, the 3D Hilbert curve index can be used as the unique sorting criterion to match and associate the local aggregated features of all superpoints at that scale in ascending (or descending) order of index values. Since each superpoint uniquely corresponds to a 3D Hilbert curve index and a set of local aggregated features, the orderliness of the index is directly transformed into the orderliness of the features. This transforms the originally discretely distributed local aggregated features of the superpoints into an ordered set that is highly consistent with the three-dimensional spatial distribution pattern of the superpoints. Ultimately, an ordered superpoint feature sequence that combines the internal details of the superpoints with the spatial correlation between superpoints is constructed, providing structured input for subsequent bidirectional state space scanning at the superpoint level to extract the global context.

[0085] Step 260-3: Perform forward state space recursive scan and reverse state space recursive scan on the ordered superpoint feature sequence to obtain the forward superpoint feature sequence and the reverse superpoint feature sequence.

[0086] Among them, the forward superpoint feature sequence refers to the feature set generated after performing a forward recursive scan of the state space on the ordered superpoint feature sequence. The feature of each superpoint implies its forward spatial dependency relationship with the preceding superpoint (such as the association of A→B→C), reflecting the forward context of the superpoint; the reverse superpoint feature sequence refers to the feature set generated after performing a reverse recursive scan of the state space on the ordered superpoint feature sequence. The feature of each superpoint implies its backward spatial dependency relationship with the following superpoint (such as the association of D→C→B), reflecting the backward context of the superpoint.

[0087] In this embodiment of the disclosure, after obtaining an ordered sequence of superpoint features sorted by 3D Hilbert curve index and containing superpoint spatial associations and internal details, a bidirectional scan can be performed on the ordered sequence: The forward state space recursive scan starts from the first element of the sequence (the superpoint feature with the smallest index) and proceeds element by element in the natural order of the sequence (index from smallest to largest). The state is continuously updated using the aforementioned state transition formula, and the forward feature corresponding to each element is output synchronously, ultimately forming a complete forward superpoint feature sequence. The reverse state space recursive scan starts from the last element of the sequence (the superpoint feature with the largest index) and proceeds recursively element by element in reverse order of the sequence (index from largest to smallest). The same state transition logic is used to pass the reverse state, and the reverse feature corresponding to each element is output, generating a reverse superpoint feature sequence. Throughout the process, the bidirectional scans share the state space model parameters, only in opposite directions, ensuring that the forward scan captures the forward spatial associations of the superpoints and the reverse scan captures the backward spatial associations, forming a complete coverage of the global spatial context of the superpoints.

[0088] Step 260-4: Concatenate or weightedly fuse the forward and reverse superpoint feature sequences to obtain the global context features of the superpoint at this scale.

[0089] Among them, global context features refer to the superpoint-level features obtained by fusing positive and negative superpoint features. They include the full spatial dependency relationship (global association) between the superpoint and all related superpoints before and after it, and also retain the internal details of the superpoint itself (such as differences in radiation / geometric features). They are a complete expression of the semantic attributes of the superpoint.

[0090] In this embodiment of the disclosure, after obtaining the forward and reverse superpoint feature sequences by recursively scanning the ordered superpoint feature sequences in the bidirectional state space, an appropriate fusion method can be selected based on the superpoint feature expression requirements. Specifically, if it is necessary to fully preserve the bidirectional context details, a concatenated fusion method can be used, directly concatenating the features of each superpoint in the forward sequence with the corresponding features of the superpoint in the reverse sequence in terms of dimension (e.g., the forward feature is d-dimensional, the reverse feature is d-dimensional, and the concatenation results in 2d-dimensionality). If it is necessary to balance the importance of the bidirectional context, a weighted fusion method can be used, assigning reasonable weights to the forward and reverse features respectively (e.g., both are 0.5, or dynamically adjusted according to the superpoint type), and obtaining the fused features through element-level weighted summation. Both fusion methods can fully integrate the forward and backward spatial association information of the superpoints, ultimately enabling each superpoint at this scale to obtain global context features containing the full-dimensional spatial context and its own internal details, providing global semantic support for subsequent multi-scale feature replay and point-level prediction.

[0091] Step 270: Map the global context features of the multi-scale superpoints back to the point-level dimension of the original large-scale point cloud according to the membership relationship of the superpoint clusters, and obtain the multi-scale point-level features.

[0092] For embodiments of this disclosure, step 270 may include the following steps:

[0093] Step 270-1: According to the scale order from smallest to largest in at least three different scales, based on the membership relationship of the superpoint cluster, map the global context features of the superpoint at the corresponding scale to all the points of the original large-scale point cloud contained in the superpoint, and obtain the single-scale point-level features of the superpoint at the corresponding scale.

[0094] After obtaining the global context features of superpoints at at least three different scales (small, medium, and large), the superpoints are processed sequentially from small to large scale: First, for small-scale superpoints, based on the membership relationships recorded during superpoint clustering (i.e., each point in the original large-scale point cloud clearly belongs to which small-scale superpoint), the global context features of the corresponding small-scale superpoint are directly assigned to all the original points contained in that superpoint, so that each original point obtains the global semantic information of the superpoint at the small scale, generating small-scale point-level features; then, medium-scale superpoints are processed using the same logic, and based on the membership relationships of medium-scale membership points, the global context features of medium-scale superpoints are mapped to the original points they contain, obtaining medium-scale point-level features; finally, large-scale superpoints are processed, and feature mapping is completed based on the membership relationships of large-scale membership points, generating large-scale point-level features.

[0095] By mapping the scale order from small to large with the membership relationship of member points, small-scale point-level features can accurately preserve the local details and relationships of the original points, while medium- and large-scale point-level features gradually supplement the original points with a broader global context, avoiding the loss of details or global information by single-scale features. At the same time, the direct mapping based on the membership relationship of member points ensures that the global context features of the superpoints can be transmitted to the original points without bias, avoiding semantic confusion caused by feature misalignment. The resulting point-level features of different scales can provide hierarchical and semantically complementary basic data for subsequent residual fusion, so that the fused point-level features can preserve the accuracy of local details and cover global spatial relationships, significantly improving the accuracy and semantic consistency of large-scale point cloud dense prediction (such as segmentation and classification).

[0096] Step 270-2: Determine at least three single-scale point-level features corresponding to different scales as multi-scale point-level features. Multi-scale point-level features include at least small-scale point-level features, medium-scale point-level features, and large-scale point-level features.

[0097] In the embodiments of this disclosure, after generating single-scale point-level features such as small-scale point-level features, medium-scale point-level features, and large-scale point-level features, the single-scale point-level features of the same original point at these three or more scales can be associated to clarify that these features belonging to different scales but pointing to the same original point together constitute the multi-scale point-level features of the original point.

[0098] Step 280: Use residual fusion to aggregate multi-scale point-level features, and finally connect the point-level task head to perform dense point-level prediction of large-scale point clouds.

[0099] For embodiments of this disclosure, step 280 may include the following steps:

[0100] Step 280-1: Aggregate multi-scale point-level features by residual connection and dimensional concatenation to obtain the aggregated target point-level features.

[0101] After obtaining small-scale, medium-scale, and large-scale point-level features of the same original point, the small-scale and medium-scale point-level features can be synchronously transferred to the large-scale aggregation stage through residual connections to ensure that the fine local semantics of the small scale and the medium spatial correlation of the medium scale are not lost. Then, the small-scale, medium-scale, and large-scale point-level features retained by the residual connections are directly superimposed on the feature dimensions by using dimensional concatenation, so that the semantic information of each scale is complementary and fused, and finally the aggregated target point-level features are obtained.

[0102] Step 280-2: Input the target point-level features into the point-level task header. Through the feature dimension adaptation layer in the point-level task header, adjust the target point-level features to meet the feature dimensions required by the classification output layer.

[0103] Among them, the point-level task head is the module responsible for transforming the target point-level features into the final task output (such as semantic classification). It includes a feature dimension adaptation layer and a classification output layer, and is the execution unit of the downstream dense prediction task.

[0104] In this embodiment of the disclosure, after obtaining the target point-level features, they can be input into the point-level task header. The feature dimension adaptation layer in the point-level task header uses linear transformation (such as fully connected or convolutional operations) to adjust the high-dimensional fusion dimension of the target point-level features to a dimension that matches the requirements of the classification output layer (such as the dimension corresponding to the number of semantic categories), so that the feature dimension is fully compatible with the input requirements of the subsequent classification output layer, and provides adapted feature input for the final point cloud semantic classification task.

[0105] Step 280-3: Using the classification output layer based on the target point-level features, perform semantic category determination on each point of the original large-scale point cloud to generate dense point-level prediction results.

[0106] In this embodiment of the disclosure, after obtaining the target point-level features adjusted by the feature dimension adaptation layer and meeting the input requirements of the classification output layer, the classification output layer processes the target point-level features corresponding to each point in the original large-scale point cloud through an activation function (such as Softmax), mapping the feature vector to the probability distribution of the corresponding semantic category (each category corresponds to a probability value of 0-1, and the sum of the probabilities of all categories is 1); then, according to the principle of maximizing probability, the category with the highest probability value is selected as the semantic category determination result of the point; by performing the above feature input-probability calculation-category selection process on all points in the original large-scale point cloud one by one, a dense point-level prediction result covering all points and each point corresponding to a unique semantic category is finally generated, realizing the transformation from multi-scale fused features to point cloud semantic parsing results.

[0107] In summary, the technical solution in this application firstly divides the original large-scale point cloud into planar subsets and non-planar subsets based on the planarity index, which lays the foundation for subsequent differentiated feature utilization, avoids clustering bias caused by confusion between planar and non-planar features, and improves the targeting of feature utilization; secondly, it adopts super-point clustering dominated by radiation features and geometric features respectively for the two types of subsets, and obtains super-point segmentation results at least three scales by adjusting parameters, which can not only make full use of the radiation information of the planar subsets and the geometric information of the non-planar subsets, but also enrich the feature receptive field through multi-scale design and reduce the accumulation of inter-layer errors; thirdly, it performs local feature aggregation with linear complexity for super-points at each scale. While ensuring computational efficiency and controllable memory, this method preserves the complete details inside the superpoints. By extracting global context features through spatial serialization combined with bidirectional state space scanning, it effectively solves the problems of high computational complexity, loss of local details, and insufficient capture of global associations in traditional methods. Finally, the global context features of multi-scale superpoints are mapped back to the point-level dimension according to the membership of member points. Combined with residual fusion to aggregate multi-scale point-level features, it can reduce feature propagation errors. Finally, it connects the point-level task head to complete dense prediction, which can realize high-precision, real-time point-level semantic parsing in scenarios with hundreds of millions of point clouds, meeting the needs of high-precision maps for autonomous driving, digital twin cities, and other fields for high-resolution, high-precision point cloud processing.

[0108] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this embodiment provides a dense point-level prediction device for large-scale point clouds, such as... Figure 3 As shown, the device may include: an analysis module 31, a clustering module 32, an extraction module 33, and a prediction module 34.

[0109] Analysis module 31 can be used to perform local geometric statistical analysis on the original large-scale point cloud, and divide the original large-scale point cloud into planar subsets and non-planar subsets based on planarity related indicators;

[0110] Clustering module 32 can be used to adopt a superpoint clustering scheme dominated by radiation features for planar subsets and a superpoint clustering scheme dominated by geometry features for non-planar subsets. By adjusting the clustering parameters, at least three different scales of superpoint segmentation results can be obtained. In each scale of superpoint segmentation results, there are radiation feature-dominated superpoints derived from planar subsets and geometry feature-dominated superpoints derived from non-planar subsets.

[0111] The extraction module 33 can be used to perform a local feature aggregation operation with linear complexity for each scale of the superpoint to obtain local aggregated features containing complete details inside the superpoint. Based on the local aggregated features, the global context features of the multi-scale superpoint are extracted at the superpoint level by combining spatial serialization processing with bidirectional state space scanning.

[0112] The prediction module 34 can be used to map the global context features of multi-scale superpoints back to the point-level dimension of the original large-scale point cloud according to the membership relationship of the superpoint clusters, to obtain multi-scale point-level features. The multi-scale point-level features are aggregated by residual fusion and finally connected to the point-level task head to perform dense point-level prediction of the large-scale point cloud.

[0113] In some embodiments of this application, the analysis module 31 can be specifically used to perform dimensional normalization processing on the coordinate information and radiation information of the original large-scale point cloud to eliminate the numerical differences between data of different dimensions; calculate the k-neighborhood data of each point in the normalized original large-scale point cloud, and construct the covariance matrix of each point based on the k-neighborhood data; perform eigenvalue decomposition on the covariance matrix of each point to obtain eigenvalues ​​that reflect the local geometric shape around the point, calculate the flatness or local curvature based on the eigenvalues, and use the flatness or local curvature as a plane criterion to determine the plane attribute of each point; and classify the points whose corresponding plane attribute determination results are not less than a preset threshold into the plane subset, and classify the remaining points into the non-plane subset to complete the point cloud dual-branch splitting.

[0114] In some embodiments of this application, the clustering module 32 can be specifically used to perform radiation-feature-dominated superpoint clustering on planar subsets to obtain planar superpoints, wherein the radiation features include at least RGB color and reflection intensity; and to perform geometry-feature-dominated superpoint clustering on non-planar subsets to obtain local superpoint fragments, wherein the geometry features include at least curvature consistency, normal matching degree, point density distribution, and spatial connectivity; by adjusting the size of the clustering parameters, the planar superpoints can form at least three radiation-feature-dominated superpoints of different scales, and the local superpoint fragments can form at least three geometry-feature-dominated superpoints of different scales.

[0115] In some embodiments of this application, when performing a linearly complex local feature aggregation operation on each scale of a superpoint to obtain local aggregated features containing complete details of the superpoint's interior, the extraction module 33 can specifically be used to collect the input feature vectors of all member points within a single superpoint at any scale; perform lightweight initial aggregation on the input feature vectors to obtain the initial aggregated features of the superpoint; randomly sort the input feature vectors of member points within the superpoint to obtain a member point feature sequence, and append the initial aggregated features as the tail aggregation token to the end of the member point feature sequence to construct the input sequence; scan the input sequence in a linear recursive manner, and selectively retain key features through an input-dependent gating mechanism to output local aggregated features containing complete details of the superpoint's interior.

[0116] In some embodiments of this application, when extracting global context features of multi-scale superpoints at the superpoint level through spatial serialization processing combined with bidirectional state space scanning based on local aggregation features, the extraction module 33 can specifically be used to calculate the centroid coordinates of all superpoints at each scale, generate a 3D Hilbert curve index for each superpoint based on the centroid coordinates, sort all superpoints at that scale according to the 3D Hilbert curve index, so that spatially adjacent superpoints form an ordered superpoint sequence, associate the local aggregation features corresponding to each superpoint in the ordered superpoint sequence, form an ordered superpoint feature sequence, perform forward state space recursive scanning and reverse state space recursive scanning on the ordered superpoint feature sequence respectively, to obtain a forward superpoint feature sequence and a reverse superpoint feature sequence, and concatenate or weightedly fuse the forward superpoint feature sequence and the reverse superpoint feature sequence to obtain the global context features of the superpoints at that scale.

[0117] In some embodiments of this application, when mapping the global context features of a multi-scale superpoint back to the point-level dimension of the original large-scale point cloud according to the membership relationship of the superpoint cluster, to obtain multi-scale point-level features, the prediction module 34 can be specifically used to map the global context features of the superpoint at the corresponding scale to all the points of the original large-scale point cloud contained in the superpoint according to the membership relationship of the superpoint cluster in ascending order of at least three different scales, to obtain the single-scale point-level features of the superpoint at the corresponding scale; and to determine the single-scale point-level features corresponding to at least three different scales as multi-scale point-level features, wherein the multi-scale point-level features include at least small-scale point-level features, medium-scale point-level features and large-scale point-level features.

[0118] In some embodiments of this application, when using residual fusion to aggregate multi-scale point-level features and finally connecting them to a point-level task head for dense point-level prediction of large-scale point clouds, the prediction module 34 can specifically be used to aggregate multi-scale point-level features through residual connection and dimension concatenation to obtain aggregated target point-level features; input the target point-level features into the point-level task head, and adjust the target point-level features to meet the feature dimensions required by the feature dimension adaptation layer in the point-level task head; and use the classification output layer to perform semantic category determination on each point of the original large-scale point cloud based on the target point-level features to generate dense point-level prediction results.

[0119] It should be noted that other corresponding descriptions of the functional units involved in the large-scale point cloud dense point-level prediction device provided in this embodiment can be found in [reference]. Figure 1 and Figure 2 The corresponding descriptions in [the document] will not be repeated here.

[0120] Based on the above, Figure 1 and Figure 2 Accordingly, this embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described method. Figure 1 and Figure 2 The method for dense point-level prediction of large-scale point clouds is shown.

[0121] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause an electronic device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0122] Based on the above, Figure 1 and Figure 2 The method shown, and Figure 3 To achieve the above objectives, the present application also provides an electronic device, specifically a personal computer, tablet computer, server, or other network device, as shown in the virtual device embodiment. This device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 and Figure 2 The method for dense point-level prediction of large-scale point clouds is shown.

[0123] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0124] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0125] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0127] This invention splits the original large-scale point cloud into planar and non-planar subsets based on planarity. It then employs super-point clustering dominated by radiative and geometric features to generate super-point segmentation results at at least three scales, enabling accurate classification of different types of point clouds. Furthermore, it preserves the complete details within super-points through linear complexity local feature aggregation, and combines spatial serialization and bidirectional state space scanning to extract global contextual features, balancing processing efficiency with the integrity of feature representation. Finally, it maps the multi-scale super-point global features back to the point level and fuses them using residuals, effectively avoiding inter-layer error accumulation and detail loss. Ultimately, the point-level task head can stably complete high-precision dense point-level prediction on large-scale point clouds, making it suitable for scenarios with high requirements for accuracy and efficiency in large-scene point cloud processing, such as autonomous driving and digital twin cities. It has strong practicality and scalability.

[0128] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0129] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for dense point-level prediction of large-scale point clouds, characterized in that, The method comprises the following steps: performing local geometric statistical analysis on the original large-scale point cloud, and dividing the original large-scale point cloud into a planar subset and a non-planar subset according to a planeness-related index; adopting a radiation-feature-dominant superpoint clustering scheme for the planar subset and a geometric-feature-dominant superpoint clustering scheme for the non-planar subset, and obtaining at least three superpoint segmentation results of different scales by adjusting clustering parameters, wherein in each scale of the superpoint segmentation results, the planar subset-derived radiation-feature-dominant superpoint and the non-planar subset-derived geometric-feature-dominant superpoint are included; performing a linear-complexity local feature aggregation operation on each scale of the superpoint to obtain local aggregated features containing complete details inside the superpoint, and based on the local aggregated features, extracting global context features of the multi-scale superpoint through spatial serialization processing combined with bidirectional state space scanning at the superpoint level; mapping the global context features of the multi-scale superpoint to the point level of the original large-scale point cloud according to the member point attribution relationship of the superpoint clustering, obtaining multi-scale point-level features, and finally connecting a point-level task head to perform dense point-level prediction of the large-scale point cloud by aggregating the multi-scale point-level features in a residual fusion manner.

2. The method of claim 1, wherein, The method of performing local geometric statistical analysis on the original large-scale point cloud and dividing the original large-scale point cloud into a planar subset and a non-planar subset according to a planeness-related index comprises the following steps: performing dimension normalization processing on the coordinate information and radiation information of the original large-scale point cloud respectively to eliminate the numerical difference of data in different dimensions; calculating the k-neighborhood data of each point in the normalized original large-scale point cloud, and constructing a covariance matrix of each point based on the k-neighborhood data; performing eigenvalue decomposition on the covariance matrix of each point to obtain eigenvalues reflecting the local geometric shape of the point, calculating the planeness or local curvature based on the eigenvalues, and determining the planar attribute of each point based on the planeness or local curvature as a planar criterion; points with a planar attribute determination result greater than or equal to a preset threshold are classified into the planar subset, and the remaining points are classified into the non-planar subset, thereby completing the point cloud double-branch division.

3. The method of claim 1, wherein, The method of adopting a radiation-feature-dominant superpoint clustering scheme for the planar subset and a geometric-feature-dominant superpoint clustering scheme for the non-planar subset, and obtaining at least three superpoint segmentation results of different scales by adjusting clustering parameters comprises the following steps: performing radiation-feature-dominant superpoint clustering on the planar subset to obtain planar superpoints, wherein the radiation features at least include RGB color and reflection intensity; performing geometric-feature-dominant superpoint clustering on the non-planar subset to obtain local superpoint segments, wherein the geometric features at least include curvature consistency, normal matching degree, point density distribution and spatial connectivity; by adjusting the size of the clustering parameter, the planar superpoints form at least three radiation-feature-dominant superpoints of different scales, and the local superpoint segments form the at least three geometric-feature-dominant superpoints of different scales.

4. The method of claim 1, wherein, The method of performing a linear-complexity local feature aggregation operation on each scale of the superpoint to obtain local aggregated features containing complete details inside the superpoint comprises the following steps: For any single superpoint at any scale, collect the input feature vectors of all member points within the superpoint; Perform lightweight initial aggregation on the input feature vectors to obtain the initial aggregated features of the superpoint; Randomly sort the input feature vectors of the member points within the superpoint to obtain a member point feature sequence, and append the initial aggregated features as tail aggregation tokens at the end of the member point feature sequence to construct an input sequence; Scan the input sequence in a linear recursive manner and selectively retain key features through an input-dependent gating mechanism to output local aggregated features containing complete details within the superpoint.

5. The method of claim 1, wherein, Based on the local aggregated features, extract the global context features of the multi-scale superpoints at the superpoint level through spatial serialization processing combined with bidirectional state space scanning, including: Calculate the center of gravity coordinates of all superpoints at each scale, and generate a 3D Hilbert curve index for each superpoint based on the center of gravity coordinates; Sort all superpoints at the scale according to the 3D Hilbert curve index to form an ordered superpoint sequence, and associate the local aggregated features corresponding to each superpoint in the ordered superpoint sequence to form an ordered superpoint feature sequence; Perform forward and backward state space recursive scanning on the ordered superpoint feature sequence to obtain forward and backward superpoint feature sequences; Concatenate or weightedly fuse the forward and backward superpoint feature sequences to obtain the global context features of the superpoint at the scale.

6. The method of claim 1, wherein, Map the global context features of the multi-scale superpoints to the point-level dimension of the original large-scale point cloud according to the member point attribution relationship of the superpoint cluster to obtain multi-scale point-level features, including: According to the scale order from small to large in the at least three different scales, respectively according to the member point attribution relationship of the superpoint cluster, map the global context features of the superpoint at the corresponding scale to all points of the original large-scale point cloud contained in the superpoint to obtain single-scale point-level features of the superpoint at the corresponding scale; Determine the single-scale point-level features corresponding to the at least three different scales as multi-scale point-level features, which at least include small-scale point-level features, medium-scale point-level features, and large-scale point-level features.

7. The method of claim 6, wherein, Aggregate the multi-scale point-level features using a residual fusion method, and finally connect a point-level task head to perform dense point-level prediction of the large-scale point cloud, including: Aggregate the multi-scale point-level features through residual connection and dimension splicing to obtain target point-level features after aggregation; Input the target point-level features into the point-level task head, adjust the target point-level features to a feature dimension that meets the requirements of the classification output layer through the feature dimension adaptation layer in the point-level task head; Use the classification output layer to determine the semantic category of each point of the original large-scale point cloud based on the target point-level features to generate a dense point-level prediction result.

8. A dense point-level prediction device for large-scale point clouds, characterized in that, including: an analysis module for performing local geometric statistical analysis on the original large-scale point cloud, and dividing the original large-scale point cloud into a planar subset and a non-planar subset according to planarity-related indicators; a clustering module configured to adopt a radiation-feature-dominant superpoint clustering scheme for the planar subset and a geometry-feature-dominant superpoint clustering scheme for the non-planar subset, and obtain superpoint segmentation results of at least three different scales by adjusting clustering parameters, wherein in each superpoint segmentation result of a scale, radiation-feature-dominant superpoints derived from the planar subset and geometry-feature-dominant superpoints derived from the non-planar subset are included; an extraction module configured to perform a local feature aggregation operation of linear complexity for each scale of superpoint to obtain local aggregated features containing complete details inside the superpoint, and based on the local aggregated features, extract global context features of multi-scale superpoints by spatial serialization processing combined with bidirectional state space scanning at a superpoint level; a prediction module configured to map the global context features of the multi-scale superpoints to a point level of the original large-scale point cloud according to a member point attribution relationship of superpoint clustering to obtain multi-scale point-level features, aggregate the multi-scale point-level features in a residual fusion manner, and finally connect a point-level task head to perform dense point-level prediction of the large-scale point cloud.

9. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the method of claim 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor executes the computer program to implement the method of claim 7.

Images