Feature extraction method and device for point cloud data, storage medium and program product

Abstract

The invention discloses a feature extraction method and device for point cloud data, a storage medium and a program product. Relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring voxel features of point cloud data, wherein the voxel features comprise semantic features and spatial features of the point cloud; clustering the voxel features based on the semantic features to obtain at least one super-class, and determining a mask rate of each super-class, the mask rate being used for representing an inter-class difference and an intra-class difference between the super-classes; and performing self-supervised learning on the point cloud feature extraction model according to the voxel features and the mask rate. By implementing the technical scheme provided by the invention, the overall performance of the automatic driving system in a complex traffic environment can be improved, and the traffic accident risk is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, storage medium and program product for extracting features from point cloud data. Background Art

[0002] With the rapid development of autonomous driving technology, the processing and analysis of 3D point cloud data has become increasingly important. Point cloud feature extraction is a key link in these applications, which directly affects the performance of downstream tasks such as object detection, semantic segmentation, and scene understanding. However, obtaining a large amount of annotated point cloud data is often time-consuming and labor-intensive, so developing efficient self-supervised learning methods to extract point cloud features has become a research hotspot.

[0003] At present, many researchers have proposed point cloud self-supervised learning methods based on masked autoencoders. These methods usually adopt a random masking strategy to randomly mask a part of the input point cloud, and then train the model to reconstruct the masked part. In this way, the point cloud feature extraction model can learn the spatial structure and local geometric features of the point cloud without manual annotation.

[0004] However, in autonomous driving scenarios, the random masking strategy commonly used by existing methods has serious flaws. Point cloud data in autonomous driving environments usually contain complex and diverse elements, such as vehicles, pedestrians, buildings, and roads, which vary significantly in importance and complexity. The above problems may cause the autonomous driving system to have a reduced perception ability in key areas in complex traffic environments, increasing the risk of traffic accidents. Summary of the invention

[0005] The present invention provides a feature extraction method for point cloud data, an electronic device, a storage medium and a program product, which can improve the overall performance of an automatic driving system in a complex traffic environment and reduce the risk of traffic accidents.

[0006] In a first aspect of the present invention, a method for extracting features from point cloud data is provided, comprising: Acquire voxel features of point cloud data, wherein the voxel features include semantic features and spatial features of the point cloud; At least one superclass is obtained by clustering the voxel features based on the semantic features, and a mask rate of each superclass is determined, where the mask rate is used to characterize inter-class differences and intra-class differences between the superclasses; The point cloud feature extraction model is self-supervised learned according to the voxel features and the mask rate.

[0007] Optionally, the obtaining of voxel features of point cloud data includes: Obtain the spatial features and semantic features of the point cloud data to obtain the point cloud features; The point cloud features are voxelized and average pooled to obtain voxel features.

[0008] Optionally, clustering the voxel features based on the semantic features to obtain at least one superclass, and determining a mask rate of each superclass, includes: Clustering the voxel features based on the semantic features to obtain at least one superclass, and determining a cluster center of each superclass; Based on the first average distance between the cluster centers corresponding to the superclasses, each superclass is divided into a plurality of superclass groups, wherein the superclass groups are used to characterize the inter-class differences between the superclasses; Obtaining a second average distance between the voxel feature within each superclass and the cluster center, wherein the second average distance is used to characterize the intra-class difference within the superclass; Based on the first average distance and the second average distance of each super class, a mask rate of each super class is determined.

[0009] Optionally, determining the mask rate of each superclass based on the first average distance and the second average distance of each superclass includes: Determining a basic masking rate based on the first average distance corresponding to each of the superclass groups; Determining an intra-class consistency coefficient based on the second average distance of each superclass, wherein the intra-class consistency coefficient is inversely proportional to the second average coefficient; The product of the basic mask rate corresponding to each superclass and the intra-class consistency coefficient is used as the mask rate.

[0010] Optionally, the self-supervised learning of the point cloud feature extraction model according to the voxel features and the mask rate includes:

[0011] performing masking on the voxel feature based on the masking rate; The codec is used to reconstruct the semantic features and the spatial features of the voxel features after the mask processing to obtain the reconstructed semantic features and the reconstructed spatial features; Obtaining loss values ​​of the reconstructed semantic features and the reconstructed spatial features; Parameters in the point cloud feature extraction model are adjusted based on the loss value.

[0012] Optionally, the performing semantic feature reconstruction and spatial feature reconstruction on the masked voxel features by a codec includes: Determine a first weight factor corresponding to the semantic feature reconstruction and a second weight factor corresponding to the spatial feature reconstruction, wherein the first weight factor is greater than the second weight factor; Based on the first weight factor and the second weight factor, semantic feature reconstruction and spatial feature reconstruction are performed on the masked voxel features through an encoder.

[0013] Optionally, the performing semantic feature reconstruction and spatial feature reconstruction on the masked voxel features by a codec includes: Activate the codec to reconstruct the spatial features of the masked voxel features; Based on the start-up time factor, the codec is activated to reconstruct the semantic features of the voxel features after mask processing, wherein the start-up time factor is determined according to the completion of spatial feature reconstruction when the codec is activated to reconstruct the semantic features during the training process of the point cloud feature extraction model.

[0014] In a second aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a feature extraction method for point cloud data as described above is implemented.

[0015] In a third aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for extracting features from point cloud data as described above is implemented.

[0016] In a fourth aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for extracting features from point cloud data as described above is implemented.

[0017] In summary, one or more technical solutions provided in the present invention have at least the following technical effects or advantages: By obtaining the voxel features of point cloud data, including semantic features and spatial features, and clustering voxel features based on semantic features to obtain superclasses, and then determining the mask rate of each superclass, the importance of different areas in point cloud data can be recognized and distinguished. Compared with the random mask strategy commonly used in the prior art, the above-mentioned adaptive mask strategy based on semantic and spatial information can more specifically guide the model to focus on the key areas in the point cloud data. By combining voxel features and mask rates for model training, this method enables the trained point cloud feature extraction model to more accurately capture the features of various complex elements in the autonomous driving scene, such as vehicles, pedestrians, buildings and roads. This method effectively solves the defects of the existing random mask strategy in processing complex and diverse point cloud data in autonomous driving environments, and improves the model's perception of key areas. Finally, by extracting features from the point cloud data of the target area through the trained point cloud feature extraction model, a more accurate and effective feature representation can be obtained, thereby providing more reliable input for downstream tasks such as target detection, semantic segmentation and scene understanding, which helps to improve the overall performance of the autonomous driving system in complex traffic environments and reduce the risk of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of a feature extraction method of point cloud data provided by an embodiment of the present invention.

[0020] Figure 2 It is a flow chart of a feature extraction method of point cloud data provided by an embodiment of the present invention.

[0021] Figure 3 It is a schematic diagram of the comparison between a true value semantic feature and an encoder output heat map provided by an embodiment of the present invention.

[0022] Figure 4 It is a schematic diagram of semantic category and mask rate distribution in a scenario provided by an embodiment of the present invention.

[0023] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Before describing the technical solution of the present invention, it is necessary to analyze in detail the deficiencies of the point cloud self-supervised learning method in the prior art. The existing point cloud self-supervised learning method mainly consists of two parts: proxy task and mask sampling strategy. The proxy task is used to guide the neural network to reconstruct the point cloud information and improve the feature extraction capability of the point cloud encoder, while the mask sampling strategy is used to perform sparse processing on the input data and determine the area to perform point cloud reconstruction to improve the learning efficiency and robustness of the model. These two parts work together to provide feature representation for downstream tasks. However, the prior art still has the following deficiencies: Disadvantages of agency tasks: 1.1 Limitations of learning dominated by spatial features: Existing proxy task methods mainly focus on the reconstruction of spatial features. For example, some methods promote model learning by predicting whether each masked voxel is occupied. The core idea is to enhance scene understanding by reasoning about the existence or non-existence of objects in the 3D scene. Other methods focus on the reconstruction of the 3D coordinates of the point cloud in each masked voxel, and achieve more accurate scene reconstruction by guiding the model to learn the fine geometric structure of the point cloud data. Some methods also introduce geometric features, such as normal information, to enhance the model's spatial understanding ability.

[0026] Although these methods can effectively capture the geometric structure of point clouds, they often ignore the importance of semantic information. Specifically: (a) Limitations of learning features: The point cloud self-supervised learning process can only learn the features needed by the shallow layer of the downstream task network, and has limited promotion effect on advanced tasks such as 3D object detection, 3D semantic segmentation, and occupancy grid prediction. For example, using the reconstruction of point cloud 3D coordinates or geometric features as a proxy task for the self-supervised learning method can only bring 0.85% and 1.06% improvement in the L2 mAP index for 3D object detection.

[0027] (b) Insufficient feature discrimination: The low amount of information in the proxy task will lead to poor feature discrimination. This makes the features of different point clouds not specific enough after the point cloud encoder obtained by the existing point cloud self-supervised learning, which brings difficulties to downstream tasks such as recognition and classification. Experiments show that the feature similarity of the deep network is nearly 50% larger than that of the shallow network.

[0028] 1.2 Lack of learning of high-level semantic relations: Although some methods try to introduce innovative proxy task designs, such as the combined use of jigsaw puzzles and masked autoencoders to understand the spatial relationship between voxels, these methods still focus on the reconstruction of spatial structure rather than the extraction of semantic information. Although this design can learn the arrangement and connection rules of voxels in three-dimensional space, thereby capturing the overall structural information of the scene at a higher level, it is still insufficient for understanding the semantic relationship and high-level concepts of objects in the scene.

[0029] Disadvantages of mask sampling strategy: 2.1 Uniformity of mask strategy: Existing mask sampling strategies, such as random masking or position-based masking, have the following problems: (a) Ignoring regional differences: These strategies treat all voxels equally and ignore the inconsistency of voxel features in different regions, which makes the model face challenges in reconstructing complex regions. Specifically, by randomly selecting voxels in 3D space for masking, although the model can infer the content of the masked voxels through neighborhood information, this method does not take into account the differences between different locations in the point cloud data, especially when dealing with areas with large distance differences in 3D scenes, it may lead to mask inhomogeneity, thus affecting the generalization ability of the model.

[0030] (b) Dispersed focus: Failure to prioritize the categories or regions of interest based on the needs of downstream tasks, but instead evenly distributes the model’s learning focus across all regions. This average distribution may lead to insufficient feature extraction of key regions, affecting the final task performance. For example, using random mask as the mask sampling strategy for self-supervised learning methods can only bring a 1.84% improvement in the L2mAP indicator for 3D object detection.

[0031] 2.2 Insufficient processing of sparse areas: Some improved methods try to solve some of the problems by considering distance factors or performing special treatments on sparse areas. For example, some methods propose that the mask rate should decrease with the distance from the self, using a higher mask rate for closer points and a lower mask rate for farther points. Other methods apply a lower mask rate in sparse areas to ensure that the model can reconstruct stably in these areas. However, these methods still fail to fully solve the above problems, especially in terms of combining semantic information and adapting to complex scenes.

[0032] 2.3 Lack of adaptability to different scales: Although some methods have proposed masking strategies for different granularities to maintain a consistent mask range, separate masking strategies are implemented according to the different granularities of point cloud data. The advantage of this strategy is that it can flexibly adjust the mask rate according to the characteristics of data at different scales, thereby ensuring that the model can effectively learn at all scales. However, this method still fails to fully consider the distribution characteristics of semantic information at different scales, which may lead to insufficient learning in some key semantic areas.

[0033] Based on the above detailed analysis, this paper proposes a new point cloud self-supervised learning method to overcome the shortcomings of the existing technology. Figure 1 , Figure 1 This is a flow chart of a feature extraction method for point cloud data provided by an embodiment of the present invention. The method can be implemented by a computer program, a single-chip microcomputer, or a self-supervised learning system based on a point cloud feature extraction model of the von Neumann system. The computer program can be integrated into an application or run as an independent tool application. Specifically, the method may include the following steps: Step 101: Acquire voxel features of point cloud data, where the voxel features include semantic features and spatial features of the point cloud.

[0034] The voxel feature refers to the feature information contained in each non-empty grid unit after discretizing the 3D point cloud data into a regular 3D grid structure. The voxel feature in the embodiment of the present invention can be understood as a comprehensive feature representation, which mainly consists of two parts: spatial feature and semantic feature.

[0035] Furthermore, semantic features refer to high-level feature representations that reflect the semantic information of objects or scenes in point cloud data. In an embodiment of the present invention, semantic features can be understood as high-dimensional feature vectors extracted by a pre-trained model. The feature vector encodes high-level semantic information such as the possible object category to which the voxel belongs, the scene structure, etc. Semantic features are used to enhance the model's ability to understand the content of the scene, provide semantic-related guidance for the adaptive mask strategy, and serve as important reconstruction targets in the feature reconstruction process. By introducing semantic features, the present invention can better capture the semantic structure in the scene and improve the performance of the model in various downstream tasks, such as three-dimensional object detection and semantic segmentation.

[0036] Correspondingly, spatial features refer to low-level features that describe the geometric structure and spatial distribution of point cloud data. In an embodiment of the present invention, spatial features can be understood as the three-dimensional coordinates of voxels, which directly reflect the position information of voxels in three-dimensional space. In addition, spatial features may also include geometric information such as the density and normal vector of points within the voxel. Spatial features are used to preserve the basic geometric structure of the point cloud and provide accurate spatial information for the model. In the process of self-supervised learning, the reconstruction of spatial features helps the model learn the local geometric features and spatial distribution of the point cloud.

[0037] In the subsequent adaptive masking process, semantic features are used to guide the formulation of masking strategies, while spatial features ensure the spatial consistency of masks. In the feature reconstruction task, the model learns to reconstruct these two features at the same time, thereby obtaining a more comprehensive understanding of the three-dimensional scene. Through the cascade reconstruction guided by the central core alignment, the present invention can also emphasize the reconstruction of spatial features and semantic features at different levels, achieving more refined feature learning.

[0038] Based on the above embodiment, as an optional embodiment, in step 101: the step of obtaining voxel features of point cloud data may further include the following steps: Step 201: Acquire the spatial features and semantic features of point cloud data to obtain point cloud features.

[0039] Specifically, the system first obtains the spatial features of the point cloud data, including the three-dimensional coordinate information of each point, which directly reflects the geometric structure and distribution of the point cloud in three-dimensional space. At the same time, in order to enhance the model's ability to understand the scene content, the present invention also introduces the extraction of semantic features. Semantic features can be extracted through a pre-trained MinkUNet model. The model can effectively aggregate the contextual information of the point cloud and generate a high-dimensional semantic feature vector.

[0040] By combining spatial features with semantic features, a comprehensive point cloud feature representation can be obtained. The above feature representation not only contains the basic geometric information of the point cloud, but also encodes high-level semantic information such as possible object categories and scene structures. In this way, the system pays attention to both the spatial structure and semantic content of the point cloud during self-supervised learning, thereby improving the performance of the model in various downstream tasks, such as 3D object detection and semantic segmentation.

[0041] It is worth noting that the semantic feature extraction process can be done offline, which helps to speed up the entire self-supervised learning process.

[0042] Step 202: voxelize and average pool the point cloud features to obtain voxel features.

[0043] Specifically, the system first performs voxelization on the acquired point cloud features. The voxelization process discretizes the continuous three-dimensional space into a regular three-dimensional grid structure, and each grid unit is called a voxel, so that the originally irregularly distributed point cloud data is mapped into a three-dimensional grid of fixed size, greatly simplifying the complexity of subsequent processing.

[0044] After voxelization, the present invention uses an average pooling method to aggregate point cloud features located in the same voxel. The average pooling operation calculates the feature average of all points in each voxel to obtain the representative feature of the voxel.

[0045] For example, the final voxel feature can be defined as:

[0046] S all ={s i |j=1,K,N v}.

[0047] Step 102: cluster the voxel features based on the semantic features to obtain at least one superclass, and determine a mask rate for each superclass.

[0048] After obtaining the voxel features of point cloud data, the present invention proposes an adaptive masking strategy based on semantic features. The strategy aims to optimize the masking process by considering the semantic attributes of voxel features, thereby improving the effect of self-supervised learning. Specifically, the present invention first clusters the voxels based on their semantic features and divides the voxels into multiple superclasses. Since the true semantic categories cannot be directly known in the self-supervised learning process, using a clustering algorithm to group semantic voxel features is an effective alternative.

[0049] Among them, the superclass refers to a set of semantically similar voxels obtained by clustering the semantic features of the point cloud data. In the embodiment of the present invention, the superclass can be understood as a group of voxels with similar semantic attributes. These voxels may be distributed in different positions in the three-dimensional space, but are close to each other in the semantic feature space. The superclass is used to group the original point cloud data according to its semantic features, so that differentiated strategies can be adopted for different semantic regions in the subsequent adaptive masking process.

[0050] Furthermore, after clustering is completed, the present invention determines a corresponding masking rate for each superclass. The masking rate refers to the proportion of masking a specific area or feature during the self-supervised learning process.

[0051] Among them, the mask rate refers to the proportion of specific areas or features that are masked during the self-supervised learning process. In the embodiment of the present application, it can be understood as the degree to which the voxel features in each superclass are masked. It is a value between 0 and 1, which indicates the proportion of masked voxels in the superclass to the total voxels. The mask rate is used to regulate the degree of information shielding in the self-supervised learning process, thereby affecting the learning difficulty of the model and the complexity of the reconstruction task. By setting different mask rates for different superclasses, the present invention can guide the model to pay more attention to key areas and complex structures in point cloud data, while avoiding over-learning in simple areas. This adaptive masking strategy helps to improve the model's ability to understand complex three-dimensional scenes, enhance the effectiveness and robustness of feature extraction, and provide better feature representation for subsequent downstream tasks such as target detection and semantic segmentation.

[0052] In order to quantify the difference between classes, the present invention adopts the farthest class sampling algorithm. The algorithm first calculates the distance between the cluster centers of each superclass, and then selects the superclass combination with the largest average inter-class distance. This method can effectively identify the superclasses with the most significant semantic differences and assign them a higher basic mask rate.

[0053] Among them, inter-class differences and intra-class differences are two important concepts that describe the characteristics of data distribution. Inter-class differences refer to the degree of feature differences between different categories or groups, while intra-class differences reflect the degree of feature changes within the same category or group. In an embodiment of the present invention, inter-class differences can be understood as the degree of difference in semantic features between different superclasses, reflecting the degree of separation of each superclass in the semantic space; intra-class differences can be understood as the degree of change in voxel features within the same superclass, reflecting the consistency of the semantics within the superclass.

[0054] There is a close relationship between inter-class differences and intra-class differences and the mask rate, which together determine the final mask rate of each superclass. Specifically, the greater the inter-class difference, the more significant the semantic difference between different superclasses. In this case, the present invention tends to assign a higher basic mask rate to superclasses with large inter-class differences. This is because areas with significant semantic differences usually contain richer information, and increasing the mask rate can prompt the model to learn more discriminative features. For intra-class differences, the present invention adopts an opposite strategy: the smaller the intra-class difference, that is, the higher the intra-class consistency, the more similar the semantic features within the superclass are. In this case, the present invention will retain more original information by reducing the mask rate because the reconstruction difficulty of these areas is relatively low.

[0055] Based on the above embodiment, as an optional embodiment, in step 102: clustering the voxel features based on the semantic features to obtain at least one superclass and determining the mask rate of each superclass may further include the following steps:

[0056] S301 . Cluster voxel features based on semantic features to obtain at least one superclass, and determine the cluster center of each superclass.

[0057] Specifically, in order to group semantically similar voxels together, it is first necessary to cluster the voxel features to form a superclass with semantic similarity. The present invention adopts a clustering method based on semantic features, such as the K-means algorithm, to group voxel features. The reason for selecting the K-means algorithm is that it is simple and efficient, and can effectively divide the data into a predetermined number of clusters without supervision. In this embodiment, the number of clusters N can be set according to the specific application scenario and the complexity of the point cloud data, and usually a value that can reasonably reflect the semantic diversity of the scene can be selected.

[0058] At the beginning of the clustering process, the system first randomly initializes N cluster centers. Then, for each voxel feature, its distance to all cluster centers is calculated, and the voxel is assigned to the superclass represented by the cluster center with the closest distance. After completing a round of assignment, the system recalculates the cluster center of each superclass, which is the average value of all voxel features in the superclass. This process is iterated until the position of the cluster center no longer changes significantly or the preset maximum number of iterations is reached.

[0059] For example, for each superclass k i , calculate its cluster center That is, the mean of the semantic features of all voxels in the superclass, and the formula is as follows:

[0060]

[0061] In the formula, Represents the superclass k i The semantic features of the j-th voxel in , Represents the superclass k i The number of voxels in .

[0062] S302 . Based on the first average distance between the cluster centers corresponding to each superclass, divide each superclass into a plurality of superclass groups, where the superclass groups are used to characterize the inter-class differences between each superclass.

[0063] Specifically, after completing the clustering of superclasses and determining the cluster centers, in order to quantify and utilize the inter-class differences between superclasses, thereby identifying superclass groups with significant semantic differences and providing more detailed guidance for the subsequent adaptive masking strategy, the present invention proposes to use the farthest class sampling algorithm to achieve this goal, which can effectively divide superclasses into groups with different degrees of semantic differences.

[0064] Furthermore, the present invention adopts the farthest class sampling algorithm to achieve this goal. First, select any n1 superclasses from all superclasses, and calculate the average distance between the cluster centers of these n1 superclasses, that is, the first average distance. By calculating the first average distance of all possible n1 superclass combinations, find the combination with the largest average inter-class distance, which is recorded as K1. This combination represents the n1 superclasses with the most significant semantic differences. Next, select n2 from the remaining superclasses, repeat the above process, and find the combination with the second largest average inter-class distance, which is recorded as K2. Finally, the remaining superclasses automatically form a third group K3, whose average inter-class distance is the smallest.

[0065] The superclass grouping method based on the first average distance can effectively identify the superclass groups with the most significant semantic differences, providing clear guidance for the subsequent masking strategy. The strategy of dividing superclasses into three groups allows a more flexible and detailed approach to the subsequent mask rate allocation, prompting the model to pay more attention to areas with significant semantic differences during the self-supervised learning process, while not neglecting areas with high semantic similarity. This method provides an implicit semantic structure guidance for the self-supervised learning process, which helps improve the model's ability to understand complex scenes and the effectiveness of feature extraction.

[0066] For example, we first define a set K = {k i |i=1,2,K,N c}, where N c is the total number of superclasses. Next, select any n1 superclasses from the set K and calculate the distances between cluster centers. The set D of average inter-class distances inter The calculation formula is as follows:

[0067]

[0068] in, Represents the distance between the centers of two super-class clusters, which can be measured using cosine similarity or other distance metrics. Select any combination of n1 superclasses from the set K.

[0069] Next, find the n1 super-class partitions with the largest average inter-class distance, denoted as K1. These super-classes have the greatest differences, so a higher base mask rate can be set for them to increase the difficulty of reconstruction and the attention of the model.

[0070] Then, we use the difference between sets K and K1 as the input for the next cycle. Select n2 superclasses from the remaining superclasses again and repeat the above steps to get the second farthest average inter-class distance partition K2. The remaining superclasses form K3, which has the smallest inter-class difference. In this way, the superclasses can be divided into three groups: K1, K2, and K3, which have the largest, second largest, and smallest inter-class differences, respectively. The larger the inter-class difference, the more difficult it is to reconstruct, so a higher base mask rate can be set.

[0071] S303 , obtaining a second average distance between the voxel features within each superclass and the cluster center, where the second average distance is used to characterize the intra-class difference within the superclass.

[0072] Specifically, after completing the division of superclasses and the quantification of inter-class differences, in order to gain an in-depth understanding of the internal structure of each superclass, more comprehensive information is provided for the subsequent adaptive masking strategy. The present invention further proposes a method for evaluating intra-class differences within each superclass. The intra-class differences are quantified by calculating the average distance between the voxel features within each superclass and their cluster centers, that is, the second average distance. The second average distance refers to a quantitative indicator within a single superclass that is used to measure the compactness of the distribution of voxel features within the superclass. In an embodiment of the present invention, the second average distance can be understood as the average value of the distances between all voxel features within a superclass and their cluster centers.

[0073] Exemplarily, the distance between a voxel and a cluster center is defined, and a second average distance D of voxels that are too far from the super cluster center is calculated. intra The formula is as follows:

[0074]

[0075] in, Representing voxel features and cluster centers , λ is the distance threshold, when the distance between the voxel feature and the cluster center exceeds λ, it will be counted, and 1{·} represents the indicator function, indicating that only the voxel features with a distance greater than the threshold are considered.

[0076] By calculating the second average distance, the present invention can evaluate the internal consistency of each superclass. A smaller second average distance indicates that the voxel features within the superclass are more densely distributed and have higher semantic consistency; while a larger second average distance indicates that there is greater semantic variation or heterogeneity within the superclass. This quantification of intra-class differences provides an important basis for subsequent adaptive masking strategies. For example, for superclasses with higher internal consistency, a lower masking rate can be used because the reconstruction difficulty of these areas is relatively low; while for superclasses with larger internal differences, a higher masking rate may be required to encourage the model to learn more complex feature structures.

[0077] S304: Determine the mask rate of each superclass based on the first average distance and the second average distance of each superclass.

[0078] Specifically, after completing the quantification of the inter-class differences and intra-class differences of the superclasses, in order to formulate an adaptive masking strategy for each superclass that takes into account both its position in the overall semantic space and its internal structure. The present invention proposes a method that comprehensively considers these two differences to determine the masking rate of each superclass. Specifically, the process may also include the following steps:

[0079] S401: Determine a basic masking rate based on a first average distance corresponding to each superclass group.

[0080] Specifically, for the three groups K1, K2, and K3, they correspond to superclass sets with different semantic differences from large to small. Different levels of basic mask rates are set for these three groups. Superclass groups with greater semantic differences should have higher base mask rates.

[0081] S402: Determine an intra-class consistency coefficient based on the second average distance of each superclass, where the intra-class consistency coefficient is inversely proportional to the second average coefficient.

[0082] Specifically, after determining the basic mask rate, the structural features within the superclass are considered, so that the mask strategy can better adapt to the complexity and diversity of point cloud data. The present invention further introduces an intra-class consistency coefficient to more finely adjust the mask rate of each superclass.

[0083] Among them, the intra-class consistency coefficient is an important numerical indicator used in the present invention to quantify the degree of semantic consistency within the superclass. In this embodiment, it can be understood as a coefficient between 0 and 1 calculated based on the second average distance of the superclass. The closer its value is to 1, the higher the semantic consistency within the superclass. By multiplying with the basic mask rate, the intra-class consistency coefficient can carefully adjust the mask strength according to the semantic structure within the superclass, so that the masking strategy can better adapt to the local semantic changes of the point cloud data. For superclasses with high internal consistency, a higher coefficient will reduce the mask rate to avoid excessive masking of these relatively simple semantic areas; for superclasses with low consistency, a lower coefficient will increase the mask rate, prompting the model to pay more attention to these complex areas, increasing the learning difficulty and the robustness of the model.

[0084] Exemplarily, the calculation formula of the intra-class consistency coefficient is as follows:

[0085]

[0086] In the formula, Represents the maximum intra-class distance among all superclasses and is used for normalization.

[0087] S403: The product of the basic mask rate corresponding to each superclass and the intra-class consistency coefficient is used as the mask rate.

[0088] Specifically, after determining the basic mask rate and the intra-class consistency coefficient, the present invention multiplies the basic mask rate and the intra-class consistency coefficient corresponding to each superclass to obtain the final mask rate of each superclass.

[0089] Exemplarily, the calculation formula is as follows:

[0090]

[0091] S103. The point cloud feature extraction model is trained according to the voxel features and the mask rate to obtain a trained point cloud feature extraction model, wherein the mask rate is used to assist the point cloud feature extraction model to focus on key areas in the point cloud data.

[0092] Specifically, Figure 2 As shown, Figure 2 A flow chart of a feature extraction method for point cloud data provided in an embodiment of the present invention. The point cloud feature extraction model refers to a deep learning model for extracting meaningful feature representations from raw point cloud data. In the embodiment of the present application, it can be understood as a codec structure based on a sparse voxel transformer for learning the spatial and semantic features of point clouds. The model includes a voxel feature extraction module, an adaptive mask module, a cascade reconstruction module, and a progressive learning module, which together constitute a complete self-supervised learning system.

[0093] The voxel feature extraction module is the basis of the entire model and is responsible for extracting initial features from the original point cloud data. The module first obtains the spatial features of the point cloud, including three-dimensional coordinate information, which directly reflects the geometric structure of the point cloud. At the same time, in order to enhance the model's ability to understand the scene content, the module also introduces the extraction of semantic features, which can be done through the pre-trained MinkUNet model. MinkUNet can effectively aggregate the contextual information of the point cloud and generate a high-dimensional semantic feature vector. Subsequently, the module voxelizes these features and discretizes the continuous three-dimensional space into a regular three-dimensional grid structure, where each grid unit is called a voxel. Through the average pooling operation, the module aggregates the point cloud features located in the same voxel, and finally obtains the voxel features that integrate spatial and semantic information.

[0094] The adaptive masking module implements an intelligent masking strategy based on voxel features. The module first clusters voxels based on semantic features to form several superclasses. A superclass can be understood as a group of voxels with similar semantic attributes. Subsequently, the module determines an appropriate masking rate for each superclass by considering the inter-class differences between superclasses and the intra-class differences within superclasses. Specifically, the module uses the farthest class sampling algorithm to quantify the inter-class differences, calculates the distance between the cluster centers of each superclass, and selects the superclass combination with the largest average inter-class distance. For intra-class differences, the module introduces an intra-class consistency coefficient, which reflects the degree of variation of voxel features within the same superclass. By combining inter-class differences and intra-class differences, the module generates an adaptive masking rate for each superclass. This strategy can better guide the model to focus on important areas and improve learning efficiency.

[0095] The cascade reconstruction module is responsible for performing the reconstruction tasks of spatial features and semantic features. The module adopts a central kernel alignment-guided strategy to weight the reconstruction of spatial features and semantic features at different network layers. Specifically, the cascade reconstruction module calculates the central kernel alignment factor of each layer, which reflects the relevance of the features of this layer to the downstream tasks. Based on this factor, the module dynamically adjusts the weights of spatial and semantic feature reconstruction. In shallow networks, the module pays more attention to the reconstruction of spatial features because these layers mainly capture local geometric information; while in deep networks, it pays more attention to the reconstruction of semantic features to learn higher-level semantic representations. This cascade reconstruction strategy enables the model to better learn spatial and semantic information at different levels, thereby improving the quality of feature representation.

[0096] The progressive learning module implements a differential gated progressive learning strategy to control the learning process of spatial feature reconstruction and semantic feature reconstruction. The module dynamically adjusts the start time of semantic feature reconstruction based on the completion of spatial feature reconstruction. Specifically, the progressive learning module evaluates the completion of spatial feature reconstruction by calculating the central kernel alignment factor of the mutual information metric between two adjacent training rounds. When the completion reaches the preset threshold, the progressive learning module activates the semantic feature reconstruction task. This progressive learning method enables the model to first master the basic spatial structure and then gradually learn complex semantic information, thereby improving the stability and effect of training.

[0097] It should be noted that the modules in the point cloud feature extraction model can be executed separately or as a whole to implement the feature extraction method of point cloud data. When executed as a whole, these modules work closely together to form an end-to-end self-supervised learning process. The voxel feature extraction module first processes the input raw point cloud data to generate voxel features containing spatial and semantic information. Then, the adaptive mask module generates masks based on these voxel features to create conditions for subsequent reconstruction tasks. Next, the cascade reconstruction module uses the generated masks to reconstruct the masked voxel features while taking into account the characteristics of different network layers. Finally, the progressive learning module controls the entire learning process to ensure that the model learns spatial and semantic features in an appropriate order and rhythm.

[0098] Through the above overall collaboration, the point cloud feature extraction model can effectively learn the comprehensive feature representation of point cloud data and provide rich and effective features for various downstream tasks. At the same time, the model also improves the overall understanding of 3D scenes and enhances the ability to handle partially occluded or noisy data. This self-supervised learning method not only reduces the dependence on a large amount of labeled data, but also provides a high-quality pre-trained model for point cloud related tasks.

[0099] Specifically, after obtaining voxel features and determining the mask rate of each superclass, the present invention further proposes a self-supervised learning method based on voxel features and mask rate to optimize the performance of the point cloud feature extraction model. This method aims to improve the model's ability to understand three-dimensional scenes and the quality of feature representation by combining spatial and semantic information and an adaptive masking strategy.

[0100] Specifically, the system first masks the voxel features according to a certain masking rate to create a mixed data set containing masked and unmasked voxels. This masking operation simulates the partial information loss that may exist in the actual scene, which helps to improve the robustness of the model. Subsequently, these processed data are input into the point cloud feature extraction model for training.

[0101] During the training process, the model is required to simultaneously reconstruct spatial features and semantic features, which reflect the geometric structure and high-level semantic information of the point cloud, respectively. In order to better utilize feature information at different levels, the present invention adopts a cascade reconstruction strategy guided by central kernel alignment. This strategy dynamically adjusts the weights of spatial feature and semantic feature reconstruction according to the relevance of different network layers to downstream tasks. Specifically, in shallow networks, the focus is on the reconstruction of spatial features because these layers mainly capture local geometric information; while in deep networks, more attention is paid to the reconstruction of semantic features to learn higher-level semantic representations.

[0102] In addition, in order to ensure the stability and effectiveness of the training process, the present invention also introduces a differential gating progressive learning strategy. This strategy monitors the completion of spatial feature reconstruction and dynamically adjusts the start timing of the semantic feature reconstruction task, avoiding the problem of training instability that may be caused by the premature introduction of complex tasks. In this way, the model can gradually learn from basic spatial structures to complex semantic concepts, achieving a smoother and more effective feature learning process. Finally, the model parameters are optimized by minimizing the reconstruction loss, where the reconstruction loss includes two parts: spatial feature reconstruction loss and semantic feature reconstruction loss.

[0103] The above comprehensive self-supervised learning method can not only improve the model's ability to understand point cloud data, but also provide richer and more effective feature representations for various downstream tasks without a large amount of labeled data. In this way, the self-supervised learning method of the present invention can significantly improve the performance and generalization ability of the point cloud feature extraction model, laying a solid foundation for complex three-dimensional scene understanding tasks.

[0104] S104: extracting features from the point cloud data of the target area using the trained point cloud feature extraction model.

[0105] Specifically, the system first obtains the original point cloud data of the target area, which may come from devices such as lidar scanning or depth cameras. Subsequently, the system preprocesses these original point cloud data, including denoising, downsampling and other operations to improve the efficiency of subsequent processing.

[0106] Next, the system inputs the preprocessed point cloud data into the trained point cloud feature extraction model. The model processes the input point cloud data based on the previously learned adaptive mask strategy and cascade reconstruction method. In this process, the model first converts the point cloud data into voxel features, which include spatial features and semantic features.

[0107] The encoder part of the model then performs multi-level feature extraction and transformation on these voxel features to generate a latent representation that contains high-level features of the point cloud data, including both geometric structure information and semantic information.

[0108] Finally, the system extracts the final feature representation from this latent representation as the feature extraction result of the point cloud data of the target area.

[0109] Based on the above embodiment, the training process of the point cloud feature extraction model may further include the following steps:

[0110] S501 . Perform mask processing on the voxel features based on a mask rate.

[0111] Specifically, for each superclass, the system first calculates the number of voxels that need to be masked based on its corresponding mask rate. Subsequently, a corresponding number of voxels are randomly selected within the superclass for masking. The masking operation involves replacing the features of the selected voxels with a specific mask token, which is usually a predefined vector or an all-zero vector. This approach ensures that the model cannot directly access the masked information during training, forcing the model to learn to infer and reconstruct the missing information from the surrounding unmasked voxels.

[0112] It is worth noting that the masking strategy of the present invention not only considers the semantic differences between superclasses, but also reflects the feature distribution within the superclass through the intra-class consistency coefficient, making the masking process more refined and targeted.

[0113] S502 , reconstructing semantic features and spatial features of the masked voxel features through a codec to obtain reconstructed semantic features and reconstructed spatial features.

[0114] After completing the mask processing of the voxel features, the present invention uses a codec structure to reconstruct the semantic features and spatial features of the masked voxel features. The model is trained through the reconstruction task so that it can infer and restore the complete scene representation from partially visible information.

[0115] Furthermore, the present invention uses a codec structure based on sparse voxel transformer (DSVT) to perform the above tasks. First, the masked voxel features are input into the encoder. The encoder consists of multiple DSVT layers, each of which can capture spatial and semantic information at different scales. The unmasked voxel features are directly input into the encoder, while the masked voxel features are replaced with specific mask tokens. The encoder generates a latent representation through multi-level feature extraction and transformation. Subsequently, this latent representation is input into the decoder. The task of the decoder is to reconstruct the complete voxel features based on this latent representation, including the masked parts.

[0116] In a feasible implementation, since the shallower layers of the encoder receive weaker information feedback from the supervisory signal, an embodiment of the present invention adds deep supervision to the point cloud self-supervision method. For the reconstruction of non-final layers, only half of the decoder layers are used to avoid expensive computational burden. In addition, since the point cloud encoder follows hierarchical feature learning, the shallow layers learn local geometric features and spatial distribution information, while the deep layers capture high-level semantic information. Not all layers are suitable for performing the same reconstruction. In order to further improve the effect and efficiency of the reconstruction, the present invention adopts a cascade reconstruction strategy guided by central kernel alignment to weight the spatial feature and semantic feature reconstruction losses of each layer.

[0117] Specifically, the process may further include the following steps:

[0118] S601. Determine a first weight factor corresponding to semantic feature reconstruction and a second weight factor corresponding to spatial feature reconstruction, wherein the first weight factor is greater than the second weight factor.

[0119] Specifically, the system calculates the central kernel alignment factor of each layer, which reflects the relevance of the layer's features to the downstream tasks. Then, the average of the central kernel alignment factors of the three downstream tasks is defined as the proportion of spatial information α i (i=1,2,…,L), where L is the total number of layers in the network. Based on this ratio, the second weight factor of spatial feature reconstruction is set to The first weight factor for semantic feature reconstruction is set to

[0120] It is worth noting that since the deep network is more inclined to capture high-level semantic information, in most cases, the first weight factor will be greater than the second weight factor. The reasons for this setting are multifaceted: First, it takes into account the hierarchical feature learning characteristics of the point cloud encoder, that is, the shallow layer mainly learns local geometric features and spatial distribution information, while the deep layer captures more high-level semantic information. By setting different weights for different layers, the feature learning ability of each layer can be better utilized.

[0121] Secondly, this strategy can alleviate the gradient decay problem in deep supervision. Since the shallower layers of the encoder receive weaker information feedback from the supervision signal, the learning of deep features can be enhanced by increasing the weight of deep semantic feature reconstruction. Furthermore, this method can balance the learning of spatial information and semantic information, avoiding the model from focusing too much on one aspect and ignoring the other.

[0122] Finally, by dynamically adjusting the weights, the model can more flexibly adapt to the needs of different types of downstream tasks. The effect of this central core alignment-guided cascade reconstruction strategy is significant. It can improve the model's feature learning ability at different levels, so that the learned features contain both rich geometric information and advanced semantic understanding. On the basis of improving the effect of self-supervised learning, it can also provide more comprehensive and effective feature representation for various downstream tasks.

[0123] S602: Based on the first weight factor and the second weight factor, perform semantic feature reconstruction and spatial feature reconstruction on the masked voxel features through an encoder.

[0124] In another feasible implementation, the reconstruction of semantic features fundamentally depends on the accurate reconstruction of spatial features, such as local geometric features. The error amplification caused by parallel learning will lead to the instability of semantic feature reconstruction. The present invention proposes a multi-task switching strategy, namely differential gated progressive learning, which adjusts the start time of semantic feature reconstruction according to the completion of spatial feature reconstruction. The activation difference between two adjacent rounds can reflect the degree of training completion.

[0125] Specifically, the process may further include the following steps:

[0126] S701, activating the codec to reconstruct the spatial features of the voxel features after mask processing.

[0127] Specifically, spatial features are the basis of point cloud data, reflecting the geometric structure of objects and scenes. Accurate spatial feature reconstruction lays a solid foundation for subsequent semantic feature reconstruction. The above method is in line with the human cognitive process, that is, first understand the shape and position of the object, and then give it semantic meaning. Secondly, spatial features are usually more stable and consistent than semantic features, and are not easily affected by annotation bias or semantic ambiguity. Starting with relatively simple and stable tasks helps the model establish a basic understanding of point cloud data and avoids the training instability that may be caused by directly processing complex semantic information.

[0128] Furthermore, by focusing on spatial feature reconstruction first, the model can build an understanding of point cloud data at a lower level, which provides a good starting point for subsequent high-level semantic feature learning. Finally, this progressive learning strategy can effectively prevent the error amplification problem that may be caused by parallel learning, especially in the semantic feature reconstruction task. By ensuring the accurate reconstruction of spatial features first, the uncertainty and instability in the semantic feature reconstruction process can be greatly reduced.

[0129] S702. Based on the start-up time factor, activate the codec to perform semantic feature reconstruction on the voxel features after mask processing, wherein the start-up time factor is determined according to the completion of spatial feature reconstruction when the codec is activated to perform semantic feature reconstruction during the training process of the point cloud feature extraction model.

[0130] Specifically, on the basis of completing the spatial feature reconstruction, the present invention proposes a semantic feature reconstruction activation mechanism based on the start time factor, which is the core component of the differential gated progressive learning strategy. The design of this mechanism is to ensure that the semantic feature reconstruction task is started at the most appropriate time, thereby improving the stability and effect of training.

[0131] The start time factor refers to a dynamically adjusted parameter used to determine when to start the semantic feature reconstruction task during the point cloud self-supervised learning process. In the embodiment of the present invention, the start time factor can be understood as a value between 0 and 1, which reflects the degree of completion of the spatial feature reconstruction and controls the start timing of the semantic feature reconstruction task accordingly.

[0132] When the start time factor is 0, it means that only spatial features are reconstructed; when it is close to 1, it means that the spatial features have been well reconstructed and the semantic feature reconstruction task can be started. The start time factor is used to implement the differential gating progressive learning strategy. By dynamically adjusting the start time of semantic feature reconstruction, it ensures that the model gradually learns more complex semantic information on the basis of mastering the basic spatial structure, thereby improving the stability and effect of training.

[0133] Specifically, the start-up time factor is dynamically determined according to the degree of completion of spatial feature reconstruction during the training of the point cloud feature extraction model. When calculating, the activation difference is first calculated by measuring the central core alignment factor through the mutual information between two adjacent training rounds. Considering that the shallow network mainly learns local geometric features, the present invention calculates the average central core alignment factor of the first four layers as the degree of completion of spatial feature reconstruction in the current round t. Exemplarily, the calculation formula of the start-up timing factor is:

[0134]

[0135] In the formula, is the completion degree of spatial feature reconstruction in the tth round, δ is the gating factor, which indicates the starting threshold of semantic feature reconstruction, and γ is the slope factor, which controls β t growth rate.

[0136] Based on the start-up time factor, the present invention activates the codec at an appropriate time to reconstruct the semantic features of the voxel features after mask processing. Specifically, when β t When the preset threshold is reached, the model begins to perform the semantic feature reconstruction task. In this process, the encoder continues to process the input voxel features, but now focuses more on extracting high-level semantic information. The decoder is trained to reconstruct these semantic features, including the masked parts.

[0137] The activation mechanism of semantic feature reconstruction based on the start-up time factor ensures that the semantic feature reconstruction task starts on the basis that the spatial features have been well understood, which greatly reduces the instability of training. The spatial features provide the necessary context and structural information for the semantic features, making the learning of semantic features more accurate and effective. Secondly, this progressive learning method simulates the learning sequence from basic shapes to complex concepts in the human cognitive process, which helps the model to establish a more reasonable and hierarchical feature representation. Furthermore, by dynamically adjusting the start timing of semantic feature reconstruction, the model can better adapt to point cloud data of different complexities. For simple scenes, it can enter the semantic learning stage faster, while for complex scenes, it gives more time to master the basic spatial structure. Finally, this method effectively prevents the error amplification problem that may be caused by parallel learning, especially when dealing with complex semantic information.

[0138] S503: Obtain loss values ​​of the reconstructed semantic features and the reconstructed spatial features.

[0139] Specifically, the spatial feature reconstruction loss can be expressed as:

[0140]

[0141] in, is the spatial feature of the masked voxel predicted by the model, is the corresponding real space feature. For semantic features, the smooth L1 loss function is also used to calculate the reconstruction error:

[0142]

[0143] in, is the semantic feature of the masked voxel predicted by the model, is the corresponding real semantic feature.

[0144] S504: Adjust parameters in the point cloud feature extraction model based on the loss value. Specifically, the spatial feature reconstruction loss space and the semantic feature reconstruction loss_semantics are first combined to form a comprehensive loss function. Then, the back propagation algorithm is used to calculate the gradient of the loss function to the parameters of each layer of the model. The present invention uses the Adam optimizer to update the model parameters. This optimizer can adaptively adjust the learning rate, which is conducive to processing the sparsity and non-uniformity of point cloud data. By minimizing the reconstruction error, the model can gradually learn more accurate and meaningful feature representations. The above embodiment introduces the self-supervised learning process of the point cloud feature extraction model. Based on the above embodiment, the above process will be exemplified in combination with the scenario of point cloud self-supervised learning in an autonomous driving scenario. As shown in Table 1, Table 1 is a parameter setting table of a self-supervisory method provided by an embodiment of the present invention. The present invention adopts a laser radar as a sensor to obtain input point cloud data. In the construction of the point cloud feature extraction model, the present invention chooses to fine-tune the encoder based on the sparse voxel transformer (DSVT). Specifically, the encoder consists of four DSVTs, each DSVT containing eight attention layers. The decoder part consists of two DSVTs forming the last layer, and another DSVT is used for decoding of non-last layers. The feature dimension of the model is set to 192, and the voxel size is set to [0.32, 0.32, 0.1875]. In the adaptive masking module guided by inter-class differences, the present invention sets the expected number of superclass partitions (n1, n2, n3) to (3, 3, 2), and the corresponding basic masking rate to (0.9, 0.45, 0). It is worth noting that this setting means that some superclasses will not be masked. For the adaptive masking guided by intra-class differences, the present invention sets the distance threshold λ to 0.6. In the cascade reconstruction module guided by the center kernel alignment, the proportion of spatial information is set to [0.97, 0.88, 0.78, 0.64, 0.55, 0.44, 0.36, 0.31]. This setting reflects the different emphasis of different network layers on spatial information and semantic information. For the differential gated progressive learning strategy, the present invention sets the slope factor γ to 10 and the gate factor δ to 0.2. The selection of these parameters is intended to achieve a smooth transition between spatial feature reconstruction and semantic feature reconstruction. The present invention chooses to perform point cloud self-supervised learning on the Waymo autonomous driving scene dataset, and evaluates the performance of 3D object detection on the same dataset. The evaluation indicators include standard 3D mean average precision (mAP) and mAP weighted by orientation accuracy (mAPH). Specifically, for the vehicle category, an intersection over union (IoU) threshold of 0.7 is used; for other categories, an IoU threshold of 0.5 is used. The performance evaluation is divided into two difficulty levels: Level 1 is applicable to target boxes containing more than five point clouds, and Level 2 is applicable to target boxes containing at least one point cloud. The present invention mainly uses L2mAP and L2mAPH as key evaluation indicators. Table 1 Parameter setting table of self-supervision method In the adaptive mask guided by inter-class differences, in order to determine the number of superclasses of the semantic feature, the embodiment of the present invention first determines the semantic categories that can represent the autonomous driving scene. Table 2 lists the correspondence between the eight semantic categories and the categories on the downstream SemanticKITTI scene, nuScenes Lidarseg scene, and Waymo scene. Based on this, the number of superclasses N is set to 8. Table 1. Correspondence between semantic categories and scene categories Please refer to Figure 3 , Figure 3 A schematic diagram of a comparison between a true value semantic feature and an encoder output heat map provided by an embodiment of the present invention, where FIGX(a) is a true value semantic feature, FIGX(b) is a heat map output by the encoder when the semantic feature is used for reconstruction, and FIGX(c) is a heat map output by the encoder when the semantic feature is not used for reconstruction. After introducing the semantic voxel feature as a proxy task, the latent features extracted by the encoder are more aligned with the semantic features and have higher mutual information, which is beneficial for downstream tasks that rely on semantic information. Please refer to Table 3, which is a comparative experimental table of reconstruction targets. When only semantic features are used, the performance is not as good as using only spatial features, which shows that it is necessary to reconstruct spatial features. Secondly, the semantic voxel features are supplemented as a proxy task, and the L2mAP is improved by 0.69% and 0.48% respectively compared with various spatial features. Table 2 Comparative experimental table of reconstruction targets Please refer to Figure 4 , Figure 4 The following is a schematic diagram of the distribution of semantic categories and mask rates in a scenario provided by an embodiment of the present invention. Figure 4 (a) and Figure 4 As shown in (b), compared with random masks, the adaptive masks guided by inter-class and intra-class differences assign mask rates with opposite trends to different regions according to the reconstruction difficulty. In addition, it can be seen that areas such as vehicles and pedestrians, which are highly concerned in downstream tasks, have higher mask rates, which forces these semantic categories to undergo complex reconstruction. In addition, areas such as roads and buildings have lower mask rates, which reduces the attention during reconstruction. The adaptive masks guided by inter-class and intra-class differences of the embodiments of the present invention promote mutual information between latent features and semantic objects. Please refer to Table 4, which is a comparative experimental table of different mask sampling strategies. The method of the embodiment of the present invention achieves an improvement of 0.51% in L2mAP. This shows that a smoother information flow between unmasked and masked features will enhance the mutual information between latent features and semantic features. Table 3 Comparative experimental table of different mask sampling strategies Please refer to Table 5, which is a comparative experimental table of different multi-task weighting strategies. On the one hand, due to the higher mutual information between each layer of potential features and the input point cloud, deep supervision can improve model performance. On the other hand, compared with the fixed weighting method, the central kernel alignment-guided cascade reconstruction proposed in the embodiment of the present invention follows hierarchical feature learning and can improve L2mAP by 0.34%. Table 4 Comparative experimental table of different multi-task weighting strategies Please refer to Table 6, which is a comparative experimental table of different multi-task switching strategies. First, all switching strategies bring improvements compared to continuously enabling semantic feature reconstruction. This shows that in multi-agent task learning, it is crucial to follow progressive learning. In addition, the derivative-based method uses the gradient of the pre-task loss to determine whether to start the semantic task. The achievement-based method defines the ratio of the pre-task accuracy to a single task as achievement, and uses it to trigger high-level tasks. The differential gated progressive learning proposed in the embodiment of the present invention deeply studies the mutual information of features, rather than relying on objective indicators such as gradients and accuracy. The more precise switching strategy of the embodiment of the present invention improves the L2 mAP by 0.21%.

[0145] Table 6 Comparative experimental table of different multi-task switching strategies

[0146] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the feature extraction method of the point cloud data.

[0147] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0148] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the feature extraction methods of point cloud data provided by the above methods.

[0149] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the feature extraction method of point cloud data provided by the above methods.

[0150] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A feature extraction method for point cloud data, characterized in that: include: Acquire voxel features of point cloud data, wherein the voxel features include semantic features and spatial features of the point cloud; At least one superclass is obtained by clustering the voxel features based on the semantic features, and a mask rate of each superclass is determined; The point cloud feature extraction model is trained according to the voxel features and the mask rate to obtain a trained point cloud feature extraction model, wherein the mask rate is used to assist the point cloud feature extraction model to focus on key areas in the point cloud data; The trained point cloud feature extraction model is used to extract features from the point cloud data of the target area.

2. The feature extraction method of point cloud data according to claim 1, characterized in that: The step of obtaining voxel features of point cloud data includes: Obtain the spatial features and semantic features of the point cloud data to obtain the point cloud features; The point cloud features are voxelized and average pooled to obtain voxel features.

3. The feature extraction method of point cloud data according to claim 1, characterized in that: The at least one superclass obtained by clustering the voxel features based on the semantic features, and determining the mask rate of each superclass, comprises: Clustering the voxel features based on the semantic features to obtain at least one superclass, and determining a cluster center of each superclass; Based on the first average distance between the cluster centers corresponding to the superclasses, each superclass is divided into a plurality of superclass groups, wherein the superclass groups are used to characterize the inter-class differences between the superclasses; Obtaining a second average distance between the voxel feature within each superclass and the cluster center, wherein the second average distance is used to characterize the intra-class difference within the superclass; Based on the first average distance and the second average distance of each super class, a mask rate of each super class is determined.

4. The feature extraction method of point cloud data according to claim 3, characterized in that: The determining the mask rate of each superclass based on the first average distance and the second average distance of each superclass comprises: Determining a basic masking rate based on the first average distance corresponding to each of the superclass groups; Determining an intra-class consistency coefficient based on the second average distance of each superclass, wherein the intra-class consistency coefficient is inversely proportional to the second average coefficient; The product of the basic mask rate corresponding to each superclass and the intra-class consistency coefficient is used as the mask rate.

5. The feature extraction method of point cloud data according to claim 1, characterized in that: The self-supervised learning of the point cloud feature extraction model according to the voxel feature and the mask rate includes: performing masking on the voxel feature based on the masking rate; The codec is used to reconstruct the semantic features and the spatial features of the voxel features after the mask processing to obtain the reconstructed semantic features and the reconstructed spatial features; Obtaining loss values ​​of the reconstructed semantic features and the reconstructed spatial features; Parameters in the point cloud feature extraction model are adjusted based on the loss value.

6. The feature extraction method of point cloud data according to claim 5, characterized in that: The method of reconstructing semantic features and spatial features of the masked voxel features by using a codec includes: Determine a first weight factor corresponding to the semantic feature reconstruction and a second weight factor corresponding to the spatial feature reconstruction, wherein the first weight factor is greater than the second weight factor; Based on the first weight factor and the second weight factor, semantic feature reconstruction and spatial feature reconstruction are performed on the masked voxel features through an encoder.

7. The feature extraction method of point cloud data according to claim 5, characterized in that: The method of reconstructing semantic features and spatial features of the masked voxel features by using a codec includes: Activate the codec to reconstruct the spatial features of the masked voxel features; Based on the start-up time factor, the codec is activated to reconstruct the semantic features of the voxel features after mask processing, wherein the start-up time factor is determined according to the completion of spatial feature reconstruction when the codec is activated to reconstruct the semantic features during the training process of the point cloud feature extraction model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the feature extraction method for point cloud data according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the feature extraction method of point cloud data as claimed in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the feature extraction method of point cloud data as claimed in any one of claims 1 to 7 is implemented.

Images