A self-attention mechanism-based anti-redundant point cloud classification method and device
The point cloud classification method using the self-attention mechanism solves the problems of point cloud data redundancy and feature fusion, thereby improving the accuracy and efficiency of point cloud classification.
Patent Information
- Application Number
- CN202310128289.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-17
- Publication Date
- 2026-01-09
- Estimated Expiration
- 2043-02-17
AI Technical Summary
Existing technologies lack efficient and accurate point cloud data classification methods, especially in dealing with data redundancy and feature fusion.
A self-attention mechanism is adopted, which integrates global and local information of point cloud data through data acquisition, preprocessing, feature extraction, anti-redundancy processing, feature fusion and regression analysis.
It improves the accuracy and effectiveness of point cloud classification, and enhances the sample richness and feature information utilization efficiency of point cloud data through data augmentation and redundancy processing.
Smart Images
Figure CN116129188B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of three-dimensional point cloud data processing, in particular to a redundant point cloud classification method and device based on a self-attention mechanism. BACKGROUND
[0002] In recent years, with the continuous progress of science and technology, the performance of sensors has been continuously improved, and the development and popularization of three-dimensional sensors such as laser radar and MEMS sensors, three-dimensional point cloud data has emerged in large quantities. Three-dimensional point cloud data has shown a wide range of applications in different fields such as autonomous driving, intelligent robots, remote sensing, and has provided a solid data foundation for real three-dimensional scene understanding. Compared with two-dimensional images, three-dimensional point clouds can more intuitively describe three-dimensional objects. The classification of three-dimensional point clouds is of great significance to the development of autonomous driving, robotics and remote sensing.
[0003] However, three-dimensional sensors still have some drawbacks. Environmental noise, missing sample points, near dense and far sparse, and sample shape distortion during data acquisition can bring difficulties to data analysis. And point cloud data is highly redundant, even if half of the points are randomly lost, the original object can still be represented. Therefore, in the prior art, there is a lack of an efficient and accurate point cloud data classification method that takes into account data redundancy and feature fusion. SUMMARY
[0004] The embodiment of the present application provides a redundant point cloud classification method and device based on a self-attention mechanism. The technical solution is as follows:
[0005] On the one hand, a redundant point cloud classification method based on a self-attention mechanism is provided, which is realized by an electronic device, and the method comprises:
[0006] Data acquisition by a sensor to obtain three-dimensional point cloud data;
[0007] Pretreatment of the three-dimensional point cloud data to obtain pretreated point cloud data;
[0008] Feature extraction based on the pretreated point cloud data to obtain first feature data;
[0009] Anti-redundancy processing based on the first feature data to obtain second feature data;
[0010] Feature fusion based on the second feature data to obtain third feature data;
[0011] Regression analysis based on the third feature data to obtain a classification result.
[0012] Optionally, the data acquisition by the sensor to obtain the three-dimensional point cloud data comprises:
[0013] Obtaining three-dimensional point cloud data of indoor static objects through a sensor device, the three-dimensional point cloud data containing position information of an observation target.
[0014] Optionally, the preprocessing of the three-dimensional point cloud data to obtain preprocessed point cloud data comprises:
[0015] Performing random scaling and random rotation operations on the three-dimensional point cloud data to obtain first point cloud data;
[0016] Performing resampling and height addition operations on the first point cloud data to obtain second point cloud data;
[0017] Adding random noise based on the second point cloud data to obtain preprocessed point cloud data.
[0018] Optionally, the feature extraction based on the preprocessed point cloud data to obtain first feature data comprises:
[0019] Performing farthest point sampling on the preprocessed point cloud data to obtain uniformly distributed center points;
[0020] Obtaining neighbor points near the center points through a ball query operation based on the center points;
[0021] Aggregating information of the neighbor points onto information of the center points through a pooling operation to obtain first feature data.
[0022] Optionally, the anti-redundancy processing based on the first feature data to obtain second feature data comprises:
[0023] Constructing an average similarity matrix of the first feature data according to the first feature data, and performing anti-redundancy processing according to the average similarity matrix of the first feature data to obtain second feature data.
[0024] Optionally, the feature fusion based on the second feature data to obtain third feature data comprises:
[0025] Obtaining local information of point cloud data based on the second feature data by using a channel attention mechanism;
[0026] Obtaining global information of point cloud data based on the second feature data by using a spatial attention mechanism;
[0027] Performing feature fusion based on the local information of point cloud data and the global information of point cloud data to obtain third feature data.
[0028] Optionally, the regression analysis based on the third feature data to obtain a classification result comprises:
[0029] According to the regression operation on the third feature data, a classification probability value of the third feature data is obtained, and a value with the maximum classification probability value is selected as a classification result of the point cloud data.
[0030] In another aspect, a self-attention mechanism-based anti-redundancy point cloud classification device is provided, which is applied to a self-attention mechanism-based anti-redundancy point cloud classification method, and the device comprises:
[0031] A data acquisition module is configured to acquire three-dimensional point cloud data through a sensor.
[0032] A data preprocessing module is configured to preprocess the three-dimensional point cloud data to obtain preprocessed point cloud data.
[0033] A feature extraction module is configured to extract features based on the preprocessed point cloud data to obtain first feature data.
[0034] An anti-redundancy processing module is configured to perform anti-redundancy processing based on the first feature data to obtain second feature data.
[0035] A feature fusion module is configured to perform feature fusion based on the second feature data to obtain third feature data.
[0036] A regression analysis module is configured to perform regression analysis based on the third feature data to obtain a classification result.
[0037] Optionally, the data acquisition module is further configured to:
[0038] The three-dimensional point cloud data of the indoor static object is acquired through a sensor device, and the three-dimensional point cloud data contains position information of an observation target.
[0039] Optionally, the data preprocessing module is further configured to:
[0040] Random scaling and random rotation operations are performed on the three-dimensional point cloud data to obtain first point cloud data.
[0041] Resampling and height addition operations are performed on the first point cloud data to obtain second point cloud data.
[0042] Random noise is added based on the second point cloud data to obtain preprocessed point cloud data.
[0043] Optionally, the feature extraction module is further configured to:
[0044] Farthest point sampling is performed based on the preprocessed point cloud data to obtain uniformly distributed center points.
[0045] Neighbor points near the center points are obtained through a ball query operation based on the center points.
[0046] obtaining first feature data by aggregating information of the neighbor points onto information of the center point through a pooling operation.
[0047] Optionally, the anti-redundancy processing module is further used for:
[0048] constructing an average similarity matrix of the first feature data according to the first feature data, and performing anti-redundancy processing according to the average similarity matrix of the first feature data to obtain second feature data.
[0049] Optionally, the feature fusion module is further used for:
[0050] obtaining local information of the point cloud data based on the second feature data by using a channel attention mechanism;
[0051] obtaining global information of the point cloud data based on the second feature data by using a spatial attention mechanism;
[0052] performing feature fusion based on the local information of the point cloud data and the global information of the point cloud data to obtain third feature data.
[0053] Optionally, the regression analysis module is further used for:
[0054] performing regression operation according to the third feature data to obtain a classification probability value of the third feature data, and selecting a value with the largest classification probability value as a classification result of the point cloud data.
[0055] In another aspect, an electronic device is provided, which includes a processor and a memory, the memory having stored therein at least one instruction, the at least one instruction being loaded and executed by the processor to implement the above-described anti-redundancy point cloud classification method based on a self-attention mechanism.
[0056] In another aspect, a computer-readable storage medium is provided, the storage medium having stored therein at least one instruction, the at least one instruction being loaded and executed by a processor to implement the above-described anti-redundancy point cloud classification method based on a self-attention mechanism.
[0057] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:
[0058] The present application provides an anti-redundancy point cloud classification method based on a self-attention mechanism, which increases the sample richness of point cloud data by means of data enhancement, performs anti-redundancy processing on redundant point cloud feature information, and fuses long and short range feature data of global information and local information of point cloud data by using a channel attention mechanism and a spatial attention mechanism, thereby effectively improving the accuracy of point cloud classification operation. BRIEF DESCRIPTION OF DRAWINGS
[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.
[0060] Figure 1 is a flow chart of a self-attention mechanism-based anti-redundant point cloud classification method provided by an embodiment of the present application;
[0061] Figure 2 is a block diagram of a self-attention mechanism-based anti-redundant point cloud classification device provided by an embodiment of the present application;
[0062] Figure 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0063] In order to make the technical problems, technical solutions and advantages of the present application more clear, the following will be described in detail with reference to the drawings and specific embodiments.
[0064] An embodiment of the present application provides a self-attention mechanism-based anti-redundant point cloud classification method, which can be realized by an electronic device, which can be a terminal or a server. Figure 1 As shown in a self-attention mechanism-based anti-redundant point cloud classification flow chart, the processing flow of the method can include the following steps:
[0065] S1, acquiring three-dimensional point cloud data through a sensor.
[0066] Optionally, the three-dimensional point cloud data acquired through the sensor includes:
[0067] The three-dimensional point cloud data of the indoor static object is acquired through a sensor device, and the three-dimensional point cloud data contains the position information of the observed target.
[0068] In a feasible implementation manner, the three-dimensional image of the indoor static object is acquired through a laser Lidar sensor, including the environment of the object, and the image data is transmitted to a computer. The image format acquired by the laser Lidar sensor is a point cloud format; the computer labels the point cloud object belonging to the category using CloudCompare.
[0069] S2, pre-processing the three-dimensional point cloud data to obtain pre-processed point cloud data.
[0070] Optionally, the 3D point cloud data is preprocessed to obtain preprocessed point cloud data, including:
[0071] Perform random scaling and random rotation operations on the 3D point cloud data to obtain the first point cloud data;
[0072] The first point cloud data is resampled and highly augmented to obtain the second point cloud data.
[0073] Random noise is added to the second point cloud data to obtain preprocessed point cloud data.
[0074] In one feasible implementation, the input point cloud data is randomly zoomed in and out and rotated at a certain horizontal angle to obtain the first point cloud data.
[0075] The first point cloud data is sampled a second time using the farthest point sampling to obtain 1200 point cloud data; the selected 1200 point cloud data are height-added, that is, the minimum value of the y coordinate is subtracted from the y coordinate of each of the selected 1200 point cloud data, and the difference is stitched to the position information to obtain the second point cloud data.
[0076] A small number of invalid random points are added to the second point cloud data as random noise to obtain preprocessed point cloud data.
[0077] S3. Based on the preprocessed point cloud data, feature extraction is performed to obtain the first feature data.
[0078] Optionally, feature extraction is performed based on the preprocessed point cloud data to obtain first feature data, including:
[0079] Based on the preprocessed point cloud data, the farthest point sampling operation is performed to obtain uniformly distributed center points;
[0080] Based on the center point, the neighboring points near the center point are obtained through ball query operation;
[0081] The information of neighboring points is aggregated into the information of the central point through pooling operations to obtain the first feature data.
[0082] In one feasible implementation, in this step, the farthest point sampling is first performed on the preprocessed point cloud data to obtain the coordinates of the 1024 center points with the farthest relative distance among the 1200 preprocessed point cloud data; then, the grouping radius and the number of groups are determined by ball query to obtain the 24 nearest neighbor points of the center point; finally, the feature information of the neighbor points is aggregated to the center point using max pooling to complete the feature aggregation process and obtain the first feature data.
[0083] S4. Perform anti-redundancy processing based on the first feature data to obtain the second feature data.
[0084] Optionally, the anti-redundancy processing based on the first feature data is performed to obtain second feature data, including:
[0085] An average similarity matrix of the first feature data is constructed according to the first feature data, and the anti-redundancy processing is performed according to the average similarity matrix of the first feature data to obtain the second feature data.
[0086] In an available implementation, mean-pooling processing is performed on the similarity matrix of the first feature data to obtain an average similarity matrix; the similarity matrix is subtracted from the average similarity matrix to obtain a differentiated similarity matrix, and the feature information with high similarity is removed to obtain the second feature matrix.
[0087] S5, performing feature fusion based on the second feature data to obtain third feature data.
[0088] Optionally, the feature fusion based on the second feature data is performed to obtain the third feature data, including:
[0089] The local information of the point cloud data is obtained based on the second feature data by using a channel attention mechanism;
[0090] The global information of the point cloud data is obtained based on the second feature data by using a spatial attention mechanism;
[0091] The feature fusion is performed based on the local information of the point cloud data and the global information of the point cloud data to obtain the third feature data.
[0092] In an available implementation, a multi-layer perception is used on the third feature data to obtain a channel-based query, key and value matrix and a spatial-based query, key and value matrix; wherein the size of the channel-based query and key matrix is B*C*C, and the size of the value matrix is B*C*N; the size of the spatial-based query and key matrix is B*N*N, and the size of the value matrix is B*N*C. The similarity matrix is obtained by multiplying the query matrix and the key matrix, and the attention preposition matrix is obtained by multiplying the similarity matrix and the value matrix, and then the point cloud channel-based self-attention matrix and the spatial-based self-attention matrix are obtained through the Relu activation function and the batch normalization function combined with the residual structure, the local feature information obtained by the channel self-attention module and the global feature information obtained by the spatial self-attention module are spliced, and the third feature data is obtained after the two kinds of features are spliced together.
[0093] S6, performing regression analysis based on the third feature data to obtain a classification result.
[0094] Optionally, the regression analysis based on the third feature data is performed to obtain the classification result, including:
[0095] According to the regression operation of the third feature data, a classification probability value of the third feature data is obtained, and the maximum value of the classification probability value is selected as a classification result of the point cloud data.
[0096] In an implementable embodiment, a combination layer is constructed based on three linear full connections, a Relu activation function and a batch normalization function, the third feature data is gradually reduced in dimension through the combination layer to the number of categories of the point cloud data classification target, and regression analysis operation is performed on the third feature data after dimension reduction using a Softmax activation function, so that the final prediction probability value is obtained.
[0097] The application provides a self-attention mechanism-based anti-redundancy point cloud classification method, which increases the sample richness of point cloud data through data enhancement, performs anti-redundancy processing on redundant point cloud feature information, and fuses long and short range feature data of global information and local information of point cloud data by using a channel attention mechanism and a spatial attention mechanism, so that the accuracy of point cloud classification operation is effectively improved.
[0098] Figure 2 is a device block diagram of a self-attention mechanism-based anti-redundancy point cloud classification device according to an example embodiment. Referring to Figure 2 , the device comprises:
[0099] The data acquisition module 210 is configured to acquire three-dimensional point cloud data through a sensor.
[0100] The data preprocessing module 220 is configured to preprocess the three-dimensional point cloud data to obtain preprocessed point cloud data.
[0101] The feature extraction module 230 is configured to extract features based on the preprocessed point cloud data to obtain first feature data.
[0102] The anti-redundancy processing module 240 is configured to perform anti-redundancy processing based on the first feature data to obtain second feature data.
[0103] The feature fusion module 250 is configured to perform feature fusion based on the second feature data to obtain third feature data.
[0104] The regression analysis module 260 is configured to perform regression analysis based on the third feature data to obtain a classification result.
[0105] Optionally, the data acquisition module 210 is further configured to:
[0106] The three-dimensional point cloud data of the indoor static object is acquired through a sensor device, and the three-dimensional point cloud data contains position information of the observation target.
[0107] Optionally, the data preprocessing module 220 is further configured to:
[0108] Randomly scaling and rotating the three-dimensional point cloud data to obtain first point cloud data;
[0109] Resampling and height adding the first point cloud data to obtain second point cloud data;
[0110] Adding random noise based on the second point cloud data to obtain preprocessed point cloud data.
[0111] Optionally, the feature extraction module 230 is further used for:
[0112] Performing farthest point sampling based on the preprocessed point cloud data to obtain uniformly distributed center points;
[0113] Obtaining neighbor points near the center points through a ball query operation based on the center points;
[0114] Aggregating information of the neighbor points to information of the center points through a pooling operation to obtain first feature data.
[0115] Optionally, the anti-redundancy processing module 240 is further used for:
[0116] Constructing an average similarity matrix of the first feature data according to the first feature data, and performing anti-redundancy processing according to the average similarity matrix of the first feature data to obtain second feature data.
[0117] Optionally, the feature fusion module 250 is further used for:
[0118] Obtaining local information of the point cloud data based on the second feature data using a channel attention mechanism;
[0119] Obtaining global information of the point cloud data based on the second feature data using a spatial attention mechanism;
[0120] Performing feature fusion based on the local information of the point cloud data and the global information of the point cloud data to obtain third feature data.
[0121] Optionally, the regression analysis module 260 is further used for:
[0122] Performing regression based on the third feature data to obtain classification probability values of the third feature data, and selecting a value with the largest classification probability value as a classification result of the point cloud data.
[0123] The present application provides an anti-redundancy point cloud classification method based on a self-attention mechanism, which increases the sample richness of point cloud data through data enhancement means, performs anti-redundancy processing on redundant point cloud feature information, and fuses long and short range feature data of global information and local information of point cloud data using a channel attention mechanism and a spatial attention mechanism. The present application effectively improves the accuracy of point cloud classification operation.
[0124] Figure 3 Figure 3 is a structural schematic diagram of an electronic device 300 provided by an embodiment of the present application. The electronic device 300 can have a large difference in configuration or performance, and can include one or more central processing units (CPUs) 301 and one or more memories 302. The memory 302 stores at least one instruction, which is loaded and executed by the processor 301 to implement the steps of the above-described anti-redundancy point cloud classification method based on a self-attention mechanism.
[0125] In an exemplary embodiment, a computer-readable storage medium, such as a memory including instructions, is also provided. The instructions can be executed by a processor in a terminal to complete the above-described anti-redundancy point cloud classification method based on a self-attention mechanism. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
[0126] Those of ordinary skill in the art can understand that all or part of the steps of the above-described embodiments can be completed by hardware, or by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, and the like.
[0127] The above description is merely preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A self-attention mechanism-based anti-redundancy point cloud classification method, characterized in that, The method comprises: data acquisition by a sensor to obtain three-dimensional point cloud data; preprocessing the three-dimensional point cloud data to obtain preprocessed point cloud data; feature extraction based on the preprocessed point cloud data to obtain first feature data; wherein the feature extraction based on the preprocessed point cloud data to obtain first feature data comprises: furthest point sampling operation based on the preprocessed point cloud data to obtain uniformly distributed center points; neighbor points near the center points are obtained through a ball query operation based on the center points; the information of the neighbor points is aggregated onto the information of the center points through a pooling operation to obtain first feature data; anti-redundancy processing based on the first feature data to obtain second feature data; wherein the anti-redundancy processing based on the first feature data to obtain second feature data comprises: constructing an average similarity matrix of the first feature data according to the first feature data, and performing anti-redundancy processing according to the average similarity matrix of the first feature data to obtain second feature data; feature fusion based on the second feature data to obtain third feature data; wherein the feature fusion based on the second feature data to obtain third feature data comprises: obtaining local information of point cloud data based on the second feature data using a channel attention mechanism; obtaining global information of point cloud data based on the second feature data using a spatial attention mechanism; performing feature fusion based on the local information of point cloud data and the global information of point cloud data to obtain third feature data; regression analysis based on the third feature data to obtain a classification result; wherein the regression analysis based on the third feature data to obtain a classification result comprises: performing a regression operation according to the third feature data to obtain a classification probability value of the third feature data, and selecting the value with the largest classification probability value as the classification result of the point cloud data.
2. The method of claim 1, wherein, The data acquisition by a sensor to obtain three-dimensional point cloud data comprises: acquiring three-dimensional point cloud data of indoor static objects through a sensor device, the three-dimensional point cloud data containing position information of an observation target.
3. The method of claim 1, wherein, The preprocessing the three-dimensional point cloud data to obtain preprocessed point cloud data comprises: random scaling operation and random rotation operation on the three-dimensional point cloud data to obtain first point cloud data; resampling operation and height addition operation on the first point cloud data to obtain second point cloud data; adding random noise based on the second point cloud data to obtain preprocessed point cloud data.
4. A self-attention mechanism based anti-redundancy point cloud classification device, characterized in that, The device comprises: a data acquisition module for data acquisition by a sensor to obtain three-dimensional point cloud data; a data preprocessing module for preprocessing the three-dimensional point cloud data to obtain preprocessed point cloud data; a feature extraction module for feature extraction based on the preprocessed point cloud data to obtain first feature data; wherein the feature extraction module is further used for: The farthest point sampling operation is performed based on the preprocessed point cloud data, and a uniformly distributed center point is obtained; neighbor points near the center point are obtained through a ball query operation based on the center point; information of the neighbor points is aggregated onto information of the center point through a pooling operation, and first feature data is obtained; The anti-redundancy processing module is configured to perform anti-redundancy processing based on the first feature data, and obtain second feature data; The anti-redundancy processing module is further configured to: construct an average similarity matrix of the first feature data according to the first feature data, and perform anti-redundancy processing according to the average similarity matrix of the first feature data, and obtain second feature data; The feature fusion module is configured to perform feature fusion based on the second feature data, and obtain third feature data; The feature fusion module is further configured to: obtain local information of the point cloud data based on the second feature data by using a channel attention mechanism; obtain global information of the point cloud data based on the second feature data by using a spatial attention mechanism; perform feature fusion based on the local information of the point cloud data and the global information of the point cloud data, and obtain third feature data; The regression analysis module is configured to perform regression analysis based on the third feature data, and obtain a classification result; The regression analysis module is further configured to: perform regression operation according to the third feature data, obtain a classification probability value of the third feature data, and select a value with the largest classification probability value as the classification result of the point cloud data.
Citation Information
Patent Citations
Convolutional neural network-based point cloud downsampling classification method and system
CN115620064A
Point cloud segmentation and virtual environment generation method and apparatus based on pointnet network
WO2022252274A1