Point cloud-based ground object extraction method, device and equipment

Abstract

The present invention discloses a method, device and equipment for extracting ground objects based on point clouds, which relates to the field of computer vision technology and can solve the technical problems of large computational overhead and low accuracy of current point cloud data extraction. The method comprises: obtaining target point cloud data to be extracted and a trained ground object classification model, wherein the ground object classification model comprises n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, wherein the i-th decoding module corresponds to the n-i+1-th encoding module, each encoding module has a downsampling layer and an encoding layer, and each decoding module has an upsampling layer, a feature splicing layer and a decoding layer; inputting the target point cloud data into the ground object classification model, performing downsampling processing and encoding processing through each encoding module in sequence, performing upsampling processing, feature splicing processing and decoding processing through each decoding module in sequence, and performing ground object classification on the output of the last decoding module through the first multi-layer perceptron; and extracting ground objects according to the ground object classification result.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device and equipment for extracting ground objects based on point clouds. Background Art

[0002] The information of feature distribution plays a guiding role in the rapid establishment of urban three-dimensional models, urban infrastructure construction and environmental protection. Through feature extraction, the distribution of features in different regions can be analyzed, the urban layout can be optimized, the quality of urban life can be improved, and land use can be rationally planned to ensure sustainable urban development. Since there are different types of features, such as buildings, vegetation, roads, and water facilities, and they have different characteristics and forms, and there is no relatively regular structure, all of these have brought difficulties to feature extraction.

[0003] As a three-dimensional structural data, point cloud can directly generate 3D coordinates of the surface of objects and realize 3D representation of objects. At present, point cloud object extraction is mainly based on projection, voxel or. Projection-based methods require 3D point clouds to be projected into 2D images, which leads to the loss of geometric details and makes it a huge challenge to perform fine-grained point cloud extraction tasks. Voxel-based methods require point clouds to be voxelized. Low-resolution voxels may lead to the loss of fine-grained details, and high-resolution voxels may cause significant computational overhead when converting irregular point clouds into regular voxels. Summary of the invention

[0004] In view of this, the present invention provides a method, device and equipment for extracting ground objects based on point cloud, which can solve the current technical problems of high computational overhead and low accuracy in point cloud data extraction.

[0005] According to a first aspect of the present invention, a method for extracting ground objects based on point clouds is provided, the method comprising:

[0006] Obtain the target point cloud data to be extracted and the trained object classification model, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each of the encoding modules includes a downsampling layer and an encoding layer, and each of the decoding modules includes an upsampling layer, a feature concatenation layer and a decoding layer;

[0007] Input the target point cloud data into the land object classification model, perform downsampling processing and encoding processing through each encoding module in turn, perform upsampling processing, feature splicing processing and decoding processing through each decoding module in turn, and perform land object classification on the target decoding result output by the last decoding module through the first multi-layer perceptron to obtain a land object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module;

[0008] The ground object is extracted according to the ground object classification result.

[0009] According to a second aspect of the present invention, a ground feature extraction device based on point cloud is provided, the device comprising:

[0010] An acquisition module is used to acquire the target point cloud data to be extracted and the trained object classification model, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each encoding module includes a downsampling layer and an encoding layer, and each decoding module includes an upsampling layer, a feature concatenation layer and a decoding layer;

[0011] A classification module, used for inputting the target point cloud data into the land object classification model, performing downsampling processing and encoding processing through each encoding module in turn, performing upsampling processing, feature splicing processing and decoding processing through each decoding module in turn, and performing land object classification on the target decoding result output by the last decoding module through the first multi-layer perceptron to obtain a land object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module;

[0012] The extraction module is used to extract the ground objects according to the ground object classification result.

[0013] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned point cloud-based ground object extraction method is implemented.

[0014] According to the fourth aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned point cloud-based object extraction method when executing the program.

[0015] By means of the above technical scheme, the present invention provides a method, device and equipment for extracting ground objects based on point clouds. First, the target point cloud data to be extracted and the trained ground object classification model are obtained, wherein the ground object classification model includes n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each of the encoding modules includes a downsampling layer and an encoding layer, and each of the decoding modules includes an upsampling layer, a feature concatenation layer and a decoding layer; then, The target point cloud data is input into the object classification model, and down-sampling and encoding are performed in sequence through each encoding module, up-sampling, feature splicing and decoding are performed in sequence through each decoding module, and the first multi-layer perceptron is used to classify the object on the target decoding result output by the last decoding module to obtain the object classification result, wherein the feature splicing process of each decoding module is to splice the up-sampling processing result of the decoding module and the encoding result of the corresponding encoding module; finally, the object is extracted according to the object classification result. Through the technical solution of the present invention, the down-sampling layer adopts random sampling, and the computational complexity is independent of the scene size of the target point cloud data, thereby improving the processing efficiency of the object classification model, and combining the neighboring point information of the random sampling point through the self-attention mechanism to prevent information loss, thereby improving the classification accuracy of the object classification model, and the encoding layer adopts three point transformer blocks to be superimposed by residual connection, so that the context information is fully considered during feature extraction, thereby improving the classification accuracy of the object classification model. Compared with the existing technology, there is no need to project the target point cloud data into a 2D image or voxelize the point cloud, which is a processing method that loses information. This improves the accuracy of the object classification results and further improves the accuracy of object extraction.

[0016] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation on the local application. In the drawings:

[0018] Figure 1 A schematic diagram of a process of extracting ground objects based on point clouds provided by an embodiment of the present invention is shown;

[0019] Figure 2A schematic flow chart of another method for extracting ground objects based on point clouds provided by an embodiment of the present invention is shown;

[0020] Figure 3 A schematic structural diagram of a ground feature extraction device based on point cloud provided by an embodiment of the present invention is shown;

[0021] Figure 4 A schematic structural diagram of another ground feature extraction device based on point cloud provided by an embodiment of the present invention is shown;

[0022] Figure 5 A structural example diagram of a land feature classification model provided by an embodiment of the present invention is shown;

[0023] Figure 6 A structural example diagram of another land feature classification model provided by an embodiment of the present invention is shown;

[0024] Figure 7 A diagram showing an example structure of a coding layer provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0026] This embodiment provides a method for extracting ground objects based on point clouds. Figure 1 As shown, the method includes:

[0027] 101. Obtain target point cloud data to be extracted and a trained object classification model, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, n is a positive integer greater than or equal to 2, the ith decoding module corresponds to the n-i+1th encoding module, i∈[1,n], each encoding module includes a downsampling layer and an encoding layer, and each decoding module includes an upsampling layer, a feature concatenation layer and a decoding layer.

[0028] It should be noted that a point cloud is a data set consisting of a large number of points. Each point represents a position in space and contains rich information, including: (1) Position information: Each point has its coordinate position in three-dimensional space, usually expressed in a Cartesian coordinate system (x, y, z). These coordinates represent the position of the point in space. (2) Reflection intensity information: When the lidar scans, the laser reflection intensity returned by each point is measured. This information represents the target surface's ability to reflect the laser and can be used to characterize the target surface's color or material. (3) Color information: Some point cloud data sets contain color information for each point, usually expressed in RGB (red, green, blue) or RGBA (red, green, blue, transparency) format. This information is usually obtained through an attached camera or other sensor.

[0029] For this embodiment, obtaining the target point cloud data to be extracted includes: collecting raw point cloud data based on laser radar; preprocessing the raw point cloud data to obtain the target point cloud data to be extracted. The principle of obtaining raw point cloud data based on airborne laser is to use a laser beam to scan the target object in space, and calculate the distance of the target object by measuring the time from the emission to the reception of the laser beam. This acquisition method is suitable for collecting large-scale scenes such as cities, roads, and forests. It mainly includes the following steps: (1) Emitting a laser beam: The laser radar device emits a laser beam, which is usually a monochromatic laser generated by a laser. (2) Receiving a reflected signal: After the laser beam is irradiated on the surface of the target object, it will be reflected, and the laser radar receives the reflected signal. (3) Calculating the time difference: The laser radar measures the time difference from the emission to the reception of the laser beam, and calculates the distance between the target object and the laser radar through the speed of light and the time difference. (4) Scanning the entire area: The laser radar device scans the entire area by rotating or moving, and continuously obtains the distance information of the target object. (5) Constructing original point cloud data: The laser radar obtains the position and distance information of each point based on the measured distance information and scanning angle, thereby obtaining the original point cloud data. The original point cloud data is then preprocessed by removing gross errors, denoising, filtering, interpolation, and eliminating data redundancy between flight strips to obtain the target point cloud data to be extracted.

[0030] It should be noted that the function of the object classification model is to classify the target point cloud data and obtain the object classification result. Specifically, classifying the target point cloud data means assigning each point in the target point cloud data to its semantic category or object category.

[0031] 102. Input the target point cloud data into the land object classification model, perform downsampling processing and encoding processing through each encoding module in turn, perform upsampling processing, feature splicing processing and decoding processing through each decoding module in turn, and perform land object classification on the target decoding result output by the last decoding module through the first multi-layer perceptron to obtain a land object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module.

[0032] For example, steps 101 and 102, Figure 5 As shown, taking n=2 as an example, there are four encoding modules and two decoding modules, which are the first encoding module, the second encoding module, the third encoding module and the fourth encoding module from top to bottom, and the first decoding module and the second decoding module from bottom to top.

[0033] 103. Extracting ground objects according to the ground object classification result.

[0034] For this embodiment, the object classification result includes the category to which each point in the target point cloud data belongs. Points belonging to the same category are assigned the same label, so that areas belonging to the same category are extracted, realizing regional division and semantic understanding of each area in terms of object category.

[0035] First, the target point cloud data to be extracted and the trained object classification model are obtained, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multilayer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each encoding module includes a downsampling layer and an encoding layer, and each decoding module includes an upsampling layer, a feature splicing layer and a decoding layer; then, the target point cloud data is input into the object classification model, downsampling processing and encoding processing are performed in sequence through each encoding module, upsampling processing, feature splicing processing and decoding processing are performed in sequence through each decoding module, and the first multilayer perceptron is used to classify the target decoding result output by the last decoding module to obtain the object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module; finally, the object is extracted according to the object classification result. Through the technical solution of the present invention, the downsampling layer adopts random sampling, and the computational complexity is independent of the scene size of the target point cloud data, thereby improving the processing efficiency of the object classification model, and combining the neighboring point information of the random sampling points through the self-attention mechanism to prevent information loss, thereby improving the classification accuracy of the object classification model, and the encoding layer adopts three point transformer blocks superimposed by residual connection, so that when extracting features, the context information is fully considered, thereby improving the classification accuracy of the object classification model. Compared with the prior art, there is no need to project the target point cloud data into a 2D image or voxelize the point cloud, which is a processing method that loses information, thereby improving the accuracy of the object classification result, and then improving the accuracy of the object extraction.

[0036] Furthermore, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another ground feature extraction method based on point cloud is provided, such as Figure 2 As shown, the method includes:

[0037] 201. Obtain target point cloud data to be extracted and a trained feature classification model, wherein the feature classification model includes n+2 encoding modules, n decoding modules and a first multilayer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each of the encoding modules includes a downsampling layer and an encoding layer, and also includes a second multilayer perceptron between the downsampling layer and the encoding layer, the encoding layer of each of the encoding modules includes a first point transformer block, a second point transformer block and a third point transformer block, and each of the decoding modules includes an upsampling layer, a feature splicing layer and a decoding layer.

[0038] For this embodiment, Figure 6 and Figure 7 As shown in the figure, the role of the downsampling layer is to randomly sample from the target point cloud data to obtain random sampling points, reduce the amount of calculation, and is suitable for large-scale target point cloud data. The role of the second multi-layer perceptron is to aggregate the feature information of the neighboring points of the random sampling points, while reducing the amount of calculation, without losing data information. The encoding layer uses three point transformer blocks to superimpose, and combined with residual connections, it can better extract features. Since the downsampled points need to be upsampled multiple times to restore to the original point cloud resolution, the upsampling layer can use the nearest neighbor difference method for upsampling, which will not be repeated here. Through the feature splicing layer, the upsampling processing results of the decoding module and the encoding results of the corresponding encoding module are feature spliced ​​(the two feature dimensions to be spliced ​​are p and q respectively, then after feature splicing, the feature dimension is p+q), the feature splicing results are decoded instead of only the upsampling processing results, which can refine the feature representation so that the network can capture global and local semantic information at the same time. The target decoding result output by the last decoding module is restored to the original resolution.

[0039] It should be noted that before inputting the target point cloud data into the land object classification model, the method also includes: constructing an untrained initial land object classification model; obtaining point cloud data samples, and using the point cloud data samples to train the initial land object classification model until a preset stopping condition is met, thereby obtaining a trained land object classification model, wherein the point cloud data samples are marked with land object categories.

[0040] Among them, the structure of the untrained initial object classification model is the same as the structure of the trained object classification model, which will not be repeated here. The method for obtaining point cloud data samples is the same as the method for obtaining target point cloud data, which will not be repeated here. The difference between point cloud data samples and target point cloud data is that each point of the point cloud data sample is marked with the real object category, for example, a certain point is a building and a certain point is vegetation, while the target point cloud data needs to be classified by training the object classification model to obtain the object classification result. The initial land object classification model is trained using point cloud data samples, including: inputting the point cloud data samples into the initial land object classification model to obtain initial land object classification results, the initial land object classification results are the predicted land object categories for each point, and calculating the loss function according to the predicted land object category and the actual land object category of the same point. If the loss function is greater than or equal to a preset value, the training is continued until the loss function is less than the preset value and the preset stop condition is reached to obtain a trained land object classification model, wherein the method of inputting the point cloud data samples into the initial land object classification model to obtain the initial land object classification results is the same as the method of inputting the target point cloud data into the land object classification model to obtain the land object classification results, which will not be repeated here.

[0041] 202. Input the target point cloud data into the land feature classification model, randomly sample the input data through the downsampling layer of the current encoding module to obtain random sampling points, and use the nearest neighbor algorithm to query the neighboring points of the random sampling points, wherein the input data of the downsampling layer of the first encoding module is the target point cloud data, and the input data of the downsampling layers of other encoding modules are the encoding results of the previous encoding module.

[0042] 203. For each of the random sampling points, the neighboring points of the random sampling point are formed into a point set, and the attention score of each feature is calculated according to the features of each of the neighboring points in the point set by using the second multilayer perceptron, and the features of each of the neighboring points in the point set are weighted summed with the corresponding attention scores to obtain a weighted summation result, and the weighted summation result and the features of the random sampling point are used as the current target point cloud sampling data corresponding to the current encoding module.

[0043] For step 202 of the embodiment, since different sampling methods have different computational complexities, computational complexity refers to the complexity of collecting some points from all the points contained in the input data. For this embodiment, since the computational complexity of the random sampling sampling method is independent of the number of points in the scene, the downsampling layer of the land object classification model adopts the random sampling sampling method, so that the land object classification model is suitable for extracting large-scale point cloud data, with high computational efficiency and fast running speed.

[0044] For step 203 of the embodiment, the random sampling points obtained are , use the nearest neighbor algorithm (k-NearestNeighbor, kNN) to find each random sampling point The m neighboring points of , j∈[1,m], point set ={ , }, point set The feature of each neighboring point in , },in, The feature correspondence , The feature correspondence ,... The feature correspondence .

[0045] Due to the sparse nature of point clouds, the attention mechanism can automatically focus on important areas and expand or shrink the receptive field to capture more important features. Therefore, a function g() is proposed to learn the attention score of each feature. The function consists of a second multi-layer perceptron. The following is the point set The calculation formula for the attention score of each feature in is:

[0046] =g( ,W),j∈[1,m]

[0047] Among them, W is the weight parameter learned by the second multi-layer perceptron.

[0048] The features of each neighboring point in the point set are weighted summed with the corresponding attention score to obtain the weighted summation result, and the features of each neighboring point of the randomly sampled point are replaced by the weighted summation result. The calculation formula is as follows:

[0049] =

[0050] By calculating each random sampling point The weighted summation result corresponding to all neighboring points aggregates representative geometric features and descriptor features from the neighboring points, retains the information of the neighboring points, and ensures the accuracy of the land object classification model.

[0051] The current target point cloud sampling data corresponding to the current encoding module is {random sampling points are Features , m weighted summation results }.

[0052] 204. Use the encoding layer of the current encoding module to encode the current target point cloud sampling data to obtain an encoding result output by the current encoding module.

[0053] like Figure 7 As shown, for this embodiment, it includes: for each of the random sampling points, using the first point transformer block of the encoding layer of the current encoding module to calculate the first query matrix corresponding to the feature of the random sampling point in the current target point cloud sampling data and the key matrix and value matrix corresponding to the weighted sum result of any one of the neighboring points, aggregating the first query matrix, the key matrix and the corresponding position code to obtain a first aggregate feature, performing nonlinear transformation on the first aggregate feature, and then normalizing it to obtain a first normalized result, aggregating the value matrix and the position code to obtain a second aggregate feature, performing point multiplication calculation on the first normalized result and the second aggregate feature to obtain a first point multiplication result, adding the first point multiplication results of all the neighboring points corresponding to the random sampling point to obtain a first point multiplication sum result, and performing linear projection on the first point multiplication sum result to obtain a first point transformer block output result corresponding to each of the random sampling points;

[0054] For this embodiment, the aggregating the first query matrix, the key matrix and the corresponding position code to obtain the first aggregated feature includes: calculating the first difference between the first query matrix and the key matrix, and calculating the sum of the first difference and the corresponding position code to obtain the first aggregated feature; the aggregating the value matrix and the corresponding position code to obtain the second aggregated feature includes: calculating the sum of the value matrix and the corresponding position code to obtain the second aggregated feature.

[0055] Among them, the features of the random sampling points in the current target point cloud sampling data are calculated The corresponding first query matrix , random sampling points Any neighboring point of The weighted sum of ,calculate The corresponding key matrix With value matrix (It should be noted that for each j∈[1,m] are the same, each of j∈[1,m] are the same), the first difference of the first query matrix minus the key matrix is , calculate the sum of the first difference and the corresponding position code, and get the first aggregate feature as , calculate the sum of the value matrix and the corresponding position encoding, and get the second aggregate feature as .

[0056] is a function that performs a nonlinear transformation on the first aggregate feature, which can be a fourth multi-layer perceptron that performs a nonlinear transformation on the first aggregate feature to obtain , then, normalization is performed ( is a normalization processing function, such as a softmax function), and the first normalization result is obtained, that is, , perform point multiplication on the first normalized result and the second aggregated feature to obtain the first point multiplication result, add the first point multiplication results of all neighboring points (j∈[1,m]) corresponding to the random sampling point, and obtain the first point multiplication sum result , contains semantic information and feature structure, ( ).

[0057] = ( )

[0058] in, is the dot product operator, and the first dot product is summed up. is a random sampling point Corresponding.

[0059] Then, a linear layer is used to perform linear projection on the result of the first point multiplication and addition, so that the dimension of the result of the first point multiplication and addition is mapped to a higher dimensional space. This transformation helps to increase the expressive power of the model, so that the model can better capture and extract feature information in the input sequence.

[0060] For this embodiment, before aggregating the first query matrix, the key matrix and the corresponding position code to obtain the first aggregated feature, the method also includes: determining the random sampling point corresponding to the first query matrix as the target random sampling point; determining the neighboring point corresponding to the key matrix as the target neighboring point; calculating the second difference between the target random sampling point and the target neighboring point in coordinates, and inputting the second difference into a third multi-layer perceptron to obtain the corresponding position code.

[0061] In the self-attention mechanism, position encoding is very critical, which enables the attention operation to adapt to the local structure in the input data. When processing 3D point cloud data, the coordinates of the points themselves contain position information and can therefore serve as natural position encodings. In order to further improve model performance, trainable parameterized position encodings can be introduced. Trainable parameterized position encodings can extract more complex position information through learning, thus going beyond simply using the coordinates of the points as position encodings. This approach enables the model to better adapt to different data features and learn position representations that are more suitable for the task. The parameterized position encoding function is usually designed as a learnable function that can be adjusted and optimized according to the requirements of the data and tasks to achieve better performance. By introducing trainable parameterized position encodings, the model can better capture the position information in the data and perform better when processing complex tasks. Position encoding function The definition is as follows:

[0062] = ( - )

[0063] in, , are the coordinates of the target random sampling points and the coordinates of the target neighboring points, - is the second difference between the target random sampling point and the target neighboring point in coordinates, the encoding function It is a third multilayer perceptron with two linear layers and one activation function nonlinearity.

[0064] It should be noted that the coding layer stacks three point transformer blocks (the first point transformer block, the second point transformer block, and the third point transformer block, and it is not a simple stacking, but a combination of residual connections, that is, the output result of the first point transformer block is combined with the output result of the second point transformer block, the output result of the first point transformer block is also combined with the output result of the third point transformer block, the output result of the second point transformer block is combined with the output result of the third point transformer block, and the input data of the first point transformer block is combined with the output result of the third point transformer block), which fully considers the contextual information and improves the accuracy of the land feature classification model. The output result of the first point transformer block should continue to be input into the second point transformer block.

[0065] For this embodiment, for each of the random sampling points, the second point transformer block of the encoding layer of the current encoding module is used to calculate the second query matrix corresponding to the output result of the first point transformer block and the key matrix and the value matrix corresponding to the weighted sum result of any one of the neighboring points, aggregate the second query matrix, the key matrix and the corresponding position code to obtain a third aggregate feature, perform nonlinear transformation on the third aggregate feature, and then perform normalization processing to obtain a second normalized result, perform point multiplication calculation on the second normalized result and the second aggregate feature to obtain a second point multiplication result, add the second point multiplication results of all the neighboring points corresponding to the random sampling point to obtain a second point multiplication sum result, perform linear projection on the second point multiplication sum result, and obtain the second point transformer block output result;

[0066] The processing process of the first point converter block, the second point converter block and the third point converter block is the same, and the random sampling point in the output result of the first point converter block is obtained. Corresponding ,calculate The corresponding second query matrix , the third aggregation feature is , the second normalization result is , the second multiplication and addition result is , the second multiplication and addition result is a random sampling point Corresponding.

[0067] = ( )

[0068] Then, a linear layer is used to linearly project the second point multiplication and addition result to obtain the second point transformer block output result.

[0069] The output result of the first point converter block and the output result of the second point converter block corresponding to the same random sampling point are added together to obtain the sum result corresponding to each random sampling point. ;

[0070] For this embodiment, for each of the random sampling points, the third point transformer block of the coding layer of the current coding module is used to calculate the third query matrix corresponding to the sum result and the key matrix and the value matrix corresponding to the weighted sum result of any one of the neighboring points, aggregate the third query matrix, the key matrix and the corresponding position code to obtain a fourth aggregate feature, perform nonlinear transformation on the fourth aggregate feature, and then perform normalization to obtain a third normalized result, perform point multiplication calculation on the third normalized result and the second aggregate feature to obtain a third point multiplication result, add the third point multiplication results of all the neighboring points corresponding to the random sampling point to obtain a third point multiplication sum result, perform linear projection on the third point multiplication sum result, and obtain an output result of the third point transformer block;

[0071] The first point converter block, the second point converter block and the third point converter block have the same processing process, and the random sampling points in the summation result are obtained. Corresponding ,calculate The corresponding third query matrix , the fourth aggregation feature is , the third normalization result is , the third multiplication and addition result is , the third point multiplication and addition result is a random sampling point Corresponding.

[0072] = ( )

[0073] Then, a linear layer is used to linearly project the third point multiplication and addition result to obtain the third point transformer block output result.

[0074] The features of the random sampling points in the current target point cloud sampling data , the first point converter block output result of the same random sampling point The second point converter block outputs the result And the third point converter block output results A multiplication calculation is performed to obtain the encoding result output by the current encoding module.

[0075] 205. Perform upsampling processing, feature splicing processing and decoding processing in turn through each of the decoding modules, and classify the target decoding result output by the last decoding module through the first multi-layer perceptron to obtain a ground object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module.

[0076] For this embodiment, the upsampling processing, feature splicing processing and decoding processing are performed in sequence through each of the decoding modules, including: upsampling processing is performed on the decoding result output by the previous decoding module through the upsampling layer of the current decoding module, wherein the input of the first decoding module is the encoding result of the last encoding module; feature splicing processing is performed on the upsampling processing result of the current decoding module and the encoding result of the corresponding encoding module through the feature splicing layer of the current decoding module; decoding processing is performed on the feature splicing result of the current decoding module through the decoding layer of the current decoding module to obtain the decoding result of the current decoding module as the input of the upsampling layer of the next decoding module, wherein the output of the last decoding module is the target decoding result.

[0077] 206. Extracting ground objects according to the ground object classification result.

[0078] For this embodiment, the object classification result includes the category to which each point in the target point cloud data belongs. Points belonging to the same category are assigned the same label, so that areas belonging to the same category are extracted, realizing regional division and semantic understanding of each area in terms of object category.

[0079] First, the target point cloud data to be extracted and the trained object classification model are obtained, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multilayer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each encoding module includes a downsampling layer and an encoding layer, and each decoding module includes an upsampling layer, a feature splicing layer and a decoding layer; then, the target point cloud data is input into the object classification model, downsampling processing and encoding processing are performed in sequence through each encoding module, upsampling processing, feature splicing processing and decoding processing are performed in sequence through each decoding module, and the first multilayer perceptron is used to classify the target decoding result output by the last decoding module to obtain the object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module; finally, the object is extracted according to the object classification result. Through the technical solution of the present invention, the downsampling layer adopts random sampling, and the computational complexity is independent of the scene size of the target point cloud data, thereby improving the processing efficiency of the object classification model, and combining the neighboring point information of the random sampling points through the self-attention mechanism to prevent information loss, thereby improving the classification accuracy of the object classification model, and the encoding layer adopts three point transformer blocks superimposed by residual connection, so that when extracting features, the context information is fully considered, thereby improving the classification accuracy of the object classification model. Compared with the prior art, there is no need to project the target point cloud data into a 2D image or voxelize the point cloud, which is a processing method that loses information, thereby improving the accuracy of the object classification result, and then improving the accuracy of the object extraction.

[0080] Further, as Figure 1 and Figure 2 The specific implementation of the method shown in the figure, the embodiment of the present invention provides a ground feature extraction device based on point cloud, such as Figure 3 As shown, the device includes: an acquisition module 31, a classification module 32, and an extraction module 33;

[0081] An acquisition module 31 is used to acquire the target point cloud data to be extracted and the trained object classification model, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each of the encoding modules includes a downsampling layer and an encoding layer, and each of the decoding modules includes an upsampling layer, a feature concatenation layer and a decoding layer;

[0082] The classification module 32 is used to input the target point cloud data into the ground object classification model, perform downsampling processing and encoding processing through each encoding module in turn, perform upsampling processing, feature splicing processing and decoding processing through each decoding module in turn, and perform ground object classification on the target decoding result output by the last decoding module through the first multi-layer perceptron to obtain a ground object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module;

[0083] The extraction module 33 is used to extract ground objects according to the ground object classification result.

[0084] Correspondingly, each of the encoding modules also includes a second multi-layer perceptron between the downsampling layer and the encoding layer. In order to perform downsampling processing and encoding processing through each of the encoding modules in turn, the classification module 32 can be specifically used to randomly sample the input data through the downsampling layer of the current encoding module to obtain random sampling points, and use the nearest neighbor algorithm to query the neighboring points of the random sampling points, wherein the input data of the downsampling layer of the first encoding module is the target point cloud data, and the input data of the downsampling layer of other encoding modules is the encoding result of the previous encoding module; for each of the random sampling points, the neighboring points of the random sampling point are formed into a point set, and the second multi-layer perceptron is used to calculate the attention score of each feature according to the features of each of the neighboring points in the point set, and the features of each of the neighboring points in the point set and the corresponding attention score are weighted summed to obtain a weighted summation result, and the weighted summation result and the features of the random sampling points are used as the current target point cloud sampling data corresponding to the current encoding module; the encoding layer of the current encoding module is used to encode the current target point cloud sampling data to obtain the encoding result output by the current encoding module.

[0085] Correspondingly, the encoding layer of each encoding module includes a first point transformer block, a second point transformer block and a third point transformer block. In order to use the encoding layer of the current encoding module to encode the current target point cloud sampling data and obtain the encoding result output by the current encoding module, the classification module 32 can be specifically used to calculate, for each of the random sampling points, the first query matrix corresponding to the feature of the random sampling point in the current target point cloud sampling data and the key matrix and value matrix corresponding to the weighted summation result of any one of the neighboring points using the first point transformer block of the encoding layer of the current encoding module, aggregate the first query matrix, the key matrix and the corresponding position code to obtain a first aggregate feature, perform nonlinear transformation on the first aggregate feature, and then perform normalization processing to obtain a first normalized result, aggregate the value matrix and the position code to obtain a second aggregate feature, perform point multiplication calculation on the first normalized result and the second aggregate feature to obtain a first point multiplication result, add the first point multiplication results of all the neighboring points corresponding to the random sampling point to obtain a first point multiplication sum result, and perform point multiplication on the first normalized result and the second aggregate feature. The first point multiplication and addition result is linearly projected to obtain the first point transformer block output result corresponding to each of the random sampling points; for each of the random sampling points, the second point transformer block of the coding layer of the current coding module is used to calculate the second query matrix corresponding to the output result of the first point transformer block and the key matrix and the value matrix corresponding to the weighted sum result of any of the neighboring points, aggregate the second query matrix, the key matrix and the corresponding position code to obtain a third aggregate feature, perform nonlinear transformation on the third aggregate feature, and then perform normalization processing to obtain a second normalized result, perform point multiplication calculation on the second normalized result and the second aggregate feature to obtain a second point multiplication result, add the second point multiplication results of all the neighboring points corresponding to the random sampling point to obtain a second point multiplication and addition result, and perform linear projection on the second point multiplication and addition result to obtain the second point transformer block output result; the first point transformer block output result and the second point transformer block output result corresponding to the same random sampling point are summed to obtain the sum result corresponding to each of the random sampling points;For each of the random sampling points, the third query matrix corresponding to the sum result and the key matrix and the value matrix corresponding to the weighted sum result of any one of the neighboring points are calculated using the third point transformer block of the encoding layer of the current encoding module, the third query matrix, the key matrix and the corresponding position code are aggregated to obtain a fourth aggregate feature, the fourth aggregate feature is nonlinearly transformed, and then normalized to obtain a third normalized result, the third normalized result and the second aggregate feature are point multiplied to obtain a third point multiplication result, the third point multiplication results of all the neighboring points corresponding to the random sampling point are added to obtain a third point multiplication sum result, and the third point multiplication sum result is linearly projected to obtain a third point transformer block output result; the features of the random sampling point in the current target point cloud sampling data, the output result of the first point transformer block, the output result of the second point transformer block and the output result of the third point transformer block of the same random sampling point are multiplied to obtain the encoding result output by the current encoding module. ;

[0086] Correspondingly, in order to aggregate the first query matrix, the key matrix and the corresponding position coding to obtain the first aggregate feature, the classification module 32 can be specifically used to calculate the first difference between the first query matrix and the key matrix, and calculate the sum of the first difference and the corresponding position coding to obtain the first aggregate feature. In order to aggregate the value matrix and the corresponding position coding to obtain the second aggregate feature, the classification module 32 can be specifically used to calculate the sum of the value matrix and the corresponding position coding to obtain the second aggregate feature.

[0087] In specific application scenarios, such as Figure 4 As shown, a ground feature extraction device based on point cloud, the device also includes: a calculation module 34, which can be specifically used to determine the random sampling points corresponding to the first query matrix as target random sampling points; determine the neighboring points corresponding to the key matrix as target neighboring points; calculate the second difference between the target random sampling points and the target neighboring points in coordinates, input the second difference into a third multi-layer perceptron, and obtain the corresponding position code.

[0088] In specific application scenarios, such as Figure 4 As shown, a point cloud-based object extraction device also includes: a training module 35, which can be used to construct an untrained initial object classification model; obtain point cloud data samples, and use the point cloud data samples to train the initial object classification model until a preset stop condition is met, to obtain a trained object classification model, wherein the point cloud data samples are marked with object categories.

[0089] Correspondingly, in order to perform upsampling processing, feature splicing processing and decoding processing through each of the decoding modules in sequence, the classification module 32 can also be used to upsample the decoding result output by the previous decoding module through the upsampling layer of the current decoding module, wherein the input of the first decoding module is the encoding result of the last encoding module; feature splicing processing is performed on the upsampling processing result of the current decoding module and the encoding result of the corresponding encoding module through the feature splicing layer of the current decoding module; the feature splicing result of the current decoding module is decoded through the decoding layer of the current decoding module to obtain the decoding result of the current decoding module as the input of the upsampling layer of the next decoding module, wherein the output of the last decoding module is the target decoding result.

[0090] It should be noted that for other corresponding descriptions of the functional units involved in the point cloud-based ground feature extraction device provided in this embodiment, reference can be made to Figure 1 to Figure 2 The corresponding description will not be repeated here.

[0091] Based on the above Figure 1 to Figure 2 The method shown in the embodiment, accordingly, also provides a storage medium, which can be volatile or non-volatile, and stores a computer program, which is executed by a processor to implement the above-mentioned Figure 1 to Figure 2 The point cloud-based feature extraction method shown.

[0092] Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present invention.

[0093] Based on the above Figure 1 to Figure 2 The method shown and Figure 3 , Figure 4 In order to achieve the above-mentioned purpose, the present embodiment further provides a computer device, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 to Figure 2 The point cloud-based feature extraction method shown.

[0094] Optionally, the computer device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0095] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0096] The storage medium may also include an operating network communication module. The operating is a program that manages the hardware and software resources of the above-mentioned computer device, supporting the operation of the information processing program and other software and / or programs. The network communication module is used to communicate between the components inside the storage medium, and to communicate with other hardware and software in the information processing entity device.

[0097] Through the description of the above implementation modes, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform, or by hardware.

[0098] First, the target point cloud data to be extracted and the trained object classification model are obtained, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multilayer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each encoding module includes a downsampling layer and an encoding layer, and each decoding module includes an upsampling layer, a feature splicing layer and a decoding layer; then, the target point cloud data is input into the object classification model, downsampling processing and encoding processing are performed in sequence through each encoding module, upsampling processing, feature splicing processing and decoding processing are performed in sequence through each decoding module, and the first multilayer perceptron is used to classify the target decoding result output by the last decoding module to obtain the object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module; finally, the object is extracted according to the object classification result. Through the technical solution of the present invention, the downsampling layer adopts random sampling, and the computational complexity is independent of the scene size of the target point cloud data, thereby improving the processing efficiency of the object classification model, and combining the neighboring point information of the random sampling points through the self-attention mechanism to prevent information loss, thereby improving the classification accuracy of the object classification model, and the encoding layer adopts three point transformer blocks superimposed by residual connection, so that when extracting features, the context information is fully considered, thereby improving the classification accuracy of the object classification model. Compared with the prior art, there is no need to project the target point cloud data into a 2D image or voxelize the point cloud, which is a processing method that loses information, thereby improving the accuracy of the object classification result, and then improving the accuracy of the object extraction.

[0099] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present invention. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0100] The above serial numbers of the present invention are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of the present invention, but the present invention is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for extracting ground objects based on point clouds, characterized in that: The method comprises: Obtain the target point cloud data to be extracted and the trained object classification model, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each of the encoding modules includes a downsampling layer and an encoding layer, and each of the decoding modules includes an upsampling layer, a feature concatenation layer and a decoding layer; Input the target point cloud data into the land object classification model, perform downsampling processing and encoding processing through each encoding module in turn, perform upsampling processing, feature splicing processing and decoding processing through each decoding module in turn, and perform land object classification on the target decoding result output by the last decoding module through the first multi-layer perceptron to obtain a land object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module; Extracting ground objects according to the ground object classification result; Each of the encoding modules further comprises a second multi-layer perceptron between the downsampling layer and the encoding layer; The downsampling process and the encoding process are performed in sequence through each of the encoding modules, including: The input data is randomly sampled through the downsampling layer of the current encoding module to obtain random sampling points, and the neighboring points of the random sampling points are queried using a nearest neighbor algorithm, wherein the input data of the downsampling layer of the first encoding module is the target point cloud data, and the input data of the downsampling layers of the other encoding modules is the encoding result of the previous encoding module; For each of the random sampling points, the neighboring points of the random sampling point are used to form a point set, and the attention score of each feature is calculated according to the features of each of the neighboring points in the point set by using the second multilayer perceptron, and the features of each of the neighboring points in the point set and the corresponding attention score are weighted summed to obtain a weighted summation result, and the weighted summation result and the features of the random sampling point are used as the current target point cloud sampling data corresponding to the current encoding module; Using the encoding layer of the current encoding module to encode the current target point cloud sampling data, to obtain an encoding result output by the current encoding module; The coding layer of each coding module includes a first point transformer block, a second point transformer block and a third point transformer block; The encoding layer of the current encoding module is used to encode the current target point cloud sampling data to obtain the encoding result output by the current encoding module, including: For each of the random sampling points, the first point transformer block of the encoding layer of the current encoding module is used to calculate the first query matrix corresponding to the feature of the random sampling point in the current target point cloud sampling data and the key matrix and value matrix corresponding to the weighted sum result of any one of the neighboring points, aggregate the first query matrix, the key matrix and the corresponding position code to obtain a first aggregate feature, perform nonlinear transformation on the first aggregate feature, perform normalization processing to obtain a first normalized result, aggregate the value matrix and the position code to obtain a second aggregate feature, perform point multiplication calculation on the first normalized result and the second aggregate feature to obtain a first point multiplication result, add the first point multiplication results of all the neighboring points corresponding to the random sampling point to obtain a first point multiplication sum result, perform linear projection on the first point multiplication sum result, and obtain the first point transformer block output result corresponding to each of the random sampling points; For each of the random sampling points, the second point transformer block of the coding layer of the current coding module is used to calculate the second query matrix corresponding to the output result of the first point transformer block and the key matrix and the value matrix corresponding to the weighted sum result of any one of the neighboring points, the second query matrix, the key matrix and the corresponding position code are aggregated to obtain a third aggregate feature, the third aggregate feature is nonlinearly transformed, and then normalized to obtain a second normalized result, the second normalized result is point multiplied with the second aggregate feature to obtain a second point multiplication result, the second point multiplication results of all the neighboring points corresponding to the random sampling point are added to obtain a second point multiplication sum result, and the second point multiplication sum result is linearly projected to obtain a second point transformer block output result; Adding the output result of the first point converter block and the output result of the second point converter block corresponding to the same random sampling point to obtain a summed result corresponding to each random sampling point; For each of the random sampling points, the third point transformer block of the coding layer of the current coding module is used to calculate the third query matrix corresponding to the sum result and the key matrix and the value matrix corresponding to the weighted sum result of any one of the neighboring points, the third query matrix, the key matrix and the corresponding position code are aggregated to obtain a fourth aggregate feature, the fourth aggregate feature is nonlinearly transformed, and then normalized to obtain a third normalized result, the third normalized result is point-multiplied with the second aggregate feature to obtain a third point-multiplied result, the third point-multiplied results of all the neighboring points corresponding to the random sampling point are added to obtain a third point-multiplied sum result, and the third point-multiplied sum result is linearly projected to obtain an output result of the third point transformer block; The characteristics of the random sampling point in the current target point cloud sampling data, the output result of the first point transformer block, the output result of the second point transformer block and the output result of the third point transformer block of the same random sampling point are multiplied to obtain the encoding result output by the current encoding module.

2. The method according to claim 1, characterized in that The aggregating the first query matrix, the key matrix, and the corresponding position encoding to obtain a first aggregated feature includes: Calculate a first difference between the first query matrix and the key matrix, and calculate a sum of the first difference and a corresponding position code to obtain a first aggregate feature; The aggregating the value matrix and the corresponding position code to obtain a second aggregate feature includes: The sum of the value matrix and the corresponding position code is calculated to obtain a second aggregate feature.

3. The method according to claim 1 or 2, characterized in that: Before aggregating the first query matrix, the key matrix, and the corresponding position codes to obtain a first aggregate feature, the method further includes: Determine the random sampling point corresponding to the first query matrix as a target random sampling point; Determine the neighboring point corresponding to the key matrix as a target neighboring point; Calculate the second difference between the target random sampling point and the target neighboring point in coordinates, input the second difference into a third multi-layer perceptron, and obtain a corresponding position code.

4. The method according to claim 1, characterized in that: Before inputting the target point cloud data into the ground object classification model, the method further includes: Construct an untrained initial object classification model; Obtain point cloud data samples, and use the point cloud data samples to train the initial ground object classification model until a preset stop condition is met, to obtain a trained ground object classification model, wherein the point cloud data samples are marked with ground object categories.

5. The method according to claim 1, characterized in that The upsampling process, feature splicing process and decoding process are performed in sequence through each of the decoding modules, including: Performing upsampling processing on the decoding result output by the previous decoding module through the upsampling layer of the current decoding module, wherein the input of the first decoding module is the encoding result of the last encoding module; Performing feature splicing processing on the upsampling processing result of the current decoding module and the encoding result of the corresponding encoding module through the feature splicing layer of the current decoding module; The feature splicing result of the current decoding module is decoded by the decoding layer of the current decoding module to obtain the decoding result of the current decoding module as the input of the upsampling layer of the next decoding module, wherein the output of the last decoding module is the target decoding result.

6. A ground feature extraction device based on point cloud, characterized in that: The device comprises: An acquisition module is used to acquire the target point cloud data to be extracted and the trained object classification model, wherein the object classification model includes n+2 encoding modules, n decoding modules and a first multi-layer perceptron in sequence, n is a positive integer greater than or equal to 2, the i-th decoding module corresponds to the n-i+1-th encoding module, i∈[1,n], each encoding module includes a downsampling layer and an encoding layer, and each decoding module includes an upsampling layer, a feature concatenation layer and a decoding layer; A classification module, used for inputting the target point cloud data into the land object classification model, performing downsampling processing and encoding processing through each encoding module in turn, performing upsampling processing, feature splicing processing and decoding processing through each decoding module in turn, and performing land object classification on the target decoding result output by the last decoding module through the first multi-layer perceptron to obtain a land object classification result, wherein the feature splicing processing of each decoding module is to splice the upsampling processing result of the decoding module and the encoding result of the corresponding encoding module; An extraction module, used for extracting ground objects according to the ground object classification result; Each of the encoding modules also includes a second multi-layer perceptron between the downsampling layer and the encoding layer. The classification module is specifically used to randomly sample the input data through the downsampling layer of the current encoding module to obtain random sampling points, and query the neighboring points of the random sampling points using the nearest neighbor algorithm, wherein the input data of the downsampling layer of the first encoding module is the target point cloud data, and the input data of the downsampling layers of other encoding modules are the encoding results of the previous encoding module; for each of the random sampling points, the neighboring points of the random sampling point are formed into a point set, and the second multi-layer perceptron is used to calculate the attention score of each feature according to the features of each of the neighboring points in the point set, and the features of each of the neighboring points in the point set and the corresponding attention scores are weighted summed to obtain a weighted summation result, and the weighted summation result and the features of the random sampling points are used as the current target point cloud sampling data corresponding to the current encoding module; the encoding layer of the current encoding module is used to encode the current target point cloud sampling data to obtain the encoding result output by the current encoding module; The encoding layer of each encoding module includes a first point transformer block, a second point transformer block and a third point transformer block. The classification module is specifically used to calculate, for each of the random sampling points, a first query matrix corresponding to the features of the random sampling point in the current target point cloud sampling data and a key matrix and a value matrix corresponding to the weighted summation result of any one of the neighboring points using the first point transformer block of the encoding layer of the current encoding module, aggregate the first query matrix, the key matrix and the corresponding position code to obtain a first aggregate feature, perform nonlinear transformation on the first aggregate feature, and then perform normalization processing to obtain a first normalized result, aggregate the value matrix and the position code to obtain a second Aggregate features, perform point multiplication calculation on the first normalized result and the second aggregated feature to obtain a first point multiplication result, add the first point multiplication results of all the neighboring points corresponding to the random sampling point to obtain a first point multiplication sum result, perform linear projection on the first point multiplication sum result to obtain a first point transformer block output result corresponding to each of the random sampling points; for each of the random sampling points, use the second point transformer block of the coding layer of the current coding module to calculate the second query matrix corresponding to the output result of the first point transformer block and the key matrix and the value matrix corresponding to the weighted sum result of any one of the neighboring points, aggregate the second query matrix, the key matrix and the corresponding Position encoding is performed to obtain a third aggregate feature, the third aggregate feature is subjected to nonlinear transformation and then normalized to obtain a second normalized result, the second normalized result is subjected to point multiplication calculation with the second aggregate feature to obtain a second point multiplication result, the second point multiplication results of all the neighboring points corresponding to the random sampling point are added to obtain a second point multiplication sum result, the second point multiplication sum result is subjected to linear projection to obtain a second point transformer block output result; the first point transformer block output result and the second point transformer block output result corresponding to the same random sampling point are added to obtain a sum result corresponding to each random sampling point; for each random sampling point, the current encoding module is used to generate a sum of the first point transformer block output result and the second point transformer block output result. The third point transformer block of the coding layer calculates the third query matrix corresponding to the sum result and the key matrix and the value matrix corresponding to the weighted sum result of any one of the neighboring points, aggregates the third query matrix, the key matrix and the corresponding position code to obtain a fourth aggregate feature, performs nonlinear transformation on the fourth aggregate feature, and then performs normalization processing to obtain a third normalized result, performs point product calculation on the third normalized result and the second aggregate feature to obtain a third point product result, adds the third point product results of all the neighboring points corresponding to the random sampling point to obtain a third point product sum result, and performs linear projection on the third point product sum result to obtain an output result of the third point transformer block;The feature of the random sampling point in the current target point cloud sampling data, the output result of the first point transformer block, the output result of the second point transformer block and the output result of the third point transformer block of the same random sampling point are multiplied to obtain the encoding result output by the current encoding module. ; 7. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the point cloud-based feature extraction method according to any one of claims 1 to 5 is implemented.

8. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the point cloud-based feature extraction method according to any one of claims 1 to 5 is implemented.

Images