A point cloud feature extraction method, device and equipment based on neighborhood geometry coding

Abstract

The present application relates to a kind of point cloud feature extraction method, device and equipment based on neighborhood geometry coding.The method includes: obtaining the initial point feature representation of initial three-dimensional point cloud.According to point feature representation and the range strategy of pre-set, multi-scale neighborhood is constructed.In the first multilayer perceptron, the correlation between point cloud features in multi-scale neighborhood is measured using scale dot product attention, and the self-attention graph is obtained.In the second multilayer perceptron, the activation value corresponding to the point cloud feature in multi-scale neighborhood in different feature channels is obtained using residual channel attention block, and the channel feature map of current feature channel is updated according to the activation value.In the third multilayer perceptron, the activated point feature representation is obtained using the truncated function according to the channel feature map and the multi-scale neighborhood.After point feature representation is normalized by softmax, the real point feature representation is obtained.This method can effectively encode neighborhood information and accurately extract target point cloud features.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional point cloud feature extraction, in particular to a point cloud feature extraction method, device and equipment based on neighborhood geometric coding. BACKGROUND

[0002] Point cloud data, as an efficient and concise signal format for describing three-dimensional objects and scenes, plays a crucial role in various applications such as three-dimensional reconstruction, scene understanding, target detection, and segmentation. Due to the vigorous development of deep learning technology, learning-based point cloud processing has received extensive attention and has given rise to many advanced methods. Although the tasks are diverse, effective point feature extraction is always a key step. For this purpose, many point feature extraction methods have been proposed. For example, based on a multi-layer perceptron (MLP) network, point features are learned from the three-dimensional coordinates of a single three-dimensional point. However, the features composed of the coordinates of a single point cannot provide sufficient details for subsequent tasks, resulting in a performance bottleneck. SUMMARY

[0003] Therefore, it is necessary to provide a point cloud feature extraction method, device and equipment based on neighborhood geometric coding, which can accurately obtain target point cloud features, in view of the above technical problems.

[0004] A point cloud feature extraction method based on neighborhood geometric coding, the method comprising:

[0005] Obtaining an initial point feature representation of an initial three-dimensional point cloud.

[0006] Constructing a multi-scale neighborhood according to the point feature representation and a preset range strategy.

[0007] On a first multi-layer perceptron, the correlation between point cloud features in the multi-scale neighborhood is measured using a scale dot product attention to obtain a self-attention map.

[0008] On a second multi-layer perceptron, a residual channel attention block is used to obtain the activation values of point cloud features in the multi-scale neighborhood in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values.

[0009] On a third multi-layer perceptron, the channel feature map and the multi-scale neighborhood are used to obtain the activated point feature representation using a truncation function.

[0010] After the point feature representation is normalized by softmax, the real point feature representation is obtained.

[0011] A point cloud feature extraction device based on neighborhood geometric coding, the device comprising:

[0012] A preprocessing module for obtaining an initial point feature representation of an initial three-dimensional point cloud.

[0013] a multi-scale neighborhood construction module configured to construct a multi-scale neighborhood according to the point feature representation and a preset range strategy.

[0014] a first-layer self-attention processing module configured to measure correlations between point cloud features in the multi-scale neighborhood by using scale dot-product attention on a first multi-layer perceptron to obtain a self-attention graph.

[0015] a second-layer channel processing module configured to obtain activation values of the point cloud features in the multi-scale neighborhood in different feature channels by using a residual channel attention block on a second multi-layer perceptron, and update a channel feature map of a current feature channel according to the activation values.

[0016] a third-layer response aggregation module configured to obtain activated point feature representations by using a clipping function on the channel feature map and the multi-scale neighborhood on a third multi-layer perceptron.

[0017] an extraction module configured to obtain real point feature representations after the point feature representations are normalized by using a softmax.

[0018] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0019] obtaining initial point feature representations of an initial three-dimensional point cloud.

[0020] constructing a multi-scale neighborhood according to the point feature representations and a preset range strategy.

[0021] measuring correlations between point cloud features in the multi-scale neighborhood by using scale dot-product attention on a first multi-layer perceptron to obtain a self-attention graph.

[0022] obtaining activation values of the point cloud features in the multi-scale neighborhood in different feature channels by using a residual channel attention block on a second multi-layer perceptron, and updating a channel feature map of a current feature channel according to the activation values.

[0023] obtaining activated point feature representations by using a clipping function on the channel feature map and the multi-scale neighborhood on a third multi-layer perceptron.

[0024] obtaining real point feature representations after the point feature representations are normalized by using a softmax.

[0025] The above-mentioned method, apparatus, and device for point cloud feature extraction based on neighborhood geometric encoding first construct a multi-scale neighborhood using point feature representation and a preset range strategy to comprehensively capture the geometric relationships of the point cloud at different scales. This enhances the diversity of feature extraction and considers both local details and global information. Furthermore, a scaled dot product attention mechanism is used to calculate the correlation between point cloud features in the multi-scale neighborhood, generating a self-attention map that clearly defines the geometric dependencies between points and their neighborhoods and highlights key point features. Next, a residual channel attention block is used to further optimize the feature extraction process. By calculating the activation values ​​of different feature channels in the multi-scale neighborhood, the importance of each channel feature is dynamically adjusted, thereby improving feature representation. Subsequently, a feature refinement module uses a truncation function to extract the activated point feature representation based on the channel feature map and the multi-scale neighborhood. Finally, softmax normalization is performed to ensure feature consistency and high discriminability. Throughout the entire process, multi-scale neighborhood construction lays the foundation for feature extraction, the attention mechanism effectively filters key information at the local and channel levels, and soft normalization ensures the stability and practicality of the feature representation. This design achieves the goal of deeply mining geometric structural relationships from point cloud data and dynamically strengthening feature representation, effectively solving the problem of insufficient features in traditional methods, providing stronger feature support for subsequent tasks such as point cloud classification and segmentation, breaking through the performance bottleneck caused by insufficient information expression, and comprehensively improving the performance of the point cloud processing network. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a diagram showing an application scenario of a point cloud feature extraction method based on neighborhood geometric coding in one embodiment;

[0027] Figure 2 1 is a flow chart of a point cloud feature extraction method based on neighborhood geometric coding in one embodiment;

[0028] Figure 3 FIG1 is a schematic diagram of the architecture of the NPFA module in one embodiment;

[0029] Figure 4 is a schematic diagram of a multi-scale feature fusion framework in one embodiment;

[0030] Figure 5 1 is a structural block diagram of a point cloud feature extraction device based on neighborhood geometric coding in one embodiment;

[0031] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] The point cloud feature extraction method based on neighborhood geometry coding provided by the present invention can be applied to Figure 1 The point cloud processing network shown in the figure consists of a segmentation network and a classification network. The segmentation network first calibrates the pose of the input point cloud. Then, three NGL modules are applied to learn point features, including cross-layer connections to connect their output point features. Finally, a new point feature representation is implemented to regress the segmentation result for each individual point. The classification network has a similar architecture, but all point features are aggregated into a global representation through a maximum pooling operation. This global representation is then used to regress the classification result.

[0034] In one embodiment, Figure 2 As shown in the figure, a point cloud feature extraction method based on neighborhood geometry coding is provided. Figure 1 The point cloud processing network (NGNet) in

[15] is used as an example to illustrate the following steps:

[0035] Step 202: Obtain initial point feature representation of the initial three-dimensional point cloud.

[0036] Step 204: construct a multi-scale neighborhood based on the point feature representation and the preset range strategy.

[0037] In step 206 , on the first multi-layer perceptron, scale dot product attention is used to measure the correlation between point cloud features in multi-scale neighborhoods to obtain a self-attention map.

[0038] In step 208, on the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values ​​corresponding to the point cloud features in the multi-scale neighborhoods in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values.

[0039] Step 210: On the third multi-layer perceptron, a truncation function is used to obtain an activated point feature representation based on the channel feature map and the multi-scale neighborhood.

[0040] In step 212, the point feature representation is normalized by softmax to obtain the true point feature representation.

[0041] In the point cloud feature extraction method based on neighborhood geometry coding, a multi-scale neighborhood is first constructed through point feature representation and a preset range strategy to comprehensively capture the geometric relationship of the point cloud at different scales and enhance the diversity of feature extraction, considering both local details and global information. On this basis, a scale point multiplication attention mechanism is used to calculate the correlation between point cloud features in the multi-scale neighborhood to generate a self-attention graph, clearly defining the geometric dependency between points and neighborhoods and highlighting key point features. Then, a residual channel attention block is used to further optimize the feature extraction process by calculating the activation values of different feature channels in the multi-scale neighborhood to dynamically adjust the importance of each channel feature, thereby improving the feature expression capability. Subsequently, through a feature refining module, the activated point feature representation is extracted using a truncation function according to the channel feature map and the multi-scale neighborhood, and finally normalized by softmax to ensure consistency and high recognition of the features. In the entire process, the multi-scale neighborhood construction lays the foundation for feature extraction, the attention mechanism effectively filters key information at the local and channel levels, and soft normalization ensures the stability and practicality of the feature representation. This design achieves the goal of deeply mining geometric structure relationships from point cloud data and dynamically enhancing feature representation, effectively solving the problem of insufficient features in traditional methods, providing stronger feature support for subsequent tasks such as point cloud classification and segmentation, breaking through the performance bottleneck caused by insufficient information expression, and comprehensively improving the performance of point cloud processing networks.

[0042] In one embodiment, a three-dimensional point cloud is processed by a multi-layer perception network to output an initial point feature representation:

[0043] , ;

[0044] wherein, is the initial point feature representation, is a parameter of the multi-layer perception network, is a feature representation of the point cloud, is a dimension of a query or key vector.

[0045] In one embodiment, the range strategy is defined as follows: according to a given target point cloud and each scale parameter set. According to the point feature representation, a multi-scale neighborhood is constructed with the target point cloud as the center and each scale parameter as the radius.

[0046] In one embodiment, the first multi-layer perceptron is: The second multi-layer perceptron is: The third multi-layer perceptron is: wherein, is the initial point feature representation, 、 、 are the parameters of the first, second and third multilayer perceptrons respectively.

[0047] In one embodiment, on the first multi-layer perceptron, the scale dot product attention is used to measure the correlation between the point cloud features in the multi-scale neighborhood to obtain a self-attention map:

[0048] ;

[0049] in, is the self-attention map, Q is the query matrix, is the transpose of the key matrix, is the dimension of the query or key vector, for n OK n A real matrix of columns, n The field of real numbers.

[0050] In one embodiment, a weighted average is performed on the third multilayer perceptron based on the self-attention map to obtain feature-related information:

[0051] ;

[0052] in, For feature related information, is the transposed matrix of the third multilayer perceptron. The feature representation of each initial point is enhanced according to the feature-related information to obtain the enhanced point feature representation:

[0053] ;

[0054] in, To enhance point feature representation, is the initial point feature representation.

[0055] In one embodiment, a residual channel attention block is used on a second multilayer perceptron to average all neighboring points of each point in the enhanced point feature representation to generate a global representation:

[0056] ;

[0057] in, is the global representation, To take the average value, To enhance point feature representation, the global representation learns channel attention mapping in a fully connected network to obtain the activation values ​​corresponding to the point cloud features in the multi-scale neighborhood:

[0058] ;

[0059] in, is the channel attention map, for the reason Parameterized fully connected network. Update the channel feature map of the current feature channel according to the activation value:

[0060] ;

[0061] in, is the channel feature map, is the channel weight vector.

[0062] In one embodiment, an architecture of an NPFA module is provided, which first averages all neighboring points to produce a global representation, i.e. ,in represents the averaging operation. Then, as Figure 3 As shown in Figure 2, a fully connected network is applied, which consists of 3 fully connected layers to learn a channel map to indicate the key channels, where , Indicated by Parameterized fully connected network. The feature map is further updated according to the channel map, namely:

[0063] ;

[0064] in, and Respectively represented by parameters and Parameterized multilayer perceptrons (MLPs) and fully connected networks.

[0065] It is worth noting that in this way, key features are emphasized by providing a larger response coefficient to explore neighboring geometries. Since this response map learning process is always permutation-equivariant and aggregation achieves permutation invariance, our network can naturally handle the disorder of 3D point clouds.

[0066] In one embodiment, Figure 4 As shown, a multi-scale feature fusion framework is provided. Specifically, Generated by cascade .Then, Regression to parameters through a fully connected network . To express preference, this method is based on Combining these features, namely:

[0067] ;

[0068] in, represents a fully connected network whose output is normalized by the softmax operation to .

[0069] ;

[0070] yes The i-th element of , which implicitly indicates the neighborhood selection and guides the feature fusion process.

[0071] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0072] In one embodiment, Figure 5 As shown, a point cloud feature extraction device based on neighborhood geometric coding is provided, comprising: a pre-processing module 502, a multi-scale neighborhood construction module 504, a first-layer self-attention processing module 506, a second-layer channel processing module 508, a third-layer response aggregation module 510, and an extraction module 512, wherein:

[0073] The pre-processing module 502 is used to obtain the initial point feature representation of the initial three-dimensional point cloud.

[0074] The multi-scale neighborhood construction module 504 is used to construct a multi-scale neighborhood according to the point feature representation and a preset range strategy.

[0075] The first layer self-attention processing module 506 is used to use scale dot product attention to measure the correlation between point cloud features in multi-scale neighborhoods on the first multi-layer perceptron to obtain a self-attention map.

[0076] The second layer channel processing module 508 is used to use the residual channel attention block on the second multi-layer perceptron to obtain the activation values ​​corresponding to the point cloud features in the multi-scale neighborhood in different feature channels, and update the channel feature map of the current feature channel according to the activation values.

[0077] The third layer response aggregation module 510 is used to obtain the activated point feature representation based on the channel feature map and the multi-scale neighborhood using a truncation function on the third multi-layer perceptron.

[0078] The extraction module 512 is used to obtain the true point feature representation after the point feature representation is normalized by softmax.

[0079] Regarding the specific definition of a point cloud feature extraction device based on neighborhood geometry coding, please refer to the definition of a point cloud feature extraction method based on neighborhood geometry coding above, which will not be repeated here. The various modules in the above-mentioned point cloud feature extraction device based on neighborhood geometry coding can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0080] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a point cloud feature extraction method based on neighborhood geometry coding is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0081] Those skilled in the art will understand that Figure 5-Figure 6 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0082] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0083] Get the initial point feature representation of the initial 3D point cloud.

[0084] Construct multi-scale neighborhoods based on point feature representation and preset range strategies.

[0085] On the first multi-layer perceptron, scaled dot product attention is used to measure the correlation between point cloud features in multi-scale neighborhoods to obtain a self-attention map.

[0086] On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values ​​corresponding to the point cloud features in the multi-scale neighborhoods in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values.

[0087] On the third multi-layer perceptron, a truncation function is used to obtain the activated point feature representation according to the channel feature map and the multi-scale neighborhood.

[0088] The point feature representation is normalized by softmax to obtain the true point feature representation.

[0089] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0090] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A point cloud feature extraction method based on neighborhood geometric coding, characterized in that: Applied to a point cloud processing network, the method includes: Obtaining initial point feature representation of the initial three-dimensional point cloud; Constructing a multi-scale neighborhood based on the point feature representation and a preset range strategy; On the first multi-layer perceptron, scale dot product attention is used to measure the correlation between point cloud features in the multi-scale domain to obtain a self-attention map; Based on the self-attention map, the third multi-layer perceptron is weighted averaged to obtain feature-related information: ； in, For feature related information, is the transposed matrix of the third multilayer perceptron; Each of the initial point feature representations is enhanced according to the feature related information to obtain an enhanced point feature representation: ； in, To enhance point feature representation, is the feature representation of the initial point; On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values ​​corresponding to the point cloud features in the multi-scale domain in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values; On a third multilayer perceptron, a truncation function is used to obtain an activated point feature representation according to the channel feature map and the multi-scale neighborhood; The point feature representation is normalized by softmax to obtain the true point feature representation.

2. The method according to claim 1, characterized in that Before the step of obtaining the initial point feature representation of the initial three-dimensional point cloud, the following steps are also included: Processing 3D point clouds through multi-layer perceptron networks , output the initial point feature representation: ； in, is the initial point feature representation, are the parameters of the multi-layer perception network, is the feature representation of the point cloud, The dimension of the query or key vector.

3. The method according to claim 1, characterized in that The preset range strategy is defined as: based on the given target point cloud With each scale parameter set composition; Constructing a multi-scale neighborhood based on the point feature representation and the preset range strategy includes: According to the point feature representation, a multi-scale domain is constructed with the target point cloud as the center and each scale parameter as the radius.

4. The method according to claim 3, characterized in that The first multilayer perceptron: ; The second multilayer perceptron: ; The third multilayer perceptron: ; in, is the initial point feature representation, 、 、 are the parameters of the first, second and third multilayer perceptrons respectively.

5. The method according to claim 4, characterized in that On the first multi-layer perceptron, the scale dot product attention is used to measure the correlation between the point cloud features in the multi-scale field, and a self-attention map is obtained, including: On the first multi-layer perceptron, the scale dot product attention is used to measure the correlation between point cloud features in the multi-scale field to obtain the self-attention map: ； in, is the self-attention map, Q is the query matrix, is the transpose of the key matrix, is the dimension of the query or key vector, for n OK n A real matrix of columns, n The field of real numbers.

6. The method according to claim 4, characterized in that On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values ​​corresponding to the point cloud features in the multi-scale domain in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values, including: On the second multi-layer perceptron, the residual channel attention block is used to average all the neighboring points of each point in the enhanced point feature representation to generate a global representation: ； in, is the global representation, To take the average value, To enhance point feature representation; The global representation learns channel attention mapping in a fully connected network to obtain activation values ​​corresponding to point cloud features in the multi-scale domain: ； in, is the channel attention map, for the reason Parameterized fully connected networks; Update the channel feature map of the current feature channel according to the activation value: ； in, is the channel feature map, is the channel weight vector.

7. The method according to claim 6, characterized in that On the third multi-layer perceptron, a truncation function is used to obtain an activated point feature representation according to the channel feature map and the multi-scale neighborhood, including: On the third multilayer perceptron, the first K response parameters are filtered using a truncation function based on the channel feature map and the multi-scale neighborhood: ； ； in, is the truncation function, for the reason Parameterized third multilayer perceptron; In the third multilayer perceptron, except for the response parameter whose parameters are set to 0, the point feature representation of the multi-scale neighborhood corresponding to the response parameter is activated by weighted average aggregation: ； in, is the point feature representation, is the response graph activation function, is an aggregate function.

8. A point cloud feature extraction device based on neighborhood geometric coding, characterized in that: The device comprises: A preprocessing module is used to obtain the initial point feature representation of the initial three-dimensional point cloud; A multi-scale neighborhood construction module, configured to construct a multi-scale neighborhood based on the point feature representation and a preset range strategy; The first layer of self-attention processing module is used to use scale dot product attention to measure the correlation between point cloud features in the multi-scale field on the first multi-layer perceptron to obtain a self-attention map; based on the self-attention map, a weighted average is performed on the third multi-layer perceptron to obtain feature-related information: ； in, For feature related information, is the transposed matrix of the third multilayer perceptron; Each of the initial point feature representations is enhanced according to the feature related information to obtain an enhanced point feature representation: ； in, To enhance point feature representation, is the initial point feature representation; A second-layer channel processing module is configured to obtain activation values ​​corresponding to the point cloud features in the multi-scale domain in different feature channels using a residual channel attention block on the second multi-layer perceptron, and update a channel feature map of the current feature channel according to the activation values; A third-layer response aggregation module is configured to obtain an activated point feature representation based on the channel feature map and the multi-scale neighborhood using a truncation function on a third multi-layer perceptron; The extraction module is used to obtain the real point feature representation after the point feature representation is normalized by softmax.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images