Point cloud feature extraction method, device and equipment based on neighborhood geometric coding

Abstract

The invention relates to a point cloud feature extraction method, device and equipment based on neighborhood geometric coding. The method comprises: obtaining an initial point feature representation of an initial three-dimensional point cloud; and constructing a multi-scale neighborhood according to the point feature representation and a preset range strategy. On the first multi-layer perceptron, the correlation between point cloud features in a multi-scale neighborhood is measured through scale dot product attention, and a self-attention graph is obtained; on the second multi-layer perceptron, activation values corresponding to the point cloud features in the multi-scale neighborhoods in different feature channels are obtained through the residual channel attention block, and the channel feature map of the current feature channel is updated according to the activation values. And on a third multi-layer perceptron, acquiring activated point feature representation by adopting a truncation function according to the channel feature map and the multi-scale neighborhood. And after the point feature representation is normalized through softmax, a real point feature representation is obtained. By adopting the method, neighborhood information can be effectively coded, and target point cloud features can be accurately extracted.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional point cloud feature extraction, and in particular to a point cloud feature extraction method, device and equipment based on neighborhood geometry coding. Background Art

[0002] As an efficient and concise signal format for describing three-dimensional objects and scenes, point cloud data plays a vital role in various applications, including 3D reconstruction, scene understanding, object detection, and segmentation. Due to the rapid development of deep learning technology, learning-based point cloud processing has received widespread attention and has spawned many advanced methods. Despite the diverse tasks, effective point feature extraction remains a key step. To this end, numerous point feature extraction methods have been proposed. For example, point features are learned from the 3D coordinates of a single 3D point using a multi-layer perceptron (MLP) network. However, the features formed by the coordinates of a single point cannot provide sufficient detail for subsequent tasks, leading to performance bottlenecks. Summary of the Invention

[0003] Based on this, it is necessary to provide a point cloud feature extraction method, device and equipment based on neighborhood geometric coding that can accurately obtain target point cloud features in response to the above technical problems.

[0004] A point cloud feature extraction method based on neighborhood geometric coding, the method comprising: Get the initial point feature representation of the initial 3D point cloud.

[0005] Construct multi-scale neighborhoods based on point feature representation and preset range strategies.

[0006] On the first multi-layer perceptron, scaled dot product attention is used to measure the correlation between point cloud features in multi-scale neighborhoods to obtain a self-attention map.

[0007] On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhoods in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values.

[0008] On the third multi-layer perceptron, a truncation function is used to obtain the activated point feature representation according to the channel feature map and the multi-scale neighborhood.

[0009] The point feature representation is normalized by softmax to obtain the true point feature representation.

[0010] A point cloud feature extraction device based on neighborhood geometric coding, the device comprising: The preprocessing module is used to obtain the initial point feature representation of the initial three-dimensional point cloud.

[0011] The multi-scale neighborhood construction module is used to construct a multi-scale neighborhood based on point feature representation and a preset range strategy.

[0012] The first layer of self-attention processing module is used to use scale dot product attention to measure the correlation between point cloud features in multi-scale neighborhoods on the first multi-layer perceptron to obtain a self-attention map.

[0013] The second-layer channel processing module is used to use the residual channel attention block on the second multi-layer perceptron to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhood in different feature channels, and update the channel feature map of the current feature channel according to the activation values.

[0014] The third layer response aggregation module is used to obtain the activated point feature representation based on the channel feature map and the multi-scale neighborhood using a truncation function on the third multi-layer perceptron.

[0015] The extraction module is used to obtain the real point feature representation after point feature representation is normalized by softmax.

[0016] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: Get the initial point feature representation of the initial 3D point cloud.

[0017] Construct multi-scale neighborhoods based on point feature representation and preset range strategies.

[0018] On the first multi-layer perceptron, scaled dot product attention is used to measure the correlation between point cloud features in multi-scale neighborhoods to obtain a self-attention map.

[0019] On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhoods in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values.

[0020] On the third multi-layer perceptron, a truncation function is used to obtain the activated point feature representation according to the channel feature map and the multi-scale neighborhood.

[0021] The point feature representation is normalized by softmax to obtain the true point feature representation.

[0022] The above-mentioned method, apparatus, and device for point cloud feature extraction based on neighborhood geometric encoding first construct a multi-scale neighborhood using point feature representation and a preset range strategy to comprehensively capture the geometric relationships of the point cloud at different scales. This enhances the diversity of feature extraction and considers both local details and global information. Furthermore, a scaled dot product attention mechanism is used to calculate the correlation between point cloud features in the multi-scale neighborhood, generating a self-attention map that clearly defines the geometric dependencies between points and their neighborhoods and highlights key point features. Next, a residual channel attention block is used to further optimize the feature extraction process. By calculating the activation values of different feature channels in the multi-scale neighborhood, the importance of each channel feature is dynamically adjusted, thereby improving feature representation. Subsequently, a feature refinement module uses a truncation function to extract the activated point feature representation based on the channel feature map and the multi-scale neighborhood. Finally, softmax normalization is performed to ensure feature consistency and high discriminability. Throughout the entire process, multi-scale neighborhood construction lays the foundation for feature extraction, the attention mechanism effectively filters key information at the local and channel levels, and soft normalization ensures the stability and practicality of the feature representation. This design achieves the goal of deeply mining geometric structural relationships from point cloud data and dynamically strengthening feature representation, effectively solving the problem of insufficient features in traditional methods, providing stronger feature support for subsequent tasks such as point cloud classification and segmentation, breaking through the performance bottleneck caused by insufficient information expression, and comprehensively improving the performance of the point cloud processing network. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a diagram showing an application scenario of a point cloud feature extraction method based on neighborhood geometric coding in one embodiment; Figure 2 1 is a flow chart of a point cloud feature extraction method based on neighborhood geometric coding in one embodiment; Figure 3 FIG1 is a schematic diagram of the architecture of the NPFA module in one embodiment; Figure 4 is a schematic diagram of a multi-scale feature fusion framework in one embodiment; Figure 5 1 is a structural block diagram of a point cloud feature extraction device based on neighborhood geometric coding in one embodiment; Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] The point cloud feature extraction method based on neighborhood geometry coding provided by the present invention can be applied to Figure 1 The point cloud processing network shown in the figure consists of a segmentation network and a classification network. The segmentation network first calibrates the pose of the input point cloud. Then, three NGL modules are applied to learn point features, including cross-layer connections to connect their output point features. Finally, a new point feature representation is implemented to regress the segmentation result for each individual point. The classification network has a similar architecture, but all point features are aggregated into a global representation through a maximum pooling operation. This global representation is then used to regress the classification result.

[0026] In one embodiment, Figure 2 As shown in the figure, a point cloud feature extraction method based on neighborhood geometry coding is provided. Figure 1 The point cloud processing network (NGNet) in

[15] is used as an example to illustrate the following steps: Step 202: Obtain initial point feature representation of the initial three-dimensional point cloud.

[0027] Step 204: construct a multi-scale neighborhood based on the point feature representation and the preset range strategy.

[0028] In step 206 , on the first multi-layer perceptron, scale dot product attention is used to measure the correlation between point cloud features in multi-scale neighborhoods to obtain a self-attention map.

[0029] In step 208, on the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhoods in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values.

[0030] Step 210: On the third multi-layer perceptron, a truncation function is used to obtain an activated point feature representation based on the channel feature map and the multi-scale neighborhood.

[0031] In step 212, the point feature representation is normalized by softmax to obtain the true point feature representation.

[0032] In this point cloud feature extraction method based on neighborhood geometric encoding, a multi-scale neighborhood is first constructed using point feature representations and a pre-set range strategy to comprehensively capture the geometric relationships of the point cloud at different scales. This enhances the diversity of feature extraction and considers both local details and global information. Furthermore, a scaled dot product attention mechanism is used to calculate the correlation between point cloud features in the multi-scale neighborhood, generating a self-attention map that clearly defines the geometric dependencies between points and their neighborhoods and highlights key point features. Next, a residual channel attention block is used to further optimize the feature extraction process. By calculating the activation values of different feature channels in the multi-scale neighborhood, the importance of each channel feature is dynamically adjusted, thereby improving feature representation. Finally, a feature refinement module extracts feature representations of the activated points using a truncation function based on the channel feature map and the multi-scale neighborhood. Finally, softmax normalization is performed to ensure consistent and highly discriminative features. Throughout the entire process, multi-scale neighborhood construction lays the foundation for feature extraction, the attention mechanism effectively filters key information at the local and channel levels, and soft normalization ensures the stability and practicality of the feature representation. This design achieves the goal of deeply mining geometric structural relationships from point cloud data and dynamically strengthening feature representation, effectively solving the problem of insufficient features in traditional methods, providing stronger feature support for subsequent tasks such as point cloud classification and segmentation, breaking through the performance bottleneck caused by insufficient information expression, and comprehensively improving the performance of the point cloud processing network.

[0033] In one embodiment, a 3D point cloud is processed by a multi-layer perceptron network. , output the initial point feature representation: , ; in, is the initial point feature representation, are the parameters of the multi-layer perception network, is the feature representation of the point cloud, The dimension of the query or key vector.

[0034] In one embodiment, the range strategy is defined as: With each scale parameter set Based on the point feature representation, a multi-scale neighborhood is constructed with the target point cloud as the center and each scale parameter as the radius.

[0035] In one embodiment, the first multilayer perceptron: . Second Multilayer Perceptron: . The third multilayer perceptron: .in, is the initial point feature representation, 、 、 are the parameters of the first, second and third multilayer perceptrons respectively.

[0036] In one embodiment, on the first multi-layer perceptron, the scale dot product attention is used to measure the correlation between the point cloud features in the multi-scale neighborhood to obtain a self-attention map: ; in, is the self-attention map, Q is the query matrix, is the transpose of the key matrix, is the dimension of the query or key vector, for n OK n A real matrix of columns, n The field of real numbers.

[0037] In one embodiment, a weighted average is performed on the third multilayer perceptron based on the self-attention map to obtain feature-related information: ; in, For feature related information, is the transposed matrix of the third multilayer perceptron. The feature representation of each initial point is enhanced according to the feature-related information to obtain the enhanced point feature representation: ; in, To enhance point feature representation, is the initial point feature representation.

[0038] In one embodiment, a residual channel attention block is used on a second multilayer perceptron to average all neighboring points of each point in the enhanced point feature representation to generate a global representation: ; in, is the global representation, To take the average value, To enhance point feature representation, the global representation learns channel attention mapping in a fully connected network to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhood: ; in, is the channel attention map, for the reason Parameterized fully connected network. Update the channel feature map of the current feature channel according to the activation value: ; in, is the channel feature map, is the channel weight vector.

[0039] In one embodiment, an architecture of an NPFA module is provided, which first averages all neighboring points to produce a global representation, i.e. ,in represents the averaging operation. Then, as Figure 3 As shown in Figure 2, a fully connected network is applied, which consists of 3 fully connected layers to learn a channel map to indicate the key channels, where , Indicated by Parameterized fully connected network. The feature map is further updated according to the channel map, namely: ; in, and Respectively represented by parameters and Parameterized multilayer perceptrons (MLPs) and fully connected networks.

[0040] It is worth noting that in this way, key features are emphasized by providing a larger response coefficient to explore neighboring geometries. Since this response map learning process is always permutation-equivariant and aggregation achieves permutation invariance, our network can naturally handle the disorder of 3D point clouds.

[0041] In one embodiment, Figure 4 As shown, a multi-scale feature fusion framework is provided. Specifically, Generated by cascade .Then, Regression to parameters through fully connected network . To express preference, this method is based on Combining these features, namely: ; in, represents a fully connected network whose output is normalized by the softmax operation to .

[0042] ; yes The i-th element of , which implicitly indicates the neighborhood selection and guides the feature fusion process.

[0043] It should be understood that although Figure 2The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0044] In one embodiment, Figure 5 As shown, a point cloud feature extraction device based on neighborhood geometric coding is provided, comprising: a pre-processing module 502, a multi-scale neighborhood construction module 504, a first-layer self-attention processing module 506, a second-layer channel processing module 508, a third-layer response aggregation module 510, and an extraction module 512, wherein: The pre-processing module 502 is used to obtain the initial point feature representation of the initial three-dimensional point cloud.

[0045] The multi-scale neighborhood construction module 504 is used to construct a multi-scale neighborhood according to the point feature representation and a preset range strategy.

[0046] The first layer self-attention processing module 506 is used to use scale dot product attention to measure the correlation between point cloud features in multi-scale neighborhoods on the first multi-layer perceptron to obtain a self-attention map.

[0047] The second layer channel processing module 508 is used to use the residual channel attention block on the second multi-layer perceptron to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhood in different feature channels, and update the channel feature map of the current feature channel according to the activation values.

[0048] The third layer response aggregation module 510 is used to obtain the activated point feature representation based on the channel feature map and the multi-scale neighborhood using a truncation function on the third multi-layer perceptron.

[0049] The extraction module 512 is used to obtain the true point feature representation after the point feature representation is normalized by softmax.

[0050] Regarding the specific definition of a point cloud feature extraction device based on neighborhood geometry coding, please refer to the definition of a point cloud feature extraction method based on neighborhood geometry coding above, which will not be repeated here. The various modules in the above-mentioned point cloud feature extraction device based on neighborhood geometry coding can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0051] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a point cloud feature extraction method based on neighborhood geometry coding is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0052] Those skilled in the art will understand that Figure 5-Figure 6 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0053] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: Get the initial point feature representation of the initial 3D point cloud.

[0054] Construct multi-scale neighborhoods based on point feature representation and preset range strategies.

[0055] On the first multi-layer perceptron, scaled dot product attention is used to measure the correlation between point cloud features in multi-scale neighborhoods to obtain a self-attention map.

[0056] On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhoods in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values.

[0057] On the third multi-layer perceptron, a truncation function is used to obtain the activated point feature representation according to the channel feature map and the multi-scale neighborhood.

[0058] The point feature representation is normalized by softmax to obtain the true point feature representation.

[0059] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0060] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A point cloud feature extraction method based on neighborhood geometric coding, characterized in that: Applied to a point cloud processing network, the method includes: Obtaining initial point feature representation of the initial three-dimensional point cloud; Constructing a multi-scale neighborhood based on the point feature representation and a preset range strategy; On the first multi-layer perceptron, scale dot product attention is used to measure the correlation between point cloud features in the multi-scale neighborhood to obtain a self-attention map; On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhood in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values; On a third multilayer perceptron, a truncation function is used to obtain an activated point feature representation according to the channel feature map and the multi-scale neighborhood; The point feature representation is normalized by softmax to obtain the true point feature representation.

2. The method according to claim 1, characterized in that Before the step of obtaining the initial point feature representation of the initial three-dimensional point cloud, the following steps are also included: Processing 3D point clouds through multi-layer perceptron networks , output the initial point feature representation: ， in, is the initial point feature representation, are the parameters of the multi-layer perception network, is the feature representation of the point cloud, The dimension of the query or key vector.

3. The method according to claim 1, characterized in that The preset range strategy is defined as: based on a given target point cloud With each scale parameter set composition; Constructing a multi-scale neighborhood based on the point feature representation and the preset range strategy includes: According to the point feature representation, a multi-scale neighborhood is constructed with the target point cloud as the center and each of the scale parameters as the radius.

4. The method according to claim 3, characterized in that The first multilayer perceptron: ; The second multilayer perceptron: ; The third multilayer perceptron: ; in, is the initial point feature representation, 、 、 are the parameters of the first, second and third multilayer perceptrons respectively.

5. The method according to claim 4, characterized in that On the first multi-layer perceptron, the scale dot product attention is used to measure the correlation between the point cloud features in the multi-scale neighborhood to obtain a self-attention map, including: On the first multi-layer perceptron, the scale dot product attention is used to measure the correlation between the point cloud features in the multi-scale neighborhood to obtain the self-attention map: in, is the self-attention map, Q is the query matrix, is the transpose of the key matrix, is the dimension of the query or key vector, for n OK n A real matrix of columns, n The field of real numbers.

6. The method according to claim 4, characterized in that On the first multi-layer perceptron, after the step of using scaled dot product attention to measure the correlation between point cloud features in the multi-scale neighborhood to obtain a self-attention map, the method further includes: Based on the self-attention map, the third multilayer perceptron is weighted averaged to obtain feature-related information: in, For feature related information, is the transposed matrix of the third multilayer perceptron; Each of the initial point feature representations is enhanced according to the feature related information to obtain an enhanced point feature representation: in, To enhance point feature representation, is the initial point feature representation.

7. The method according to claim 6, characterized in that On the second multi-layer perceptron, the residual channel attention block is used to obtain the activation values corresponding to the point cloud features in the multi-scale neighborhood in different feature channels, and the channel feature map of the current feature channel is updated according to the activation values, including: On the second multi-layer perceptron, the residual channel attention block is used to average all the neighboring points of each point in the enhanced point feature representation to generate a global representation: in, is the global representation, To take the average value, To enhance point feature representation; The global representation learns channel attention mapping in a fully connected network to obtain activation values corresponding to point cloud features in the multi-scale neighborhood: in, is the channel attention map, for the reason Parameterized fully connected networks; Update the channel feature map of the current feature channel according to the activation value: in, is the channel feature map, is the channel weight vector.

8. The method according to claim 7, characterized in that On the third multi-layer perceptron, a truncation function is used to obtain an activated point feature representation according to the channel feature map and the multi-scale neighborhood, including: On the third multilayer perceptron, the first K response parameters are filtered using a truncation function based on the channel feature map and the multi-scale neighborhood: in, is the truncation function, for the reason Parameterized third multilayer perceptron; In the third multilayer perceptron, except for the response parameter whose parameters are set to 0, the point feature representation of the multi-scale neighborhood corresponding to the response parameter is activated by weighted average aggregation: in, is the point feature representation, is the response graph activation function, is an aggregate function.

9. A point cloud feature extraction device based on neighborhood geometric coding, characterized in that: The device comprises: A preprocessing module is used to obtain the initial point feature representation of the initial three-dimensional point cloud; A multi-scale neighborhood construction module, configured to construct a multi-scale neighborhood based on the point feature representation and a preset range strategy; A first-layer self-attention processing module is configured to use scaled dot product attention to measure the correlation between point cloud features in the multi-scale neighborhood on the first multi-layer perceptron to obtain a self-attention map; A second-layer channel processing module is configured to obtain activation values corresponding to the point cloud features in the multi-scale neighborhood in different feature channels using the residual channel attention block on the second multi-layer perceptron, and update the channel feature map of the current feature channel according to the activation values; A third-layer response aggregation module is configured to obtain an activated point feature representation based on the channel feature map and the multi-scale neighborhood using a truncation function on a third multi-layer perceptron; The extraction module is used to obtain the real point feature representation after the point feature representation is normalized by softmax.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images