Feature Extraction Method, Device and Computer Equipment Based on Point Cloud Segmentation

Abstract

The present application relates to a feature extraction method, device, computer device, and storage medium based on point cloud segmentation. The method includes: obtaining the perceptual features of each point of the point cloud to be segmented at a preset grid scale, performing multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features; performing dual feature iterative processing on the mixed perceptual features to obtain point features after dual feature iterative processing; and performing fusion processing on the mixed perceptual features and the point features obtained after dual feature iterative processing to obtain fused point features. Using this method can improve the accuracy of feature extraction.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and particularly to a feature extraction method, apparatus, computer device, and storage medium based on point cloud segmentation. Background Art

[0002] With the development of computer technology, the number of digital images has been increasing day by day, and the demand for digital image processing technology has also been increasing. Digital images can be specifically represented by point clouds, and the segmentation processing of point clouds is also an important branch of digital image processing technology. Point cloud segmentation is to divide point clouds according to features such as space, geometry, and texture, so that the point clouds in the same division have similar features. A better point cloud segmentation method will facilitate many subsequent applications. Point cloud segmentation is a key project in point cloud tasks for semantic interaction and understanding with the real world.

[0003] Existing point cloud segmentation mainly uses methods such as mathematical model fitting, region growing, minimum segmentation, and Euclidean clustering. These methods are simple and easy to implement, but have poor flexibility, and when there is noise in the point cloud data, it will greatly affect the feature extraction effect, resulting in the defect of low feature extraction accuracy. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a feature extraction method, apparatus, computer device, and storage medium based on point cloud segmentation that can improve the feature extraction accuracy.

[0005] A feature extraction method based on point cloud segmentation, the method includes:

[0006] Obtain the perceptual features of each point of the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perceptual features, and obtain mixed perceptual features;

[0007] Perform dual feature iterative processing on the mixed perceptual features to obtain point features after dual feature iterative processing;

[0008] Perform fusion processing on the mixed perceptual features and the point features obtained after dual feature iterative processing to obtain fused point features.

[0009] In one embodiment, it further includes: processing the perceptual features based on a preset multi-layer perceptron to obtain perceptual features after multi-layer perceptron processing;

[0010] Perform dispersion processing on the perceptual features after multi-layer perceptron processing based on a preset dispersion operator to obtain dispersed perceptual features;

[0011] Perform aggregation processing on the dispersed perceptual features based on a preset aggregation operator to obtain aggregated perceptual features;

[0012] Perform a tensor concatenation operation on the perceptual features after multi-layer perception processing and the aggregated perceptual features to obtain the perceptual features after tensor concatenation;

[0013] Generate the mixed perceptual features based on the perceptual features after tensor concatenation.

[0014] In one embodiment, it further includes: performing multi-scale pooling processing on the mixed perceptual features based on a preset multi-scale pooling layer to obtain first point features;

[0015] Perform a dispersion operation on the first point features based on a preset dispersion operator to obtain dispersed voxel features;

[0016] Extract point features from the dispersed voxel features to obtain second point features, and output the second point features as the point features after double feature iterative processing.

[0017] In one embodiment, it further includes: determining whether the second point features meet a preset condition;

[0018] When the second point features meet the preset condition, output the second point features as the point features after double feature iterative processing;

[0019] When the second point features do not meet the preset condition, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the second point features in sequence to obtain third point features, and output the second point features and the third point features as the point features after double feature iterative processing.

[0020] In one embodiment, it further includes: performing a tensor concatenation operation on the mixed perceptual features and the point features obtained after double feature iterative processing to obtain the point features after tensor concatenation processing;

[0021] Process the point features after tensor concatenation processing based on a preset multi-layer perceptron to obtain the fused point features.

[0022] In one embodiment, it further includes: performing a dispersion operation on the mixed perceptual features based on a preset dispersion operator to obtain the mixed perceptual features after dispersion processing;

[0023] Perform an aggregation operation on the mixed perceptual features after dispersion processing based on a preset aggregation operator to obtain the mixed perceptual features after aggregation processing;

[0024] Perform a tensor concatenation operation on the mixed perceptual features after aggregation processing and the mixed perceptual features to obtain the mixed perceptual features after tensor concatenation processing;

[0025] Process the mixed perception features after tensor cascading processing based on a multi-layer perceptron to obtain the first point features.

[0026] In one embodiment, it further includes: calculating the weight product of the first point features and a preset learnable weight matrix;

[0027] Based on a preset dispersion operator, perform dispersion processing on the weight product to obtain the result of the dispersion processing as the dispersed voxel features.

[0028] In one embodiment, it further includes: processing the dispersed voxel features based on a preset sparse residual network to obtain high-dimensional dispersed voxel features;

[0029] Obtain the second point features based on the high-dimensional dispersed voxel features and a preset geometric weight; the geometric weight is the product of the learnable weight matrix and the mixed perception features.

[0030] A feature extraction device based on point cloud segmentation, the device includes:

[0031] An acquisition module, configured to acquire the perception features of each point of the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perception features to obtain mixed perception features;

[0032] An iteration module, configured to perform dual feature iteration processing on the mixed perception features to obtain the point features after the dual feature iteration processing;

[0033] A fusion module, configured to perform fusion processing on the mixed perception features and the point features obtained after the dual feature iteration processing to obtain the fused point features.

[0034] A computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Acquire the perception features of each point of the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perception features to obtain mixed perception features;

[0036] Perform dual feature iteration processing on the mixed perception features to obtain the point features after the dual feature iteration processing;

[0037] Perform fusion processing on the point features obtained after the dual feature iteration processing to obtain the fused point features.

[0038] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0039] Obtain the perceptual features of each point of the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features;

[0040] Perform dual-feature iterative processing on the mixed perceptual features to obtain point features after dual-feature iterative processing;

[0041] Perform fusion processing on the point features obtained after dual-feature iterative processing to obtain fused point features.

[0042] The above-mentioned feature extraction method, device, computer device, and storage medium based on point cloud segmentation first obtain the perceptual features of each point of the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features; and perform dual-feature iterative processing on the mixed perceptual features to obtain point features after dual-feature iterative processing; finally, perform fusion processing on the mixed perceptual features and the point features obtained after dual-feature iterative processing to obtain fused point features. Through the dual-feature iterative processing of the mixed perceptual features, the mutual iterative processing of point features and voxel features is realized, as well as the fusion of the point features after iterative processing, improving the accuracy of feature extraction. Description of the Drawings

[0043] Figure 1 It is an application environment diagram of the feature extraction method based on point cloud segmentation in an embodiment;

[0044] Figure 2 It is a flowchart of the feature extraction method based on point cloud segmentation in an embodiment;

[0045] Figure 3 It is a flowchart of the feature extraction steps based on point cloud segmentation in another embodiment;

[0046] Figure 4 It is a structural block diagram of the feature extraction device based on point cloud segmentation in an embodiment;

[0047] Figure 5 It is an internal structure diagram of a computer device in an embodiment. Detailed Embodiments

[0048] In order to make the purpose, technical solutions, and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The feature extraction method based on point cloud segmentation provided by the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The terminal 102 and the server 104 can be separately used to execute the point cloud segmentation method provided by this application; the terminal 102 and the server 104 can also be used to cooperate in executing the point cloud segmentation method provided by this application. For example, the server 104 obtains the perception features of each point of the point cloud to be segmented at a preset grid scale, performs multi-scale enhancement processing on the perception features to obtain mixed perception features; performs dual feature iterative processing on the mixed perception features to obtain point features after dual feature iterative processing; performs fusion processing on the mixed perception features and the point features obtained after dual feature iterative processing to obtain fused point features.

[0050] Among them, the terminal 102 can be but is not limited to a terminal device with a point cloud scanning function. For example: an autonomous driving vehicle with a point cloud scanning function. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0051] In one embodiment, as Figure 2 shown, a feature extraction method based on point cloud segmentation is provided. Taking the terminal in Figure 1 as an example, the method includes the following steps:

[0052] Step 202: Obtain the perception features of each point of the point cloud to be segmented at a preset grid scale, and perform multi-scale enhancement processing on the perception features to obtain mixed perception features.

[0053] Specifically, the point cloud to be segmented can be represented by an unordered point set, such as {p1, p2,..., p N}}. Among them, the points in the point cloud to be segmented The perception feature at a given grid scale s is expressed as:

[0054]

[0055] Among them, the set V i represents the set of point coordinates in the same voxel as the point p i ; that is: is the voxel pointer for calculating the point p i at the grid scale s, where, is the floor function; s represents the scale size of each voxel in the xyz directions. c i =(x i , y i , z i ) is the point p iThe point coordinates; N is the number of points in the point cloud to be segmented. After obtaining the perceptual features of each point, perform multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features.

[0056] Step 204: Perform double feature iterative processing on the mixed perceptual features to obtain the point features after double feature iterative processing.

[0057] Specifically, after obtaining the mixed perceptual features, perform double feature iterative processing on the mixed perceptual features. The double feature iterative processing includes voxel feature extraction processing and point feature extraction processing. The double feature iterative processing first performs voxel feature extraction processing on the mixed perceptual features to obtain the mixed perceptual features after voxel feature extraction processing, and then performs point feature extraction processing on the mixed perceptual features after voxel feature extraction processing to obtain the mixed perceptual features after point feature extraction processing. Further, use the mixed perceptual features after point feature extraction processing as the new mixed perceptual features to re-perform voxel feature extraction processing and point feature extraction processing. After each voxel feature extraction processing and point feature extraction processing, the point features after double feature iterative processing are obtained. The point features after the first double feature iterative processing are the mixed perceptual features after point feature extraction processing. When the double feature iterative processing is performed a preset number of times, the point features after the preset number of double feature iterative processing are obtained.

[0058] Step 206: Perform fusion processing on the mixed perceptual features and the point features obtained after double feature iterative processing to obtain the fused point features.

[0059] Specifically, after the double feature iterative processing is performed a preset number of times, the point features after the preset number of double feature iterative processing are obtained. Perform fusion processing on these point features after double feature iterative processing and the mixed perceptual features to obtain the fused point features; as for the specific fusion method, it includes concatenating the point features after each double feature iterative processing and processing through a preset multi-layer perceptron.

[0060] In the above feature extraction method based on point cloud segmentation, first obtain the perceptual features of each point in the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features; perform double feature iterative processing on the mixed perceptual features to obtain the point features after double feature iterative processing; finally, perform fusion processing on the mixed perceptual features and the point features obtained after double feature iterative processing to obtain the fused point features. Through the double feature iterative processing of the mixed perceptual features, the iterative processing of point features and voxel features is realized, and the accuracy of feature extraction is improved.

[0061] In one embodiment, the performing multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features includes:

[0062] Process the perception features based on a preset multi-layer perceptron to obtain the perception features after multi-layer perceptron processing;

[0063] Perform dispersion processing on the perception features after multi-layer perceptron processing based on a preset dispersion operator to obtain the dispersed perception features;

[0064] Perform aggregation processing on the dispersed perception features based on a preset aggregation operator to obtain the aggregated perception features;

[0065] Perform tensor concatenation processing on the perception features after multi-layer perceptron processing and the aggregated perception features to obtain the perception features after tensor concatenation;

[0066] Generate the mixed perception features based on the perception features after tensor concatenation.

[0067] Specifically, when performing multi-scale enhancement processing on the perception features, input the perception features into a preset multi-layer perceptron (MLP, Multilayer Perceptron), and output the perception features after multi-layer perceptron processing. Then perform dispersion processing on the perception features after multi-layer perceptron processing based on a preset dispersion operator to obtain the dispersed perception features; according to the preset dispersion operator Perform dispersion processing on the perception features after multi-layer perceptron processing, where Φ is defined as the mean or maximum operation. After the dispersion processing is completed, based on the preset aggregation operator Perform aggregation processing on the dispersed perception features to obtain the aggregated perception features; where The dispersion operator and the aggregation operator establish a coordinate space mapping system between points and voxels {p, v} for inline feature indexing, realizing the transformation between voxel features and point features between. Where C is the number of feature channels.

[0068] After obtaining the aggregated perception features, perform tensor concatenation processing on the perception features after multi-layer perceptron processing and the aggregated perception features to obtain the perception features after tensor concatenation. Finally, calculate the sum of the perception features after tensor concatenation for each grid scale in the scale list as the mixed perception features. The calculation formula for the mixed perception features can be referred to the following formula:

[0069]

[0070] Where, G p is the mixed perception feature; h is the multi-layer perceptron processing based on MLP; S is the scale list, represents the tensor concatenation processing.

[0071] In this embodiment, the perceptual features are processed based on a preset multi-layer perceptron to obtain the perceptual features after multi-layer perceptron processing; the perceptual features after multi-layer perceptron processing are dispersed based on a preset dispersion operator to obtain the dispersed perceptual features, and the dispersed perceptual features are aggregated based on a preset aggregation operator to obtain the aggregated perceptual features; the perceptual features after multi-layer perceptron processing and the aggregated perceptual features are subjected to tensor concatenation processing to obtain the perceptual features after tensor concatenation, and finally the sum of the perceptual features after tensor concatenation at each point is calculated as the mixed perceptual features, realizing the accurate extraction of the mixed perceptual features and improving the accuracy of the obtained mixed perceptual features.

[0072] In one embodiment, the double feature iteration processing of the mixed perceptual features includes:

[0073] The mixed perceptual features are subjected to multi-scale pooling processing based on a preset multi-scale pooling layer to obtain the first point features; the first point features are dispersed based on a preset dispersion operator to obtain the dispersed voxel features; point feature extraction is performed on the dispersed voxel features to obtain the second point features, and the second point features are output as the point features after double feature iteration processing.

[0074] Specifically, when performing double feature iteration processing on the mixed perceptual features, as Figure 3 shown in the sparse point-voxel feature extraction step in, first, the mixed perceptual features are subjected to multi-scale pooling processing through a preset multi-scale pooling layer to obtain the first point features generated after multi-scale pooling processing; then, the first point features are dispersed based on a preset dispersion operator to obtain the dispersed voxel features; then, point feature extraction is performed on the dispersed voxel features, and the dispersed voxel features after point feature extraction are used as the second point features, and the second point features are output as the point features after double feature iteration processing.

[0075] In this embodiment, the mixed perceptual features are respectively subjected to multi-scale pooling processing, dispersion processing, and point feature extraction processing to obtain the second point features as the point features after double feature iteration processing. The acquisition of the point features after double feature iteration processing is realized, and the efficiency and accuracy of the processing of the mixed perceptual features are improved.

[0076] In one embodiment, after performing point feature extraction on the dispersed voxel features to obtain the second point features and outputting the second point features as the point features after double feature iteration processing, it further includes:

[0077] Determine whether the second point features meet the preset conditions;

[0078] When the second point features meet the preset conditions, the second point features are output as the point features after double feature iteration processing;

[0079] When the second point feature does not meet the preset condition, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the second point feature in sequence to obtain a third point feature, and output the second point feature and the third point feature as the point feature after the dual feature iterative processing.

[0080] Specifically, after outputting the second point feature as the point feature after the dual feature iterative processing, it is also necessary to determine whether the second point feature meets the preset condition. If the second point feature meets the preset condition, output the second point feature as the point feature after the dual feature iterative processing; where the preset condition is the accuracy requirement for the second point feature; if the second point feature meets the preset accuracy requirement, output the second point feature as the point feature after the dual feature iterative processing. If the second point feature does not meet the preset accuracy requirement, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the second point feature in sequence to obtain a third point feature; output the second point feature and the third point feature as the point feature after the dual feature iterative processing. After obtaining the third point feature by performing multi-scale pooling processing, dispersion processing, and point feature extraction on the second point feature in sequence, it further includes determining whether the third point feature meets the preset accuracy requirement. If it meets the preset accuracy requirement, output the second point feature and the third point feature as the point feature after the dual feature iterative processing; otherwise, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the third point feature in sequence to obtain a fourth point feature, and determine whether the fourth point feature meets the preset accuracy requirement. If it meets the preset accuracy requirement, output the second point feature, the third point feature, and the fourth point feature as the point feature after the dual feature iterative processing. Repeat the above determination of whether the finally generated point feature meets the preset accuracy requirement. If it does not meet the requirement, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the finally generated point feature in sequence until the Nth point feature that meets the preset accuracy requirement is obtained. At this time, output the second point feature, the third point feature... to the Nth point feature as the point feature after the dual feature iterative processing.

[0081] In this embodiment, by determining whether the second point feature meets the preset accuracy, when the second point feature meets the preset accuracy requirement, output the second point feature as the point feature after the dual feature iterative processing; otherwise, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the second point feature in sequence to obtain a third point feature, and output the second point feature and the third point feature as the point feature after the dual feature iterative processing. The acquisition of the point feature after the dual feature iterative processing is realized, and the efficiency and accuracy of the mixed perception feature processing are improved.

[0082] In one embodiment, the fusing the mixed perception feature and the point feature obtained after the dual feature iterative processing to obtain a fused point feature includes:

[0083] Perform tensor concatenation processing on the point features obtained after iteratively processing the mixed perception features and the dual features to obtain the point features after tensor concatenation processing;

[0084] Based on a preset multi-layer perceptron, process the point features after tensor concatenation processing to obtain the fused point features.

[0085] Specifically, after the mixed perception features are iteratively processed by the dual features, fusion processing is performed according to the point features obtained after the dual feature iterative processing. The mixed perception features and the point features obtained after the dual feature iterative processing are subjected to tensor concatenation processing to obtain the point features after tensor concatenation processing. Then, the point features after tensor concatenation processing are input into a preset multi-layer perceptron for processing, and the processed features are output as the fused point features.

[0086] In this embodiment, by performing tensor concatenation processing on the point features obtained after iteratively processing the dual features, and based on a preset multi-layer perceptron to process the point features after tensor concatenation processing, the fused point features are obtained, realizing the deep fusion of the point features obtained after iterative processing and improving the accuracy of feature extraction.

[0087] In one embodiment, the multi-scale pooling processing of the mixed perception features based on a preset multi-scale pooling layer to obtain the first point features includes:

[0088] Based on a preset dispersion operator, perform dispersion processing on the mixed perception features to obtain the mixed perception features after dispersion processing;

[0089] Based on a preset aggregation operator, perform aggregation processing on the mixed perception features after dispersion processing to obtain the mixed perception features after aggregation processing;

[0090] Perform tensor concatenation processing on the mixed perception features after aggregation processing and the mixed perception features to obtain the mixed perception features after tensor concatenation processing;

[0091] Based on a multi-layer perceptron, process the mixed perception features after tensor concatenation processing to obtain the first point features.

[0092] Specifically, when performing multi-scale pooling processing on the mixed perception features, first, based on a preset dispersion operator perform dispersion processing on the mixed perception features to obtain the mixed perception features after dispersion processing; then, based on a preset aggregation operator Perform aggregation processing on the mixed perception features after dispersion processing to obtain the mixed perception features after aggregation processing. Finally, perform tensor concatenation processing on the mixed perception features after aggregation processing for each grid scale, and process the mixed perception features after tensor concatenation processing based on a preset multi-layer perceptron, that is, input the mixed perception features after tensor concatenation processing into the preset multi-layer perceptron to obtain the mixed perception features after tensor concatenation processing with multi-scale pooling as the first point feature.

[0093] In this embodiment, the multi-scale pooling layer preset to include a dispersion operator, an aggregation operator, a tensor concatenation processing operator, and a multi-layer perceptron performs multi-scale pooling processing on the mixed perception features to obtain the first point feature, realizing the acquisition of the first point feature required for voxel feature extraction and improving the accuracy of feature extraction.

[0094] In one embodiment, the dispersion processing of the first point feature based on a preset dispersion operator to obtain dispersed voxel features includes:

[0095] Calculate the weighted product of the first point feature and a preset learnable weight matrix;

[0096] Based on a preset dispersion operator, perform dispersion processing on the weighted product to obtain the result of dispersion processing as the dispersed voxel feature.

[0097] Specifically, the formula for obtaining the dispersed voxel feature by performing dispersion processing on the first point feature based on a preset dispersion operator can be referred to the following formula:

[0098]

[0099] Among them, W is a preset learnable weight matrix; f is the first point feature; when calculating the weighted product of the first point feature and the preset learnable weight matrix, the weighted product is Wf;

[0100] The calculation formula for performing dispersion processing on the weighted product based on a preset dispersion operator can be referred to the following formula:

[0101]

[0102] Among them, is the voxel index; the result of the dispersion processing obtained after performing dispersion processing on the weighted product is the dispersed voxel feature.

[0103] In this embodiment, by calculating the weighted product of the first point feature and a preset learnable weight matrix and performing dispersion processing on the weighted product based on a preset dispersion operator to obtain the result of dispersion processing as the dispersed voxel feature, the extraction of the dispersed voxel feature is realized, and the accuracy of voxel feature extraction for the mixed perception features is improved.

[0104] In one embodiment, the extraction of point features from the dispersed voxel features to obtain second point features includes:

[0105] Processing the dispersed voxel features based on a preset sparse residual network to obtain high-dimensional dispersed voxel features;

[0106] Obtaining the second point features based on the high-dimensional dispersed voxel features and a preset geometric weight; the geometric weight is the product of the learnable weight matrix and the mixed perception feature.

[0107] Specifically, after obtaining the dispersed voxel features, when extracting point features from the dispersed voxel features, as Figure 3 shown in the sparse voxel - point feature extraction step, first process the dispersed voxel features based on a preset sparse residual network to obtain high-dimensional dispersed voxel features; wherein, the preset sparse residual network is obtained by replacing the 2D sparse convolution in ResNet with 3D convolution. After obtaining the high-dimensional dispersed voxel features, perform an exclusive NOR operation on the high-dimensional dispersed voxel features and the preset geometric weight to obtain the second point features of each perception feature; wherein, the geometric weight is the product of the learnable weight matrix and the mixed perception feature. The specific calculation formula can refer to the following formula:

[0108] F att = WG p ,

[0109] F out = F ⊙ F att

[0110] wherein, W is a preset learnable weight matrix; F att is the geometric weight; F is the high-dimensional dispersed voxel feature; F out is the output second point feature.

[0111] In this embodiment, by processing the dispersed voxel features based on a preset sparse residual network to obtain high-dimensional dispersed voxel features, and obtaining the second point features based on the high-dimensional dispersed voxel features and a preset geometric weight, the extraction of the second point features is realized.

[0112] It should be understood that although Figure 2-3 the steps in the flowchart of Figure 2-3At least some of the steps may include multiple steps or multiple stages, and these steps or stages do not necessarily need to be executed and completed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0113] In one embodiment, as Figure 4 shown, a feature extraction device based on point cloud segmentation is provided, including: an acquisition module 401, an iterative module 402, and a fusion module 403, where:

[0114] The acquisition module 401 is configured to acquire the perception features of each point of the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perception features, and acquire mixed perception features.

[0115] The iterative module 402 is configured to perform dual feature iterative processing on the mixed perception features to acquire point features after dual feature iterative processing.

[0116] The fusion module 403 is configured to perform fusion processing on the mixed perception features and the point features acquired after dual feature iterative processing to acquire fused point features.

[0117] In one of the embodiments, the acquisition module 401 is further configured to process the perception features based on a preset multi-layer perceptron to acquire perception features after multi-layer perceptron processing; perform dispersion processing on the perception features after multi-layer perceptron processing based on a preset dispersion operator to acquire dispersed perception features; perform aggregation processing on the dispersed perception features based on a preset aggregation operator to acquire aggregated perception features; perform tensor concatenation processing on the perception features after multi-layer perceptron processing and the aggregated perception features to acquire tensor concatenated perception features; and generate the mixed perception features based on the tensor concatenated perception features.

[0118] In one of the embodiments, the iterative module 402 is further configured to perform multi-scale pooling processing on the mixed perception features based on a preset multi-scale pooling layer to acquire first point features; perform dispersion processing on the first point features based on a preset dispersion operator to acquire dispersed voxel features; extract point features from the dispersed voxel features to acquire second point features, and output the second point features as the point features after dual feature iterative processing.

[0119] In one embodiment, the iterative module 402 is further configured to determine whether the second point feature meets a preset condition; when the second point feature meets the preset condition, output the second point feature as the point feature after double-feature iterative processing; when the second point feature does not meet the preset condition, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the second point feature in sequence to obtain a third point feature, and output the second point feature and the third point feature as the point feature after double-feature iterative processing.

[0120] In one embodiment, the fusion module 403 is further configured to perform tensor concatenation processing on the hybrid perception feature and the point feature obtained after double-feature iterative processing to obtain a point feature after tensor concatenation processing; and process the point feature after tensor concatenation processing based on a preset multi-layer perceptron to obtain a fused point feature.

[0121] In one embodiment, the iterative module 402 is further configured to perform dispersion processing on the hybrid perception feature based on a preset dispersion operator to obtain a hybrid perception feature after dispersion processing; perform aggregation processing on the hybrid perception feature after dispersion processing based on a preset aggregation operator to obtain a hybrid perception feature after aggregation processing; perform tensor concatenation processing on the hybrid perception feature after aggregation processing and the hybrid perception feature to obtain a hybrid perception feature after tensor concatenation processing; and process the hybrid perception feature after tensor concatenation processing based on a multi-layer perceptron to obtain a first point feature.

[0122] In one embodiment, the iterative module 402 is further configured to calculate a weighted product of the first point feature and a preset learnable weight matrix; perform dispersion processing on the weighted product based on a preset dispersion operator, and obtain the result of dispersion processing as a dispersed voxel feature.

[0123] In one embodiment, the iterative module 402 is further configured to process the dispersed voxel feature based on a preset sparse residual network to obtain a high-dimensional dispersed voxel feature; obtain the second point feature based on the high-dimensional dispersed voxel feature and a preset geometric weight; and the geometric weight is the product of the learnable weight matrix and the hybrid perception feature.

[0124] The above-mentioned feature extraction device based on point cloud segmentation first obtains the perceptual features of each point of the point cloud to be segmented at a preset grid scale, performs multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features, and performs dual-feature iterative processing on the mixed perceptual features to obtain point features after dual-feature iterative processing. Finally, the mixed perceptual features and the point features obtained after dual-feature iterative processing are fused to obtain fused point features. Through the dual-feature iterative processing of the mixed perceptual features, the mutual iterative processing of point features and voxel features is realized, and the fusion of the point features after iterative processing is performed, improving the extraction accuracy of features.

[0125] For the specific limitations of the feature extraction device based on point cloud segmentation, reference can be made to the limitations on the point cloud segmentation method in the above text, which will not be elaborated here. Each module in the above-mentioned feature extraction device based on point cloud segmentation can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0126] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a feature extraction method based on point cloud segmentation.

[0127] Those skilled in the art can understand that Figure 5 the structure shown in

[0128] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0129] Obtain the perceptual features of each point of the point cloud to be segmented at a preset grid scale, perform multi-scale enhancement processing on the perceptual features to obtain mixed perceptual features;

[0130] Perform double - feature iterative processing on the mixed perception features to obtain point features after double - feature iterative processing;

[0131] Perform fusion processing on the mixed perception features and the point features obtained after double - feature iterative processing to obtain fused point features.

[0132] In one embodiment, when the processor executes the computer program, the following steps are also implemented: Process the perception features based on a preset multi - layer perceptron to obtain perception features after multi - layer perceptron processing; Perform dispersion processing on the perception features after multi - layer perceptron processing based on a preset dispersion operator to obtain dispersed perception features; Perform aggregation processing on the dispersed perception features based on a preset aggregation operator to obtain aggregated perception features; Perform tensor concatenation processing on the perception features after multi - layer perceptron processing and the aggregated perception features to obtain perception features after tensor concatenation; Generate the mixed perception features based on the perception features after tensor concatenation.

[0133] In one embodiment, when the processor executes the computer program, the following steps are also implemented: Perform multi - scale pooling processing on the mixed perception features based on a preset multi - scale pooling layer to obtain first - point features; Perform dispersion processing on the first - point features based on a preset dispersion operator to obtain dispersed voxel features; Perform point - feature extraction on the dispersed voxel features to obtain second - point features, and output the second - point features as the point features after double - feature iterative processing.

[0134] In one embodiment, when the processor executes the computer program, the following steps are also implemented: Determine whether the second - point features meet a preset condition; When the second - point features meet the preset condition, output the second - point features as the point features after double - feature iterative processing; When the second - point features do not meet the preset condition, perform multi - scale pooling processing, dispersion processing, and point - feature extraction on the second - point features in sequence to obtain third - point features, and output the second - point features and the third - point features as the point features after double - feature iterative processing.

[0135] In one embodiment, when the processor executes the computer program, the following steps are also implemented: Perform tensor concatenation processing on the mixed perception features and the point features obtained after double - feature iterative processing to obtain point features after tensor concatenation processing; Process the point features after tensor concatenation processing based on a preset multi - layer perceptron to obtain fused point features.

[0136] In one embodiment, when the processor executes the computer program, the following steps are further implemented: performing dispersion processing on the mixed perception feature based on a preset dispersion operator to obtain the mixed perception feature after dispersion processing; performing aggregation processing on the mixed perception feature after dispersion processing based on a preset aggregation operator to obtain the mixed perception feature after aggregation processing; performing tensor concatenation processing on the mixed perception feature after aggregation processing and the mixed perception feature to obtain the mixed perception feature after tensor concatenation processing; and processing the mixed perception feature after tensor concatenation processing based on a multi-layer perceptron to obtain the first point feature.

[0137] In one embodiment, when the processor executes the computer program, the following steps are further implemented: calculating the weight product of the first point feature and a preset learnable weight matrix; and performing dispersion processing on the weight product based on a preset dispersion operator to obtain the result of dispersion processing as the dispersed voxel feature.

[0138] In one embodiment, when the processor executes the computer program, the following steps are further implemented: processing the dispersed voxel feature based on a preset sparse residual network to obtain a high-dimensional dispersed voxel feature; and obtaining the second point feature based on the high-dimensional dispersed voxel feature and a preset geometric weight, where the geometric weight is the product of the learnable weight matrix and the mixed perception feature.

[0139] The above computer device first obtains the perception features of each point of the point cloud to be segmented at a preset grid scale, performs multi-scale enhancement processing on the perception features to obtain a mixed perception feature; performs dual-feature iterative processing on the mixed perception feature to obtain the point feature after dual-feature iterative processing; and finally performs fusion processing on the mixed perception feature and the point feature obtained after dual-feature iterative processing to obtain the fused point feature. Through the dual-feature iterative processing of the mixed perception feature, the mutual iterative processing of the point feature and the voxel feature and the fusion of the point feature after iterative processing are realized, and the accuracy of feature extraction is improved.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0141] Obtaining the perception features of each point of the point cloud to be segmented at a preset grid scale, and performing multi-scale enhancement processing on the perception features to obtain a mixed perception feature;

[0142] Performing dual-feature iterative processing on the mixed perception feature to obtain the point feature after dual-feature iterative processing;

[0143] Performing fusion processing on the mixed perception feature and the point feature obtained after dual-feature iterative processing to obtain the fused point feature.

[0144] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: processing the perception features based on a preset multi-layer perceptron to obtain the perception features after multi-layer perceptron processing; performing dispersion processing on the perception features after multi-layer perceptron processing based on a preset dispersion operator to obtain the dispersed perception features; performing aggregation processing on the dispersed perception features based on a preset aggregation operator to obtain the aggregated perception features; performing tensor concatenation processing on the perception features after multi-layer perceptron processing and the aggregated perception features to obtain the perception features after tensor concatenation; generating the hybrid perception features based on the perception features after tensor concatenation.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: performing multi-scale pooling processing on the hybrid perception features based on a preset multi-scale pooling layer to obtain the first point features; performing dispersion processing on the first point features based on a preset dispersion operator to obtain the dispersed voxel features; performing point feature extraction on the dispersed voxel features to obtain the second point features, and outputting the second point features as the point features after double feature iterative processing.

[0146] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: determining whether the second point features meet a preset condition; when the second point features meet the preset condition, outputting the second point features as the point features after double feature iterative processing; when the second point features do not meet the preset condition, performing multi-scale pooling processing, dispersion processing, and point feature extraction on the second point features in sequence to obtain the third point features, and outputting the second point features and the third point features as the point features after double feature iterative processing.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: performing tensor concatenation processing on the hybrid perception features and the point features obtained after double feature iterative processing to obtain the point features after tensor concatenation processing; processing the point features after tensor concatenation processing based on a preset multi-layer perceptron to obtain the fused point features.

[0148] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: performing dispersion processing on the hybrid perception features based on a preset dispersion operator to obtain the hybrid perception features after dispersion processing; performing aggregation processing on the hybrid perception features after dispersion processing based on a preset aggregation operator to obtain the hybrid perception features after aggregation processing; performing tensor concatenation processing on the hybrid perception features after aggregation processing and the hybrid perception features to obtain the hybrid perception features after tensor concatenation processing; processing the hybrid perception features after tensor concatenation processing based on a multi-layer perceptron to obtain the first point features.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: calculating a weighted product of the first point feature and a preset learnable weight matrix; performing a dispersion process on the weighted product based on a preset dispersion operator, and obtaining a result of the dispersion process as a dispersed voxel feature.

[0150] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: processing the dispersed voxel feature based on a preset sparse residual network to obtain a high-dimensional dispersed voxel feature; obtaining the second point feature based on the high-dimensional dispersed voxel feature and a preset geometric weight; the geometric weight is a product of the learnable weight matrix and the mixed perception feature.

[0151] The above storage medium first obtains the perception features of each point of the point cloud to be segmented at a preset grid scale, performs a multi-scale enhancement process on the perception features to obtain a mixed perception feature; and performs a dual feature iteration process on the mixed perception feature to obtain point features after the dual feature iteration process; finally, performs a fusion process on the mixed perception feature and the point features obtained after the dual feature iteration process to obtain fused point features. Through the dual feature iteration process on the mixed perception feature, the mutual iterative processing of the point features and the voxel features is realized, and the fusion of the point features after the iterative processing is performed, improving the accuracy of feature extraction.

[0152] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0153] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0154] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A feature extraction method based on point cloud segmentation, characterized in that, The method includes: Obtaining the perceptual features of each point of the point cloud to be segmented at a preset grid scale, processing the perceptual features based on a preset multi-layer perceptron, and obtaining the perceptual features after multi-layer perceptron processing; Performing dispersion processing on the perceptual features after multi-layer perceptron processing based on a preset dispersion operator to obtain the dispersed perceptual features; Performing aggregation processing on the dispersed perceptual features based on a preset aggregation operator to obtain the aggregated perceptual features; Performing tensor concatenation processing on the perceptual features after multi-layer perceptron processing and the aggregated perceptual features to obtain the perceptual features after tensor concatenation; Generating mixed perceptual features based on the perceptual features after tensor concatenation; Performing multi-scale pooling processing on the mixed perceptual features based on a preset multi-scale pooling layer to obtain the first point features; Performing dispersion processing on the first point features based on a preset dispersion operator to obtain dispersed voxel features; Performing point feature extraction on the dispersed voxel features to obtain second point features, and outputting the second point features as the point features after double feature iterative processing; Performing fusion processing on the mixed perceptual features and the point features after double feature iterative processing to obtain the fused point features.

2. The method according to claim 1, characterized in that, After performing the point feature extraction on the dispersed voxel features to obtain second point features and outputting the second point features as the point features after double feature iterative processing, it further includes: Judging whether the second point features meet a preset condition; When the second point features meet the preset condition, outputting the second point features as the point features after double feature iterative processing; When the second point features do not meet the preset condition, performing multi-scale pooling processing, dispersion processing, and point feature extraction on the second point features in sequence to obtain third point features, and outputting the second point features and the third point features as the point features after double feature iterative processing.

3. The method according to claim 1, characterized in that, The performing fusion processing on the mixed perceptual features and the point features after double feature iterative processing to obtain the fused point features includes: Performing tensor concatenation processing on the mixed perceptual features and the point features obtained after double feature iterative processing to obtain the point features after tensor concatenation processing; Processing the point features after tensor concatenation processing based on a preset multi-layer perceptron to obtain the fused point features.

4. The method according to claim 1, characterized in that, The performing multi-scale pooling processing on the mixed perceptual features based on a preset multi-scale pooling layer to obtain the first point features includes: Performing dispersion processing on the mixed perceptual features based on a preset dispersion operator to obtain the mixed perceptual features after dispersion processing; Performing aggregation processing on the mixed perceptual features after dispersion processing based on a preset aggregation operator to obtain the mixed perceptual features after aggregation processing; Performing tensor concatenation processing on the mixed perceptual features after aggregation processing and the mixed perceptual features to obtain the mixed perceptual features after tensor concatenation processing; Processing the mixed perceptual features after tensor concatenation processing based on a multi-layer perceptron to obtain the first point features.

5. The method according to claim 1, characterized in that, The performing dispersion processing on the first point features based on a preset dispersion operator to obtain the dispersed voxel features includes: Calculating the weighted product of the first point features and a preset learnable weight matrix; Perform dispersion processing on the weighted product based on a preset dispersion operator, and obtain the result of the dispersion processing as the dispersed voxel feature.

6. The method according to claim 1, characterized in that, Performing point feature extraction on the dispersed voxel feature to obtain the second point feature includes: Processing the dispersed voxel feature based on a preset sparse residual network to obtain a high-dimensional dispersed voxel feature; Obtaining the second point feature based on the high-dimensional dispersed voxel feature and a preset geometric weight; the geometric weight is the product of a preset learnable weight matrix and the mixed perception feature.

7. A feature extraction device based on point cloud segmentation, characterized in that, The device includes: An acquisition module, configured to acquire the perception features of each point of the point cloud to be segmented at a preset grid scale, process the perception features based on a preset multi-layer perceptron to obtain the perception features after multi-layer perceptron processing; perform dispersion processing on the perception features after multi-layer perceptron processing based on a preset dispersion operator to obtain the dispersed perception features; perform aggregation processing on the dispersed perception features based on a preset aggregation operator to obtain the aggregated perception features; perform tensor concatenation processing on the perception features after multi-layer perceptron processing and the aggregated perception features to obtain the perception features after tensor concatenation; generate a mixed perception feature based on the perception features after tensor concatenation; An iteration module, configured to perform multi-scale pooling processing on the mixed perception feature based on a preset multi-scale pooling layer to obtain a first point feature; perform dispersion processing on the first point feature based on a preset dispersion operator to obtain a dispersed voxel feature; perform point feature extraction on the dispersed voxel feature to obtain a second point feature, and output the second point feature as the point feature after double feature iteration processing; A fusion module, configured to perform fusion processing on the mixed perception feature and the point feature after double feature iteration processing to obtain the fused point feature.

8. The device according to claim 7, characterized in that, The iteration module is further configured to: Determine whether the second point feature meets a preset condition; When the second point feature meets the preset condition, output the second point feature as the point feature after double feature iteration processing; When the second point feature does not meet the preset condition, perform multi-scale pooling processing, dispersion processing, and point feature extraction on the second point feature in sequence to obtain a third point feature, and output the second point feature and the third point feature as the point feature after double feature iteration processing.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images