Three-dimensional point cloud data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a three-dimensional point cloud data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: projecting three-dimensional point cloud data to N views of different visual angles, and performing position coding on the projection data to obtain position coding information of the N different visual angles, N being a positive integer greater than 1; calling a pre-trained two-dimensional image processing model to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and performing local feature aggregation processing on the three-dimensional point cloud data to obtain aggregation features corresponding to the three-dimensional point cloud data; and performing aggregation processing on the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data. According to the embodiment of the invention, the computing resource consumption of the three-dimensional point cloud data and the three-dimensional data processing cost can be reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for processing three-dimensional point cloud data. Background Art

[0002] At present, the processing method of three-dimensional point cloud data is usually a three-dimensional model processing method. The existing 3D models are usually obtained by directly using 3D data and performing 3D model training.

[0003] When 3D data is used directly to train 3D models, the cost of data collection and data annotation for model training is higher, because 3D data collection may involve laser scanning, professional photogrammetry technology, or multi-view stereo reconstruction, etc. In addition, 3D data annotation is more complex and time-consuming than 2D image annotation, requiring professional knowledge and specialized tools. In addition, 3D data has a higher dimension and a larger data volume, which means that the training of 3D models often requires a large amount of computing resources. Summary of the invention

[0004] The technical problem to be solved by the embodiments of the present application is to provide a method, device, electronic device and storage medium for processing three-dimensional point cloud data, so as to reduce the computing resource consumption of three-dimensional point cloud data and reduce the cost of three-dimensional data processing.

[0005] In a first aspect, an embodiment of the present application provides a method for processing three-dimensional point cloud data, the method comprising:

[0006] Projecting the three-dimensional point cloud data onto N views at different viewing angles, and position encoding the projection data to obtain position encoding information of N different viewing angles, where N is a positive integer greater than 1;

[0007] Calling a pre-trained two-dimensional image processing model to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and performing local feature aggregation processing on the three-dimensional point cloud data to obtain aggregation features corresponding to the three-dimensional point cloud data;

[0008] Aggregation processing is performed on the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data.

[0009] Optionally, the two-dimensional image processing model includes: a feature interaction network and a local feature aggregation network,

[0010] The calling of the pre-trained two-dimensional image processing model to perform local feature interactive processing on the three-dimensional point cloud data based on the position coding information, and performing local feature aggregation processing on the three-dimensional point cloud data to obtain aggregate features corresponding to the three-dimensional point cloud data, includes:

[0011] Calling the feature interaction network to perform local feature interaction processing on the three-dimensional point cloud data based on the position encoding information to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data;

[0012] The local feature aggregation network is called to perform local feature interaction processing on the three-dimensional point cloud data based on the position encoding information to obtain aggregation features corresponding to the three-dimensional point cloud data.

[0013] Optionally, the feature interaction network includes: a multi-layer perceptron, a two-dimensional local grouping network layer, a self-attention mechanism layer and an average pooling layer.

[0014] The calling of the feature interaction network to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data includes:

[0015] Calling the multilayer perceptron to map the point cloud features corresponding to the three-dimensional point cloud data to a feature space of a preset dimension using a nonlinear activation function and the position encoding information, so as to obtain target point cloud features of the three-dimensional point cloud data under the preset dimension;

[0016] Calling the two-dimensional local grouping network layer to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features;

[0017] Calling the self-attention mechanism layer to process the point cloud features of the multiple local areas to obtain attention mechanism features;

[0018] The average pooling layer is called to perform average pooling processing on the attention mechanism features to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data.

[0019] Optionally, the local feature aggregation network includes: a local grouping network layer, a pooling layer and a propagation layer,

[0020] The calling of the local feature aggregation network to perform local feature interactive processing on the three-dimensional point cloud data based on the position encoding information to obtain aggregation features corresponding to the three-dimensional point cloud data includes:

[0021] Calling the local grouping network layer to perform feature grouping on the three-dimensional point cloud data according to the position coding information and data features of the three-dimensional point cloud data to obtain local grouping features of the three-dimensional point cloud data;

[0022] Calling the pooling layer to perform downsampling processing on the local grouping features to obtain downsampled features;

[0023] The propagation layer is called to process the down-sampled features to obtain aggregated features corresponding to the three-dimensional point cloud data.

[0024] Optionally, the two-dimensional image processing model includes: an adaptive integrated network layer,

[0025] The aggregating the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data includes:

[0026] The adaptive integrated network layer is called to perform aggregation processing on the three-dimensional semantic features and the aggregation features to obtain the three-dimensional information output result.

[0027] Optionally, the calling of the adaptive integrated network layer to perform aggregation processing on the three-dimensional semantic features and the aggregation features to obtain the three-dimensional information output result includes:

[0028] Calling the adaptive integrated network layer to calculate the cosine similarity between the three-dimensional semantic feature and the aggregated feature;

[0029] Adaptively perform feature aggregation on the three-dimensional semantic features and the aggregation features based on the cosine similarity to obtain the three-dimensional information output result.

[0030] Optionally, projecting the three-dimensional point cloud data onto views at N different viewing angles, and position encoding the projection data to obtain position encoding information at N different viewing angles, includes:

[0031] Acquire the three-dimensional point cloud data acquired by an acquisition device and device acquisition parameters of the acquisition device;

[0032] Projecting the three-dimensional point cloud data onto the N views at different perspectives based on the device acquisition parameters to obtain two-dimensional images at N different perspectives;

[0033] Based on a pre-selected encoding method, position encoding is performed on each pixel point in the N two-dimensional images of different viewing angles to obtain position encoding information corresponding to the N two-dimensional images of different viewing angles.

[0034] In a second aspect, an embodiment of the present application provides a three-dimensional point cloud data processing device, the device comprising:

[0035] A position coding acquisition module is used to project the three-dimensional point cloud data onto views of N different viewing angles, and perform position coding on the projection data to obtain position coding information of N different viewing angles, where N is a positive integer greater than 1;

[0036] A three-dimensional data processing module, configured to call a pre-trained two-dimensional image processing model to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and to perform local feature aggregation processing on the three-dimensional point cloud data to obtain aggregate features corresponding to the three-dimensional point cloud data;

[0037] The three-dimensional result acquisition module is used to aggregate the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data.

[0038] Optionally, the two-dimensional image processing model includes: a feature interaction network and a local feature aggregation network,

[0039] The three-dimensional data processing module includes:

[0040] A semantic feature acquisition unit, configured to call the feature interaction network to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, so as to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data;

[0041] The aggregation feature acquisition unit is used to call the local feature aggregation network to perform local feature interactive processing on the three-dimensional point cloud data based on the position coding information to obtain the aggregation features corresponding to the three-dimensional point cloud data.

[0042] Optionally, the feature interaction network includes: a multi-layer perceptron, a two-dimensional local grouping network layer, a self-attention mechanism layer and an average pooling layer.

[0043] The semantic feature acquisition unit comprises:

[0044] A first feature acquisition subunit is used to call the multilayer perceptron to map the point cloud features corresponding to the three-dimensional point cloud data to a feature space of a preset dimension by using a nonlinear activation function and the position encoding information, so as to obtain target point cloud features of the three-dimensional point cloud data under the preset dimension;

[0045] A second feature acquisition subunit is used to call the two-dimensional local grouping network layer to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features;

[0046] A third feature acquisition subunit is used to call the self-attention mechanism layer to process the point cloud features of the multiple local areas to obtain attention mechanism features;

[0047] The semantic feature acquisition subunit is used to call the average pooling layer to perform average pooling processing on the attention mechanism features to obtain the three-dimensional semantic features corresponding to the three-dimensional point cloud data.

[0048] Optionally, the local feature aggregation network includes: a local grouping network layer, a pooling layer and a propagation layer,

[0049] The aggregate feature acquisition unit includes:

[0050] A grouping feature acquisition subunit is used to call the local grouping network layer to perform feature grouping on the three-dimensional point cloud data according to the position coding information and the data features of the three-dimensional point cloud data, so as to obtain local grouping features of the three-dimensional point cloud data;

[0051] A sampling feature acquisition subunit, used for calling the pooling layer to perform downsampling processing on the local grouping features to obtain downsampling features;

[0052] The aggregate feature acquisition subunit is used to call the propagation layer to process the down-sampled features to obtain aggregate features corresponding to the three-dimensional point cloud data.

[0053] Optionally, the two-dimensional image processing model includes: an adaptive integrated network layer,

[0054] The three-dimensional result acquisition module includes:

[0055] The three-dimensional result acquisition unit is used to call the adaptive integrated network layer to perform aggregation processing on the three-dimensional semantic features and the aggregation features to obtain the three-dimensional information output result.

[0056] Optionally, the three-dimensional result acquisition unit includes:

[0057] A similarity calculation subunit, used for calling the adaptive integrated network layer to calculate the cosine similarity between the three-dimensional semantic feature and the aggregated feature;

[0058] The three-dimensional result acquisition subunit is used to perform adaptive feature aggregation on the three-dimensional semantic features and the aggregation features based on the cosine similarity to obtain the three-dimensional information output result.

[0059] Optionally, the position code acquisition module includes:

[0060] A three-dimensional point cloud data acquisition unit, used to acquire the three-dimensional point cloud data acquired by an acquisition device and device acquisition parameters of the acquisition device;

[0061] A two-dimensional image acquisition unit, configured to project the three-dimensional point cloud data onto the N views at different viewing angles based on the device acquisition parameters, to obtain two-dimensional images at N different viewing angles;

[0062] The position coding acquisition unit is used to perform position coding on each pixel point in the N two-dimensional images of different viewing angles based on a pre-selected coding method to obtain position coding information corresponding to the N two-dimensional images of different viewing angles.

[0063] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0064] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the above-mentioned methods for processing three-dimensional point cloud data when executing the program.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned methods for processing three-dimensional point cloud data.

[0066] Compared with the prior art, the embodiments of the present application have the following advantages:

[0067] In an embodiment of the present application, the three-dimensional point cloud data is projected onto views of N different perspectives, and the projection data is position-encoded to obtain position coding information of N different perspectives, wherein N is a positive integer greater than 1. A pre-trained two-dimensional image processing model is called to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and to perform local feature aggregation processing on the three-dimensional point cloud data to obtain aggregate features corresponding to the three-dimensional point cloud data. The three-dimensional semantic features and the aggregate features are aggregated to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data. In an embodiment of the present application, the three-dimensional point cloud data is projected onto a two-dimensional view to obtain a position code, and the position code and the three-dimensional point cloud data are input into a two-dimensional image processing model for processing. There is no need to use a three-dimensional model for processing, which can greatly reduce the consumption of computing resources in the model processing process. At the same time, there is no need to annotate the three-dimensional data, which can reduce the cost of three-dimensional data processing.

[0068] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A flowchart of a method for processing three-dimensional point cloud data provided in an embodiment of the present application;

[0070] Figure 2 A schematic diagram of a three-dimensional point cloud data projection process provided in an embodiment of the present application;

[0071] Figure 3 A schematic diagram of a model processing flow provided in an embodiment of the present application;

[0072] Figure 4 A schematic diagram of the structure of a three-dimensional point cloud data processing device provided in an embodiment of the present application;

[0073] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0075] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0076] Embodiment 1

[0077] Reference Figure 1 , which shows a flowchart of a method for processing three-dimensional point cloud data provided by an embodiment of the present application, such as Figure 1 As shown, the method for processing three-dimensional point cloud data may include: step 101, step 102 and step 103.

[0078] Step 101: Project the three-dimensional point cloud data onto views of N different perspectives, and position encode the projection data to obtain position encoding information of N different perspectives, where N is a positive integer greater than 1.

[0079] In this embodiment, 3D point cloud data refers to a set of vectors in a 3D coordinate system. These vectors record the scan data in the form of points, and each point contains 3D coordinates (X, Y, Z).

[0080] In some examples, the three-dimensional point cloud data may be point cloud data collected by a data collection device such as a laser radar. For example, a laser radar is usually installed on a car or other equipment, and the laser radar emits a laser beam and measures the time and intensity of the reflected light to obtain point cloud data on the surface of the object.

[0081] In some examples, the three-dimensional point cloud data may be obtained by photographing the same object with multiple cameras, for example, by simultaneously photographing the same object with two or more cameras and calculating the three-dimensional coordinates of each point on the surface of the object using the parallax principle.

[0082] It is understandable that the above examples are only examples listed for better understanding of the technical solutions of the embodiments of the present application. In the prior art, other methods for acquiring three-dimensional point cloud data may also be used, such as projecting structured light sources such as gratings or coded light strips onto the surface of an object, and using a camera to capture the reflected or deformed light strip information, thereby calculating the three-dimensional coordinates of the surface of the object, etc. Specifically, the method for acquiring three-dimensional point cloud data may be determined according to business needs, and this embodiment does not limit this.

[0083] After obtaining the three-dimensional point cloud data, the three-dimensional point cloud data can be projected onto views of N different perspectives, and the projection data can be position-encoded to obtain position coding information of N different perspectives, where N is a positive integer greater than 1.

[0084] In the specific implementation, the process of projecting 3D point cloud data to a 2D view image is essentially a perspective projection or an orthogonal projection. Perspective projection simulates the way the human eye or camera observes the 3D world, taking into account the effect of the distance of the object on the imaging; while orthogonal projection does not consider the distance of the object and directly maps the 3D point to the 2D plane. The acquisition process of the projection and position coding can be described in detail in combination with the following specific implementation method.

[0085] In a specific implementation of the present application, the above step 101 may include:

[0086] Sub-step S1: Acquire the three-dimensional point cloud data acquired by an acquisition device and the device acquisition parameters of the acquisition device.

[0087] In this embodiment, three-dimensional point cloud data of an object collected by a laser scanner, a depth camera, etc. can be obtained. Such data usually exists in the form of three-dimensional coordinates (x, y, z) and may contain other information such as color and intensity.

[0088] After obtaining the three-dimensional point cloud data collected by the device, the three-dimensional point cloud data can be preprocessed, such as performing denoising, filtering, dimensionality reduction and other preprocessing operations on the point cloud data to improve the quality and efficiency of subsequent processing.

[0089] At the same time, the device acquisition parameters of the acquisition device may be obtained. In this example, the device acquisition parameters may be, but are not limited to, the device position, orientation, focal length, and the like.

[0090] After obtaining the three-dimensional point cloud data collected by the collection device and the device collection parameters of the collection device, sub-step S2 is executed.

[0091] Sub-step S2: projecting the three-dimensional point cloud data onto the N views at different perspectives based on the device acquisition parameters to obtain two-dimensional images at N different perspectives.

[0092] After obtaining the three-dimensional point cloud data collected by the acquisition device and the device acquisition parameters of the acquisition device, the three-dimensional point cloud data can be projected onto views of N different perspectives based on the device acquisition parameters to obtain two-dimensional images of N different perspectives.

[0093] First, the projection plane and the viewing angle can be determined. Specifically, N different projection planes (the plane is a two-dimensional plane) and the corresponding viewing angles need to be determined. These viewing angles can be any angles rotating around the object, or they can be fixed key viewing angles (such as front, back, left, right, up, down, etc.).

[0094] Then, the projection matrix can be calculated. According to the internal and external parameters of each view (camera position, orientation, focal length, etc.), the corresponding projection matrix is ​​calculated. This matrix will be used to transform the points in the 3D point cloud data from the world coordinate system to the image coordinate system.

[0095] Finally, the projection matrix is ​​applied. Each point in the three-dimensional point cloud data is transformed by the projection matrix to obtain the coordinates on the two-dimensional image plane of the corresponding viewing angle.

[0096] Sub-step S3: performing position encoding on each pixel point in the N two-dimensional images of different viewing angles based on a pre-selected encoding method to obtain position encoding information corresponding to the N two-dimensional images of different viewing angles.

[0097] Furthermore, each pixel point in the two-dimensional images of N different viewing angles can be position-encoded based on the pre-selected encoding method to obtain the position encoding information corresponding to the two-dimensional images of N different viewing angles. For the two-dimensional image of each viewing angle, each pixel point in the image is position-encoded according to the selected encoding method. The encoding result will reflect the position information of the pixel point in the original three-dimensional space.

[0098] The projection process can be as follows Figure 2 As shown, the 3D point cloud data is projected onto n planes at n (n is a positive integer) angles (i.e., viewing angles), so that the appropriate position encoding of the corresponding points can be obtained, i.e. Figure 2 As shown in the figure, Pos embed1, Posembed2, ...Pos embedn, etc. Finally, 3D Pos embed, i.e., three-dimensional position embedding information, can be obtained. 3D PosEmbed can help the model understand and locate the target in three-dimensional space more accurately.

[0099] 3D to any modality virtual projection module such as Figure 2As shown: Since the space initially trained by the pre-trained model is in 2D space, if a new 3D position code is designed, there will be a semantic mismatch with the frozen weights of the pre-trained model during the training process. In order to solve this problem, in this embodiment, the 3D Token (that is, the information point cloud in the three-dimensional space is converted into a two-dimensional image that can be processed by the machine learning model) is virtually projected back to the source modality to obtain the appropriate position code of the corresponding point. When processing the data of the 2D modality, Transformer is used for processing. A simple projection method in PointCLIP that does not require learning parameters is used to virtually project each 3D coordinate onto N different views to obtain diverse position coordinate information in the 2D space, and multi-view images are not actually generated. After obtaining these 2D position codes, they can be averaged into a unified position code. This module injects the position information of the source modality into the 3D token (that is, the three-dimensional point cloud data and position code processed by the model), reducing the information loss caused by the real projection.

[0100] After projecting the three-dimensional point cloud data onto views at N different viewing angles and performing position encoding on the projection data to obtain position encoding information at N different viewing angles, step 102 is executed.

[0101] Step 102: Call a pre-trained two-dimensional image processing model to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and perform local feature aggregation processing on the three-dimensional point cloud data to obtain aggregation features corresponding to the three-dimensional point cloud data.

[0102] After projecting the three-dimensional point cloud data onto views of N different perspectives and position encoding the projection data to obtain position encoding information of N different perspectives, the pre-trained two-dimensional image processing model can be called to perform local feature interaction processing on the three-dimensional point cloud data based on the position encoding information, and perform local feature aggregation processing on the three-dimensional point cloud data to obtain the aggregated features corresponding to the three-dimensional point cloud data. In this example, the two-dimensional image processing model can include two branch networks, namely a feature interaction network and a local feature aggregation network. The processing process of the three-dimensional point cloud data can be described in detail in combination with the following specific implementation method.

[0103] In a specific implementation of the present application, the above step 102 may include:

[0104] Sub-step A1: calling the feature interaction network to perform local feature interaction processing on the three-dimensional point cloud data based on the position encoding information to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data.

[0105] In this embodiment, after obtaining the position coding information of the three-dimensional point cloud data at N different viewing angles, the feature interaction network can be called to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information to obtain the three-dimensional semantic features corresponding to the three-dimensional point cloud data. In this example, the feature interaction network includes: a multi-layer perceptron, a two-dimensional local grouping network layer, a self-attention mechanism layer, and an average pooling layer. The three-dimensional semantic features of the three-dimensional point cloud data can be obtained through the processing of these network layers. The implementation process can be described in detail in combination with the following specific implementation method.

[0106] In another specific implementation of the present application, the above sub-step A1 may include:

[0107] Sub-step B1: calling the multilayer perceptron to use a nonlinear activation function and the position encoding information to map the point cloud features corresponding to the three-dimensional point cloud data to a feature space of a preset dimension, so as to obtain the target point cloud features of the three-dimensional point cloud data under the preset dimension.

[0108] In this embodiment, the feature interaction network may include: a multi-layer perceptron, a two-dimensional local grouping network layer, a self-attention mechanism layer, and an average pooling layer. Figure 2 As shown, the feature interaction network is the branch network of the left half of the model, which includes: MLP (multi-layer perceptron), 2D Guided Local Grouping (two-dimensional local grouping network layer), SelfAttention (self-attention mechanism layer), and Average Pooling (average pooling layer).

[0109] After obtaining the position coding information of N different perspectives, the position coding information and the three-dimensional point cloud data can be input into the model together, and then the multi-layer perceptron can be called to use the nonlinear activation function and the position coding information to map the point cloud features corresponding to the three-dimensional point cloud data to the feature space of a preset dimension, so as to obtain the target point cloud features of the three-dimensional point cloud data under the preset dimension.

[0110] In the specific implementation, MLP (Multi-Layer Perceptron) is a feedforward artificial neural network model, also known as a multi-layer feedforward network or a deep feedforward network. MLP contains multiple fully connected layers and uses nonlinear activation functions (such as ReLU, etc.) between layers. MLP can perform feature extraction based on the position-encoded three-dimensional point cloud data through forward propagation. The specific extraction method is to map the point cloud features corresponding to the three-dimensional point cloud data to a feature space of a preset dimension (such as a high dimension), thereby extracting the target point cloud features.

[0111] After calling the multilayer perceptron to use the nonlinear activation function and position encoding information to map the point cloud features corresponding to the three-dimensional point cloud data to the feature space of the preset dimension, and obtaining the target point cloud features of the three-dimensional point cloud data under the preset dimension, sub-step B2 is executed.

[0112] Sub-step B2: calling the two-dimensional local grouping network layer to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features.

[0113] After calling a multilayer perceptron to use a nonlinear activation function and position coding information to map the point cloud features corresponding to the three-dimensional point cloud data to a feature space of a preset dimension, and obtaining the target point cloud features of the three-dimensional point cloud data under the preset dimension, the two-dimensional local grouping network layer can be called to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features. Specifically, 2D Guided Local Grouping can first use two-dimensional information (i.e., position coding information) to identify or locate key features or areas in the point cloud. Then, these two-dimensional information are used as a guide to perform more refined local area grouping in the three-dimensional point cloud, that is, based on the guidance of the position coding information, local area grouping is performed in the three-dimensional point cloud, and points with similar attributes and / or spatial relationships are divided into the same group, thereby obtaining point cloud features of multiple local areas.

[0114] After calling the two-dimensional local grouping network layer to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features, sub-step B3 is executed.

[0115] Sub-step B3: calling the self-attention mechanism layer to process the point cloud features of the multiple local areas to obtain attention mechanism features.

[0116] After calling the two-dimensional local grouping network layer to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features, the self-attention mechanism layer can be called to process the point cloud features of multiple local areas to obtain the attention mechanism features.

[0117] In the specific implementation, the Self attention mechanism allows the model to consider the information of all other elements when processing each element, thereby enhancing the model's ability to capture global information. Here, it can help the model better understand the relationship between the local area and its surrounding environment. In this example, for each feature point in each local area or the entire view, Self attention calculates the attention weights between that point and all other points. These weights reflect the correlation and importance between different points and are used to weighted sum the features of all points to generate a feature vector containing global context information.

[0118] After calling the self-attention mechanism layer to process the point cloud features of multiple local areas and obtaining the attention mechanism features, sub-step B4 is executed.

[0119] Sub-step B4: Call the average pooling layer to perform average pooling processing on the attention mechanism features to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data.

[0120] After calling the self-attention mechanism layer to process the point cloud features of multiple local areas to obtain the attention mechanism features, the average pooling layer can be called to perform average pooling processing on the attention mechanism features to obtain the three-dimensional semantic features corresponding to the three-dimensional point cloud data.

[0121] In the specific implementation, Average pooling is a downsampling operation used to reduce the size of the feature map and reduce the amount of computation. At the same time, it can also help the model extract features with translation invariance. In this example, Averagepooling is used to aggregate features in a local area or the entire view. It divides each local area or view into several sub-areas and calculates the average value of all feature points in each sub-area as the output of the sub-area. In this way, each local area or view will be converted into a fixed-size feature vector, which contains the average information of all feature points in the area or view, which is helpful for subsequent classification, regression or other tasks.

[0122] Sub-step A2: calling the local feature aggregation network to perform local feature interaction processing on the three-dimensional point cloud data based on the position encoding information to obtain aggregation features corresponding to the three-dimensional point cloud data.

[0123] After obtaining the position coding information of the 3D point cloud data at N different viewing angles, the local feature aggregation network can be called to perform local feature interactive processing on the 3D point cloud data based on the position coding information to obtain the aggregation features corresponding to the 3D point cloud data. Figure 2 As shown, the local feature aggregation network can be Figure 2The branch network in the right half of the figure includes: a local grouping network layer (Local Grouping), a pooling layer (Pooling) and a propagation layer (Propagation). The data processing process of the local feature aggregation network can be described in detail in combination with the following specific implementation methods.

[0124] In another specific implementation of the present application, the above sub-step A2 may include:

[0125] Sub-step C1: calling the local grouping network layer to perform feature grouping on the three-dimensional point cloud data according to the position coding information and the data features of the three-dimensional point cloud data, so as to obtain local grouping features of the three-dimensional point cloud data.

[0126] In this embodiment, the local grouping network layer can be called to perform feature grouping on the three-dimensional point cloud data according to the position coding information and the data features of the three-dimensional point cloud data to obtain the local grouping features of the three-dimensional point cloud data.

[0127] In the specific implementation, Local Grouping is a commonly used technology in 3D point cloud data processing. It aims to group the points in the point cloud data according to certain rules or features for subsequent feature extraction, classification, recognition and other tasks.

[0128] In this example, position coding information is extracted from the image data projected to M different views. These position coding information may include pixel coordinates, depth values, normal directions, etc., which together describe the positional relationship of points in three-dimensional space. The three-dimensional point cloud data itself is analyzed to extract its geometric features (such as curvature, normal direction, surface roughness, etc.), color features (if the point cloud data contains color information) and other possible features (such as intensity value, timestamp, etc.). These features describe the local and global properties of each point in the point cloud data and are an important basis for subsequent grouping. According to the position coding information and the data features of the three-dimensional point cloud data, appropriate algorithms (such as K-means clustering, DBSCAN density clustering, hierarchical clustering, etc.) are used to group the point cloud data. The purpose of grouping is to cluster points with similar features or located in similar spatial positions to form several local areas or groups. For each local area or group obtained by grouping, its local grouping features are further extracted. These features may include statistical information such as the average coordinates of the points in the area, the average normal direction, the average color value, and the point cloud density. Local grouping features are a high degree of generalization and abstraction of the original point cloud data, and they can better reflect the properties and characteristics of the local area.

[0129] After calling the local grouping network layer to perform feature grouping on the three-dimensional point cloud data according to the position coding information and the data features of the three-dimensional point cloud data to obtain the local grouping features of the three-dimensional point cloud data, sub-step C2 is executed.

[0130] Sub-step C2: calling the pooling layer to downsample the local grouping features to obtain downsampled features.

[0131] After calling the local grouping network layer to perform feature grouping on the three-dimensional point cloud data according to the position coding information and the data features of the three-dimensional point cloud data to obtain the local grouping features of the three-dimensional point cloud data, the pooling layer can be called to downsample the local grouping features to obtain the downsampled features.

[0132] In the specific implementation, the pooling layer, also known as the downsampling layer, plays an important role in the convolutional neural network (CNN). It mainly downsamples the feature map output by the convolutional layer to reduce the size of the feature map, reduce the amount of model calculation, and help prevent overfitting.

[0133] After calling the pooling layer to downsample the local grouping features to obtain downsampled features, sub-step C3 is executed.

[0134] Sub-step C3: calling the propagation layer to process the down-sampled features to obtain aggregated features corresponding to the three-dimensional point cloud data.

[0135] After calling the pooling layer to downsample the local grouping features to obtain the downsampled features, the propagation layer can be called to process the downsampled features to obtain the aggregation features corresponding to the three-dimensional point cloud data.

[0136] In a specific implementation, in the processing of three-dimensional point cloud data, propagation (propagation or upsampling) is usually used to process and transfer downsampled features (or high-level features) to a finer-grained or lower-level representation to obtain aggregated features corresponding to the three-dimensional point cloud data.

[0137] The processing process of propagation for downsampled features can be as follows: first, feature extraction is performed on the original three-dimensional point cloud data through a series of convolution, pooling and other operations, and a downsampled feature map or point cloud set is obtained. These downsampled features usually contain higher-level semantic information, but have lower spatial resolution. The function of the Feature Propagation module is to propagate or upsample these high-level downsampled features to the original point cloud data or a finer-grained feature representation. Specifically, a portion of points or features can be sampled from the high-level feature representation as the basis for propagation, and the high-level features can be interpolated according to the coordinate information of the sampling points to obtain the feature values ​​of the corresponding positions of the low-level or original point cloud data. Interpolation methods may include linear interpolation, nearest neighbor interpolation, trilinear interpolation, etc. Finally, the interpolated feature values ​​can be fused with the feature representation of the low-level or original point cloud data (such as splicing, weighted sum, etc.) to generate aggregated features containing high-level semantic information.

[0138] After obtaining the three-dimensional semantic features and aggregation features corresponding to the three-dimensional point cloud data, step 103 is executed.

[0139] Step 103: Aggregate the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data.

[0140] After obtaining the three-dimensional semantic features and aggregation features corresponding to the three-dimensional point cloud data, the three-dimensional semantic features and aggregation features can be aggregated to obtain the three-dimensional information output results corresponding to the three-dimensional point cloud data. In this embodiment, the two-dimensional image processing model can also include: an adaptive integrated network layer. When performing feature aggregation, the adaptive integrated network layer can be called to aggregate the three-dimensional semantic features and aggregation features to obtain the three-dimensional information output results. In this example, the three-dimensional information output result can be information such as the category of the object. Specifically, the three-dimensional information output result varies according to different application scenarios. Specifically, it can be determined according to the actual situation, and this embodiment does not limit this. The feature aggregation process can be described in detail in conjunction with the following specific implementation method.

[0141] In a specific implementation of the present application, calling the adaptive integrated network layer to aggregate the three-dimensional semantic features and the aggregate features to obtain the three-dimensional information output result includes:

[0142] Sub-step D1: calling the adaptive integrated network layer to calculate the cosine similarity between the three-dimensional semantic feature and the aggregated feature.

[0143] In this embodiment, the adaptive integrated network layer may be called first to calculate the cosine similarity between the three-dimensional semantic features and the aggregated features.

[0144] In this example, after obtaining the three-dimensional semantic features (i.e., 3D features) and the aggregate features (i.e., 2D features), cosine similarity can be used to calculate the similarity of each 2D feature relative to the 3D feature baseline. Cosine similarity measures the degree of similarity between two vectors (i.e., the two features output are two vector representations) in direction, regardless of their size. In this context, it can help us determine the importance of the features of each perspective or line for the overall 3D representation.

[0145] After calling the adaptive integration network layer to calculate the cosine similarity between the three-dimensional semantic features and the aggregated features, sub-step D2 is performed.

[0146] Sub-step D2: performing adaptive feature aggregation on the three-dimensional semantic features and the aggregation features based on the cosine similarity to obtain the three-dimensional information output result.

[0147] After calling the adaptive integration network layer to calculate the cosine similarity between the 3D semantic features and the aggregated features, the 3D semantic features and the aggregated features can be adaptively aggregated based on the cosine similarity to obtain the 3D information output result. Specifically, adaptive feature aggregation can be performed based on the relative weights calculated in the previous step. This means that features with higher weights will occupy a larger proportion in the final output. The aggregation operation can be a weighted summation, weighted average, or other forms of feature fusion, depending on the specific application scenarios and requirements of the 3D point cloud data. Ultimately, the output obtained by adaptive feature aggregation will be a more robust and comprehensive 3D representation that incorporates information from different perspectives or lines and is weighted according to the importance of this information. This output can be used for a variety of downstream tasks such as 3D reconstruction, object recognition, scene understanding, etc.

[0148] like Figure 2As shown in the figure, the adaptive ensemble network layer is Adaptive Ensemble. Adaptive Ensemble dynamically adjusts the weights or selections of each model in the ensemble according to a certain strategy to adapt to different data distributions and task requirements. The core idea of ​​this method is to achieve better prediction results by adaptively adjusting the ensemble members. In the specific implementation, the feature aggregation network can convert 3D tokens into feature baselines of adaptive ensemble, calculate the relative weights of different perspectives / lines by cosine similarity, and finally aggregate their features to obtain the final output. Specifically, the converted 3D tokens are used as a feature baseline, which can provide a unified reference framework to aggregate information from different perspectives or lines (or more generally, different dimensions or feature sets). In order to aggregate features from different perspectives / lines, the method uses cosine similarity to calculate the similarity between them and the feature baseline. Cosine similarity is a measure of the degree of similarity between two vectors in direction. For feature aggregation, it can reflect the correlation and importance between different feature sets. Using the weights calculated above, features from different perspectives / lines are weighted summed (or other forms of aggregation) to generate the final feature representation. This process utilizes complementary information from multiple perspectives and improves the comprehensiveness and robustness of feature representation.

[0149] The method for processing three-dimensional point cloud data provided in the embodiment of the present application is to project the three-dimensional point cloud data onto views of N different perspectives, and position encode the projection data to obtain position encoding information of N different perspectives, wherein N is a positive integer greater than 1. The pre-trained two-dimensional image processing model is called to perform local feature interactive processing on the three-dimensional point cloud data based on the position encoding information, and to perform local feature aggregation processing on the three-dimensional point cloud data to obtain aggregate features corresponding to the three-dimensional point cloud data. The three-dimensional semantic features and the aggregate features are aggregated to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data. The embodiment of the present application projects the three-dimensional point cloud data onto a two-dimensional view to obtain a position code, and inputs the position code and the three-dimensional point cloud data into a two-dimensional image processing model for processing. There is no need to use a three-dimensional model for processing, which can greatly reduce the consumption of computing resources in the model processing process. At the same time, there is no need to annotate the three-dimensional data, which can reduce the cost of three-dimensional data processing.

[0150] Embodiment 2

[0151] Reference Figure 4 , which shows a schematic diagram of the structure of a three-dimensional point cloud data processing device provided in an embodiment of the present application, such as Figure 4 As shown, the 3D point cloud data processing device 400 may include the following modules:

[0152] The position coding acquisition module 410 is used to project the three-dimensional point cloud data onto N views of different viewing angles, and perform position coding on the projection data to obtain position coding information of N different viewing angles, where N is a positive integer greater than 1;

[0153] The three-dimensional data processing module 420 is used to call a pre-trained two-dimensional image processing model to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and to perform local feature aggregation processing on the three-dimensional point cloud data to obtain aggregation features corresponding to the three-dimensional point cloud data;

[0154] The three-dimensional result acquisition module 430 is used to perform aggregation processing on the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data.

[0155] Optionally, the two-dimensional image processing model includes: a feature interaction network and a local feature aggregation network,

[0156] The three-dimensional data processing module includes:

[0157] A semantic feature acquisition unit, configured to call the feature interaction network to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, so as to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data;

[0158] The aggregation feature acquisition unit is used to call the local feature aggregation network to perform local feature interactive processing on the three-dimensional point cloud data based on the position coding information to obtain the aggregation features corresponding to the three-dimensional point cloud data.

[0159] Optionally, the feature interaction network includes: a multi-layer perceptron, a two-dimensional local grouping network layer, a self-attention mechanism layer and an average pooling layer.

[0160] The semantic feature acquisition unit comprises:

[0161] A first feature acquisition subunit is used to call the multilayer perceptron to map the point cloud features corresponding to the three-dimensional point cloud data to a feature space of a preset dimension by using a nonlinear activation function and the position encoding information, so as to obtain target point cloud features of the three-dimensional point cloud data under the preset dimension;

[0162] A second feature acquisition subunit is used to call the two-dimensional local grouping network layer to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features;

[0163] A third feature acquisition subunit is used to call the self-attention mechanism layer to process the point cloud features of the multiple local areas to obtain attention mechanism features;

[0164] The semantic feature acquisition subunit is used to call the average pooling layer to perform average pooling processing on the attention mechanism features to obtain the three-dimensional semantic features corresponding to the three-dimensional point cloud data.

[0165] Optionally, the local feature aggregation network includes: a local grouping network layer, a pooling layer and a propagation layer,

[0166] The aggregate feature acquisition unit includes:

[0167] A grouping feature acquisition subunit is used to call the local grouping network layer to perform feature grouping on the three-dimensional point cloud data according to the position coding information and the data features of the three-dimensional point cloud data, so as to obtain local grouping features of the three-dimensional point cloud data;

[0168] A sampling feature acquisition subunit, used for calling the pooling layer to perform downsampling processing on the local grouping features to obtain downsampling features;

[0169] The aggregate feature acquisition subunit is used to call the propagation layer to process the down-sampled features to obtain aggregate features corresponding to the three-dimensional point cloud data.

[0170] Optionally, the two-dimensional image processing model includes: an adaptive integrated network layer,

[0171] The three-dimensional result acquisition module includes:

[0172] The three-dimensional result acquisition unit is used to call the adaptive integrated network layer to perform aggregation processing on the three-dimensional semantic features and the aggregation features to obtain the three-dimensional information output result.

[0173] Optionally, the three-dimensional result acquisition unit includes:

[0174] A similarity calculation subunit, used for calling the adaptive integrated network layer to calculate the cosine similarity between the three-dimensional semantic feature and the aggregated feature;

[0175] The three-dimensional result acquisition subunit is used to perform adaptive feature aggregation on the three-dimensional semantic features and the aggregation features based on the cosine similarity to obtain the three-dimensional information output result.

[0176] Optionally, the position code acquisition module includes:

[0177] A three-dimensional point cloud data acquisition unit, used to acquire the three-dimensional point cloud data acquired by an acquisition device and device acquisition parameters of the acquisition device;

[0178] A two-dimensional image acquisition unit, configured to project the three-dimensional point cloud data onto the N views at different viewing angles based on the device acquisition parameters, to obtain two-dimensional images at N different viewing angles;

[0179] The position coding acquisition unit is used to perform position coding on each pixel point in the N two-dimensional images of different viewing angles based on a pre-selected coding method to obtain position coding information corresponding to the N two-dimensional images of different viewing angles.

[0180] The processing device of three-dimensional point cloud data provided in the embodiment of the present application projects the three-dimensional point cloud data onto views of N different perspectives, and position encodes the projection data to obtain position encoding information of N different perspectives, wherein N is a positive integer greater than 1. The pre-trained two-dimensional image processing model is called to perform local feature interactive processing on the three-dimensional point cloud data based on the position encoding information, and local feature aggregation processing on the three-dimensional point cloud data to obtain the aggregated features corresponding to the three-dimensional point cloud data. The three-dimensional semantic features and the aggregated features are aggregated to obtain the three-dimensional information output result corresponding to the three-dimensional point cloud data. The embodiment of the present application obtains the position encoding by projecting the three-dimensional point cloud data onto a two-dimensional view, and inputs the position encoding and the three-dimensional point cloud data into the two-dimensional image processing model for processing. There is no need to use a three-dimensional model for processing, which can greatly reduce the consumption of computing resources in the model processing process. At the same time, there is no need to annotate the three-dimensional data, which can reduce the cost of three-dimensional data processing.

[0181] Embodiment 3

[0182] An embodiment of the present application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned three-dimensional point cloud data processing method when executed by the processor.

[0183] Figure 5 FIG. 5 is a schematic diagram showing the structure of an electronic device 500 according to an embodiment of the present invention. Figure 5 As shown, the electronic device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 502 or computer program instructions loaded from a storage unit 508 into a random access memory (RAM) 503. In RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0184] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, a microphone, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0185] The various processes and processing described above may be performed by the processing unit 501. For example, the method of any of the above embodiments may be implemented as a computer software program, which is tangibly contained in a computer-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the CPU 501, one or more actions in the method described above may be performed.

[0186] Embodiment 4

[0187] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned three-dimensional point cloud data processing method is implemented.

[0188] The above is a detailed introduction to a method for processing three-dimensional point cloud data, a device for processing three-dimensional point cloud data, an electronic device and a computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for processing three-dimensional point cloud data, characterized in that: The method comprises: Projecting the three-dimensional point cloud data onto N views at different viewing angles, and position encoding the projection data to obtain position encoding information of N different viewing angles, where N is a positive integer greater than 1; Calling a pre-trained two-dimensional image processing model to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and performing local feature aggregation processing on the three-dimensional point cloud data to obtain aggregation features corresponding to the three-dimensional point cloud data; Aggregation processing is performed on the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data.

2. The method according to claim 1, characterized in that The two-dimensional image processing model includes: a feature interaction network and a local feature aggregation network, The calling of the pre-trained two-dimensional image processing model to perform local feature interactive processing on the three-dimensional point cloud data based on the position coding information, and performing local feature aggregation processing on the three-dimensional point cloud data to obtain aggregate features corresponding to the three-dimensional point cloud data, includes: Calling the feature interaction network to perform local feature interaction processing on the three-dimensional point cloud data based on the position encoding information to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data; The local feature aggregation network is called to perform local feature interaction processing on the three-dimensional point cloud data based on the position encoding information to obtain aggregation features corresponding to the three-dimensional point cloud data.

3. The method according to claim 2, characterized in that The feature interaction network includes: a multi-layer perceptron, a two-dimensional local grouping network layer, a self-attention mechanism layer and an average pooling layer. The calling of the feature interaction network to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data includes: Calling the multilayer perceptron to map the point cloud features corresponding to the three-dimensional point cloud data to a feature space of a preset dimension using a nonlinear activation function and the position encoding information, so as to obtain target point cloud features of the three-dimensional point cloud data under the preset dimension; Calling the two-dimensional local grouping network layer to divide the target point cloud features into point cloud features of multiple local areas according to the similarity and spatial relationship between the target point cloud features; Calling the self-attention mechanism layer to process the point cloud features of the multiple local areas to obtain attention mechanism features; The average pooling layer is called to perform average pooling processing on the attention mechanism features to obtain three-dimensional semantic features corresponding to the three-dimensional point cloud data.

4. The method according to claim 2, characterized in that: The local feature aggregation network includes: a local grouping network layer, a pooling layer and a propagation layer, The calling of the local feature aggregation network to perform local feature interactive processing on the three-dimensional point cloud data based on the position encoding information to obtain aggregation features corresponding to the three-dimensional point cloud data includes: Calling the local grouping network layer to perform feature grouping on the three-dimensional point cloud data according to the position coding information and data features of the three-dimensional point cloud data to obtain local grouping features of the three-dimensional point cloud data; Calling the pooling layer to perform downsampling processing on the local grouping features to obtain downsampled features; The propagation layer is called to process the down-sampled features to obtain aggregated features corresponding to the three-dimensional point cloud data.

5. The method according to claim 1, characterized in that The two-dimensional image processing model includes: an adaptive integrated network layer, The aggregating the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data includes: The adaptive integrated network layer is called to perform aggregation processing on the three-dimensional semantic features and the aggregation features to obtain the three-dimensional information output result.

6. The method according to claim 5, characterized in that The calling of the adaptive integrated network layer to aggregate the three-dimensional semantic features and the aggregate features to obtain the three-dimensional information output result includes: Calling the adaptive integrated network layer to calculate the cosine similarity between the three-dimensional semantic feature and the aggregated feature; Adaptively perform feature aggregation on the three-dimensional semantic features and the aggregation features based on the cosine similarity to obtain the three-dimensional information output result.

7. The method according to claim 1, characterized in that The projecting of the three-dimensional point cloud data onto N views at different viewing angles and position encoding of the projection data to obtain position encoding information at N different viewing angles includes: Acquire the three-dimensional point cloud data acquired by an acquisition device and device acquisition parameters of the acquisition device; Projecting the three-dimensional point cloud data onto the N views at different perspectives based on the device acquisition parameters to obtain two-dimensional images at N different perspectives; Based on a pre-selected encoding method, position encoding is performed on each pixel point in the N two-dimensional images of different viewing angles to obtain position encoding information corresponding to the N two-dimensional images of different viewing angles.

8. A three-dimensional point cloud data processing device, characterized in that: The device comprises: A position coding acquisition module is used to project the three-dimensional point cloud data onto views of N different viewing angles, and perform position coding on the projection data to obtain position coding information of N different viewing angles, where N is a positive integer greater than 1; A three-dimensional data processing module, configured to call a pre-trained two-dimensional image processing model to perform local feature interaction processing on the three-dimensional point cloud data based on the position coding information, and to perform local feature aggregation processing on the three-dimensional point cloud data to obtain aggregate features corresponding to the three-dimensional point cloud data; The three-dimensional result acquisition module is used to aggregate the three-dimensional semantic features and the aggregation features to obtain a three-dimensional information output result corresponding to the three-dimensional point cloud data.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for processing three-dimensional point cloud data according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for processing three-dimensional point cloud data according to any one of claims 1 to 7.

Images