Point cloud data global information calculation method, application, device, equipment and medium

Abstract

The invention relates to a point cloud data global information calculation method, an application, a device, equipment and a medium. The method is based on a global information calculation model, and comprises the following steps: acquiring point cloud data and a bird's-eye view and a distance view of the point cloud data, extracting a first feature of the bird's-eye view, a second feature of the distance view and a third feature of the point cloud data, fusing the first feature and the second feature to obtain a fused feature, and calculating the point cloud data according to the fused feature. The global attention is calculated based on the fusion feature and the third feature, and the global information of the point cloud data is calculated according to the global attention, so that the features of multiple views can be fused, the features of the points and the features of the multiple views are combined, the operand is reduced, the global information of the point cloud data can be obtained, and the accuracy of an identification result is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, application, device, equipment and medium for calculating global information of point cloud data. Background Art

[0002] Point cloud semantic segmentation is to label each point in the point cloud with a corresponding category. Existing practices are mainly divided into three directions. The first is to extract features directly based on the points in the point cloud without processing the points in the point cloud. This method will not lose the information in the original point cloud, but the amount of calculation is relatively large; the second is to convert the points in the point cloud into voxels, extract features based on voxels, and use the center point of the voxel to represent the information of the surrounding points, making the point cloud sparse. In the process of voxelization, the information in the original point cloud will inevitably be lost; the third is to use the method of multi-projection views to project the point cloud data onto different two-dimensional planes. This method is fast, but when projecting high-dimensional point cloud data onto low-dimensional planes, some information in the data will inevitably be lost.

[0003] In the related technologies, voxel-based and multi-projection view-based methods have low computational complexity, but they will lose the original information in the point cloud, resulting in poor results. In order to avoid excessive computational complexity, point-based methods generally add global average pooling to the last layer to obtain global information. This method simply adds up each position and calculates the average, which cannot fully obtain global information. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related technology, the present application provides a method, application, device, equipment and medium for calculating the global information of point cloud data, which can fuse the features of multiple views and combine the features of points and multiple views, thereby reducing the amount of calculation and obtaining the global information of point cloud data, thereby improving the accuracy of recognition results.

[0005] The first aspect of the present application provides a method for calculating global information of point cloud data, based on a global information calculation model, the method comprising:

[0006] Obtain point cloud data, as well as bird's-eye view and distance view of the point cloud data;

[0007] extracting a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data;

[0008] fusing the first feature and the second feature to obtain a fused feature;

[0009] Calculate global attention based on the fused feature and the third feature;

[0010] The global information of the point cloud data is calculated according to the global attention.

[0011] Preferably, the extracting the first feature of the bird's-eye view, the second feature of the distance view, and the third feature of the point cloud data comprises:

[0012] Using a first convolution module to convolve the bird's-eye view and the distance view respectively to obtain the first feature and the second feature;

[0013] The point cloud data is convolved using a second convolution module to obtain the third feature.

[0014] Preferably, the calculating the global attention based on the fused feature and the third feature comprises:

[0015] Dividing the points in the point cloud data into first-category points and second-category points with equal spacing;

[0016] respectively calculating a first attention of the first type of points and a second attention of the second type of points; the first attention and the second attention are calculated based on the fusion feature and the third feature;

[0017] The global attention is calculated based on the first attention and the second attention.

[0018] Preferably, calculating the global attention based on the first attention and the second attention includes:

[0019] Team up two adjacent points;

[0020] Calculate the fused attention of adjacent points in the team;

[0021] The global attention is calculated based on the fused attention.

[0022] Preferably, the fusing the first feature and the second feature to obtain a fused feature comprises:

[0023] Inputting the first feature into an attention module to obtain an attention weight;

[0024] The fused feature is calculated according to the attention weight, the first feature and the second feature.

[0025] Preferably, the global information calculation model is trained by the following method:

[0026] Acquire training samples, and input the training samples into a global information calculation model;

[0027] Sampling the fused features to obtain a predicted label;

[0028] Calculate the cross entropy loss based on the predicted label and the preset label;

[0029] The global information calculation model is adjusted based on the cross entropy loss, and the adjusted global information calculation model is re-trained using the training samples until the cross entropy loss meets a preset condition, thereby obtaining a trained global information calculation model.

[0030] A second aspect of the present application provides an application of a method for calculating global information of point cloud data, which is applied to point cloud data segmentation and recognition.

[0031] The third aspect of the present application provides a point cloud data global information calculation device, based on a global information calculation model, the device comprises:

[0032] An acquisition module, used for acquiring point cloud data, and a bird's-eye view and a distance view of the point cloud data;

[0033] an extraction module, configured to extract a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data;

[0034] A fusion module, used for fusing the first feature and the second feature to obtain a fusion feature;

[0035] An attention module, configured to calculate global attention based on the fused feature and the third feature;

[0036] A global information module is used to calculate the global information of the point cloud data according to the global attention.

[0037] A fourth aspect of the present application provides an electronic device, including:

[0038] Processor; and

[0039] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0040] A fifth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0041] The technical solution provided by the present application may include the following beneficial effects: An embodiment of the present application discloses a method for calculating global information of point cloud data, which is based on a global information calculation model and includes: obtaining point cloud data, as well as a bird's-eye view and a distance view of the point cloud data, extracting a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data, fusing the first feature with the second feature to obtain a fused feature, calculating global attention based on the fused feature and the third feature, calculating the global information of the point cloud data based on the global attention, being able to fuse features of multiple views and combine features of points and multiple views, thereby reducing the amount of computation, and being able to obtain global information of the point cloud data and improve the accuracy of the recognition results.

[0042] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0044] Figure 1 It is a flow chart of a method for calculating global information of point cloud data shown in an embodiment of the present application;

[0045] Figure 2 is another flow chart of the method for calculating global information of point cloud data shown in an embodiment of the present application;

[0046] Figure 3 is a schematic diagram of point cloud data division shown in an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of a global information calculation model shown in an embodiment of the present application;

[0048] Figure 5 It is a structural schematic diagram of a point cloud data global information calculation device shown in an embodiment of the present application;

[0049] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0051] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0052] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0053] Voxel-based and multi-projection view-based methods have low computational complexity, but they lose the original information in the point cloud, resulting in poor results. In order to avoid excessive computational complexity, point-based methods generally add global average pooling to the last layer to obtain global information. This method simply adds up the values ​​of each position and calculates the mean, which cannot fully obtain global information.

[0054] In response to the above problems, an embodiment of the present application provides a method for calculating global information of point cloud data, which can fuse the features of multiple views and combine the features of points and multiple views, thereby reducing the amount of calculation and obtaining global information of point cloud data.

[0055] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0056] Figure 1 It is a flow chart of a method for calculating global information of point cloud data shown in an embodiment of the present application.

[0057] See also Figure 1 , the method is based on a global information calculation model, the method comprises:

[0058] Step 101 , obtaining point cloud data, as well as a bird's-eye view and a distance view of the point cloud data.

[0059] Point cloud is a data set of points in a certain coordinate system. There are many fields involving point cloud, such as computer vision and robotics. The embodiments of this application can be applied to the autonomous driving perception part of intelligent vehicles, such as high-precision map construction, robotic vision and other scenarios. Point cloud data can be obtained by scanning the laser radar (Li DAR) of intelligent vehicles.

[0060] The Bird's Eye View (BEV) is a three-dimensional image drawn from a certain point in a high place, based on the principle of perspective, using a high viewpoint perspective method to look down on the undulations of the ground. Simply put, it is an image of a certain area seen from the air, which is more realistic than a plane map. In the field of autonomous driving and robotics, data obtained by sensors (such as Li DAR and cameras) are usually converted into BEV representations to better perform tasks such as object detection and path planning. BEV can simplify complex three-dimensional environments into two-dimensional images, which is especially important for efficient computing in real-time systems.

[0061] Range View (RV) is a representation method for LiDAR point cloud data, which converts 3D (3D) point cloud data into a structured 2D (2D) visual representation. This representation method has the following characteristics: compactness, Range View losslessly organizes 3D data into 2D representation, making the data more compact and efficient; information integrity, compared with other point cloud representation methods, Range View has an advantage in preserving the original information of the point cloud; fast neighborhood query, Range View can quickly perform neighborhood query, improving processing efficiency; multi-task potential, Range View has significantly higher consistency between segmentation and detection tasks, suitable for multi-task learning framework; structured representation, Range View converts LiDAR point cloud data into an image-like structured format, where each pixel represents the characteristics of the reflection point at an angle, such as distance, reflection intensity, etc.; applicability, Range View is applicable to a variety of 3D perception tasks, including object detection, semantic segmentation, and panoptic segmentation. Through specific network architectures and strategies, Range View can improve detection performance, especially in the detection of distant objects.

[0062] In this embodiment, the acquired point cloud data may be projected at multiple angles to obtain a bird's-eye view BEV and a distance view RV of the point cloud data.

[0063] Step 102 , extracting a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data.

[0064] Different convolution modules may be used to perform convolution on the bird's-eye view, the distance view, and the point cloud data, respectively, so as to extract the first feature of the bird's-eye view, the second feature of the distance view, and the third feature of the point cloud data.

[0065] Step 103: fuse the first feature and the second feature to obtain a fused feature.

[0066] After obtaining the first feature and the second feature, the first feature and the second feature may be fused to obtain a fused feature.

[0067] Step 104, calculating the global attention based on the fused feature and the third feature.

[0068] When calculating attention, in order to avoid taking up too much memory due to too large a matrix due to too many points in the point cloud, a sparse matrix can be used to calculate sparse attention, and then the global attention can be calculated based on the sparse attention.

[0069] Step 105, calculating the global information of the point cloud data based on the global attention.

[0070] Based on global attention, the global information of point cloud data can be calculated. The global information can reflect the category of each point in the point cloud data. After obtaining the global information, the point cloud data can be segmented more accurately. For example, in the process of autonomous driving, the roads, vehicles, pedestrians, etc. in the point cloud data can be segmented for navigation and obstacle avoidance; in the process of robot navigation, segmented point cloud data can help the robot understand its surrounding environment and perform path planning.

[0071] An embodiment of the present application discloses a method for calculating global information of point cloud data, which is based on a global information calculation model and includes: acquiring point cloud data, a bird's-eye view and a distance view of the point cloud data, extracting a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data, fusing the first feature with the second feature to obtain a fused feature, calculating global attention based on the fused feature and the third feature, calculating the global information of the point cloud data based on the global attention, being able to fuse features of multiple views and combine features of points and multiple views, thereby reducing the amount of computation, and being able to obtain the global information of the point cloud data and improving the accuracy of the recognition results.

[0072] Figure 2 It is another flow chart of the method for calculating global information of point cloud data shown in an embodiment of the present application.

[0073] See also Figure 2 , the method is based on a global information calculation model, the method comprises:

[0074] Step 201 : acquiring point cloud data, as well as a bird's-eye view and a distance view of the point cloud data.

[0075] In this embodiment, the acquired point cloud data may be projected at multiple angles to obtain a bird's-eye view and a distance view of the point cloud data.

[0076] Step 202: Use a first convolution module to convolve the bird's-eye view and the distance view respectively to obtain a first feature and a second feature.

[0077] Each convolutional layer (Constitutional layer) in a convolutional neural network is composed of several convolutional units, and the parameters of each convolutional unit are optimized through the back-propagation algorithm. The purpose of the convolution operation is to extract different features of the input. The first convolutional layer may only extract some low-level features such as edges, lines, and corners. More layers of the network can iteratively extract more complex features from low-level features.

[0078] The first convolution module may be composed of multiple layers of 3×3 convolution layers, and the first convolution module is used to convolve the bird's-eye view and the distance view respectively, so as to obtain a first feature B of the bird's-eye view and a second feature R of the distance view.

[0079] Step 203: Use the second convolution module to convolve the point cloud data to obtain the third feature.

[0080] The second convolution module can be composed of multiple layers of 1×1 convolution layers. The second convolution module is used to perform a convolution operation on the point cloud data to obtain the third feature Q.

[0081] Step 204: input the first feature into the attention module to obtain the attention weight.

[0082] If we understand the Attention mechanism superficially, the core logic is to shift from paying attention to the whole to paying attention to the key points, which allows the model to focus on the most important part of the input data when processing information.

[0083] In the embodiment of the present application, the first feature B can also be input into the attention module to obtain the corresponding attention weight W.

[0084] Step 205, calculating a fusion feature according to the attention weight, the first feature and the second feature.

[0085] After obtaining the attention weight W, the attention weight W can be multiplied by the first feature B, and then the multiplication result can be fused with the second feature R to obtain the fused feature KEY.

[0086] Step 206: divide the points in the point cloud data into first-category points and second-category points with equal spacing.

[0087] like Figure 3 The figure shows the schematic diagram of point cloud data division, where "1" represents the first type of points and "2" represents the second type of points. When calculating attention, the point cloud data is evenly spaced to obtain evenly spaced first type and second type points.

[0088] Step 207, respectively calculate the first attention of the first type of points and the second attention of the second type of points; the first attention and the second attention are calculated based on the fusion feature and the third feature.

[0089] When calculating the attention, the third feature is used as the value VALUE and the query QUERY, and the first attention of each first-category point and the second attention of each second-category point are calculated separately based on the fused features KEY, VALUE and QUERY.

[0090] Step 208, calculating the global attention based on the first attention and the second attention.

[0091] According to the first attention of the first type of points and the second attention of the second type of points, the global attention of the point cloud data can be calculated.

[0092] In an optional embodiment of the present application, step 208 includes:

[0093] Sub-step 2081, group two adjacent points.

[0094] When forming a team, you can group two adjacent points into teams in pairs according to the following method: "1, 2", "1, 2"...

[0095] Sub-step 2082, calculate the fusion attention of adjacent points in the team.

[0096] After forming a team, the fused attention of the two points in the team can be calculated based on the first attention of the first type of points and the second attention of the second type of points.

[0097] Sub-step 2083, calculate global attention based on fused attention.

[0098] According to the fused attention after teaming, the global attention of the point cloud data can be calculated. When calculating the attention, in order to avoid the matrix being too large due to too many point cloud points and occupying too much memory, the sparse matrix is ​​used to calculate the sparse attention, and then the global attention is calculated based on the sparse attention.

[0099] Step 209, calculating the global information of the point cloud data based on the global attention.

[0100] Based on global attention, the global information of point cloud data can be calculated. The global information can reflect the category of each point in the point cloud data. After obtaining the global information, the point cloud data can be segmented more accurately. For example, in the process of autonomous driving, the roads, vehicles, pedestrians, etc. in the point cloud data can be segmented for navigation and obstacle avoidance; in the process of robot navigation, segmented point cloud data can help the robot understand its surrounding environment and perform path planning.

[0101] In an optional embodiment of the present application, the global information calculation model is trained by the following method:

[0102] Step S1, obtaining training samples, and inputting the training samples into the global information calculation model.

[0103] The training samples may be composed of a number of point cloud data, and the acquired training samples are input into the initial global information calculation model.

[0104] Step S2: Sample the fused features to obtain the predicted labels.

[0105] After the first feature of the bird's-eye view and the second feature of the range view are fused to generate a fused feature, the fused feature can be sampled to obtain a predicted label.

[0106] Step S3, calculating the cross entropy loss based on the predicted label and the preset label.

[0107] The preset label can be a real label marked by humans. When sampling the fused features, bilinear interpolation or deconvolution can be used to obtain a result with the same shape as the preset label. The cross entropy loss loss can be calculated based on the predicted label and the preset label.

[0108] Step S4, adjusting the global information calculation model based on the cross entropy loss, re-training the adjusted global information calculation model using the training samples until the cross entropy loss meets the preset conditions, thereby obtaining a trained global information calculation model.

[0109] The global information calculation model can be adjusted according to the calculated cross entropy loss, and then the adjusted global information calculation model is re-adopted until the cross entropy loss meets the preset condition. The preset condition may be that the cross entropy loss is not greater than a preset threshold, thereby obtaining a trained global information calculation model.

[0110] Reference Figure 4 , is a schematic diagram of the global information calculation model shown in the embodiment of the present application (the "×" in the figure indicates matrix multiplication, and "+" represents matrix addition). After acquiring the point cloud data, the point cloud data is projected from multiple angles to obtain a bird's-eye view BEV and a distance view RV of the point cloud data. The first feature B of BEV and the second feature R of RV are extracted, and the first feature B is input into the attention module to obtain the corresponding attention weight W. The attention weight W is multiplied by the first feature B, and then the multiplied result is fused with the second feature R to obtain the fused feature KEY. The fused feature KEY is sampled to obtain a predicted label, and the cross entropy loss loss is calculated based on the predicted label and the preset label. The point cloud data is convolved to obtain the third feature Q, and the third feature Q is used as VALUE and QUERY. Attention is calculated based on the fused features KEY, VALUE and QUERY. The global information of the point cloud data can be calculated based on the global attention.

[0111] The embodiment of the present application discloses a method for calculating global information of point cloud data, which is based on a global information calculation model and includes: acquiring point cloud data, and a bird's-eye view and a distance view of the point cloud data, using a first convolution module to convolve the bird's-eye view and the distance view respectively to obtain a first feature and a second feature, using a second convolution module to convolve the point cloud data to obtain a third feature, inputting the first feature into an attention module to obtain an attention weight, calculating a fusion feature according to the attention weight, the first feature and the second feature, dividing the points in the point cloud data into a first type of points and a second type of points at equal intervals, and calculating a first attention of the first type of points and a second attention of the second type of points respectively; the first attention and the second attention are calculated based on the fusion feature and the third feature, calculating a global attention based on the first attention and the second attention, calculating the global information of the point cloud data according to the global attention, being able to fuse the features of multiple views, and combining the features of points and multiple views, taking points at equal intervals and using a sparse matrix to calculate the attention, thereby reducing the amount of calculation, and obtaining the global information of the point cloud data, thereby improving the accuracy of the recognition result.

[0112] The method for calculating the global information of point cloud data provided in this application is applied to point cloud data segmentation and recognition. Global information can reflect the category of each point in the point cloud data. After obtaining the global information, the point cloud data can be segmented more accurately. For example, in the process of autonomous driving, roads, vehicles, pedestrians, etc. in the point cloud data can be segmented for navigation and obstacle avoidance; in the process of robot navigation, segmenting point cloud data can help the robot understand its surrounding environment and perform path planning.

[0113] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a point cloud data global information calculation device, an electronic device and corresponding embodiments.

[0114] Figure 5 It is a structural schematic diagram of a point cloud data global information calculation device shown in an embodiment of the present application.

[0115] See also Figure 5 , the device is based on a global information calculation model, and the device includes:

[0116] The acquisition module 501 is used to acquire point cloud data, as well as a bird's-eye view and a distance view of the point cloud data.

[0117] In this embodiment, the acquired point cloud data may be projected at multiple angles to obtain a bird's-eye view BEV and a distance view RV of the point cloud data.

[0118] The extraction module 502 is used to extract the first feature of the bird's-eye view, the second feature of the distance view, and the third feature of the point cloud data.

[0119] Different convolution modules may be used to perform convolution on the bird's-eye view, the distance view, and the point cloud data, respectively, so as to extract the first feature of the bird's-eye view, the second feature of the distance view, and the third feature of the point cloud data.

[0120] The fusion module 503 is used to fuse the first feature and the second feature to obtain a fused feature.

[0121] After obtaining the first feature and the second feature, the first feature and the second feature may be fused to obtain a fused feature.

[0122] The attention module 504 is used to calculate the global attention based on the fused feature and the third feature.

[0123] When calculating attention, in order to avoid taking up too much memory due to too large a matrix due to too many points in the point cloud, a sparse matrix can be used to calculate sparse attention, and then the global attention can be calculated based on the sparse attention.

[0124] The global information module 505 is used to calculate the global information of the point cloud data based on the global attention.

[0125] Based on global attention, the global information of point cloud data can be calculated. The global information can reflect the category of each point in the point cloud data. After obtaining the global information, the point cloud data can be segmented more accurately. For example, in the process of autonomous driving, the roads, vehicles, pedestrians, etc. in the point cloud data can be segmented for navigation and obstacle avoidance; in the process of robot navigation, segmented point cloud data can help the robot understand its surrounding environment and perform path planning.

[0126] In an optional embodiment of the present application, the extraction module 502 includes:

[0127] A first convolution module, used to perform convolution on the bird's-eye view and the distance view respectively to obtain a first feature and a second feature;

[0128] The second convolution module is used to convolve the point cloud data to obtain the third feature.

[0129] In an optional embodiment of the present application, the attention module 504 includes:

[0130] A division submodule, used for dividing the points in the point cloud data into first-category points and second-category points with equal spacing;

[0131] A calculation submodule, used to calculate the first attention of the first type of points and the second attention of the second type of points respectively; the first attention and the second attention are calculated based on the fusion feature and the third feature;

[0132] The attention submodule is used to calculate the global attention based on the first attention and the second attention.

[0133] In an optional embodiment of the present application, the attention submodule includes:

[0134] A teaming unit is used to team two adjacent points;

[0135] A calculation unit, used to calculate the fusion attention of adjacent points in the team;

[0136] Attention unit, used to calculate global attention based on fused attention.

[0137] In an optional embodiment of the present application, the fusion module 503 includes:

[0138] A weight submodule, used for inputting the first feature into the attention module to obtain an attention weight;

[0139] The fusion submodule is used to calculate the fusion feature according to the attention weight, the first feature and the second feature.

[0140] In an optional embodiment of the present application, the device further includes:

[0141] An input module, used to obtain training samples and input the training samples into the global information calculation model;

[0142] The sampling module is used to sample the fusion features to obtain the predicted labels;

[0143] The loss module is used to calculate the cross entropy loss based on the predicted label and the preset label;

[0144] The adjustment module is used to adjust the global information calculation model based on the cross entropy loss, and re-adopt the training samples to train the adjusted global information calculation model until the cross entropy loss meets the preset conditions, thereby obtaining the trained global information calculation model.

[0145] An embodiment of the present application discloses a point cloud data global information calculation device, which is based on a global information calculation model and includes: acquiring point cloud data, a bird's-eye view and a distance view of the point cloud data, extracting a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data, fusing the first feature with the second feature to obtain a fused feature, calculating global attention based on the fused feature and the third feature, calculating the global information of the point cloud data based on the global attention, being able to fuse features of multiple views and combine features of points and multiple views, thereby reducing the amount of calculation, and being able to obtain the global information of the point cloud data and improve the accuracy of the recognition results.

[0146] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0147] Figure 6It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0148] See also Figure 6 , the electronic device 600 includes a memory 610 and a processor 620 .

[0149] The processor 620 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0150] The memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 620 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 610 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 610 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., CD-ROM, double-layer CD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0151] The memory 610 stores executable codes, and when the executable codes are processed by the processor 620 , the processor 620 can execute part or all of the methods described above.

[0152] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0153] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0154] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for calculating global information of point cloud data, characterized in that: Based on the global information calculation model, the method includes: Obtain point cloud data, as well as bird's-eye view and distance view of the point cloud data; extracting a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data; fusing the first feature and the second feature to obtain a fused feature; Calculate global attention based on the fused feature and the third feature; The global information of the point cloud data is calculated according to the global attention.

2. The method according to claim 1, characterized in that The extracting the first feature of the bird's-eye view, the second feature of the distance view, and the third feature of the point cloud data includes: Using a first convolution module to convolve the bird's-eye view and the distance view respectively to obtain the first feature and the second feature; The point cloud data is convolved using a second convolution module to obtain the third feature.

3. The method according to claim 1, characterized in that The calculating of the global attention based on the fused feature and the third feature comprises: Dividing the points in the point cloud data into first-category points and second-category points with equal spacing; respectively calculating a first attention of the first type of points and a second attention of the second type of points; the first attention and the second attention are calculated based on the fusion feature and the third feature; The global attention is calculated based on the first attention and the second attention.

4. The method according to claim 3, characterized in that The calculating the global attention based on the first attention and the second attention comprises: Team up two adjacent points; Calculate the fused attention of adjacent points in the team; The global attention is calculated based on the fused attention.

5. The method according to claim 1, characterized in that The fusing the first feature and the second feature to obtain a fused feature includes: Inputting the first feature into an attention module to obtain an attention weight; The fused feature is calculated according to the attention weight, the first feature and the second feature.

6. The method according to claim 1, characterized in that The global information calculation model is trained by the following method: Acquire training samples, and input the training samples into a global information calculation model; Sampling the fused features to obtain a predicted label; Calculate the cross entropy loss based on the predicted label and the preset label; The global information calculation model is adjusted based on the cross entropy loss, and the adjusted global information calculation model is re-trained using the training samples until the cross entropy loss meets a preset condition, thereby obtaining a trained global information calculation model.

7. The method for calculating global information of point cloud data as described in any one of claims 1 to 6 is applied to point cloud data segmentation and recognition.

8. A device for calculating global information of point cloud data, characterized in that: Based on the global information calculation model, the device includes: An acquisition module, used for acquiring point cloud data, and a bird's-eye view and a distance view of the point cloud data; an extraction module, configured to extract a first feature of the bird's-eye view, a second feature of the distance view, and a third feature of the point cloud data; A fusion module, used for fusing the first feature and the second feature to obtain a fusion feature; An attention module, configured to calculate global attention based on the fused feature and the third feature; A global information module is used to calculate the global information of the point cloud data according to the global attention.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method as claimed in any one of claims 1-6 and 7.

10. A computer-readable storage medium having executable codes stored thereon, wherein when the executable codes are executed by a processor of an electronic device, the processor is caused to execute the method according to any one of claims 1 to 6 and 7.

Images