Point cloud target detection method and device, electronic equipment and storage medium
By acquiring and splicing multiple bird's-eye feature maps and combining feature processing technology, the problem of poor three-dimensional point cloud target detection effect from a single perspective is solved, and the accuracy of three-dimensional object detection in the autonomous driving environment perception is improved.
Patent Information
- Application Number
- CN202410814000.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-11-26
AI Technical Summary
The existing three-dimensional point cloud object detection method is based on a single-view point cloud view, which loses some features of the three-dimensional target, resulting in poor detection results and cannot meet the needs of autonomous driving.
By obtaining multiple bird's-eye feature maps of three-dimensional point cloud data, and performing dimension alignment and stitching, a richer bird's-eye feature map is generated, combining bilinear interpolation and maximum pooling, multi-grid pooling and cross-grid attention, Improve the accuracy of three-dimensional object detection.
The three-dimensional object detection effect of three-dimensional point cloud data is enhanced, and more accurate target information is provided to meet the requirements of autonomous driving environment perception.
Smart Images

Figure CN119027907A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of target detection for autonomous driving, and in particular to a point cloud target detection method, device, electronic device and storage medium. Background Art
[0002] Detecting three-dimensional targets through radar's three-dimensional (3D) point cloud data and identifying the vehicle's surroundings is an important way to achieve environmental perception in autonomous driving. Related technologies use three-dimensional target detection based on three-dimensional point cloud data, and use a point cloud detection model to perform three-dimensional target detection using a bird's-eye view or front view of three-dimensional point cloud data. This is three-dimensional target detection based on a point cloud view from a single perspective. The point cloud view from a single perspective loses some of the features of the three-dimensional target, resulting in poor three-dimensional target detection by the point cloud detection model, which cannot meet the requirements for three-dimensional target detection in autonomous driving. Summary of the invention
[0003] In order to solve or partially solve the problems existing in the related art, the present application provides a point cloud target detection method, device, electronic device and storage medium, which can accurately detect three-dimensional targets in three-dimensional point cloud data and improve the three-dimensional target detection effect based on three-dimensional point cloud data.
[0004] The first aspect of the present application provides a point cloud target detection method, the method comprising:
[0005] Respectively obtain a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data;
[0006] Obtaining a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map;
[0007] A three-dimensional target of the three-dimensional point cloud data is detected according to the bird's-eye view feature map of the three-dimensional point cloud data.
[0008] Preferably, the step of respectively acquiring the first bird's-eye view feature map, the second bird's-eye view feature map, and the third bird's-eye view feature map of the three-dimensional point cloud data includes:
[0009] Obtaining a transformed equivalent pixel feature map of the three-dimensional point cloud data;
[0010] According to the transformed isomorphic pixel feature map, a first bird's-eye view feature map, a second bird's-eye view feature map and a third bird's-eye view feature map of the three-dimensional point cloud data are respectively obtained.
[0011] Preferably, the step of respectively acquiring a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data according to the transformed isomorphic pixel feature map comprises:
[0012] According to the transformed equivalent pixel feature map, compressing the transformed equivalent pixel feature map along the height dimension to obtain the first bird's-eye view feature map;
[0013] According to the transformed equivalent pixel feature map, compressing the transformed equivalent pixel feature map along the width dimension to obtain the second bird's-eye view feature map;
[0014] According to the transformed equivalent pixel feature map, the transformed equivalent pixel feature map is compressed along the depth dimension to obtain the third bird's-eye view feature map.
[0015] Preferably, obtaining the bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map comprises:
[0016] According to the dimension information of the first bird's-eye view feature map, aligning the dimensions of the second bird's-eye view feature map and the third bird's-eye view feature map with the dimensions of the first bird's-eye view feature map respectively;
[0017] The second bird's-eye view feature map and the third bird's-eye view feature map whose dimensions are aligned with the first bird's-eye view feature map are spliced with the first bird's-eye view feature map to obtain the bird's-eye view feature map of the three-dimensional point cloud data.
[0018] Preferably, obtaining the bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map comprises:
[0019] Obtaining the number of channels of the first bird's-eye view feature map according to the dimension information of the first bird's-eye view feature map;
[0020] According to the number of channels of the second bird's-eye view feature map, converting the second bird's-eye view feature map into a second to-be-joined bird's-eye view feature map having the same number of channels as the first bird's-eye view feature map;
[0021] According to the number of channels of the third bird's-eye view feature map, converting the third bird's-eye view feature map into a third to-be-joined bird's-eye view feature map having the same number of channels as the first bird's-eye view feature map;
[0022] The first bird's-eye view feature map, the second bird's-eye view feature map to be spliced, and the third bird's-eye view feature map to be spliced are spliced to obtain a bird's-eye view feature map of the three-dimensional point cloud data.
[0023] Preferably, detecting the three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data comprises:
[0024] According to the bird's-eye view feature map of the three-dimensional point cloud data, bilinear interpolation and maximum pooling are used to obtain a lightweight feature map;
[0025] According to the lightweight feature map, a plurality of three-dimensional object detection frame suggestions for detecting the three-dimensional object are obtained;
[0026] The three-dimensional object of the three-dimensional point cloud data is detected according to the three-dimensional object suggestion detection boxes of the multiple three-dimensional objects.
[0027] Preferably, the detecting the three-dimensional object of the three-dimensional point cloud data according to the three-dimensional object suggestion detection frames for the plurality of three-dimensional object detections comprises:
[0028] According to the plurality of three-dimensional object detection proposal detection boxes for detecting the three-dimensional objects and the transformed isomorphic pixel feature map, multi-grid pooling and / or cross-grid attention are used to obtain a feature map for three-dimensional object detection;
[0029] The three-dimensional object of the three-dimensional point cloud data is detected according to the feature map of the three-dimensional object detection.
[0030] A second aspect of the present application provides a point cloud target detection device, the device comprising:
[0031] A first acquisition module is used to respectively acquire a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data;
[0032] A bird's-eye view acquisition module, configured to obtain a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map acquired by the first acquisition module;
[0033] The target detection module is used to detect the three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data obtained by the bird's-eye view acquisition module.
[0034] A third aspect of the present application provides an electronic device, including:
[0035] Processor; and
[0036] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.
[0037] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.
[0038] The technical solution provided by this application may have the following beneficial effects:
[0039] The technical solution of the present application obtains a bird's-eye feature map of three-dimensional point cloud data based on a first bird's-eye feature map, a second bird's-eye feature map and a third bird's-eye feature map of the three-dimensional point cloud data; detects three-dimensional targets of the three-dimensional point cloud data based on the bird's-eye feature map of the three-dimensional point cloud data; the three-dimensional targets of the bird's-eye feature map contain richer information, can accurately detect the three-dimensional targets of the three-dimensional point cloud data, and improve the three-dimensional target detection effect based on the three-dimensional point cloud data.
[0040] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.
[0042] Figure 1 It is a flow chart of a point cloud target detection method shown in an embodiment of the present application;
[0043] Figure 2 is another flow chart of the point cloud target detection method shown in an embodiment of the present application;
[0044] Figure 3 is a schematic diagram of the structure of a point cloud target detection device shown in an embodiment of the present application;
[0045] Figure 4 is another structural schematic diagram of a point cloud target detection device shown in an embodiment of the present application;
[0046] Figure 5 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION
[0047] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.
[0048] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0049] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0050] Detecting three-dimensional targets through radar's three-dimensional (3D) point cloud data and identifying the vehicle's surroundings is an important way to achieve environmental perception in autonomous driving. Related technologies use three-dimensional target detection based on three-dimensional point cloud data, and use a point cloud detection model to perform three-dimensional target detection using a bird's-eye view or front view of three-dimensional point cloud data. This is three-dimensional target detection based on a point cloud view from a single perspective. The point cloud view from a single perspective loses some of the features of the three-dimensional target, resulting in poor three-dimensional target detection by the point cloud detection model, which cannot meet the requirements for three-dimensional target detection in autonomous driving.
[0051] For example, the related art detects three-dimensional objects based on three-dimensional point cloud data using TED (Transformation-Equivariant 3D Detector) to obtain the BEV (Bird's Eye View) feature map of the three-dimensional point cloud data; and detects three-dimensional objects in the three-dimensional point cloud data based on the BEV feature map. The related art detects three-dimensional objects based on three-dimensional point cloud data using TED. The BEV feature map of a single perspective loses some features of the three-dimensional object, and the BEV feature map does not contain enough information about the three-dimensional object, resulting in poor results for three-dimensional object detection based on TED, which cannot meet the requirements for three-dimensional object detection in autonomous driving.
[0052] In response to the above problems, the embodiments of the present application provide a point cloud target detection method, device, electronic device and storage medium, which can accurately detect three-dimensional targets in three-dimensional point cloud data and improve the three-dimensional target detection effect based on three-dimensional point cloud data.
[0053] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0054] Figure 1 It is a flow chart of the point cloud target detection method shown in an embodiment of the present application.
[0055] See also Figure 1 , a point cloud target detection method, comprising:
[0056] Step 101 , respectively obtain a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of three-dimensional point cloud data.
[0057] In one implementation, three-dimensional point cloud data can be input into a three-dimensional point cloud target detection model, and a transformed equivalent pixel feature map of the three-dimensional point cloud data can be obtained through the backbone network backbone of the three-dimensional point cloud target detection model; according to the transformed equivalent pixel feature map of the three-dimensional point cloud data, different transformations and convolutions are performed on the transformed equivalent pixel feature map of the three-dimensional point cloud data to obtain a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data.
[0058] Step 102: Obtain a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map of the three-dimensional point cloud data.
[0059] In one implementation, a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data can be aligned through a three-dimensional point cloud target detection model; the aligned first bird's-eye view feature map, the second bird's-eye view feature map, and the third bird's-eye view feature map are fused to obtain a bird's-eye view feature map of the three-dimensional point cloud data.
[0060] Step 103: Detect the three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data.
[0061] In one implementation, a three-dimensional point cloud target detection model may be used to detect a three-dimensional target of the three-dimensional point cloud data based on a bird's-eye view feature map of the three-dimensional point cloud data.
[0062] The point cloud target detection method of the embodiment of the present application obtains a bird's-eye feature map of the three-dimensional point cloud data based on the first bird's-eye feature map, the second bird's-eye feature map and the third bird's-eye feature map of the three-dimensional point cloud data; detects the three-dimensional target of the three-dimensional point cloud data based on the bird's-eye feature map of the three-dimensional point cloud data; the three-dimensional target of the bird's-eye feature map contains richer information, can accurately detect the three-dimensional target of the three-dimensional point cloud data, and improves the three-dimensional target detection effect based on the three-dimensional point cloud data.
[0063] Figure 2 It is another flow chart of the point cloud target detection method shown in an embodiment of the present application.
[0064] See also Figure 2 , a point cloud target detection method, comprising:
[0065] Step 201, obtaining a transformed isomorphic pixel feature map of three-dimensional point cloud data.
[0066] In one embodiment, 3D (three-dimensional) point cloud data collected by a radar such as a laser radar may be acquired. The 3D point cloud data includes but is not limited to the point cloud coordinates (x, y, z) and reflection intensity of the point cloud.
[0067] In one embodiment, 3D point cloud data can be input into a point cloud target detection model, and the point cloud target detection model can be TED; through the backbone network backbone of TED, transformation-equivariant sparse convolution is performed on the 3D point cloud data to obtain a transformation-equivariant pixel feature map of the 3D point cloud data, wherein the backbone network backbone of TED includes TeSpConv (Transformation-equivariant Sparse Convolution).
[0068] In one implementation, the 3D point cloud data can be divided into voxel sets through the backbone network of TED; each voxel in the voxel set is encoded as a transform equivariant voxel, and a transform equivariant sparse convolution is performed to obtain a transform equivariant voxel feature map of the 3D point cloud data.
[0069] In one embodiment, a 3D spatial coordinate system can be constructed along the X, Y, and Z axes in the point cloud space. The 3D spatial coordinate system takes the front of the vehicle as the positive direction of the X axis, the left of the vehicle as the positive direction of the Y axis, and the top of the vehicle as the positive direction of the Z axis. The positive direction of the Z axis corresponds to the height dimension h of the point cloud data, the positive direction of the Y axis corresponds to the width dimension w of the point cloud data, and the positive direction of the X axis corresponds to the depth dimension d of the point cloud data. The transformed equivariant pixel feature map of the 3D point cloud data can be a feature map of dimensions (n, c, d, w, h), where n represents n pieces of data of the transformed equivariant pixel feature map (corresponding to n voxels of the 3D point cloud data voxel set), c represents the number of channels of the transformed equivariant pixel feature map, d represents the height of the 3D point cloud data, w represents the width of the 3D point cloud data, and h represents the depth of the 3D point cloud data.
[0070] Step 202 , respectively obtaining a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data according to the transformed isomorphic pixel feature map of the three-dimensional point cloud data.
[0071] In one embodiment, TED may include TeBEV Pooling (Transformation-equivariant BEV Pooling). According to the transformation-equivariant pixel feature map of the three-dimensional point cloud data, the first bird's-eye view feature map, the second bird's-eye view feature map, and the third bird's-eye view feature map of the three-dimensional point cloud data may be obtained through TeBEV Pooling of TED.
[0072] In one embodiment, the first BEV feature map, the second BEV feature map and the third BEV feature map of the 3D point cloud data can be obtained respectively according to the transformed equivalent pixel feature map with dimensions (n, c, d, w, h) through TeBEV Pooling of TED.
[0073] In one embodiment, the transformed equivalent pixel feature map of the three-dimensional point cloud data can be compressed along the height dimension to obtain a first bird's-eye view feature map of the three-dimensional point cloud data. TeBEV Pooling may include a first branch Pooling. According to the transformed equivalent pixel feature map of the 3D point cloud data with a dimension of (n, c, d, w, h), through the first branch Pooling of TeBEV Pooling, the transformed equivalent pixel feature map with a dimension of (n, c, d, w, h) can be compressed along the height dimension in the Z-axis direction of the 3D point cloud data, and the information of the height dimension of the transformed equivalent pixel feature map is attributed to the channel information to obtain the first BEV feature map of the 3D point cloud data. The first BEV feature map may be a feature map with a dimension of (n, c*d, w, h).
[0074] In one embodiment, the transformed equivalent pixel feature map of the three-dimensional point cloud data can be compressed along the width dimension to obtain a second bird's-eye view feature map of the three-dimensional point cloud data. TeBEV Pooling may also include a second branch Pooling. According to the transformed equivalent pixel feature map of the 3D point cloud data with a dimension of (n, c, d, w, h), the transformed equivalent pixel feature map with a dimension of (n, c, d, w, h) can be compressed along the width dimension in the Y-axis direction of the 3D point cloud data through the second branch Pooling of TeBEV Pooling of TED, and the information of the width dimension of the transformed equivalent pixel feature map is attributed to the channel information to obtain the second BEV feature map of the 3D point cloud data. The second BEV feature map may be a feature map with a dimension of (n, c*w, d, h).
[0075] In one embodiment, the transformed equivalent pixel feature map of the three-dimensional point cloud data can be compressed along the depth dimension to obtain a third bird's-eye view feature map of the three-dimensional point cloud data. TeBEV Pooling may also include a third branch Pooling. According to the transformed equivalent pixel feature map of the 3D point cloud data with a dimension of (n, c, d, w, h), the transformed equivalent pixel feature map with a dimension of (n, c, d, w, h) can be compressed along the depth dimension in the X-axis direction through the third branch Pooling of TeBEV Pooling of TED to obtain a third BEV feature map of the 3D point cloud data. The third BEV feature map may be a feature map with a dimension of (n, c*h, d, w).
[0076] Step 203: Obtain a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map of the three-dimensional point cloud data.
[0077] In one embodiment, the dimensions of the second and third bird's-eye feature maps can be aligned with the dimensions of the first bird's-eye feature map based on the dimensional information of the first bird's-eye feature map of the three-dimensional point cloud data; the second and third bird's-eye feature maps whose dimensions are aligned with the first bird's-eye feature map are spliced with the first bird's-eye feature map to obtain a bird's-eye feature map (BEV feature map) of the three-dimensional point cloud data.
[0078] In one embodiment, a second BEV feature map with a dimension of (n, c*w, d, h) can be convolved according to the dimensional information (n, c*d, w, h) of the first bird's-eye view feature map of the three-dimensional point cloud data, and the dimension of the second BEV feature map can be adjusted so that the dimension of the second BEV feature map is equal to the dimension of the first BEV feature map, and the dimension of the second BEV feature map is aligned with the dimension of the first bird's-eye view feature map.
[0079] In one embodiment, a third BEV feature map with dimensions (n, c*h, d, w) can be convolved according to the dimensional information (n, c*d, w, h) of the first bird's-eye view feature map of the three-dimensional point cloud data, and the dimension of the third BEV feature map can be adjusted so that the dimension of the third BEV feature map is equal to the dimension of the first BEV feature map, and the dimension of the third BEV feature map is aligned with the dimension of the first bird's-eye view feature map.
[0080] In one embodiment, the second bird's-eye view feature map and the third bird's-eye view feature map whose dimensions are aligned with the first bird's-eye view feature map can be fused with the first bird's-eye view feature map, and the fusion method includes but is not limited to splicing to obtain a bird's-eye view feature map (BEV feature map) of three-dimensional point cloud data.
[0081] In one embodiment, according to the dimension information (n, c*d, w, h) of the first bird's-eye view feature map of the three-dimensional point cloud data, a second BEV feature map with a dimension (n, c*w, d, h) and a third BEV feature map with a dimension (n, c*h, d, w) can be convolved through a convolution block, and the dimensions of the second BEV feature map and the third BEV feature map can be adjusted respectively, and the dimensions of the second BEV feature map and the third BEV feature map are aligned with the dimensions of the first bird's-eye view feature map; wherein the convolution block can be a convolution network with an equal number of input channels and an output channel number and a convolution size of 3×3.
[0082] In a specific implementation, the number of channels of the first bird's-eye feature map can be obtained based on the dimensional information of the first bird's-eye feature map of the three-dimensional point cloud data; based on the number of channels of the second bird's-eye feature map, the second bird's-eye feature map is converted into a second bird's-eye feature map to be spliced having the same number of channels as the first bird's-eye feature map; based on the number of channels of the third bird's-eye feature map, the third bird's-eye feature map is converted into a third bird's-eye feature map to be spliced having the same number of channels as the first bird's-eye feature map; the first bird's-eye feature map, the second bird's-eye feature map to be spliced, and the third bird's-eye feature map to be spliced having the same number of channels are spliced to obtain a bird's-eye feature map (BEV feature map) of the three-dimensional point cloud data.
[0083] Step 204 , detecting a three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data.
[0084] In one implementation, a lightweight feature map can be obtained based on a bird's-eye view feature map of three-dimensional point cloud data by using bilinear interpolation and maximum pooling; based on the lightweight feature map, multiple three-dimensional target suggestion detection frames for detecting three-dimensional targets are obtained; and based on the multiple three-dimensional target suggestion detection frames for detecting three-dimensional targets, the three-dimensional target of the three-dimensional point cloud data is detected.
[0085] In one embodiment, a feature map of three-dimensional target detection can be obtained based on a plurality of three-dimensional target detection box proposals, transformation and other variant pixel feature maps, by using multi-grid pooling, and / or cross-grid attention; and the three-dimensional target of the three-dimensional point cloud data can be detected based on the feature map of the three-dimensional target detection.
[0086] In a specific implementation, a lightweight feature map A can be obtained based on the BEV feature map of the three-dimensional point cloud data through TeBEVPooling of TED, using bilinear interpolation and max-pooling. * ; According to the lightweight feature map A *, through TED's RPN (Region Proposal Network), we obtain multiple 3D object proposal detection boxes (object proposals B * ).
[0087] In a specific implementation, the TiVoxel pooling (Transformation-Invariant Voxel Pooling) of TED can be used to propose detection boxes (object proposals B) based on multiple 3D object detections. * ), extract multiple instance-level feature maps (multiple instance-level features) from the transformed variant pixel feature map; obtain multiple grid feature maps (grid-wise features) based on the multiple instance-level feature maps; obtain a feature map for 3D target detection based on the multiple grid feature maps (grid-wise features); detect 3D targets in 3D point cloud data based on the feature map for 3D target detection.
[0088] In a specific implementation, TiVoxel pooling includes Multi-grid pooling and Cross-grid attention. According to the output of TeSpConv, TeBEV Pooling and RPN of the backbone network of TED, a feature map of 3D target detection can be obtained through Multi-grid pooling and Cross-gridattention; according to the feature map of 3D target detection, 3D targets of 3D point cloud data can be detected.
[0089] The point cloud target detection method of the embodiment of the present application obtains a bird's-eye feature map of the three-dimensional point cloud data based on the first bird's-eye feature map, the second bird's-eye feature map and the third bird's-eye feature map of the three-dimensional point cloud data; detects the three-dimensional target of the three-dimensional point cloud data based on the bird's-eye feature map of the three-dimensional point cloud data; the three-dimensional target of the bird's-eye feature map contains richer information, can accurately detect the three-dimensional target of the three-dimensional point cloud data, and improves the three-dimensional target detection effect based on the three-dimensional point cloud data.
[0090] Furthermore, the point cloud target detection method of the embodiment of the present application compresses the transformed equivalent pixel feature map along the height dimension according to the transformed equivalent pixel feature map of the three-dimensional point cloud data to obtain a first bird's-eye view feature map of the three-dimensional point cloud data; compresses the transformed equivalent pixel feature map along the width dimension according to the transformed equivalent pixel feature map of the three-dimensional point cloud data to obtain a second bird's-eye view feature map of the three-dimensional point cloud data; compresses the transformed equivalent pixel feature map along the depth dimension according to the transformed equivalent pixel feature map of the three-dimensional point cloud data to obtain a third bird's-eye view feature map of the three-dimensional point cloud data; obtains a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye feature map, the second bird's-eye feature map and the third bird's-eye feature map of the three-dimensional point cloud data; The method further provides a method for detecting three-dimensional objects in three-dimensional point cloud data by using a bird's-eye view feature map of three-dimensional point cloud data; the second bird's-eye view feature map and the third bird's-eye view feature map provide more three-dimensional point cloud target information for the bird's-eye view feature map of three-dimensional point cloud data, so that the bird's-eye view feature map of three-dimensional point cloud data includes information of different dimensions of three-dimensional point cloud data, thereby avoiding the loss of information of different dimensions of three-dimensional point cloud data when obtaining the bird's-eye view feature map of three-dimensional point cloud data, enriching the information of three-dimensional point cloud targets in the bird's-eye view feature map of three-dimensional point cloud data, enabling the point cloud target detection model to better detect three-dimensional objects in three-dimensional point cloud data, accurately detect three-dimensional objects in three-dimensional point cloud data, and improving the three-dimensional target detection effect of the point cloud target detection model based on three-dimensional point cloud data.
[0091] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a point cloud target detection device, an electronic device and corresponding embodiments.
[0092] Figure 3 It is a structural schematic diagram of a point cloud target detection device shown in an embodiment of the present application.
[0093] See also Figure 3 A point cloud target detection device includes a first acquisition module 301, a bird's-eye view acquisition module 302, and a target detection module 303.
[0094] The first acquisition module 301 is used to respectively acquire a first bird's-eye view feature map, a second bird's-eye view feature map and a third bird's-eye view feature map of the three-dimensional point cloud data.
[0095] The bird's-eye view acquisition module 302 is used to obtain a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map acquired by the first acquisition module 301 .
[0096] The target detection module 303 is used to detect the three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data obtained by the bird's-eye view acquisition module 302.
[0097] In one embodiment, the three-dimensional point cloud data can be input into the three-dimensional point cloud target detection model, and the transformed isomorphic pixel feature map of the three-dimensional point cloud data can be obtained through the backbone network backbone of the three-dimensional point cloud target detection model. The first acquisition module 301 can perform different transformations and convolutions on the transformed isomorphic pixel feature map of the three-dimensional point cloud data according to the transformed isomorphic pixel feature map of the three-dimensional point cloud data, and obtain the first bird's-eye view feature map, the second bird's-eye view feature map, and the third bird's-eye view feature map of the three-dimensional point cloud data.
[0098] In one embodiment, the bird's-eye view acquisition module 302 can align the first bird's-eye view feature map, the second bird's-eye view feature map, and the third bird's-eye view feature map of the three-dimensional point cloud data acquired by the first acquisition module 301 through a three-dimensional point cloud target detection model; and fuse the aligned first bird's-eye view feature map, the second bird's-eye view feature map, and the third bird's-eye view feature map to obtain a bird's-eye view feature map of the three-dimensional point cloud data.
[0099] In one implementation, the target detection module 303 may detect the three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data through a three-dimensional point cloud target detection model.
[0100] The technical solution of the embodiment of the present application obtains a bird's-eye feature map of the three-dimensional point cloud data based on the first bird's-eye feature map, the second bird's-eye feature map and the third bird's-eye feature map of the three-dimensional point cloud data; detects the three-dimensional target of the three-dimensional point cloud data based on the bird's-eye feature map of the three-dimensional point cloud data; the three-dimensional target of the bird's-eye feature map contains richer information, can accurately detect the three-dimensional target of the three-dimensional point cloud data, and improves the three-dimensional target detection effect based on the three-dimensional point cloud data.
[0101] Figure 4 It is another structural schematic diagram of the point cloud target detection device shown in an embodiment of the present application.
[0102] See also Figure 4 A point cloud target detection device includes a voxel feature map acquisition module 401, a first acquisition module 301, a bird's-eye view acquisition module 302, and a target detection module 303.
[0103] The voxel feature map acquisition module 401 is used to acquire a transformed voxel feature map of the three-dimensional point cloud data.
[0104] The first acquisition module 301 is further used to respectively acquire a first bird's-eye view feature map, a second bird's-eye view feature map and a third bird's-eye view feature map of the three-dimensional point cloud data according to the transformed isomorphic voxel feature map acquired by the voxel feature map acquisition module 401.
[0105] In one implementation, the first acquisition module 301 is further used to compress the transformed equivalent voxel feature map along the height dimension according to the transformed equivalent voxel feature map acquired by the voxel feature map acquisition module 401 to acquire the first bird's-eye view feature map.
[0106] In one implementation, the first acquisition module 301 is further configured to compress the transformed equivalent voxel feature map along the width dimension according to the transformed equivalent voxel feature map acquired by the voxel feature map acquisition module 401 to acquire the second bird's-eye view feature map.
[0107] In one implementation, the first acquisition module 301 is further configured to compress the transformed equivalent voxel feature map along the depth dimension according to the transformed equivalent voxel feature map acquired by the voxel feature map acquisition module 401 to acquire a third bird's-eye view feature map.
[0108] The bird's-eye view acquisition module 302 is also used to align the dimensions of the second bird's-eye view feature map and the third bird's-eye view feature map obtained by the first acquisition module 301 with the dimensions of the first bird's-eye view feature map according to the dimensional information of the first bird's-eye view feature map obtained by the first acquisition module 301; and to splice the second bird's-eye feature map and the third bird's-eye feature map whose dimensions are aligned with the first bird's-eye feature map with the first bird's-eye feature map to obtain a bird's-eye view feature map of the three-dimensional point cloud data.
[0109] The target detection module 303 is used to detect the three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data obtained by the bird's-eye view acquisition module 302.
[0110] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.
[0111] Figure 5 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.
[0112] See also Figure 5 , the electronic device 1000 includes a memory 1010 and a processor 1020 .
[0113] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
[0114] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.
[0115] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.
[0116] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.
[0117] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.
[0118] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A point cloud object detection method, It is characterized in that include: Respectively obtain a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data; Obtaining a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map; A three-dimensional target of the three-dimensional point cloud data is detected according to the bird's-eye view feature map of the three-dimensional point cloud data.
2. The method according to claim 1, It is characterized in that The method of respectively obtaining a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data includes: Obtaining a transformed equivalent pixel feature map of the three-dimensional point cloud data; According to the transformed isomorphic pixel feature map, a first bird's-eye view feature map, a second bird's-eye view feature map and a third bird's-eye view feature map of the three-dimensional point cloud data are respectively obtained.
3. The method according to claim 2, It is characterized in that The step of respectively obtaining a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data according to the transformed isomorphic pixel feature map comprises: According to the transformed equivalent pixel feature map, compressing the transformed equivalent pixel feature map along the height dimension to obtain the first bird's-eye view feature map; According to the transformed equivalent pixel feature map, compressing the transformed equivalent pixel feature map along the width dimension to obtain the second bird's-eye view feature map; According to the transformed equivalent pixel feature map, the transformed equivalent pixel feature map is compressed along the depth dimension to obtain the third bird's-eye view feature map.
4. The method according to claim 1, It is characterized in that The step of obtaining the bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map comprises: According to the dimension information of the first bird's-eye view feature map, aligning the dimensions of the second bird's-eye view feature map and the third bird's-eye view feature map with the dimensions of the first bird's-eye view feature map respectively; The second bird's-eye view feature map and the third bird's-eye view feature map whose dimensions are aligned with the first bird's-eye view feature map are spliced with the first bird's-eye view feature map to obtain the bird's-eye view feature map of the three-dimensional point cloud data.
5. The method according to claim 3, It is characterized in that The step of obtaining the bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map comprises: Obtaining the number of channels of the first bird's-eye view feature map according to the dimension information of the first bird's-eye view feature map; According to the number of channels of the second bird's-eye view feature map, converting the second bird's-eye view feature map into a second to-be-joined bird's-eye view feature map having the same number of channels as the first bird's-eye view feature map; According to the number of channels of the third bird's-eye view feature map, converting the third bird's-eye view feature map into a third to-be-joined bird's-eye view feature map having the same number of channels as the first bird's-eye view feature map; The first bird's-eye view feature map, the second bird's-eye view feature map to be spliced, and the third bird's-eye view feature map to be spliced are spliced to obtain a bird's-eye view feature map of the three-dimensional point cloud data.
6. The method according to claim 2, It is characterized in that Detecting a three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data includes: According to the bird's-eye view feature map of the three-dimensional point cloud data, bilinear interpolation and maximum pooling are used to obtain a lightweight feature map; According to the lightweight feature map, a plurality of three-dimensional object detection frame suggestions for detecting the three-dimensional object are obtained; The three-dimensional object of the three-dimensional point cloud data is detected according to the three-dimensional object suggestion detection boxes of the multiple three-dimensional objects.
7. The method according to claim 6, It is characterized in that The detecting the three-dimensional object of the three-dimensional point cloud data according to the three-dimensional object suggestion detection frames for detecting the three-dimensional objects comprises: According to the plurality of three-dimensional object detection proposal detection boxes for detecting the three-dimensional objects and the transformed isomorphic pixel feature map, multi-grid pooling and / or cross-grid attention are used to obtain a feature map for three-dimensional object detection; The three-dimensional object of the three-dimensional point cloud data is detected according to the feature map of the three-dimensional object detection.
8. A point cloud target detection device, It is characterized in that include: A first acquisition module is used to respectively acquire a first bird's-eye view feature map, a second bird's-eye view feature map, and a third bird's-eye view feature map of the three-dimensional point cloud data; A bird's-eye view acquisition module, configured to obtain a bird's-eye view feature map of the three-dimensional point cloud data according to the first bird's-eye view feature map, the second bird's-eye view feature map and the third bird's-eye view feature map acquired by the first acquisition module; The target detection module is used to detect the three-dimensional target of the three-dimensional point cloud data according to the bird's-eye view feature map of the three-dimensional point cloud data obtained by the bird's-eye view acquisition module.
9. An electronic device, It is characterized in that include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, Features: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as claimed in any one of claims 1 to 7.
