Three-dimensional object detection method, detection device and electronic equipment

Abstract

The application discloses a three-dimensional object detection method, a detection device and electronic equipment. The detection method of the application first samples the original bird's-eye view feature map of the to-be-detected region to obtain initial feature data corresponding to each sampling point; then, for each sampling point, a dynamic sampling kernel corresponding to the sampling point is determined, and the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel are resampled to obtain target feature data corresponding to the resampled sampling point; and finally, the target bird's-eye view feature map of the to-be-detected region is determined according to the target feature data corresponding to all the sampling points, so that the three-dimensional object detection is performed on the to-be-detected region according to the target bird's-eye view feature map. The application fuses the feature data of the sampling point and the feature data of the dynamic sampling kernel through resampling to obtain the target bird's-eye view feature map with rich feature data, thereby improving the detection rate of the three-dimensional object and the reliability of the three-dimensional object detection.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a three-dimensional object detection method, detection device and electronic equipment. Background Art

[0002] Advanced Driver Assistance Systems (ADAS) use on-board sensors to monitor the vehicle's surroundings in real time, provide the driver with auxiliary information or automatically intervene in driving behavior, and improve driving safety and comfort.

[0003] The automated parking system, a module within the ADAS system, automatically identifies parking spaces and controls the vehicle's steering, speed, and braking, enabling the vehicle to automatically park in or out of the space. If a dynamic object, such as a pedestrian, suddenly appears during the process, the vehicle must brake promptly to avoid an accident. Traditional 3D object detection methods rely on feature detection from a bird's-eye view projected image. This can lead to missed detection of objects with insufficient feature information after projection, resulting in low reliability. Summary of the Invention

[0004] Purpose of the invention: The embodiments of the present application provide a three-dimensional object detection method, detection device and electronic device, which can improve the detection rate of three-dimensional objects during automatic parking and improve the reliability of three-dimensional object detection.

[0005] Technical solution: In the first aspect, a three-dimensional object detection method described in an embodiment of the present application includes: sampling the original bird's-eye view feature map of the area to be detected to obtain initial feature data corresponding to each sampling point; for each sampling point, determining the dynamic sampling kernel corresponding to the sampling point, and resampling based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel to obtain target feature data corresponding to the sampling point after resampling; determining the target bird's-eye view feature map of the area to be detected based on the target feature data corresponding to all the sampling points, and performing three-dimensional object detection on the area to be detected based on the target bird's-eye view feature map.

[0006] Optionally, the method further includes: performing weighted processing on the target feature data corresponding to the sampling points according to an aggregation weight to obtain weighted target feature data; determining the target bird's-eye view feature map of the area to be detected according to the target feature data corresponding to all the sampling points includes: determining the target bird's-eye view feature map of the area to be detected according to the weighted target feature data corresponding to all the sampling points; wherein the aggregation weight is determined according to the target feature data corresponding to the sampling point and the target feature data corresponding to the neighboring sampling points adjacent to the sampling point.

[0007] Optionally, the neighborhood sampling point is a sampling point within a preset range adjacent to the sampling point.

[0008] Optionally, determining the dynamic sampling kernel corresponding to the sampling point includes: determining a dynamic offset and a dynamic offset weight corresponding to the sampling point, and obtaining the dynamic sampling kernel according to the dynamic offset and the dynamic offset weight.

[0009] Optionally, the resampling is performed based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling core to obtain the target feature data corresponding to the sampling point after resampling, including: fusing the dynamic feature data corresponding to the dynamic sampling core with the initial feature data corresponding to the sampling point to obtain fused feature data; and resampling the fused feature data according to a preset sampling function to obtain the target feature data corresponding to the sampling point.

[0010] Optionally, the preset sampling function includes at least one of the following: a bilinear interpolation sampling function, a nearest neighbor interpolation sampling function, and a bicubic interpolation sampling function.

[0011] Optionally, the sampling of the original bird's-eye view feature map of the area to be detected to obtain initial feature data corresponding to each sampling point includes: generating a sampling grid in the original bird's-eye view feature map, the sampling grid including a plurality of sampling points; sampling the original bird's-eye view feature map according to each sampling point of the sampling grid to obtain initial feature data corresponding to each sampling point.

[0012] Optionally, the original bird's-eye view feature map is obtained by projecting the two-dimensional image features of the area to be detected onto a bird's-eye view.

[0013] In a second aspect, a three-dimensional object detection device described in an embodiment of the present application includes: an initial sampling module, used to sample the original bird's-eye view feature map of the area to be detected to obtain initial feature data corresponding to each sampling point; a resampling module, used to determine the dynamic sampling kernel corresponding to each sampling point, and resample based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel to obtain target feature data corresponding to the sampling point after resampling; a feature fusion module, used to determine the target bird's-eye view feature map of the area to be detected based on the target feature data corresponding to all the sampling points, and perform three-dimensional object detection on the area to be detected based on the target bird's-eye view feature map.

[0014] In a third aspect, an electronic device described in an embodiment of the present application includes: a memory on which a computer program or instruction is stored; and a processor for executing the computer program or instruction in the memory to implement the three-dimensional object detection method described in the first aspect above.

[0015] Beneficial effects: Compared with the prior art, the technical solution proposed in the embodiment of the present application first samples the original bird's-eye view feature map of the area to be detected to obtain initial feature data corresponding to each sampling point; then, for each sampling point, the dynamic sampling kernel corresponding to the sampling point is determined, and resampling is performed based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel to obtain target feature data corresponding to the resampled sampling point; the target bird's-eye view feature map of the area to be detected is determined based on the target feature data corresponding to all sampling points, thereby performing three-dimensional object detection on the area to be detected based on the target bird's-eye view feature map. The three-dimensional object detection method of the embodiment of the present application resamples the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel, breaking through the limitation of fixed sampling points, and fusing the feature data of the sampling point with the feature data of the dynamic sampling kernel to obtain a target bird's-eye view feature map with rich feature data, which can improve the detection rate of three-dimensional objects and improve the reliability of three-dimensional object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 1 is a flow chart of a three-dimensional object detection method provided in an embodiment of the present application;

[0018] Figure 2 This is a schematic projection diagram of a bird's-eye view provided in an embodiment of the present application;

[0019] Figure 3 is a flow chart of another three-dimensional object detection method provided in an embodiment of the present application;

[0020] Figure 4 is a structural diagram of a three-dimensional object detection device provided in an embodiment of the present application;

[0021] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0023] In Advanced Driver Assistance Systems (ADAS), automated parking can automatically identify parking spaces and control the vehicle's steering, speed, and braking, enabling it to automatically park in or out of the space. If a moving object, such as a pedestrian, appears during automated parking, the vehicle must brake promptly to avoid an accident.

[0024] Currently, automated parking systems primarily use Bird's Eye View Detection (BEVDet) for 3D object detection. The BEVDet algorithm first calculates the projection points from a bird's eye view (BEV). It then projects 2D image features onto the BEV to generate a BEV feature map. 3D object detection is then performed based on this BEV feature map. However, objects with insufficient feature information after projection may be missed, resulting in low reliability.

[0025] In view of this, embodiments of the present application provide a three-dimensional object detection method, a detection device, and an electronic device, which can improve the detection rate of three-dimensional objects during automatic parking and improve the reliability of three-dimensional object detection.

[0026] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a three-dimensional object detection method provided in an embodiment of the present application. Figure 1 As shown, the three-dimensional object detection method of the embodiment of the present application includes the following steps S100 to S300.

[0027] Step S100, sampling the original bird's-eye view feature map of the area to be detected to obtain initial feature data corresponding to each sampling point;

[0028] Step S200: for each sampling point, determining a dynamic sampling kernel corresponding to the sampling point, performing resampling based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel, to obtain target feature data corresponding to the resampled sampling point;

[0029] At step S300, a target bird's eye view feature map of the to-be-detected region is determined according to the target feature data corresponding to all the sampling points, and the to-be-detected region is detected in three dimensions according to the target bird's eye view feature map.

[0030] The to-be-detected region represents a region that needs to be detected in three dimensions. In the automatic parking scenario, the to-be-detected region can be a region related to the planned path when the vehicle is automatically parked. The embodiments of the present application optimize the BEVDet algorithm to detect the three-dimensional objects in the to-be-detected region according to the optimized three-dimensional object detection method, which can improve the detection rate of dynamic targets such as pedestrians while ensuring detection efficiency, and balances detection efficiency and detection rate.

[0031] The bird's eye view feature map (BEV feature map) is a bird's eye view feature map generated when the target is detected according to the BEVDet algorithm. Sampling the bird's eye view feature map can obtain feature data in the bird's eye view feature map, so that three-dimensional object detection can be realized in the decoder such as the detection head according to the extracted feature data. The BEVDet algorithm simplifies complex three-dimensional image recognition to target detection in two-dimensional images, reduces the amount of data processed during target detection, improves the calculation efficiency of the detection algorithm, and also saves the storage space of the feature data.

[0032] The original bird's eye view feature map represents the bird's eye view feature map before the three-dimensional object detection method according to the embodiments of the present application is optimized. In the original bird's eye view feature map, objects with less sampled feature data may be missed. The embodiments of the present application improve the feature data of small targets by fusing the sampled feature data, thereby improving the detection rate of three-dimensional objects.

[0033] The dynamic sampling kernel is a dynamic offset interval corresponding to the sampling point. The embodiments of the present application determine the dynamic sampling kernel according to the sampling point, and perform resampling based on the initial feature data of the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel, aggregate the feature data around the sampling point through resampling, enrich the feature data of each sampling point in the bird's eye view, break through the grid limit of the fixed sampling point, and improve the detection rate of small three-dimensional objects.

[0034] After resampling, the target bird's eye view feature map is formed according to the target feature data of each sampling point after resampling, and the target bird's eye view feature map is input into the decoder such as the detection head. The target bird's eye view feature map can be decoded and detected by the decoder to obtain the type, position, size, direction and other information of the three-dimensional object.

[0035] The embodiment of the application enriches the feature data of small targets in the target bird's-eye view feature map by setting a dynamic sampling kernel, fusing the feature data of the sampling points and the feature data of the dynamic sampling kernel, and improves the detection rate of small targets and the reliability of three-dimensional object detection.

[0036] In some embodiments, the original bird's-eye view feature map is obtained by projecting two-dimensional image features of a region to be detected to a bird's-eye view.

[0037] The bird's-eye view feature map can be obtained based on a two-dimensional image of a region to be detected. For example, a two-dimensional image of a region to be detected is obtained by a camera, a radar or the like, image features of the two-dimensional image are extracted by an image encoder, and the image features are projected to a bird's-eye view by a view converter to obtain an original bird's-eye view feature map. It can be understood that the embodiment of the application is based on the bird's-eye view feature map for three-dimensional object recognition, and the method for determining the bird's-eye view feature map is not limited, and the generation mode of the BEV feature map can be determined according to the actual situation in the application.

[0038] In some embodiments, the original bird's-eye view feature map of the region to be detected in step S100 is sampled to obtain initial feature data corresponding to each sampling point, including: generating a sampling grid in the original bird's-eye view feature map, the sampling grid including a plurality of sampling points; sampling the original bird's-eye view feature map according to each sampling point of the sampling grid to obtain initial feature data corresponding to each sampling point.

[0039] The sampling grid corresponds to the original bird's-eye view feature map and is used to extract feature data in the original bird's-eye view feature map. By extracting features from the original bird's-eye view feature map, three-dimensional object detection can be performed in the detection head decoder based on the extracted feature data.

[0040] For example, a two-dimensional sampling grid is constructed based on the original bird's-eye view feature map, the resolution of the sampling grid determines the size of each cell in the sampling grid, the higher the resolution of the sampling grid, the smaller the cell, and the more details each cell can represent. A plurality of sampling points are obtained based on the sampling grid, for example, a 10x10 sampling grid corresponds to 100 sampling points. Feature extraction is performed on the original bird's-eye view feature map at each sampling point to obtain initial feature data corresponding to each sampling point. It should be noted that the feature data after sampling in the embodiment of the application can be image feature data, and for different bird's-eye view feature maps, the feature data after sampling can also be point cloud feature data, depth feature data, etc.

[0041] In some embodiments, the dynamic sampling kernel corresponding to the sampling point is determined in step S200, including: determining a dynamic offset and a dynamic offset weight corresponding to the sampling point, and obtaining the dynamic sampling kernel based on the dynamic offset and the dynamic offset weight.

[0042] Dynamic sampling kernels correspond to sampling points. Combining these kernels with the sampling points allows the feature data near the sampling points, corresponding to the dynamic offset of the kernel, to be fused with the feature data near the sampling points. For example, if the original bird's-eye view feature map includes 100 sampling points, the corresponding number of dynamic sampling kernels is also 100. Each dynamic sampling kernel corresponds to a single sampling point. The dynamic sampling kernel includes a dynamic offset and a dynamic offset weight. Dynamic sampling kernels are variable, meaning that the dynamic offset and dynamic offset weight within the kernel may vary for different bird's-eye view feature maps or in different application scenarios.

[0043] Dynamic sampling kernel can be used by K Expressed as:

[0044]

[0045] in, and Indicates the dynamic offset of different coordinate axes on a two-dimensional plane. Indicates the dynamic offset weight. In the dynamic sampling kernels corresponding to different sampling points, the dynamic offset can be the same or different. The dynamic offset weights of different dynamic sampling kernels are generally different. N Indicates the number of sampling points.

[0046] In some embodiments, the dynamic offset and dynamic offset weight of the dynamic sampling kernel can be predicted based on a lightweight Multi-Layer Perception (MLP) network. Multi-Layer Perception networks can be used to simulate complex nonlinear relationships. In embodiments of the present application, the dynamic offset and dynamic offset weight of the dynamic sampling kernel are predicted based on MLP perception, ensuring that the impact of the generated dynamic sampling kernel on the computational complexity of the three-dimensional object detection method is within a preset range. In other words, the present application uses MLP to achieve lightweight dynamic offset and dynamic offset weight prediction, improving the detection rate of three-dimensional objects while adding a small amount of computation, thus meeting the requirements for automatic parking control in extreme operating conditions.

[0047] In some embodiments, in step S200, resampling is performed based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling core to obtain the target feature data corresponding to the resampled sampling point, including: fusing the dynamic feature data corresponding to the dynamic sampling core with the initial feature data corresponding to the sampling point to obtain fused feature data; and resampling the fused feature data according to a preset sampling function to obtain the target feature data corresponding to the sampling point.

[0048] When the initial feature data of the sampling point and the dynamic feature data of the dynamic sampling kernel are known, the embodiment of the present application fuses the initial feature data and the dynamic feature data to obtain fused feature data, and resamples the fused feature data according to a preset sampling function to enrich the feature data of the sampling point. The feature data obtained after resampling is the target feature data corresponding to the sampling point.

[0049] In some embodiments, the preset sampling function includes at least one of the following: a bilinear interpolation sampling function, a nearest neighbor interpolation sampling function, and a bicubic interpolation sampling function.

[0050] Bilinear interpolation calculates the feature data of a sampling point through linear interpolation based on the feature data of the four pixels surrounding the sampling point. Nearest neighbor interpolation calculates the feature data of a sampling point based on the feature data of the pixels closest to the sampling point. Bicubic interpolation calculates the feature data of a sampling point through cubic polynomial interpolation based on the feature data of the 16 pixels surrounding the sampling point. It will be understood that the embodiments of the present application do not limit the preset sampling function for the fused feature data. In actual applications, the preset sampling function can be determined based on image quality and image processing requirements.

[0051] In some embodiments, the three-dimensional object detection method further includes: weighting the target feature data corresponding to the sampling points according to the aggregation weight to obtain weighted target feature data; determining the target bird's-eye view feature map of the area to be detected according to the target feature data corresponding to all the sampling points, including: determining the target bird's-eye view feature map of the area to be detected according to the weighted target feature data corresponding to all the sampling points; wherein the aggregation weight is determined based on the target feature data corresponding to the sampling point and the target feature data corresponding to the neighborhood sampling points adjacent to the sampling point.

[0052] After performing feature fusion on the feature data of the sampling points based on dynamic sampling and obtaining target feature data, embodiments of the present application may further perform weighted processing on the target feature data corresponding to each sampling point based on an aggregation weight. The target bird's-eye view feature map is optimized using the aggregation weight, reducing the distance between feature data of the same attribute in the bird's-eye view feature map and increasing the distance between feature data of different attributes. This results in an optimized bird's-eye view feature map, which serves as the target bird's-eye view feature map. The aggregation weight may be determined based on the target feature data corresponding to the sampling point and the target feature data of the neighboring samples adjacent to the sampling point.

[0053] Exemplarily, if the similarity of the target feature data of the sampling point and the target feature data of the neighbor sampling point is high, it is considered that the feature data corresponding to the sampling point and the neighbor sampling point has the same attribute, and a high aggregation weight is set. If the similarity of the target feature data of the sampling point and the target feature data of the neighbor sampling point is low, it is considered that the feature data corresponding to the sampling point and the neighbor sampling point does not have the same attribute, and a low aggregation weight is set. For example, when the sampling point falls on the body of a pedestrian, the feature data of the neighbor sampling point corresponding to the head or the leg has high similarity, and a high aggregation weight is set, for example, the aggregation weight is set to 0.8; the feature data of the neighbor sampling point corresponding to the background and the ground has low similarity, and a low aggregation weight is set, for example, the aggregation weight is set to 0.1.

[0054] In some embodiments, the neighbor sampling point is a sampling point within a preset range adjacent to the sampling point.

[0055] The neighbor sampling point represents a sampling point adjacent to the sampling point, and the distance between the neighbor sampling point and the sampling point is within a preset range. The neighbor sampling point can be determined by setting a neighbor window. For example, for a 3x3 neighbor window, the neighbor sampling points representing the sampling point are 8 neighbor sampling points, that is, the eight sampling points adjacent to the sampling point are taken as the neighbor sampling points.

[0056] The feature fusion method of the sampling point according to the embodiments of the present application is described below through an example.

[0057] After determining the dynamic sampling kernel corresponding to the sampling point, feature fusion is performed for each sampling point, and the fused target bird's eye view feature map can be represented as:

[0058]

[0059] wherein, represents a preset sampling function, such as a bilinear interpolation sampling function; represents an original bird's eye view feature map, and represents the initial two-dimensional coordinates of the sampling point, i represents the sequence number of the sampling point; and represents the dynamic offset of the dynamic sampling kernel, represents the aggregation weight. According to the target bird's eye view feature map, it can be seen that the feature data of the dynamic sampling kernel is combined with the feature data of the sampling point, and the combined feature data is resampled by the preset sampling function to enhance the feature data of the sampling point. According to the aggregation weight, the resampled target feature data is weighted, so as to obtain the target bird's eye view feature map with optimized feature data, thereby improving the detection rate of the three-dimensional object by enriching the feature data and improving the reliability of the detection.

[0060] The aggregation weight may be expressed as:

[0061]

[0062] wherein, denotes the target feature data of the sampling point; denotes the target feature data of the neighborhood sampling point, i denotes the serial number of the sampling point; k denotes the serial number of the neighborhood sampling point; denotes all the neighborhood sampling points participating in the aggregation, i.e., the number of neighborhood sampling points, for 8-neighborhood sampling points, ; denotes a scaling factor, used to control the range of dot product values, preventing gradient explosion.

[0063] The embodiment of the present application can strengthen the associated feature data in the aerial view feature map by setting the aggregation weight, weaken the noise or background data, and improve the detection rate of the three-dimensional object.

[0064] The three-dimensional object detection method of the embodiment of the present application will be described below through a specific example. Please refer to Figure 2 , Figure 2 is a projection schematic diagram of an aerial view provided in the embodiment of the present application. As shown in the projection schematic diagram of the aerial view, Figure 2 the pedestrian is stereoscopic in the projected image, and in the projected aerial view feature map, a pull-up shape is formed as shown by the red arrow, and the corresponding feature data is less. The embodiment of the present application fuses the feature data of the red arrow region and the green arrow region, enriches the feature data of the small target, and can increase the detection rate of the three-dimensional object and improve the reliability of the detection.

[0065] Please refer to Figure 3 , Figure 3 is a flowchart of another three-dimensional object detection method provided in the embodiment of the present application. As shown in the flowchart of the three-dimensional object detection method, Figure 3 the three-dimensional object detection method includes the following steps S310 to S370.

[0066] Step S310, an original two-dimensional image of a to-be-detected region is acquired.

[0067] Step S320, feature extraction and view transformation are performed based on the original two-dimensional image, and an original aerial view feature map is obtained.

[0068] Step S330, a sampling grid corresponding to the original aerial view feature map is generated, and the sampling grid includes a plurality of sampling points.

[0069] Step S340, a dynamic sampling kernel is generated for each sampling point of the sampling grid.

[0070] The dynamic sampling kernel is predicted based on the lightweight MLP, including dynamic offset and dynamic offset weight.

[0071] Step S350 , fusing the initial feature data corresponding to the sampling point with the dynamic feature data corresponding to the dynamic sampling kernel to obtain target feature data corresponding to the sampling point.

[0072] Step S360 , weighting the target feature data corresponding to the sampling point according to the aggregation weight to obtain the weighted target feature data corresponding to the sampling point.

[0073] Step S370 , obtaining a target bird's-eye view feature map based on the weighted target feature data of all sampling points, and inputting the target bird's-eye view feature map into a detection head to obtain a target detection result.

[0074] The embodiment of the present application predicts the dynamic sampling kernel through a lightweight MLP. Compared with the traditional three-dimensional object detection method, the embodiment of the present application can increase the feature data of small targets through feature fusion while increasing the computational complexity by only 5%, thereby improving the detection rate of dynamic targets such as pedestrians during automatic parking and meeting the needs of automatic parking under extreme working conditions.

[0075] The three-dimensional object detection method of the embodiment of the present application resamples the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel, breaking through the limitation of fixed sampling points, and fusing the feature data of the sampling point with the feature data of the dynamic sampling kernel to obtain a target bird's-eye view feature map with rich feature data, which can improve the detection rate of three-dimensional objects and improve the reliability of three-dimensional object detection.

[0076] Based on the same inventive concept, embodiments of the present application also provide a three-dimensional object detection device for implementing the three-dimensional object detection method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more of the three-dimensional object detection device embodiments provided below can be found in the above-mentioned limitations of the three-dimensional object detection method and will not be further elaborated here.

[0077] In some embodiments, see Figure 4 , Figure 4 Schematic diagram of a three-dimensional object detection device provided in an embodiment of the present application. Figure 4 As shown, a three-dimensional object detection device according to an embodiment of the present application includes: an initial sampling module 410, a resampling module 420 and a feature fusion module 430; wherein:

[0078] An initial sampling module 410 is used to sample the original bird's-eye view feature map of the area to be detected to obtain initial feature data corresponding to each sampling point;

[0079] A resampling module 420 is configured to determine, for each sampling point, a dynamic sampling kernel corresponding to the sampling point, and perform resampling based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel to obtain target feature data corresponding to the resampled sampling point.

[0080] The feature fusion module 430 is used to determine a target bird's-eye view feature map of the area to be detected based on the target feature data corresponding to all sampling points, and perform three-dimensional object detection on the area to be detected based on the target bird's-eye view feature map.

[0081] Each module in the above-mentioned three-dimensional object detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0082] Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Correspondingly, this embodiment of the present application also provides an electronic device, please refer to Figure 5 The electronic device includes a memory, a processor, and a computer program or instructions stored in the memory and executable by the processor. When the processor executes the computer program or instructions, the steps of the above-mentioned three-dimensional object detection method are implemented. Since the three-dimensional object detection method has been described in detail above, it will not be repeated here.

[0083] Accordingly, embodiments of the present application further provide a computer-readable storage medium storing a computer program or instructions that, when executed by a processor, implement the steps of the above-described three-dimensional object detection method. Since the three-dimensional object detection method has been described in detail above, it will not be repeated here.

[0084] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of exemplary embodiments and may not be to scale. The modules or processes in the drawings are not necessarily required to implement the present application and therefore cannot be used to limit the scope of protection of the present application.

[0086] The above is a detailed introduction to the three-dimensional object detection method, detection device and electronic device provided in the embodiments of the present application, and specific examples are used to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the technical solutions and core ideas of the present application; ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A three-dimensional object detection method, characterized in that: include: Sampling the original bird's-eye view feature map of the area to be detected to obtain the initial feature data corresponding to each sampling point; For each sampling point, determining a dynamic sampling kernel corresponding to the sampling point, performing resampling based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel, to obtain target feature data corresponding to the sampling point after resampling; Determining a target bird's-eye view feature map of the area to be detected based on the target feature data corresponding to all the sampling points, and performing three-dimensional object detection on the area to be detected based on the target bird's-eye view feature map; The method further comprises: Performing weighted processing on the target feature data corresponding to the sampling points according to the aggregation weight to obtain weighted target feature data; The determining of the target bird's-eye view feature map of the area to be detected based on the target feature data corresponding to all the sampling points includes: Determine a target bird's-eye view feature map of the area to be detected based on the weighted target feature data corresponding to all the sampling points; The aggregation weight is determined according to the target feature data corresponding to the sampling point and the similarity between the target feature data corresponding to the neighborhood sampling points adjacent to the sampling point.

2. The method according to claim 1, characterized in that The neighborhood sampling points are sampling points within a preset range adjacent to the sampling point.

3. The method according to claim 1, characterized in that The determining of the dynamic sampling kernel corresponding to the sampling point includes: A dynamic offset and a dynamic offset weight corresponding to the sampling point are determined, and the dynamic sampling kernel is obtained according to the dynamic offset and the dynamic offset weight.

4. The method according to claim 1, wherein The resampling based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling core to obtain the target feature data corresponding to the sampling point after resampling includes: Fusing the dynamic feature data corresponding to the dynamic sampling core with the initial feature data corresponding to the sampling point to obtain fused feature data; The fused feature data is resampled according to a preset sampling function to obtain the target feature data corresponding to the sampling point.

5. The method according to claim 4, characterized in that The preset sampling function includes at least one of the following: a bilinear interpolation sampling function, a nearest neighbor interpolation sampling function, and a bicubic interpolation sampling function.

6. The method according to claim 1, characterized in that The sampling of the original bird's-eye view feature map of the area to be detected to obtain initial feature data corresponding to each sampling point includes: Generating a sampling grid on the original bird's-eye view feature map, wherein the sampling grid includes a plurality of sampling points; The original bird's-eye view feature map is sampled according to each sampling point of the sampling grid to obtain initial feature data corresponding to each sampling point.

7. The method according to claim 1, characterized in that The original bird's-eye view feature map is obtained by projecting the two-dimensional image features of the area to be detected onto a bird's-eye view.

8. A three-dimensional object detection device, characterized in that: include: The initial sampling module is used to sample the original bird's-eye view feature map of the detection area to obtain the initial feature data corresponding to each sampling point; A resampling module is configured to determine, for each sampling point, a dynamic sampling kernel corresponding to the sampling point, and perform resampling based on the initial feature data corresponding to the sampling point and the dynamic feature data corresponding to the dynamic sampling kernel to obtain target feature data corresponding to the sampling point after resampling; a feature fusion module, configured to determine a target bird's-eye view feature map of the area to be detected based on the target feature data corresponding to all the sampling points, and perform three-dimensional object detection on the area to be detected based on the target bird's-eye view feature map; The feature fusion module is further configured to: perform weighted processing on the target feature data corresponding to the sampling points according to the aggregation weight to obtain weighted target feature data; Determining the target bird's-eye view feature map of the area to be detected based on the target feature data corresponding to all the sampling points includes: determining the target bird's-eye view feature map of the area to be detected based on weighted target feature data corresponding to all the sampling points; wherein the aggregation weight is determined based on the target feature data corresponding to the sampling point and the similarity of the target feature data corresponding to the neighboring sampling points adjacent to the sampling point.

9. An electronic device, characterized in that: include: Memory on which computer programs or instructions are stored; A processor, configured to execute the computer program or instructions in the memory to implement the three-dimensional object detection method according to any one of claims 1 to 7.

Images