Target detection method, device, electronic device and storage medium

By gradually obtaining the first and second dimension features of the laser point cloud, cropping the corresponding areas, solving the problem of high computational volume in the prior art, and improving the efficiency of 3D object detection.

CN112686167BActive Publication Date: 2024-11-08YUANLI JUHE (CHONGQING) ROBOTICS TECHNOLOGY CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202011629467.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-30
Publication Date
2024-11-08
Estimated Expiration
2040-12-30

AI Technical Summary

Technical Problem

The existing 3D object detection method based on laser point cloud requires three-dimensional convolution operations on all laser points in the laser point cloud in three dimensions, resulting in large calculation amounts and low efficiency.

Method used

By first determining the first dimension feature of the laser point cloud, performing target detection and cropping the corresponding area, then determining the second dimension feature, gradually obtaining the three-dimensional detection result of the target object, and reducing the processing amount of the entire laser point cloud.

Benefits of technology

It effectively reduces the calculation amount of 3D object detection results and improves detection efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112686167B_ABST
    Figure CN112686167B_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide a target detection method, device, electronic device and storage medium, the method comprising: determining first dimensional features corresponding to multiple voxels of a laser point cloud, the laser point cloud comprising: laser points corresponding to a target object; performing target detection based on the first dimensional features to obtain a first target detection result; based on the first target detection result, cutting a corresponding area from the laser point cloud; determining second dimensional features corresponding to multiple regional voxels of the corresponding area, and determining a second target detection result of the corresponding area in the second dimension based on the second dimensional features; and obtaining a three-dimensional detection result of the target object based on the first target detection result and the second target detection result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer vision, and specifically to target detection methods, devices, electronic devices and storage media. Background Art

[0002] Point Cloud 3D Object Detection based on laser point cloud is widely used in autonomous driving, robotics and other fields.

[0003] 3D target detection based on laser point cloud is used to obtain the 3D target detection results of the target object. The 3D target detection results of the target object include the type of the target object, the X-axis coordinate value, Y-axis coordinate value, and Z-axis coordinate value of the center point of the three-dimensional bounding box that surrounds the target object in three-dimensional space. At present, 3D target detection neural networks such as VoxelNet and PointRCNN need to perform three-dimensional convolution operations on the relevant features of all laser points in the laser point cloud in three dimensions, namely, X-dimension, Y-dimension, and Z-dimension, to obtain the corresponding items when obtaining each item in the 3D target detection result of the target object, resulting in a large amount of computation consumed in obtaining the 3D target detection results of the target object. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present application provides a target detection method, device, electronic device and storage medium.

[0005] According to a first aspect of an embodiment of the present application, a target detection method is provided, including:

[0006] Determining first dimensional features corresponding to a plurality of voxels of a laser point cloud, the laser point cloud comprising: laser points corresponding to a target object;

[0007] Performing target detection based on the first dimensional features to obtain a first target detection result;

[0008] According to the first target detection result, cutting a corresponding area from the laser point cloud;

[0009] Determine a second dimensional feature corresponding to a plurality of regional voxels of the corresponding region, and determine a second target detection result of the corresponding region in the second dimension based on the second dimensional feature;

[0010] A three-dimensional detection result of the target object is obtained based on the first target detection result and the second target detection result.

[0011] According to a second aspect of an embodiment of the present application, there is provided a target detection device, including:

[0012] A determination unit is configured to determine first dimensional features corresponding to a plurality of voxels of a laser point cloud, the laser point cloud comprising: laser points corresponding to a target object;

[0013] A first detection unit is configured to perform target detection based on the first dimensional feature to obtain a first target detection result;

[0014] A cutting unit, configured to cut a corresponding area from the laser point cloud according to the first target detection result;

[0015] A second detection unit is configured to determine a second dimensional feature corresponding to a plurality of regional voxels of the corresponding region, and determine a second target detection result of the corresponding region in a second dimension based on the second dimensional feature;

[0016] The three-dimensional detection result acquisition unit is configured to obtain a three-dimensional detection result of the target object based on the first target detection result and the second target detection result.

[0017] The target detection method and device provided in the embodiments of the present application realizes that when obtaining the 3D target detection result, the corresponding detection results of the target object are gradually obtained, and the 3D target detection result of the target object is finally obtained. Compared with obtaining the 3D target detection result of the target object based on the relevant features of all laser points in the laser point cloud in three dimensions, the amount of calculation consumed in obtaining the 3D target detection result is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] Figure 1 A flow chart of a target detection method provided by an embodiment of the present application is shown;

[0020] Figure 2 The structure block diagram of the target detection device provided by the embodiment of the present application is shown;

[0021] Figure 3 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] Figure 1 A flow chart of a target detection method provided in an embodiment of the present application is shown, and the method includes:

[0025] Step 101, determining first dimensional features corresponding to a plurality of voxels of a laser point cloud.

[0026] In an embodiment of the present application, the laser point cloud includes: laser points of the target object scanned by the laser radar.

[0027] The laser point cloud is in frames. The laser radar scans one circle and obtains one frame of laser point cloud.

[0028] The purpose of 3D object detection is to obtain the type of the target object, the three-dimensional bounding box of the target object, etc. The three-dimensional bounding box of the target object surrounds the target object in three-dimensional space. The target object can be located in three-dimensional space by obtaining the three-dimensional bounding box of the target object.

[0029] During the driving process of an object equipped with a laser radar, such as a vehicle or a robot, the laser radar can scan multiple objects around the object equipped with the laser radar, and each object scanned by the laser radar can be called a target object.

[0030] Therefore, the laser point cloud targeted by steps 101 - 105 does not specifically refer to a certain frame of laser point cloud, and the target object targeted by steps 101 - 105 does not specifically refer to a certain object.

[0031] Steps 101-105 are exemplary descriptions of operations performed in the process of obtaining the type of a target object and a three-dimensional bounding box of a target object using a frame of laser point cloud for 3D target detection.

[0032] For any frame of laser point cloud collected for 3D target detection, the type of each target object and the three-dimensional bounding box of each target object can be obtained through steps 101-105.

[0033] In the embodiment of the present application, the plurality of voxels of the laser point cloud are obtained by dividing the laser point cloud into voxels.

[0034] The three-dimensional area occupied by the laser point cloud in the three-dimensional space has length, width, and height information. Each laser point in the laser point cloud is in the three-dimensional area occupied by the laser point cloud in the three-dimensional space.

[0035] In the embodiment of the present application, the laser point cloud can be voxelized to obtain a plurality of voxels of the laser point cloud. The voxel of the laser point cloud is equivalent to a cube located in the three-dimensional area occupied by the laser point cloud in the three-dimensional space. The size of each voxel of the laser point cloud is a unit size. For example, each voxel of the laser point cloud has a length of 0.1m in the world coordinate system, a width of 0.1m in the world coordinate system, and a height of 0.1m in the world coordinate system.

[0036] In an embodiment of the present application, when performing target detection on a target object, the first dimensional features corresponding to multiple voxels of the laser point cloud can be determined. The associated features of each voxel can be determined. For example, the associated features of a voxel may include: the median of the X-axis coordinate value of the three-dimensional coordinates of all laser points in the voxel in the world coordinate system, the median of the Y-axis coordinate value of the three-dimensional coordinates of all laser points in the voxel in the world coordinate system, and the median of the Z-axis coordinate value of the three-dimensional coordinates of all laser points in the voxel in the world coordinate system. Then, the first dimensional features corresponding to multiple voxels can be determined based on the association relationship between the first dimensional features and the associated features of each voxel. The first dimensional features may include: the length feature of the target object and the width feature of the target object.

[0037] In some embodiments, determining the first dimensional features corresponding to multiple voxels of the laser point cloud includes: separately determining the first associated features of each of the multiple voxels of the laser point cloud; performing a two-dimensional convolution operation based on the first associated features of each of the multiple voxels of the laser point cloud to obtain the first dimensional features corresponding to the multiple voxels.

[0038] The feature extraction network for extracting the first dimensional feature can perform a two-dimensional convolution operation on the first associated features of the plurality of voxels of the laser point cloud to obtain the first dimensional features corresponding to the plurality of voxels. The feature extraction network for extracting the first dimensional feature can include a plurality of two-dimensional convolution layers and a plurality of pooling layers.

[0039] The first associated feature of each voxel in the plurality of voxels of the laser point cloud may be used as an input to a feature extraction network for extracting first dimensional features.

[0040] When the first associated features of the plurality of voxels of the laser point cloud are subjected to a two-dimensional convolution operation by the feature extraction network for extracting the first dimensional features, a feature map representing the first associated features of the plurality of voxels of the laser point cloud can be first generated. Then, the feature map representing the first associated features of the plurality of voxels of the laser point cloud is input into the feature extraction network for extracting the first dimensional features.

[0041] When the first dimensional feature includes the length feature of the target object and the width feature of the target object, the input of the feature extraction network for extracting the first dimensional feature only includes the relevant features in the X dimension, the relevant features in the Y dimension, and the relevant features in the reflection intensity dimension. The feature extraction network for extracting the first dimensional feature performs a two-dimensional convolution operation on the first associated features of multiple voxels of the laser point cloud, which is equivalent to performing a convolution in the XY plane, and extracts the first dimensional feature of the target object in the case of looking down at the target object, that is, in a bird's-eye view.

[0042] In some embodiments, for each voxel of multiple voxels of the laser point cloud, the first associated feature of the voxel includes: the average value of the X-axis coordinate value of the three-dimensional coordinate of the laser point in the voxel in the world coordinate system, the average value of the Y-axis coordinate value of the three-dimensional coordinate of the laser point in the voxel in the world coordinate system, and the average value of the reflection intensity of the laser point in the voxel.

[0043] For each voxel in the multiple voxels of the laser point cloud, the sum of the X-axis coordinate values ​​of the three-dimensional coordinates of all the laser points in the voxel in the world coordinate system can be calculated, and the calculated sum can be divided by the number of laser points in the voxel to obtain the average value of the X-axis coordinate values ​​of the three-dimensional coordinates of the laser points in the voxel in the world coordinate system. The sum of the Y-axis coordinate values ​​of the three-dimensional coordinates of all the laser points in the voxel in the world coordinate system can be calculated, and the calculated sum can be divided by the number of laser points in the voxel to obtain the average value of the Y-axis coordinate values ​​of the three-dimensional coordinates of the laser points in the voxel in the world coordinate system. The sum of the reflection intensities of all the laser points in the voxel can be calculated, and the calculated sum can be divided by the number of laser points in the voxel to obtain the average value of the reflection intensities of the laser points in the voxel.

[0044] Step 102: Perform target detection based on the first dimensional feature to obtain a first target detection result.

[0045] The neural network used to obtain the first target detection result of the target object can perform target detection based on the first dimensional feature to obtain the first target detection result.

[0046] The first dimensional feature may include a length feature of the target object and a width feature of the target object. The first target detection result may include the type of the target object, the length of the three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, and the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system. The first dimensional feature may be input into a neural network for obtaining the first target detection result of the target object, and the length of the three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, and the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system output by the neural network for obtaining the first target detection result of the target object are obtained.

[0047] The neural network used to obtain the first target detection result of the target object can be trained in advance using training samples and labeled data of the training samples. The training samples are first dimensional features, and the labeled data are detection results as target outputs. Each training session inputs the training samples into the neural network used to obtain the first target detection result of the target object, obtains the predicted detection result output by the neural network used to obtain the first target detection result of the target object, calculates the loss between the predicted detection result and the detection result as the target output, and performs back propagation to update the parameter values ​​of the parameters of the neural network used to obtain the first target detection result of the target object according to the loss between the predicted detection result and the detection result as the target output.

[0048] In an embodiment of the present application, the first dimensional feature includes: a length feature of the target object, a width feature of the target object; and target detection is performed based on the first dimensional feature to obtain a first target detection result including: inputting a feature map representing the first dimensional feature into a target detection network to obtain the length of a three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, and the orientation of the target object; inputting the feature map representing the first dimensional feature into a classification network to obtain the type of the target object output by the classification network.

[0049] In an embodiment of the present application, a neural network used to obtain a first target detection result of a target object may include a classification network and a target detection network.

[0050] The feature graph representing the first dimensional feature output by the feature extraction network for extracting the first dimensional feature can be input into the classification network and the target detection network respectively. The feature graph representing the first dimensional feature output by the feature extraction network for extracting the first dimensional feature can also be further subjected to dimensionality reduction processing using a convolution layer including a 1x1 convolution kernel. Then, the feature graph output by the convolution layer including a 1x1 convolution kernel is input into the classification network and the target detection network respectively.

[0051] The classification network predicts the type of the target object based on the input feature map. The target detection network predicts the length of the target object's 3D bounding box, the width of the target object's 3D bounding box, the X-axis coordinate value of the 3D coordinate of the center point of the target object's 3D bounding box in the world coordinate system, and the Y-axis coordinate value of the 3D coordinate of the center point of the target object's 3D bounding box in the world coordinate system based on the input feature map.

[0052] The classification network outputs identifications of multiple candidate types of the target object and the probabilities corresponding to the candidate types. The type of the target object can be determined by NMS (non-maximum suppression) based on the identifications of multiple candidate types of the target object and the probabilities corresponding to the candidate types. The network used to obtain the first target detection result outputs multiple candidate two-dimensional bounding boxes that surround the target object on the XY plane of the world coordinate system. The two-dimensional bounding box of the target object on the XY plane of the world coordinate system can be determined by NMS based on multiple candidate two-dimensional bounding boxes that surround the target object on the XY plane of the world coordinate system. The two-dimensional bounding box of the target object is the part of the three-dimensional bounding box of the target object that is located on the XY plane of the world coordinate system. Therefore, the length of the two-dimensional bounding box of the target object on the XY plane of the world coordinate system can be used as the length of the three-dimensional bounding box of the target object, and the width of the two-dimensional bounding box of the target object on the XY plane of the world coordinate system can be used as the width of the three-dimensional bounding box of the target object. The X-axis coordinate value of the center point of the two-dimensional bounding box of the target object on the XY plane of the world coordinate system in the world coordinate system can be used as the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system. The Y-axis coordinate value of the center point of the two-dimensional bounding box of the target object on the XY plane of the world coordinate system in the world coordinate system can be used as the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system.

[0053] Step 103: Cut out a corresponding area from the laser point cloud according to the first target detection result.

[0054] In an embodiment of the present application, when a corresponding area is cut from a laser point cloud according to a first target detection result, the corresponding area in the three-dimensional space can be first determined according to the first target detection result. The corresponding area is an area in the three-dimensional space associated with the target object. The first target detection result may include the length of the three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, and the type of the target object. The length of the corresponding area may be the length of the three-dimensional bounding box of the target object, and the width of the corresponding area may be the width of the three-dimensional bounding box of the target object. The preset height corresponding to the type of the target object may be determined according to the type of the target object. The preset height corresponding to the type of the target object may be greater than the usual actual height of the target object. The height of the corresponding area in the three-dimensional space may be the preset height corresponding to the type of the target object.

[0055] In the embodiment of the present application, after the corresponding area in the three-dimensional space is determined, the corresponding area is cut from the laser point cloud. Thus, only the corresponding area is retained, and the area outside the corresponding area is removed. Accordingly, only the laser points in the corresponding area are retained, and only the laser points in the corresponding area are used to obtain the calculation of the second target detection result, which greatly reduces the amount of calculation compared to directly involving the entire laser point cloud in the calculation.

[0056] Step 104 , determining second dimensional features corresponding to a plurality of regional voxels of the corresponding region, and determining a second target detection result of the corresponding region in the second dimension based on the second dimensional features.

[0057] In the embodiment of the present application, the laser points in the corresponding area can be voxelized to obtain a plurality of regional voxels of the corresponding area. The voxels in the corresponding area are referred to as regional voxels.

[0058] The size of each region voxel is a unit size. For example, the length of each region voxel in the world coordinate system is 0.1m, the width in the world coordinate system is 0.1m, and the height in the world coordinate system is 0.1m.

[0059] In an embodiment of the present application, the second dimensional features corresponding to the multiple regional voxels of the corresponding area can be determined, and the second dimensional features corresponding to the multiple regional voxels of the corresponding area include the height features of the target object. The associated features of each regional voxel can be determined. For example, for each voxel in the multiple regional voxels of the corresponding area, the associated features of the voxel include: the average value of the Z-axis coordinate values ​​of the three-dimensional coordinates of all laser points in the voxel in the world coordinate system. Then, the second dimensional features corresponding to the multiple regional voxels can be determined based on the association relationship between the second dimensional features and the associated features of each regional voxel. The second dimensional features can be the height features corresponding to the multiple regional voxels.

[0060] In some embodiments, determining the second dimensional features corresponding to multiple regional voxels of the corresponding area includes: determining the second associated features of each of the multiple regional voxels of the corresponding area respectively; based on the second associated features of each regional voxel, performing a two-dimensional convolution operation of the first plane and a two-dimensional convolution operation of the second plane respectively, to obtain the second dimensional features corresponding to the multiple voxels of the corresponding area, the second dimensional features including the first plane associated features and the second plane associated features.

[0061] In an embodiment of the present application, when determining the second dimensional features corresponding to the multiple regional voxels of the corresponding region, the second associated features of each of the multiple regional voxels of the corresponding region can be determined respectively. When a two-dimensional convolution operation of the first plane and a two-dimensional convolution operation of the second plane are performed respectively based on the second associated features of each regional voxel to obtain the second dimensional features corresponding to the multiple voxels of the corresponding region, a feature extraction network for extracting the second dimensional features can be used to perform a two-dimensional convolution operation on the second associated features of the multiple regional voxels of the corresponding region to obtain the second dimensional features.

[0062] The second associated features of the plurality of regional voxels of the corresponding region may be used as inputs of a feature extraction network for extracting features of the second dimension.

[0063] The input of the feature extraction network for extracting the second dimension features only includes the relevant features in the Z dimension and the relevant features in the reflection intensity dimension.

[0064] In an embodiment of the present application, the first plane may be an XZ plane of a world coordinate system, and the second plane may be a YZ plane of a world coordinate system. Based on the second associated features of each regional voxel, a two-dimensional convolution operation of the first plane and a two-dimensional convolution operation of the second plane are performed respectively, that is, a two-dimensional convolution operation is performed on the XZ plane and a two-dimensional convolution operation is performed on the YZ plane respectively, and features related to the height of the target object are extracted from the side of the target object and the front of the target object respectively. The features obtained by performing the two-dimensional convolution operation on the first plane can be referred to as first plane associated features, and the features obtained by performing the two-dimensional convolution operation on the second plane can be referred to as second plane associated features. The second dimensional features are composed of the first plane associated features and the second plane associated features, that is, the features related to the height of the target object extracted from the side and the features related to the height of the target object extracted from the front constitute the second dimensional features.

[0065] When the feature extraction network for extracting the second dimensional feature performs a two-dimensional convolution operation on the features of the plurality of regional voxels of the corresponding region, a feature map representing the associated features of the plurality of regional voxels of the corresponding region can be first generated. Then, the feature map representing the associated features of the plurality of regional voxels of the corresponding region is input into the feature extraction network for extracting the second dimensional feature.

[0066] The feature extraction network for extracting second dimensional features may include multiple two-dimensional convolutional layers and multiple pooling layers.

[0067] The feature extraction network for extracting the second dimensional features can perform a two-dimensional convolution operation on the input feature map, and the feature extraction network for extracting the second dimensional features can output a feature map representing the second dimensional features.

[0068] In some embodiments, for each regional voxel of multiple regional voxels in the corresponding area, the second associated feature of the regional voxel includes: the average value of the Z-axis coordinate value of the three-dimensional coordinate of the laser point in the regional voxel in the coordinate system associated with the target object, and the average value of the reflection intensity of the laser point in the regional voxel, wherein the direction of the X-axis or Y-axis of the coordinate system associated with the target object is consistent with the direction of the target object.

[0069] For each of the multiple regional voxels of the corresponding region, the sum of the Z-axis coordinate values ​​of the three-dimensional coordinates of all the laser points in the voxel in the coordinate system associated with the target object can be calculated, and the calculated sum is divided by the number of laser points in the voxel to obtain the average value of the Z-axis coordinate values ​​of the three-dimensional coordinates of the laser points in the voxel in the coordinate system associated with the target object. For each of the multiple regional voxels of the corresponding region, the sum of the reflection intensities of all the laser points in the regional voxel can be calculated, and the calculated sum is divided by the number of laser points in the voxel to obtain the average value of the reflection intensities of the laser points in the regional voxel.

[0070] In the embodiment of the present application, the coordinate origin of the coordinate system associated with the target object is the XY plane in the world coordinate system, and the coordinate origin of the coordinate system associated with the target object has the X-axis coordinate value and the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system. The three-dimensional coordinate of the laser point located in the corresponding area in the world coordinate system can be transformed into the three-dimensional coordinate in the coordinate associated with the target object.

[0071] The direction of one of the X-axis or Y-axis in the coordinate system associated with the target object is consistent with the orientation of the target object. The direction of one of the X-axis or Y-axis in the coordinate system associated with the target object is consistent with the orientation of the target object, and the other of the X-axis or Y-axis is perpendicular to the one of the coordinate axes consistent with the orientation of the target object.

[0072] For example, the target object is a vehicle, and the coordinate origin of the coordinate system associated with the vehicle is the XY plane in the world coordinate system, and the coordinate origin of the coordinate system associated with the vehicle has the X-axis coordinate value and the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the vehicle in the world coordinate system. The orientation of one of the X-axis or Y-axis in the coordinate system associated with the vehicle is consistent with the orientation of the vehicle, and the other of the X-axis or Y-axis is perpendicular to the one of the coordinate axes consistent with the orientation of the vehicle.

[0073] In an embodiment of the present application, the second target detection result of the corresponding area in the second dimension may include the height of the three-dimensional bounding box of the target object and the Z-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system.

[0074] In an embodiment of the present application, a neural network for obtaining a second target detection result of a target object may determine a second target detection result of a corresponding area in the second dimension based on a second dimensional feature.

[0075] The second target detection result of the corresponding area in the second dimension may include the height of the three-dimensional bounding box of the target object and the Z-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system.

[0076] The second dimensional feature can be input into a neural network for obtaining a second target detection result of the target object to obtain the height of the three-dimensional bounding box of the target object and the Z-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system output by the neural network for obtaining the second target detection result of the target object.

[0077] The neural network used to obtain the second target detection result of the target object can be trained in advance using training samples and labeled data of the training samples. The training samples are second dimensional features, and the labeled data are detection results as target outputs. Each training session inputs the training samples into the neural network used to obtain the second target detection result of the target object, obtains the predicted detection result output by the neural network used to obtain the second target detection result of the target object, calculates the loss between the predicted detection result and the detection result as the target output, and performs back propagation to update the parameter values ​​of the parameters of the neural network used to obtain the second target detection result of the target object according to the loss between the predicted detection result and the detection result as the target output.

[0078] In an embodiment of the present application, based on the second dimensional features, determining the second target detection result of the corresponding area in the second dimension includes: performing position detection according to the first plane association features and the second plane association features; based on the result of the position detection, obtaining the height of the three-dimensional bounding box of the target object and the Z-axis coordinate value of the three-dimensional coordinates of the center point of the three-dimensional bounding box of the target object in the world coordinate system.

[0079] In an embodiment of the present application, when position detection is performed based on the first plane association feature and the second plane association feature, a feature map representing the first plane association feature and a feature map representing the second plane association feature can be generated, and the feature map representing the first plane association feature and the feature map representing the second plane association feature are input into a regressor, such as a regressor including a 1x1 convolution kernel, and the regressor generates a result of position detection. Based on the result of position detection, the regressor predicts the height of the three-dimensional bounding box of the target object and the coordinate value of the Z axis of the three-dimensional coordinates of the center point of the three-dimensional bounding box of the target object in the world coordinate system.

[0080] Step 105: Obtain a three-dimensional detection result of the target object based on the first target detection result and the second target detection result.

[0081] The first target detection result may include the type of the target object, the length of the three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, and the direction of the target object. The second target detection result may include the height of the three-dimensional bounding box of the target object and the Z-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system.

[0082] When the three-dimensional detection result of the target object is obtained based on the first target detection result and the second target detection result, the length of the three-dimensional bounding box of the target object in the first detection result, the width of the three-dimensional bounding box of the target object, and the height of the three-dimensional bounding box of the target object in the second detection result can be judged according to the prior three-dimensional bounding box information of the type of the target object as the length, width, and height of the three-dimensional bounding box of the target object in the three-dimensional detection result of the target object. The prior three-dimensional bounding box information indicates the range of the three-dimensional bounding box of the object of the type of the target object in terms of length, width, and height. If the length of the three-dimensional bounding box of the target object in the first detection result and the width of the three-dimensional bounding box of the target object are both within the corresponding range indicated by the prior three-dimensional bounding box information, the three-dimensional detection result of the target object, i.e., the 3D target detection result, includes: the length and width of the three-dimensional bounding box of the target object in the first detection result, and the X-axis coordinate value and the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system. If the height of the three-dimensional bounding box of the target object in the second detection result is within the corresponding range indicated by the prior three-dimensional bounding box information, the 3D target detection result of the target object includes: the height of the three-dimensional bounding box of the target object in the second detection result and the coordinate value of the Z axis of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system. The 3D target detection result of the target object also includes: the type of the target object and the orientation of the target object.

[0083] In an embodiment of the present application, obtaining a three-dimensional detection result of the target object based on the first target detection result and the second target detection result includes: combining the first target detection result and the second target detection result to obtain a three-dimensional detection result of the target object.

[0084] After obtaining the second target detection result of the target object, the first target detection result of the target object and the second target detection result of the target object can be directly combined to obtain a 3D target detection result of the target object. The 3D target detection result of the target object may include: the type of the target object, the orientation of the target object, the length of the three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, the height of the three-dimensional bounding box of the target object, the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, the Y-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, and the Z-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system.

[0085] Please refer to Figure 2 , which shows a structural block diagram of the target detection device provided by the embodiment of the present application. The device includes: a determination unit 201, a first detection unit 202, a cutting unit 203, a second detection unit 204, and a three-dimensional detection result acquisition unit 205.

[0086] The determination unit 201 is configured to determine first dimensional features corresponding to a plurality of voxels of a laser point cloud, the laser point cloud comprising: laser points corresponding to a target object;

[0087] The first detection unit 202 is configured to perform target detection based on the first dimensional feature to obtain a first target detection result;

[0088] The cutting unit 203 is configured to cut a corresponding area from the laser point cloud according to the first target detection result;

[0089] The second detection unit 204 is configured to determine a second dimensional feature corresponding to a plurality of regional voxels of the corresponding region, and determine a second target detection result of the corresponding region in the second dimension based on the second dimensional feature;

[0090] The three-dimensional detection result acquisition unit 205 is configured to obtain a three-dimensional detection result of the target object based on the first target detection result and the second target detection result.

[0091] In some embodiments, the determination unit 201 is further configured to respectively determine the first associated features of each of the multiple voxels of the laser point cloud; perform a two-dimensional convolution operation based on the first associated features of each of the multiple voxels of the laser point cloud to obtain the first dimensional features corresponding to the multiple voxels.

[0092] In some embodiments, the first associated feature of the voxel includes: the average value of the X-axis coordinate value of the three-dimensional coordinate of the laser point in the voxel in the world coordinate system, the average value of the Y-axis coordinate value of the three-dimensional coordinate of the laser point in the voxel in the world coordinate system, and the average value of the reflection intensity of the laser point in the voxel.

[0093] In some embodiments, the first dimensional features include: a length feature of the target object, a width feature of the target object; the first detection unit 202 is configured to input a feature map representing the first dimensional features into a target detection network, and obtain the length of a three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, the Y-axis coordinate value of the three-dimensional coordinate, and the orientation of the target object output by the target detection network; and input the feature map representing the first dimensional features into a classification network, and obtain the type of the target object output by the classification network.

[0094] In some embodiments, the second detection unit 204 is further configured to respectively determine the second association features of each of the multiple regional voxels of the corresponding area; based on the second association features, perform a two-dimensional convolution operation of the first plane and a two-dimensional convolution operation of the second plane to obtain a second dimensional feature corresponding to the multiple regional voxels of the corresponding area, and the second dimensional feature includes a first plane association feature and a second plane association feature.

[0095] In some embodiments, the second associated features of the region voxels include: the average value of the Z-axis coordinate values ​​of the three-dimensional coordinates of the laser points in the region voxels in the coordinate system associated with the target object, and the average value of the reflection intensity of the laser points in the region voxels, wherein the direction of the X-axis or Y-axis of the coordinate system associated with the target object is consistent with the direction of the target object.

[0096] In some embodiments, the second detection unit 204 is further configured to perform position detection based on the first plane association feature and the second plane association feature; based on the result of the position detection, the height of the three-dimensional bounding box of the target object and the Z-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system are obtained.

[0097] In some embodiments, the three-dimensional detection result acquisition unit 205 is further configured to combine the first target detection result and the second target detection result to obtain a three-dimensional detection result of the target object.

[0098] Figure 3 3 is a block diagram of an electronic device provided by the present application. The electronic device includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332 for storing instructions executable by the processing component 322, such as an application. The application stored in the memory 332 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 322 is configured to execute instructions to perform the above method.

[0099] The electronic device may also include a power supply component 326 configured to perform power management of the electronic device, a wired or wireless network interface 350 configured to connect the electronic device to a network, and an input / output (I / O) interface 358. The electronic device may operate based on an operating system stored in the memory 332, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™ or the like.

[0100] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by an electronic device to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0101] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0102] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A target detection method, characterized in that: The method comprises: Determining first dimensional features corresponding to a plurality of voxels of a laser point cloud, the laser point cloud comprising: laser points corresponding to a target object; Performing target detection based on the first dimensional features to obtain a first target detection result; According to the first target detection result, cutting a corresponding area from the laser point cloud; Determining second dimensional features corresponding to a plurality of regional voxels of the corresponding region, and determining a second target detection result of the corresponding region in the second dimension based on the second dimensional features; Obtaining a three-dimensional detection result of the target object based on the first target detection result and the second target detection result; The determining of first-dimensional features corresponding to a plurality of voxels of the laser point cloud includes: determining a first correlation feature for each of a plurality of voxels of the laser point cloud; Performing a two-dimensional convolution operation based on the first associated features to obtain first dimensional features corresponding to the plurality of voxels; The corresponding area is an area in three-dimensional space associated with the target object; The first target detection result includes the length and width of the three-dimensional bounding box of the target object and the type of the target object.

2. The method according to claim 1, characterized in that The first associated feature of the voxel includes: the average value of the X-axis coordinate value of the laser point in the voxel in the world coordinate system, the average value of the Y-axis coordinate value of the laser point in the voxel in the world coordinate system, and the average value of the reflection intensity of the laser point in the voxel.

3. The method according to any one of claims 1 to 2, characterized in that: The first dimensional features include: a length feature of the target object, a width feature of the target object; and The performing target detection based on the first dimensional feature to obtain a first target detection result includes: Inputting the feature map representing the first dimensional feature into the target detection network, obtaining the length of the three-dimensional bounding box of the target object, the width of the three-dimensional bounding box of the target object, the X-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system, the Y-axis coordinate value of the three-dimensional coordinate, and the orientation of the target object output by the target detection network; The feature map representing the first dimension feature is input into the classification network to obtain the type of target object output by the classification network.

4. The method according to any one of claims 1 to 3, characterized in that: Determining the second dimensional features corresponding to the plurality of regional voxels in the corresponding region includes: respectively determining a second correlation feature of each of the plurality of regional voxels in the corresponding region; Based on the second association feature, a two-dimensional convolution operation of the first plane and a two-dimensional convolution operation of the second plane are performed respectively to obtain a second dimensional feature corresponding to multiple regional voxels in the corresponding area, where the second dimensional feature includes the first plane association feature and the second plane association feature.

5. The method according to claim 4, characterized in that The second associated features of the regional voxels include: the average value of the Z-axis coordinate values ​​of the laser points in the regional voxels in the coordinate system associated with the target object, and the average value of the reflection intensity of the laser points in the regional voxels, wherein the direction of the X-axis or Y-axis of the coordinate system associated with the target object is consistent with the direction of the target object.

6. The method according to claim 4, characterized in that The determining, based on the second dimensional feature, a second target detection result of the corresponding area in the second dimension includes: Performing position detection according to the first plane association feature and the second plane association feature; Based on the result of the position detection, the height of the three-dimensional bounding box of the target object and the Z-axis coordinate value of the three-dimensional coordinate of the center point of the three-dimensional bounding box of the target object in the world coordinate system are obtained.

7. The method according to any one of claims 1 to 6, characterized in that: Obtaining a three-dimensional detection result of the target object based on the first target detection result and the second target detection result includes: The first target detection result and the second target detection result are combined to obtain a three-dimensional detection result of the target object.

8. The method according to any one of claims 1 to 7, characterized in that: The target object is a vehicle.

9. A target detection device, characterized in that: The device comprises: a determining unit configured to determine first dimensional features corresponding to a plurality of voxels of a laser point cloud, the laser point cloud comprising: laser points corresponding to a target object; a first detection unit, configured to perform target detection based on the first dimensional feature to obtain a first target detection result; a cropping unit configured to crop a corresponding area from the laser point cloud according to a first target detection result; a second detection unit configured to determine a second dimensional feature corresponding to a plurality of regional voxels of the corresponding region, and determine a second target detection result of the corresponding region in the second dimension based on the second dimensional feature; a three-dimensional detection result acquiring unit, configured to obtain a three-dimensional detection result of the target object based on the first target detection result and the second target detection result; The determining unit is further configured to respectively determine a first associated feature for each of the plurality of voxels of the laser point cloud; Performing a two-dimensional convolution operation based on the first associated feature of each of the multiple voxels in the laser point cloud to obtain first dimensional features corresponding to the multiple voxels; The corresponding area is an area in three-dimensional space associated with the target object; The first target detection result includes the length and width of the three-dimensional bounding box of the target object and the type of the target object.

10. An electronic device comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8. 11 . A storage medium, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to claim 1 .

Citation Information

Patent Citations

  • Unmanned ship false target detection method based on three-dimensional laser radar

    CN108562913A

  • Sample data labeling method and device for machine learning

    CN110136273A

  • Three-dimensional target detection method and device, computer equipment and storage medium

    CN111199206A

  • Vehicle target detection method and device, computer equipment and storage medium

    CN111401264A

  • Obstacle detection method and device and storage medium

    CN111598034A