A target detection method, device, terminal and computer readable storage medium
By adaptively adjusting the lidar point cloud data and extracting multi-scale features from the target detection model, the problem of low generalization performance of point cloud target detection methods is solved, achieving higher detection accuracy and adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2022-12-29
- Publication Date
- 2026-06-05
AI Technical Summary
Existing point cloud target detection methods have low generalization performance, and the differences between the actual scenarios during model training and inference phases lead to poor detection results.
By acquiring LiDAR point cloud data, adaptive adjustments are made to obtain effective point cloud data. A target detection model is then used for voxelization, feature encoding, compression, and multi-scale feature extraction. Target category detection and location regression are performed in conjunction with two-dimensional feature maps. Focal Loss and Smooth L1 loss functions are used for model training, and data augmentation is performed to improve generalization performance.
It improves the generalization performance and accuracy of object detection and enhances the model's adaptability in different scenarios.
Smart Images
Figure CN116299312B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a target detection method, apparatus, terminal, and computer-readable storage medium. Background Technology
[0002] Point cloud object detection is a crucial task in fields such as autonomous driving and vehicle-to-everything (V2X) communication. It requires processing and analyzing point cloud data collected from LiDAR to obtain information such as the category, position, size, and orientation of targets of interest in the scene. Point cloud object detection is a vital component of perception and is of great significance for ensuring road safety.
[0003] Current point cloud object detection methods mostly rely on the scene, requiring repeated data collection and model training. The difference between the actual scene in the model training and inference stages makes it difficult to directly apply the trained model to inference. Summary of the Invention
[0004] The main technical problem solved by this invention is to provide a target detection method, device, terminal and computer-readable storage medium, thereby solving the problem of low generalization performance of existing target detection methods.
[0005] To solve the above-mentioned technical problems, the first technical solution adopted by the present invention is: to provide a target detection method, the target detection method comprising:
[0006] Acquire LiDAR point cloud data containing the target object;
[0007] The lidar point cloud data is adaptively adjusted to obtain effective point cloud data; the effective point cloud data is the point cloud data within a set area in the lidar point cloud data; the preset area is determined based on the position coordinates of each laser point in the lidar point cloud data.
[0008] An object detection model is used to detect valid point cloud data to obtain object detection information, which includes object detection bounding boxes and object detection categories.
[0009] Specifically, an object detection model is used to detect valid point cloud data to obtain object detection information, which includes object detection bounding boxes and object detection categories, including:
[0010] A three-dimensional feature map is obtained by performing voxelization and voxel feature encoding on effective point cloud data using an object detection model.
[0011] The 3D feature map corresponding to the effective point cloud data is compressed to obtain the 2D bird's-eye view features;
[0012] Multi-scale feature extraction is performed on the features of the two-dimensional bird's-eye view to obtain multiple two-dimensional feature maps of different resolutions corresponding to the effective point cloud data;
[0013] Target category detection and target location regression are performed based on two-dimensional feature maps to obtain the target detection box and target detection category corresponding to the target object.
[0014] Specifically, target category detection and target location regression are performed based on two-dimensional feature maps to obtain the target detection box and target detection category corresponding to the target object, including:
[0015] Position and category prediction are performed based on the two-dimensional feature map to obtain the category probability value and offset corresponding to each anchor box; the anchor box is the frame configured at each pixel position in the two-dimensional feature map;
[0016] The preset category corresponding to the largest category probability value is used as the target detection category;
[0017] The target detection box is determined based on the position information of the anchor box corresponding to the maximum category probability value and the offset.
[0018] Specifically, target category detection and target location regression are performed based on two-dimensional feature maps to obtain the target detection box and target detection category corresponding to the target object, including:
[0019] If the proportion of the target object in the two-dimensional bird's-eye view feature map exceeds a preset ratio, then the two-dimensional feature map with the first resolution is selected for target category detection and target location regression.
[0020] If the proportion of the target object in the two-dimensional bird's-eye view feature map does not exceed a preset ratio, then the two-dimensional feature map with the second resolution is selected for target category detection and target location regression; wherein, the second resolution is higher than the first resolution.
[0021] Among them, target detection methods also include:
[0022] If there are multiple target detection boxes, the target detection boxes are filtered based on non-maximum suppression.
[0023] This involves adaptively adjusting the lidar point cloud data to obtain effective point cloud data, including:
[0024] Based on the position coordinates of each laser point in the lidar point cloud data, the point cloud distribution histogram of the lidar point cloud data in each preset direction is determined.
[0025] Select all point cloud data within a preset range from the origin in each preset direction as valid point cloud data; the origin is the intersection of the backward extensions of each preset direction.
[0026] Before the step of using an object detection model to detect valid point cloud data and obtain object detection information, the following steps are also included:
[0027] Training the object detection model; specifically including:
[0028] Acquire training point cloud data; the training point cloud data is associated with bounding boxes and label categories of the included targets; the bounding boxes include the center point position, the size of the bounding box, and the yaw angle;
[0029] The target detection model sequentially performs voxelization, voxel feature encoding, and compression on the training point cloud data to obtain a two-dimensional feature map corresponding to the training point cloud data.
[0030] Anchor boxes are configured for each pixel position in the 2D feature map;
[0031] Based on the two-dimensional feature map, position and category prediction are performed to obtain the category prediction value and prediction offset corresponding to each anchor box;
[0032] Based on the predicted category value and predicted offset corresponding to the anchor box, the predicted bounding box and predicted category of the target are determined;
[0033] The target detection model is iteratively trained based on the category error value calculated from the category prediction value corresponding to the point cloud data used for training, and the positional error value between the predicted bounding box and the labeled bounding box.
[0034] Training the object detection model also includes:
[0035] The anchor box category is determined based on the intersection-union ratio between the anchor box and the label box; the anchor box categories include positive sample anchor boxes and negative sample anchor boxes.
[0036] The target detection model is iteratively trained based on the category error value calculated from the category prediction value corresponding to the training point cloud data and the positional error value between the predicted bounding box and the labeled bounding box, including:
[0037] The Focal Loss function is used to determine the class error value based on the class prediction values of each positive sample anchor box and each negative sample anchor box;
[0038] The Smooth L1 loss function is used based on the positional error between the predicted bounding box and the labeled bounding box corresponding to the target.
[0039] The target detection model is iteratively trained based on the sum of the location error value and the category error value.
[0040] Training the object detection model also includes:
[0041] Data augmentation is performed on the point cloud data used for training; the data augmentation methods include at least one of global rotation, translation, scaling, and masking.
[0042] To solve the above-mentioned technical problems, the second technical solution adopted by the present invention is: to provide a target detection device, the target detection device comprising:
[0043] The acquisition module is used to acquire LiDAR point cloud data containing the target object;
[0044] The preprocessing module is used to adaptively adjust the lidar point cloud data to obtain effective point cloud data. The effective point cloud data is the point cloud data within a set area in the lidar point cloud data. The preset area is determined based on the position coordinates of each laser point in the lidar point cloud data.
[0045] The detection module is used to detect valid point cloud data using an object detection model to obtain object detection information, which includes object detection bounding boxes and object detection categories.
[0046] To solve the above-mentioned technical problems, the third technical solution adopted by the present invention is to provide a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. The processor is used to execute program data to implement the steps in the above-mentioned target detection method.
[0047] To solve the above-mentioned technical problems, the fourth technical solution adopted by the present invention is to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the above-mentioned target detection method.
[0048] The beneficial effects of this invention are as follows: Unlike existing technologies, it provides a target detection method, apparatus, terminal, and computer-readable storage medium. The target detection method includes: acquiring lidar point cloud data containing a target object; adaptively adjusting the lidar point cloud data to obtain effective point cloud data; the effective point cloud data is point cloud data within a predetermined region of the lidar point cloud data; the predetermined region is determined based on the position coordinates of each laser point in the lidar point cloud data; and using a target detection model to detect the effective point cloud data to obtain target detection information, which includes a target detection bounding box and a target detection category. In this application, by adaptively adjusting the lidar point cloud data and adaptively selecting a predetermined region based on the position coordinates of each laser point in the lidar point cloud data within the scene to obtain effective point cloud data, and then performing target detection on the effective point cloud data using a target detection model, the adaptive selection of the predetermined region corresponding to the lidar point cloud data improves the generalization performance of target detection, thereby increasing the accuracy of target detection. Attached Figure Description
[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 This is a flowchart illustrating the target detection method provided by the present invention;
[0051] Figure 2 This is a schematic diagram of a specific embodiment of the target detection method provided by the present invention;
[0052] Figure 3 This is a flowchart illustrating a specific embodiment of the training method for the target detection model provided by the present invention;
[0053] Figure 4 This is the non-adaptive adjustment of lidar point cloud data provided by the present invention;
[0054] Figure 5 This is the effective point cloud data provided by the present invention;
[0055] Figure 6 This is the output result corresponding to the valid point cloud data provided by this invention;
[0056] Figure 7 This is a schematic diagram of the framework of an embodiment of the target detection device provided by the present invention;
[0057] Figure 8 This is a schematic diagram of the framework of an embodiment of the terminal provided by the present invention;
[0058] Figure 9 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided by the present invention. Detailed Implementation
[0059] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0060] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.
[0061] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.
[0062] To enable those skilled in the art to better understand the technical solution of the present invention, the target detection method provided by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0063] Please see Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating the target detection method provided by the present invention; Figure 2 This is a schematic diagram of a specific embodiment of the target detection method provided by the present invention.
[0064] This embodiment provides a target detection method, which includes the following steps.
[0065] S1: Acquire LiDAR point cloud data containing the target object.
[0066] S2: Adaptively adjust the lidar point cloud data to obtain effective point cloud data; the effective point cloud data is the point cloud data within a set area in the lidar point cloud data; the preset area is determined based on the position coordinates of each laser point in the lidar point cloud data.
[0067] S3: Use an object detection model to detect valid point cloud data and obtain object detection information, which includes object detection boxes and object detection categories.
[0068] In this embodiment, multiple lidars are deployed in a preset scenario, and time synchronization is performed on each lidar. The timestamps of each laser point in the lidar point cloud data collected by each lidar are used to correct the laser points. Based on this, spatial synchronization is performed on the data streams collected by different lidars, and the transformation relationship between the coordinate systems corresponding to different lidars is calculated, thus obtaining the corresponding extrinsic parameter matrix between the coordinate systems of each lidar. Time and spatial synchronization of each lidar facilitates the subsequent stitching of the lidar point cloud data collected by each lidar.
[0069] The target object is scanned by various LiDAR sensors, yielding raw LiDAR point cloud data for each sensor. Each laser point in the raw LiDAR point cloud data carries its position coordinates and laser reflection intensity. Abnormal and invalid points in the raw LiDAR point cloud data are removed based on the laser reflection intensity of the laser point. Specifically, laser points with laser reflection intensities lower than a preset value are removed to prevent abnormal distribution of the raw LiDAR point cloud data from affecting subsequent processes. The position coordinates of the laser points in each LiDAR coordinate system are transformed to a unified global coordinate system based on the corresponding extrinsic parameter matrix between the LiDAR coordinate systems, facilitating the stitching of the LiDAR data. The global coordinate system can be the world coordinate system.
[0070] The lidar data corresponding to each lidar is stitched together to obtain the lidar data to be tested.
[0071] Please see Figure 3 , Figure 3 This is a flowchart illustrating a specific embodiment of the training method for the target detection model provided by the present invention.
[0072] In one embodiment, the training method for the object detection model includes the following steps.
[0073] S11: Obtain training point cloud data; the training point cloud data is associated with the bounding boxes and label categories of the included targets; the bounding boxes include the center point position, the size of the bounding box, and the yaw angle.
[0074] Specifically, to avoid the impact of single-channel data loss on the detection accuracy of the target detection model, a drop rate can be set for the LiDAR data corresponding to each LiDAR channel during training. This involves discarding a certain proportion of the LiDAR data corresponding to a particular channel, allowing the target detection network to detect targets in scenes where some LiDAR data is missing without affecting detection accuracy. The drop rate can be set to 0.05.
[0075] After discarding the lidar data corresponding to a certain lidar path, the lidar data corresponding to each lidar are then stitched together to obtain the point cloud data for training.
[0076] The training point cloud data is normalized to a preset range, for example, the preset range is [x min y min , z min x max y max , z max ]. Among them, [x min x max ]、[y min y max ]、[z min , z max [ ] represents the point cloud range corresponding to the x-axis, y-axis, and z-axis in the global coordinate system, respectively. Based on the position coordinates of each laser point in the training point cloud data, the training point cloud data is preprocessed, retaining only laser points within a preset range and deleting those outside the preset range. After this reduction process, the effective point cloud data corresponding to the preset range is obtained.
[0077] To improve the generalization performance and training effectiveness of object detection models, data augmentation can be performed on the point cloud data used for training. Data augmentation methods include at least one of global rotation, translation, scaling, and masking.
[0078] Specifically, augmented point cloud data is obtained by rotating or translating the training point cloud data along a preset direction, or by scaling the training point cloud data with reference to the origin. The augmented point cloud data is also used as the training point cloud data. The rotation angle range, translation range, and scaling rate range of the training point cloud data can be set to [-π / 2, π / 2], [-1.5m, 1.5m], and [0.8, 1.2] respectively, but are not limited to the set range and can be set according to the actual situation.
[0079] In one embodiment, a mask of arbitrary shape is set at the edge of the top view of the point cloud corresponding to the training point cloud data, with the center point of the mask located at any position on the edge of the top view. The size of the mask is randomly generated within the edge region. Specifically, the size of the mask is related to a preset range. For example, the range of the mask's length and width is set to include, but is not limited to, [5m, 20m]. The laser point cloud in the area covered by the mask in the training point cloud data is removed, and the mask area is regarded as the occluded area. The training point cloud data with the occluded area continues to be used as enhanced training point cloud data.
[0080] In this embodiment, the training point cloud data includes bounding boxes and label categories for the targets. The bounding boxes include the center point location, box size, and yaw angle. The label categories are the target categories, such as vehicles and pedestrians. The target bounding boxes and label categories can be manually labeled or detected using other depth models.
[0081] S12: The training point cloud data is sequentially voxelized, voxel feature encoded, and compressed using the object detection model to obtain the two-dimensional feature map corresponding to the training point cloud data.
[0082] Specifically, the training point cloud data is input into the object detection model. The object detection network calculates the differences between the maximum and minimum values of the corresponding laser positions in the training point cloud data along the x, y, and z axes based on the position coordinates of each laser position in the training point cloud data. Then, the length, width, and height of the initial voxel are determined based on the three differences on the x, y, and z axes, thereby automatically creating the initial voxel, which already contains all the laser positions in the training point cloud data.
[0083] The initial voxels are divided into smaller voxels, and the remaining voxels form the 3D model corresponding to the training point cloud data. Specifically, the initial voxels are decomposed into N smaller voxels, and the Bresenham algorithm is used to remove invalid voxels from the initial voxels. The remaining voxels constitute the 3D model corresponding to the training point cloud data.
[0084] Voxel feature encoding is performed on the 3D model corresponding to the training point cloud data to obtain the 3D feature map corresponding to the 3D model. The 3D feature map corresponding to the 3D model is compressed into a 2D bird's-eye view feature. Multi-scale feature extraction is performed on the 2D bird's-eye view feature through the backbone network of the object detection model to obtain 2D feature maps at different scales. Specifically, a feature pyramid structure is used to perform multi-scale feature extraction on the 2D bird's-eye view feature to obtain 2D feature maps at different scales. Among them, the 2D feature maps at different scales are 2D feature maps of different resolutions.
[0085] Based on the target object corresponding to the training point cloud data, select a two-dimensional feature map of a corresponding resolution from multiple two-dimensional feature maps of different resolutions corresponding to the training point cloud data.
[0086] If the target falls within a first size range, a two-dimensional feature map with a first resolution is selected for classification and regression. If the target falls within a second size range, a two-dimensional feature map with a second resolution is selected for classification and regression; the first size range is larger than the second size range, and the second resolution is higher than the first resolution.
[0087] In another embodiment, the resolution of the selected two-dimensional feature map is determined based on the proportion of the target region in the two-dimensional bird's-eye view features. If the proportion of the target region in the two-dimensional bird's-eye view features exceeds a preset proportion, the two-dimensional feature map corresponding to a first resolution is selected for classification and regression; if the proportion of the target region in the two-dimensional bird's-eye view features does not exceed the preset proportion, the two-dimensional feature map corresponding to a second resolution is selected for classification and regression. The second resolution is higher than the first resolution. Specifically, the preset proportion can be set to multiple preset proportion ranges, each corresponding to a preset resolution. When the proportion of the target region in the two-dimensional bird's-eye view features is within a preset proportion range, the two-dimensional feature map corresponding to that range is selected for classification and regression. For example, when the training point cloud data contains targets with a relatively small proportion, such as pedestrians, a high-resolution two-dimensional feature map is selected for classification and regression; when the training point cloud data contains targets with a relatively large proportion, such as vehicles, a low-resolution two-dimensional feature map is selected for classification and regression. The actual resolution selection of the feature map is related to the target size, the preset range of the point cloud, and the accuracy and efficiency requirements. For example, pedestrian and vehicle categories can be predicted in two-dimensional feature maps of W / 4*H / 4 and W / 16*H / 16, respectively. Here, W and H represent the width and height of the two-dimensional bird's-eye view feature map, respectively.
[0088] S13: Configure anchor boxes for each pixel position in the two-dimensional feature map.
[0089] Specifically, anchor boxes are pre-defined on the two-dimensional feature map. Specifically, an anchor box is configured centered on each pixel position in the selected two-dimensional feature map. The length and width of the anchor box are determined according to a preset size corresponding to the category. The anchor box has two directions: horizontal and vertical. In this embodiment, each pixel position is assigned two anchor boxes, one horizontal and the other vertical.
[0090] S14: Based on the two-dimensional feature map, perform position and category prediction to obtain the category prediction value and prediction offset corresponding to each anchor box.
[0091] Specifically, the category of each anchor box is determined based on the intersection-union ratio (IU) between the anchor box corresponding to each pixel position and the target's bounding box. In response to the target object region's proportion in the 2D bird's-eye view features exceeding a preset ratio (e.g., if the target object is a vehicle), anchor boxes with an IU greater than 0.65 are marked as positive sample anchor boxes, and anchor boxes with an IU less than 0.35 are marked as negative sample anchor boxes.
[0092] If the proportion of the target object region in the two-dimensional bird's-eye view features does not exceed a preset ratio, such as if the target object is a pedestrian, then the anchor boxes with an intersection-union ratio greater than 0.45 are marked as positive sample anchor boxes, and the anchor boxes with an intersection-union ratio less than 0.2 are marked as negative sample anchor boxes.
[0093] Based on local features within the positive and negative anchor boxes in the 2D feature map, category prediction and offset prediction are performed respectively, yielding the predicted category value and predicted offset for each anchor box. Specifically, the predicted category value is the probability value that the target contained within the corresponding positive or negative anchor box belongs to a preset category. The predicted offset is the degree of positional offset between the corresponding positive or negative anchor box and the predicted target box.
[0094] S15: Based on the predicted category value and predicted offset corresponding to the anchor box, determine the predicted bounding box and predicted category of the target.
[0095] Specifically, based on the anchor box position corresponding to the maximum probability value and the predicted offset, the predicted bounding box of the target in the two-dimensional feature map is obtained through regression. The predicted bounding box specifically includes the center point position (x, y, z), the box size (w, h, l), and the yaw angle r. Specifically, the regression parameters are normalized, and the yaw angle is regressed using a sine wave to avoid angle conflict. During regression, vehicles with high pixel counts and similar sizes share the same feature extraction layer, and further category differentiation is achieved through fine-grained vehicle type classification to alleviate the problem of insufficient samples for certain categories.
[0096] S16: Iteratively train the target detection model based on the category error value calculated from the category prediction value corresponding to the training point cloud data and the positional error value between the predicted box and the labeled box.
[0097] Specifically, the probability values of the predicted categories corresponding to the positive and negative anchor boxes in the 2D feature map are respectively input into the Focal Loss loss function to calculate the loss value for each anchor box. The category error value is determined by summing the loss values for the positive and negative anchor boxes. The positional error value between the predicted and labeled boxes for the same target is calculated using the SmoothL1 loss function. The target detection model is then iteratively trained based on the sum of the positional and category error values.
[0098] Furthermore, the object detection model can be trained using the Adam optimizer to improve the training of the object detection network and enable faster convergence. The initial learning rate of the object detection model is 0.001.
[0099] In one embodiment, the specific implementation of obtaining lidar point cloud data containing the target object in step S1 includes the following steps.
[0100] Please see Figure 4 , Figure 4 This is the unadaptive LiDAR point cloud data provided by the present invention.
[0101] Laser beams are emitted from lidar sensors positioned within a pre-defined scene to target an object, and point cloud data is acquired from each lidar sensor. The position coordinates of each laser point in the point cloud data acquired by each lidar sensor are then transformed to the same global coordinate system based on an extrinsic parameter matrix.
[0102] In one embodiment, the specific implementation of obtaining effective point cloud data in step S2 includes the following steps.
[0103] Please see Figure 5 , Figure 5 This is the effective point cloud data provided by the present invention.
[0104] Based on the position coordinates of each laser point in the lidar point cloud data, the point cloud distribution histogram of the lidar point cloud data in each preset direction is determined; all point cloud data within a preset range from the origin in each preset direction are selected as valid point cloud data; the origin is the intersection of the backward extensions of each preset direction.
[0105] Specifically, N frames of data are collected from different time periods, and the corresponding LiDAR point cloud data are stitched together to obtain the point cloud data to be detected. N can be 50, 100, etc. The distribution of the point cloud data to be detected is determined based on the position coordinates of each laser point in the point cloud data. To ensure that the distribution of the point cloud data to be detected is consistent with the actual application scenario, the LiDAR acquisition interval should be large enough, especially for mobile platforms such as unmanned vehicles.
[0106] Based on the position coordinates of each laser point in the point cloud data to be detected, histograms of point cloud distribution along the x, y, and z axes are calculated. Data boundaries along the x, y, and z axes are determined using a bisection method. Laser points within 5% of the scene edge are filtered out to obtain valid point cloud data. The data range of the valid point cloud data is [x...]. min y min , z min x max y max , z max ].
[0107] To ensure that the target size is consistent with the physical world, avoid making too many adjustments to the anchor frame size, and instead adjust the scene range.
[0108] When the actual scene can be completely contained within the point cloud range during the training phase, the point cloud during the inference phase is placed at the center of the training coordinate system, and the part without point cloud is considered as occlusion; when the actual application scene exceeds the training range, the scene is cropped and divided into multiple frames, which are then input into the object detection model for object detection.
[0109] In one embodiment, the specific implementation of obtaining target detection information in step S3 includes the following steps.
[0110] Specifically, the effective point cloud data is voxelized and voxel feature encoded using an object detection model to obtain a 3D feature map; the 3D feature map corresponding to the effective point cloud data is compressed to obtain a 2D bird's-eye view feature; multi-scale feature extraction is performed on the 2D bird's-eye view feature to obtain multiple 2D feature maps of different resolutions corresponding to the effective point cloud data; object category detection and object location regression are performed based on the 2D feature map to obtain the object detection box and object detection category corresponding to the object.
[0111] In one specific embodiment, in response to the fact that the proportion of the target object in the two-dimensional bird's-eye view feature map exceeds a preset ratio, a two-dimensional feature map of a first resolution is selected for target category detection and target location regression; in response to the fact that the proportion of the target object in the two-dimensional bird's-eye view feature map does not exceed the preset ratio, a two-dimensional feature map of a second resolution is selected for target category detection and target location regression; wherein, the second resolution is higher than the first resolution.
[0112] In one specific embodiment, position and category prediction are performed based on the two-dimensional feature map to obtain the category probability value and offset corresponding to each anchor box; the preset category corresponding to the largest category probability value is taken as the target detection category; and the target detection box is determined based on the position information and offset of the anchor box corresponding to the largest category probability value.
[0113] The effective point cloud data is input into the target detection model, which detects at least one target detection box, the detection confidence of each target detection box, and the target detection category based on the effective point cloud data.
[0114] In one embodiment, in response to the presence of multiple target detection boxes, the target detection boxes are filtered based on non-maximum suppression.
[0115] Please see Figure 6 , Figure 6 This is the output result corresponding to the valid point cloud data provided by this invention.
[0116] Specifically, the cross-union ratio (CUP) between target detection boxes is determined. If the CUP is greater than a preset CUP, the target detection box with the lowest detection confidence among the two target detection boxes corresponding to the CUP is deleted. The target detection boxes corresponding to the valid point cloud data are transformed to the current coordinate system, and the positions of the target detection boxes and the target categories are output.
[0117] The target detection method provided in this embodiment includes: acquiring LiDAR point cloud data containing the target object; adaptively adjusting the LiDAR point cloud data to obtain effective point cloud data; the effective point cloud data is the point cloud data within a predetermined area in the LiDAR point cloud data; the predetermined area is determined based on the position coordinates of each laser point in the LiDAR point cloud data; and using a target detection model to detect the effective point cloud data to obtain target detection information, which includes a target detection bounding box and a target detection category. In this application, by adaptively adjusting the LiDAR point cloud data and adaptively selecting a predetermined area based on the position coordinates of each laser point in the LiDAR point cloud data within the scene to obtain effective point cloud data, and then performing target detection on the effective point cloud data using a target detection model, the generalization performance of target detection is improved by adaptively selecting the predetermined area corresponding to the LiDAR point cloud data, thereby improving the accuracy of target detection.
[0118] See Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of the target detection device provided by the present invention. This embodiment provides a target detection device 60, which includes an acquisition module 61, a preprocessing module 62, and a detection module 63.
[0119] The acquisition module 61 is used to acquire lidar point cloud data containing the target object.
[0120] The preprocessing module 62 is used to adaptively adjust the lidar point cloud data to obtain effective point cloud data; the effective point cloud data is the point cloud data within a set area in the lidar point cloud data; the preset area is determined according to the position coordinates of each laser point in the lidar point cloud data.
[0121] The detection module 63 is used to detect effective point cloud data using a target detection model to obtain target detection information, which includes target detection boxes and target detection categories.
[0122] The target detection device provided in this embodiment adaptively adjusts the lidar point cloud data, adaptively selects a preset area based on the position coordinates of each laser point in the lidar point cloud data in the scene to obtain effective point cloud data, and then performs target detection on the effective point cloud data according to the target detection model. By adaptively selecting the preset area corresponding to the lidar point cloud data, the generalization performance of target detection is improved, thereby improving the accuracy of target detection.
[0123] Please see Figure 8 , Figure 8 This is a schematic diagram of a terminal embodiment provided by the present invention. The terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-described target detection method embodiments. In a specific implementation scenario, the terminal 80 may include, but is not limited to, a microcomputer or a server. In addition, the terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.
[0124] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the above-described target detection method embodiments. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.
[0125] Please see Figure 9 , Figure 9 This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present invention. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the above-described embodiments of the target detection method.
[0126] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0127] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.
[0128] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.
[0129] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0130] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0131] The above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A target detection method, characterized in that, The target detection method includes: Acquire LiDAR point cloud data containing the target object; The lidar point cloud data is adaptively adjusted to obtain effective point cloud data; the effective point cloud data is the point cloud data within a preset area in the lidar point cloud data; the preset area is determined according to the position coordinates of each laser point in the lidar point cloud data. The effective point cloud data is detected using an object detection model to obtain object detection information, which includes object detection bounding boxes and object detection categories. The adaptive adjustment of the lidar point cloud data to obtain effective point cloud data includes: Based on the position coordinates of each laser point in the lidar point cloud data, determine the point cloud distribution histogram of the lidar point cloud data in each preset direction; All point cloud data within a preset range from the origin in each preset direction are selected as the valid point cloud data; the origin is the intersection of the backward extensions of each preset direction.
2. The target detection method according to claim 1, characterized in that, The effective point cloud data is detected using a target detection model to obtain target detection information, which includes target detection bounding boxes and target detection categories, including: The target detection model is used to perform voxelization and voxel feature encoding on the effective point cloud data to obtain a three-dimensional feature map. The three-dimensional feature map corresponding to the effective point cloud data is compressed to obtain a two-dimensional bird's-eye view feature map; Multi-scale feature extraction is performed on the two-dimensional bird's-eye view feature map to obtain multiple two-dimensional feature maps of different resolutions corresponding to the effective point cloud data; Based on the two-dimensional feature map, target category detection and target location regression are performed to obtain the target detection box and the target detection category corresponding to the target object.
3. The target detection method according to claim 2, characterized in that, The step of performing target category detection and target location regression based on the two-dimensional feature map to obtain the target detection box and the target detection category corresponding to the target object includes: Based on the two-dimensional feature map, position and category prediction are performed to obtain the category probability value and offset corresponding to each anchor box; the anchor box is the frame configured at each pixel position in the two-dimensional feature map; The preset category corresponding to the largest category probability value is taken as the target detection category; The target detection box is determined based on the position information of the anchor box corresponding to the maximum probability value of the category and the offset.
4. The target detection method according to claim 2 or 3, characterized in that, The step of performing target category detection and target location regression based on the two-dimensional feature map to obtain the target detection box and the target detection category corresponding to the target object includes: If the proportion of the target object in the two-dimensional bird's-eye view feature map exceeds a preset ratio, then a two-dimensional feature map of the first resolution is selected for target category detection and target location regression. If the proportion of the target object in the two-dimensional bird's-eye view feature map does not exceed the preset ratio, then a two-dimensional feature map with a second resolution is selected for target category detection and target location regression; wherein, the second resolution is higher than the first resolution.
5. The target detection method according to claim 1, characterized in that, Before the step of using a target detection model to detect the effective point cloud data and obtain target detection information, the method further includes: Training the target detection model includes: Acquire training point cloud data; the training point cloud data is associated with bounding boxes and label categories of the included targets; the bounding boxes include the center point position, the size of the bounding box, and the yaw angle; The target detection model sequentially performs voxelization, voxel feature encoding, and compression on the training point cloud data to obtain a two-dimensional feature map corresponding to the training point cloud data. Anchor frames are configured for each pixel position in the two-dimensional feature map; Based on the two-dimensional feature map, position and category prediction are performed to obtain the category prediction value and prediction offset corresponding to each anchor frame; Based on the predicted category value and the predicted offset corresponding to the anchor frame, the predicted bounding box and predicted category of the target are determined; The target detection model is iteratively trained based on the category error value calculated from the category prediction value corresponding to the training point cloud data and the positional error value between the prediction box and the annotation box.
6. The target detection method according to claim 5, characterized in that, Training the target detection model further includes: The category of the anchor frame is determined based on the intersection-union ratio between the anchor frame and the label frame; the category of the anchor frame includes positive sample anchor frames and negative sample anchor frames. The iterative training of the target detection model based on the category error value calculated from the category prediction value corresponding to the training point cloud data and the positional error value between the predicted bounding box and the labeled bounding box includes: The class error value is determined using the Focal Loss function based on the class prediction values of each of the positive sample anchor boxes and each of the negative sample anchor boxes; The Smooth L1 loss function is used based on the positional error between the predicted bounding box and the labeled bounding box corresponding to the target. The target detection model is iteratively trained based on the sum of the location error value and the category error value.
7. A target detection device, characterized in that, The target detection device includes: The acquisition module is used to acquire LiDAR point cloud data containing the target object; The preprocessing module is used to adaptively adjust the lidar point cloud data to obtain effective point cloud data. The effective point cloud data is the point cloud data within a preset region of the lidar point cloud data. The preset region is determined based on the position coordinates of each lidar point in the lidar point cloud data. The module is also used to determine a point cloud distribution histogram of the lidar point cloud data in each preset direction based on the position coordinates of each lidar point in the lidar point cloud data. All point cloud data within a preset distance from the origin in each preset direction are selected as the effective point cloud data. The origin is the intersection of the backward extensions of each preset direction. The detection module is used to detect the effective point cloud data using a target detection model to obtain target detection information, which includes target detection boxes and target detection categories.
8. A terminal, characterized in that, The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, the processor being used to execute program data to implement the steps in the target detection method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the target detection method as described in any one of claims 1 to 6.