A target detection model training method, a target detection method, a device, and a vehicle

Abstract

The application relates to the technical field of automatic driving, and discloses a target detection model training method, a target detection method, a device and a vehicle. The target detection model training method learns a reference heat map constructed by a center-based detector on a target detection frame label, and carries out confrontation through first machine learning and second machine learning, so that the first machine learning can learn the distribution of the reference heat map, and finally, the detector generates a heat map and predicts the category and parameters of the target detection frame based on the heat map, so that target detection of three-dimensional point cloud data is realized. The whole training process not only enables the first machine learning model to better learn the distribution of the reference heat map, but also is beneficial to the convergence of the model parameters of the first machine learning model and the second machine learning model.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a target detection model training method, target detection method, device, and vehicle. Background Technology

[0002] In the field of computer vision, object detection has become a major focus in recent years. Deep learning technology enables computers to detect and recognize target objects, significantly reducing the consumption of human resources. Object detection is divided into two-dimensional (2D) and three-dimensional (3D) object detection, depending on the data processing dimension and output results. 3D object detection, in particular, is more widely used in practical applications such as remote sensing mapping, military reconnaissance, autonomous driving, biomedical detection, and cultural heritage preservation. In the field of autonomous driving, perception systems are used to detect targets in the surrounding environment to obtain accurate, robust, and reliable information. These perception systems typically consist of many different sensors, commonly including cameras and LiDAR. LiDAR describes the surrounding environment by constructing point cloud data in three-dimensional space. Compared to cameras, LiDAR is unaffected by lighting conditions, has higher data throughput, and is more suitable for target detection tasks in outdoor environments.

[0003] In the field of target detection using 3D point cloud data, anchor-based target detection methods are commonly used. These methods train the detector by pre-generating anchor bounding boxes of different sizes for each type of target and assigning labels to different anchors, comparing the labels with the detector output. However, traditional anchor-based target detection methods suffer from slow detection speed. Summary of the Invention

[0004] To address the aforementioned technical problems, this application discloses a method for training an object detection model, comprising:

[0005] Obtain the sample point cloud dataset of the target object and the detection bounding box label of the target object;

[0006] Generate a reference heatmap of the target object based on the detection box label of the target object;

[0007] Based on the sample point cloud dataset of the target object, the first machine learning model is used to train the target detection and obtain the predicted heatmap.

[0008] Based on the predicted heatmap and the reference heatmap, a second machine learning model is used for target matching training to obtain the objective function;

[0009] Adjust the first model parameters of the first machine learning model and the second model parameters of the second machine learning model to optimize the value of the objective function to the preset value;

[0010] The first machine learning model corresponding to the current first model parameters is used as the object detection model; the first machine learning model and the second machine learning model are established based on a generative adversarial network.

[0011] Optionally, the acquisition of the sample point cloud dataset of the target object and the detection box label of the target object includes:

[0012] Obtain sample point cloud data for each of multiple objects and the corresponding detection box label for each object;

[0013] Based on the preset target category and the corresponding preset point cloud range, the sample point cloud dataset of the target object and the sample point cloud dataset of the non-target object are determined from the sample point cloud data of each object among the multiple objects;

[0014] Based on the sample point cloud dataset of the non-target object, the detection box labels of each object among the multiple objects are filtered to obtain the detection box labels of the target object.

[0015] Optionally, before obtaining the predicted heatmap by training the first machine learning model for object detection using the sample point cloud dataset based on the target object, the method further includes:

[0016] The sample point cloud dataset of the target object is sequentially processed by voxelization and feature extraction to obtain voxel features.

[0017] Optionally, the first machine learning model includes an image generation module, an encoder-decoder, a feature extraction network, and a detection module connected in sequence;

[0018] Based on the sample point cloud dataset of the target object, a first machine learning model is used for target detection training to obtain a predicted heatmap, including:

[0019] The voxel features were processed using an image generation model to obtain a pseudo-two-dimensional image;

[0020] The pseudo-2D image is convolved and deconvolved using the encoder-decoder to obtain the first feature map.

[0021] Based on this feature extraction network, features are extracted from the first feature map to obtain the second feature map;

[0022] The detection module is used to detect and classify the second feature map to generate the predicted heatmap.

[0023] Optionally, generating a reference heatmap of the target object based on the detection box label of the target object includes:

[0024] The scale mapping ratio between the pseudo-2D image and the predicted heatmap is determined based on the pseudo-2D image and the predicted heatmap.

[0025] The position of the center of the detection box of the target object in the reference heatmap is determined based on the position of the center of the detection box of the target object in the pseudo-2D image and the scale mapping ratio.

[0026] Based on the position of the center of the target object's detection box in the reference heatmap and the remaining labels of the target object's detection box, the target feature points of the pseudo-2D image are processed using a Gaussian radius assignment algorithm to generate the reference heatmap; the scale of the reference heatmap is the same as the scale of the predicted heatmap; the remaining labels of the target object's detection box are the detection box labels excluding the center position of the detection box; the target feature points are the pixels located within the detection box in the pseudo-2D image.

[0027] Optionally, the objective function may include an adversarial loss function, an object classification loss function, and a bounding box parameter regression loss function;

[0028] The adversarial loss function Loss GAN It is determined according to the following formula:

[0029]

[0030] The target classification loss function Loss CLS It is determined according to the following formula:

[0031]

[0032] The detection box parameters regression loss function Loss REG It is determined according to the following formula:

[0033]

[0034] Where G is the first machine learning model; D is the second machine learning model; x and t represent the input point cloud data and reference heatmap labels, respectively; Px and Pt represent the probability distributions of the point cloud data and the reference heatmap, respectively; y represents the interpolation term for the bounding box labels of the predicted heatmap and the reference heatmap; α is a uniformly distributed random number from 0 to 1; and λ is the regularization coefficient of the gradient penalty term; HM CLS and These represent the category channels of the reference heatmap and the category channels of the predicted heatmap, respectively; HM REG and These represent the detection box parameter channels of the reference heatmap and the detection box parameter channels of the predicted heatmap, respectively; N LabelThe number of detection boxes for the target object; mask is the mask; the mask is determined based on the center position of the detection boxes for the target object.

[0035] On the other hand, this application discloses a target detection method, which includes:

[0036] Obtain the point cloud dataset around the vehicle;

[0037] The point cloud dataset is input into the object detection model for detection processing, and the output is a heatmap corresponding to the point cloud dataset; the object detection model is trained based on the object detection model training method described above.

[0038] On the other hand, this application discloses a target detection model training device, which includes:

[0039] The first acquisition module is used to acquire the sample point cloud dataset of the target object and the detection box label of the target object;

[0040] The reference heatmap generation module is used to generate a reference heatmap of the target object based on the detection box label of the target object.

[0041] The predictive heatmap generation module is used to train the first machine learning model for target detection based on the sample point cloud dataset of the target object to obtain the predictive heatmap.

[0042] The objective function determination module is used to perform target matching training using a second machine learning model based on the predicted heatmap and the reference heatmap to obtain the objective function;

[0043] The optimization module is used to adjust the first model parameters of the first machine learning model and the second model parameters of the second machine learning model, and optimize the value of the objective function to a preset value.

[0044] The target detection model determination module is used to take the first machine learning model corresponding to the current first model parameters as the target detection model; the first machine learning model and the second machine learning model are established based on a generative adversarial network.

[0045] On the other hand, this application discloses a target detection device, which includes:

[0046] The second acquisition module is used to acquire the point cloud dataset around the vehicle;

[0047] The heatmap generation module is used to input the point cloud dataset into the object detection model for detection processing and output a heatmap corresponding to the point cloud dataset; the object detection model is trained based on the object detection model training method described above.

[0048] On the other hand, this application discloses a vehicle that includes the aforementioned target detection device.

[0049] By adopting the above technical solution, the target detection model training method provided in this application has the following beneficial effects:

[0050] The process involves: acquiring a sample point cloud dataset of the target object and its bounding box labels; generating a reference heatmap of the target object based on its bounding box labels; training a first machine learning model for target detection based on the sample point cloud dataset of the target object to obtain a predicted heatmap; training a second machine learning model for target matching based on the predicted heatmap and the reference heatmap to obtain an objective function; adjusting the first model parameters of the first machine learning model and the second model parameters of the second machine learning model to optimize the objective function to a preset value; using the first machine learning model corresponding to the current first model parameters as the target detection model; and establishing the first machine learning model and the second machine learning model based on a generative adversarial network. Subsequently, target detection can be completed directly using point cloud data. Moreover, this adversarial generative network model enables the first machine learning model to learn the distribution of the reference heatmap well, which is beneficial to the convergence of the model parameters of the first machine learning model and the second machine learning model. Furthermore, the target detection model obtained based on the above training method is a center point target detection method, which has rotation equivalence and rotation invariance. Compared with anchor-based detection methods, the target detection model provided in this application reduces the number of model parameters and improves the detection speed. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is an application scenario diagram provided in an embodiment of this application;

[0053] Figure 2 This is a flowchart illustrating a target detection model training method provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of a generative adversarial network framework provided in an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of the structure of a first machine learning model provided in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram of a process for generating a predictive heatmap according to an embodiment of this application.

[0057] Figure 6 This is a schematic diagram of a process for generating a reference heatmap provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the structure of a second machine learning model provided in an embodiment of this application;

[0059] Figure 8 This is a schematic flowchart of a target detection method provided in an embodiment of this application;

[0060] Figure 9 This is a schematic diagram of the structure of a target detection model training device provided in an embodiment of this application;

[0061] Figure 10 This is a schematic diagram of the structure of a target detection device provided in an embodiment of this application. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0063] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0064] See Figure 1 As shown, Figure 1This is an optional application scenario diagram of this application. The scenario includes a terminal 10 and an object detection model training module 101 located on the terminal; the object detection model training module 101 is used to acquire a sample point cloud dataset of the target object and the detection box label of the target object; generate a reference heatmap of the target object based on the detection box label of the target object; train the first machine learning model for object detection based on the sample point cloud dataset of the target object to obtain a predicted heatmap; train the second machine learning model for object matching based on the predicted heatmap and the reference heatmap to obtain an objective function; adjust the first model parameters of the first machine learning model and the second model parameters of the second machine learning model to optimize the value of the objective function to a preset value; use the first machine learning model corresponding to the current first model parameters as the object detection model; the first machine learning model and the second machine learning model are established based on a generative adversarial network.

[0065] Optionally, the model training device can be configured in a server or other terminal, or it can be configured independently of the server.

[0066] The terminal can be a physical device such as a smartphone, computer (e.g., desktop computer, tablet computer, laptop computer), digital assistant, intelligent voice interaction device (e.g., smart speaker), smart wearable device, in-vehicle terminal, etc., or it can be software running on the physical device, such as a computer program. The operating system corresponding to the first client can be Android, iOS (a mobile operating system developed by Apple), Linux (an operating system), Microsoft Windows, etc.

[0067] The server side can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server may include network communication units, processors, and memory, etc. The server side can provide backend services to the corresponding clients.

[0068] The following describes a specific embodiment of a target detection model training method according to this application. Figure 2This is a flowchart illustrating a method for training an object detection model according to an embodiment of this application. This specification provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server products, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 2 As shown, the method may include:

[0069] S201: Obtain the sample point cloud dataset of the target object and the detection box label of the target object.

[0070] Optionally, the sample point cloud dataset can be obtained directly from a historical point cloud database or based on radar. When performing target detection based on the trained target detection model, target detection can be achieved directly based on the point cloud data collected by the LiDAR on the vehicle. This method is unaffected by lighting conditions and has high data accuracy.

[0071] Optionally, the sample point cloud dataset may include point cloud datasets at different timestamps and corresponding object detection box labels. Each point cloud in the point cloud data has its three-dimensional coordinates (x, y, z) and other attributes. Each detection box label in the corresponding object includes the target category, the three-dimensional coordinate position of the detection box center (xc, yc, zc), the three-dimensional size of the detection box (dx, dy, dz), the rotation angle, and other attributes. Specifically, when the vehicle is moving, the LiDAR can collect the road scene data (i.e., point cloud dataset) of the current vehicle. This road scene may include other vehicles, pedestrians, fences, etc. Optionally, other vehicles can be used as target objects, and the target category is vehicles. The rotation angle specifically refers to the length, width, and height of the target relative to the three axes (yaw, pitch, roll) of the camera coordinate system. Here, yaw is the heading angle, which controls the left and right rotation of the target; pitch is the pitch angle, which controls the up and down rotation of the target; and roll is the roll angle, which controls the left and right roll of the target. These three angles together can be called Euler angles.

[0072] In one feasible embodiment, step S201 is a process of preprocessing the directly acquired initial sample point cloud dataset. Step S201 may include: acquiring sample point cloud data of each object in multiple objects and the corresponding detection box label for each object; determining the sample point cloud dataset of the target object and the sample point cloud dataset of non-target objects from the sample point cloud data of each object in the multiple objects based on a preset target category and a corresponding preset point cloud range; and filtering the detection box labels of each object in the multiple objects based on the sample point cloud dataset of non-target objects to obtain the detection box label of the target object. That is, the sample point cloud dataset of the target object and the sample point cloud data of non-target objects can be distinguished from the sample point cloud data of multiple objects based on a preset target category (such as vehicles) and a corresponding preset point cloud range. At this time, for a detection box of a target object, there may still be sample point cloud data of non-target objects. Therefore, the detection boxes containing point cloud data of non-target objects can be further removed based on the sample point cloud dataset of non-target objects to obtain the detection box label of the target object. By filtering the directly acquired sample point cloud data and corresponding detection box labels, the effectiveness of the data is improved, thereby increasing the training efficiency of the detection model.

[0073] In another feasible embodiment, points that exceed the preset point cloud range can be removed first, and these points can be recorded to obtain the initial target object point cloud dataset and the non-target object point cloud dataset under each timestamp; then, the detection box labels of the points that exceed the preset point cloud range (i.e., non-target object points) within the detection box are removed, and at the same time, based on the target category, points whose detection box categories do not meet the target category requirements and their corresponding detection boxes are removed to obtain the filtered target object point cloud data and the corresponding target detection box labels.

[0074] In other words, step S201 can specifically filter the point cloud dataset and corresponding detection box labels under each time stamp according to the target category and preset point cloud range set in the detection task, thereby obtaining the point cloud data of the target object and the corresponding target detection box labels.

[0075] Optionally, points that are outside the point cloud range and points that do not belong to the target category can be removed first, and the removed points can be recorded to directly obtain the point cloud data of the target object and the point cloud dataset of the non-target object; then the detection box labels containing these removed points can be removed to obtain the corresponding detection box labels of the target object.

[0076] To further ensure the accuracy of subsequent training results, the number of point cloud data points N under each timestamp can be set. After the above filtering, if the number of point cloud data points under a timestamp is less than N, the point cloud data is padded with zeros. When each point contains F attributes, the tensor size of the point cloud dataset under that timestamp is N*F. The maximum number of boxes that the corresponding detection box can contain is set to P. Similarly, for cases where this condition is not met, the detection boxes are padded with zeros (for example, when P=5, but the actual number of detection boxes is 4, the process of padding the detection boxes is to add 5 detection boxes, and set all attribute parameters of the added detection box to 0). The number of attributes for each detection box is F. C The size of the bounding box label tensor of the obtained target object is P*F. C Otherwise, random sampling is performed on the point cloud data and the bounding box labels, so that the number of sampled point cloud data and the number of bounding boxes are N and P, respectively.

[0077] Optionally, the point cloud data can be augmented by shuffling the input order of the point cloud dataset of the target object, so that the subsequent processing order of the point cloud data is different from the filtering order, thereby enabling training to obtain the same output based on point cloud data under different orders.

[0078] In this embodiment, the point cloud dataset under one timestamp will be described in detail below. However, as needed, the sample dataset may include point cloud datasets under different timestamps to improve the accuracy of the training results.

[0079] S203: Generate a reference heatmap of the target object based on the detection box label of the target object.

[0080] S205: Based on the sample point cloud dataset of the target object, the first machine learning model is used for target detection training to obtain a predicted heatmap.

[0081] In one feasible embodiment, before step S205, the method further includes: sequentially performing voxelization and feature extraction processing on the sample point cloud dataset of the target object to obtain voxel features. This further enhances the sample point cloud dataset and improves subsequent training efficiency.

[0082] In this embodiment, voxel features can be generated based on the following process: First, voxelization is performed, generating multiple voxels and voxel information corresponding to each voxel based on the sample point cloud dataset of the target object. The specific process is as follows: The point cloud is allocated to its respective voxel range according to the specified voxel size, and the points in each voxel are sampled or zero-padded according to the specified maximum number of voxel points. Finally, the voxel tensor size is M*k*F, where M is the number of voxels divided and k is the maximum number of point clouds contained in each voxel. At the same time, the spatial position and actual number of points contained in each voxel are recorded during the voxel generation process to obtain an M-dimensional voxel index tensor and a voxel point count tensor. Voxels can be generated in a cylindrical manner, and the generated voxels are located in a plane. The voxel index tensor size is M*2. The voxel information includes the voxel tensor, the voxel index tensor, and the voxel point count tensor.

[0083] Secondly, for each of the aforementioned voxels, feature extraction is performed based on the voxel information to generate initial voxel features. Based on these initial voxel features, the voxel features corresponding to the sample point cloud dataset are generated. The specific process is as follows: A Voxel Feature Encoder (VFE) layer is used to extract local features from each voxel, thereby extracting the overall features of all point clouds within each voxel as the voxel's overall features. The VFE consists of a weight-sharing fully connected layer and a max-pooling layer. For each voxel generated above, which can be a columnar voxel, the spatial center point position and the average position of all points within the voxel are determined based on the voxel information. For each point in the voxel, a first offset vector (xi) is determined based on the point's position and the spatial center point position. c ,y c ,z c The second offset vector (x) is determined based on the position of the point and the average position of all points within the aforementioned voxel. p ,y p ,z p Based on the first and second offset vectors mentioned above, the extended features are determined, namely, a 6-dimensional vector (xc, yc, zc, xp, yp, zp). The initial voxel features are determined based on the extended features and voxel information of each point in the voxel. The size of the voxel tensor is updated to M*k*(F+6). Then, a mask is added to the zero-padded points. The features of each point in the initial voxel features are calculated by increasing the dimensionality through a weight-sharing fully connected layer. The most representative point is extracted as the voxel feature of the voxel through a max pooling layer. Based on the voxel features of each voxel, the voxel features corresponding to the sample point cloud dataset are generated. The size of the voxel feature tensor is M*C, where C is the number of voxel features.

[0084] Optionally, the point with the largest eigenvalue in the voxel can be taken as the most representative point.

[0085] In one possible embodiment, see [reference] Figure 4-5 , Figure 4 This is a schematic diagram of the structure of a first machine learning model provided in an embodiment of this application; Figure 5 This is a schematic diagram of a process for generating a predicted heatmap according to an embodiment of this application. The first machine learning model includes an image generation module, an encoder-decoder, a feature extraction network, and a detection module connected in sequence. Step S205 above may include:

[0086] S2051: Use an image generation model to perform feature mapping on the voxel features to obtain a pseudo-two-dimensional image.

[0087] In this embodiment, the image generation module can be a Bird's-Eye View (BEV) module. The voxels mentioned above are stacked cylinders. In order to further extract and enhance features from these cylinders, the BEV module can be used to map the voxel feature tensor into a pseudo-two-dimensional image based on the voxel index tensor. The size of the mapped pseudo-two-dimensional image is C*W*H, where C is the number of channels of the voxel feature tensor output by the VFE, and W and H are the length and width of the voxel segmentation.

[0088] S2053: Use the encoder-decoder to perform convolution and deconvolution on the pseudo-2D image to obtain the first feature map.

[0089] In this embodiment, the encoder-decoder can be a feature pyramid network, including a multi-level encoder and a decoder. The multi-level encoder includes, for example: Figure 4 The encoders shown are encoder 1, encoder 2, and encoder 3; the decoder includes decoder 1, decoder 2, and decoder 3; each encoder consists of multiple convolutional modules, and each convolutional module consists of a convolutional layer, a BatchNorm layer (i.e., a BN layer), and a ReLU activation function; the decoder includes, as shown... Figure 4 The decoders 1, 2 and 3 shown are each composed of a convolutional module and multiple deconvolutional modules.

[0090] Optionally, the convolutional module in the decoder is similar to the convolutional module in the encoder, and the stride of feature extraction is the ratio of the target size to the corresponding encoder output image size.

[0091] Optionally, step S2053 can be specifically described as follows: for each level of encoder-decoder, the encoder performs convolution processing on the pseudo-two-dimensional image to obtain a first intermediate feature map; the decoder performs convolution and deconvolution processing on the intermediate feature map to obtain a second intermediate feature map; and the second intermediate feature maps of each level of encoder-decoder in the multi-level encoder-decoder are fused to obtain a first feature map.

[0092] Optionally, in the process of generating the first intermediate feature map, a feature pyramid network is used to extract the first intermediate features of the pseudo-image at different scales. Since each encoder consists of multiple convolutional modules, and each convolutional module consists of a convolutional layer, a BatchNorm layer, and a ReLU activation function, for the first convolutional module, the pseudo-2D image can be downsampled. The convolutional kernel size of the convolutional layer is 3, the stride is 2, the image edges are zero-padded outwards with a length of 1, the number of channels becomes twice that of the original image, and the image length and width become half that of the original image. For the subsequent multiple convolutional modules, the convolutional kernel size of their respective convolutional layers is 3, the stride is 1, the image edges are zero-padded outwards with a length of 1, and the pseudo-2D image size remains unchanged.

[0093] In the process of generating the second intermediate feature map, the decoder module consists of a convolutional module and multiple deconvolutional modules. It upsamples the feature map output by the corresponding encoder to the target size, ensuring that the feature maps output by multiple decoders have the same size. The decoder's convolutional module is similar to the encoder's convolutional module, with a stride equal to the ratio of the target size to the corresponding encoder output image size. The deconvolutional module consists of a deconvolutional layer, a BatchNorm layer, and a ReLU activation function. The deconvolutional layer has a kernel size of 3, a stride of 1, and no padding at the image edges. Subsequently, by using the output of each encoder as the input to the corresponding decoder, the final output feature map size for each decoder is C. O *W O *H O Finally, the outputs of multiple decoders are concatenated at the channel level, and the final output feature map obtained from the feature pyramid has a size of (K*C). O )*W O *H O , where K is the number of encoders or decoders.

[0094] S2055: Based on this feature extraction network, feature extraction is performed on the first feature map to obtain the second feature map.

[0095] In this embodiment, as Figure 4 As shown, this feature extraction network is used to further extract spatial features from the first feature map output by the encoder-decoder. This feature extraction network can contain a convolutional layer, a BatchNorm layer, and a ReLU activation function. The convolutional layer has a kernel size of 3, a stride of 1, and a zero-padding length of 1. The input is the second feature map output from the aforementioned feature pyramid, and the number of output channels is set to C. S The size of the output second feature map is C. S *W O *H O .

[0096] S2057: Use the detection module to detect and classify the second feature map to generate the predicted heatmap.

[0097] See Figure 4 The detection module includes multiple sub-detection modules, which can divide the target detection category into multiple groups according to the object size. Each group corresponds to a sub-detection module for target classification and regression of detection box parameters.

[0098] Optionally, the input to each sub-detection module is the second feature map output by the aforementioned feature extraction network. Each sub-detection module contains six parallel convolutional modules, which respectively detect the target object's category, horizontal position, height, size, orientation, and other attributes of the target object's bounding box. The number of output categories corresponds to the number of convolutional modules, where the number of channels for the target object's position, size, orientation, and other attributes are 3, 3, 2, and F, respectively. C -8, the kernel size of each convolutional layer in each convolutional module is 3, the stride is 1, and the zero-padding length is 1, without changing the size of the input feature map. Finally, the outputs of each convolutional module are concatenated at the channel to obtain the predicted heatmap, and the size of the predicted heatmap is (N C +F C )*W O *H O , where N C The number of categories that each sub-detection module needs to detect.

[0099] In one possible embodiment, see [reference] Figure 6 , Figure 6 This is a schematic diagram of a process for generating a reference heatmap according to an embodiment of this application. To further improve the accuracy of the judgment result, a reference heatmap with the same size as the predicted heatmap is constructed. This reference heatmap can be set using a Gaussian radius, which more smoothly represents the relationship between the category confidence and other attributes near the detection box and the center label attribute of the detection box. Step S203 can specifically include:

[0100] S2031: Determine the scale mapping ratio between the pseudo-2D image and the predicted heatmap based on the pseudo-2D image and the predicted heatmap.

[0101] In this embodiment, the C*W*H pseudo-2D image generated in step S2051 and the size obtained in step S2057 are (N C +F C )*W O *H O The predicted heatmap is calculated, and the scale mapping ratio R between the predicted heatmap and the pseudo-2D image is W / W.O .

[0102] S2033: Determine the position of the center of the detection box of the target object in the reference heatmap based on the position of the center of the detection box of the target object in the pseudo-2D image and the scale mapping ratio.

[0103] In this embodiment, the center position of the detection box in the reference heatmap can be determined based on the spatial position of the target detection box in the pseudo-two-dimensional image and the mapping ratio R.

[0104] S2035: Based on the position of the center of the detection box of the target object in the reference heatmap and the remaining labels of the detection box of the target object, the target feature points of the pseudo-2D image are processed using the Gaussian radius assignment algorithm to generate the reference heatmap; the scale of the reference heatmap is the same as the scale of the predicted heatmap; the remaining labels of the detection box of the target object are the labels of the detection boxes other than the center position of the detection box; the target feature points are the pixels in the pseudo-2D image located within the detection box.

[0105] In this embodiment, by analyzing the Nth reference heatmap k The center of each channel is assigned a value of 1, where N k The target category of the detection box is assigned a Gaussian radius near its center, with the radius following a Gaussian distribution. The Gaussian radius is determined by the size of the detection box and is expressed as max(||dx,dy||2,2), where 2 is the minimum radius set after referencing the heatmap F. C The attribute channels of each detection box are assigned values, and the detection box position (x, y, z) within the aforementioned Gaussian radius is encoded as a deviation value (xx). c yy c ,zz c ), where (x c ,y c ,z c ) represents the spatial location of the position in the reference heatmap mapped to the pseudo-2D image. Within the Gaussian radius, the target object's size, orientation, and other attribute channel codes in the reference heatmap are (log(dx), log(dy), log(dz), cos(rot), sin(rot), ...), respectively. Thus, a reference heatmap with the same size as the predicted heatmap output by the detection module is obtained.

[0106] In this embodiment, the prediction heatmap generated based on the first machine learning model can characterize the confidence level of each location in the space belonging to a certain type of target object and the parameter values ​​of each attribute of the target object's detection box; similarly, the reference heatmap can also characterize the above content, thereby determining the matching result between the two by matching the similarity of the target object's confidence level and the target object's detection box label. As can be seen from the above, the reference heatmap can be specifically obtained based on the target object's detection box label and the encoding of each attribute parameter.

[0107] S207: Based on the predicted heatmap and the reference heatmap, a second machine learning model is used to train target matching to obtain the objective function.

[0108] See Figure 7 , Figure 7 This is a schematic diagram of the structure of a second machine learning model provided in an embodiment of this application. The second machine learning module may include a Markov discriminator, and its network is composed of multiple convolutional modules connected in series, such as... Figure 7 The algorithm consists of a first convolutional module, a second convolutional module, a third convolutional module, and so on, all connected in sequence. Each convolutional module contains a convolutional layer, a BatchNorm layer, and a LeakyReLU activation function. The stride of the convolutional layer is set to 1, and the slope of the negative half-axis of the LeakyReLU activation function is set to 0.2. Then, another convolutional layer is connected, with the number of output channels set to 1. Other parameters can be set by the user. Finally, a global average pooling layer is connected to obtain a unified score that represents the degree of matching between the reference heatmap and the predicted heatmap, and the corresponding objective function can also be obtained at the same time.

[0109] In one feasible embodiment, the objective function includes an adversarial loss function, an object classification loss function, and a bounding box parameter regression loss function;

[0110] Among them, the adversarial loss function Loss GAN It is determined according to the following formula:

[0111]

[0112] G represents the first machine learning model; D represents the second machine learning model; x and t represent the input point cloud data and the reference heatmap labels, respectively; Px and Pt represent the probability distribution of the point cloud data and the probability distribution of the reference heatmap, respectively; y represents the interpolation term of the detection box labels of the predicted heatmap and the reference heatmap; α is a uniformly distributed random number from 0 to 1; and λ is the regularization coefficient of the gradient penalty term.

[0113] Wherein, the target classification loss function Loss CLS The top N values ​​in the predicted heatmap output by the first machine learning model G CThe top N of the predicted channel feature maps and reference heatmaps for each category C The cross-entropy loss of the predicted channel feature maps for each category, and the target classification loss function Loss CLS Specifically, it can be determined using the following formula:

[0114]

[0115] HM CLS and These represent the category channels of the reference heatmap and the category channels of the predicted heatmap, respectively.

[0116] Among them, the detection box parameter regression loss function Loss REG The L1 norm of the difference between the predicted and reference heatmaps can be calculated by multiplying both the predicted and reference heatmaps by the mask, and then dividing the result by the number of detection boxes. This can be expressed as the following formula:

[0117]

[0118] HM REG and These represent the detection box parameter channels of the reference heatmap and the detection box parameter channels of the predicted heatmap, respectively; N Label The number of detection boxes for the target object; mask is the mask; the mask is determined based on the center position of the detection boxes for the target object.

[0119] Based on the above three formulas, the target loss function Loss can be expressed as:

[0120] Loss = Loss GAN +λ CLS Loss CLS +λ REG Loss REG

[0121] Where, λ CLS and λ REG These are the regularization coefficients for the target classification loss function and the detection box parameter regression loss function, respectively.

[0122] S209: Adjust the first model parameters of the first machine learning model and the second model parameters of the second machine learning model to optimize the value of the objective function to the preset value.

[0123] S211: The first machine learning model corresponding to the current first model parameters is used as the target detection model; the first machine learning model and the second machine learning model are established based on a generative adversarial network.

[0124] Optionally, step S209 can be specifically described as follows: based on the values ​​of the first model parameters updated in the previous iteration and the objective function, update the values ​​of the second model parameters; based on the values ​​of the second model parameters updated in the previous iteration and the objective function, update the values ​​of the first model parameters; iterate along the gradient descent direction of the objective function, and repeat the above two steps until the value of the objective function reaches a preset value.

[0125] Optionally, the adversarial loss function of the first and second machine learning models can be constructed using WGA loss with gradient penalty, and the overall objective loss function of the generative adversarial model can be constructed by introducing the class cross-entropy loss of the heatmap and the L1 regression loss of the detection box parameters of the heatmap, i.e., the objective function mentioned above.

[0126] In this embodiment, the first machine learning model can also be called the detection model, and the second machine learning model can also be called the discriminant model. The model parameters of the detection model G and the discriminant model D are randomly initialized, and the optimizers for the discriminant model G and the discriminant model D are initialized as ADAM optimizers, denoted as O. G and O D The voxel features generated from the point cloud data are input into the detection model G for forward computation, outputting a predicted heatmap. A reference heatmap is calculated based on the bounding box labels, thereby calculating the class cross-entropy loss between the reference heatmap and the predicted heatmap, and the L1 regression loss of the bounding box parameters. The predicted heatmap and reference heatmap output by the detection model G are input into the discriminant model D for forward computation, causing the discriminant model to output the confidence score of the predicted heatmap. This allows the calculation of the adversarial loss function between the detection model G and the discriminant model D. Combining the class cross-entropy loss and the L1 regression loss of the bounding box parameters, the overall loss values ​​of models G and D can be obtained. Gradient backpropagation is then used to calculate the gradient values ​​of each network parameter. Subsequently, by fixing the second model parameters of the discriminant model D, the optimizer O... G Update the first model parameters of the detection model; then fix the first model parameters of the detection model G and use the optimizer O D Update the second model parameters of the discrimination model; repeatedly execute the above steps of predicting the heatmap based on the output of the first machine learning model, updating the first model parameters, and updating the second model parameters until the loss value of the above target loss function converges. At this time, the local optimal solution of the first model parameters can be obtained. Using the trained detector, the optimal estimate of the heatmap of the point cloud data can be calculated.

[0127] This application applies a generative adversarial neural network (GAN) framework to the field of 3D point cloud object detection. A center-based object detection method is used to construct a detection model that generates a predictive heatmap. A reference heatmap is used, with the detection box labels set to Gaussian radii for the heatmap's category channel values ​​and detection box parameter channel values. In other words, the center-based detection model learns from the reference heatmap constructed using the object detection box labels. Because the reference heatmap values ​​are set with Gaussian radii, they more smoothly represent the relationship between the category confidence and other attributes near the detection box and the attributes of the detection box's center label. Finally, the predictive heatmap and the reference heatmap are used in adversarial training against the detection model through a discriminant model. During training, adversarial loss, classification loss, and regression loss are introduced to calculate the loss value and update the network parameters. This allows the detection model to better learn the distribution of the reference heatmap, which is beneficial for the convergence of the detection model network and the discriminant model network parameters.

[0128] See Figure 8 , Figure 8 This is a schematic flowchart of a target detection method provided in an embodiment of this application. In another aspect, the target detection method includes:

[0129] S801: Obtain the point cloud dataset around the vehicle.

[0130] S803: Input the point cloud dataset into the object detection model for detection processing, and output a heatmap corresponding to the point cloud dataset; the object detection model is trained based on the object detection model training method described above.

[0131] In this embodiment, refer to Figure 3 The detection process shown, after step S803, further includes post-processing the heatmap. Specifically, it extracts the confidence scores of the top k positions for each category channel in the heatmap, decodes the detection box parameters corresponding to each detection box position to generate its own detection box, and then removes redundant detection boxes to obtain the final detection box containing the predicted category and its various attributes such as position, size, and orientation. Specifically, the process of removing redundant detection boxes can be as follows: detection boxes whose centers exceed the preset point cloud range and whose classification scores are less than a first threshold are removed. When the number of remaining detection boxes is greater than or equal to K, non-maximum suppression (NMS) is applied to the remaining detection boxes. This involves filtering detection boxes whose Intersection over Union (IOU) ratio is greater than a second threshold, retaining the top k detection boxes with the highest classification scores; otherwise, the remaining detection boxes are directly output.

[0132] See Figure 9 , Figure 9This is a schematic diagram of the structure of an object detection model training device provided in an embodiment of this application. In another aspect, this application discloses an object detection model training device, which includes:

[0133] The first acquisition module 901 is used to acquire the sample point cloud dataset of the target object and the detection box label of the target object;

[0134] Reference heatmap generation module 903 is used to generate a reference heatmap of the target object based on the detection box label of the target object;

[0135] The predictive heatmap generation module 905 is used to train the first machine learning model for target detection based on the sample point cloud dataset of the target object to obtain the predictive heatmap.

[0136] The objective function determination module 907 is used to perform target matching training using a second machine learning model based on the predicted heatmap and the reference heatmap to obtain the objective function;

[0137] The optimization module 909 is used to adjust the first model parameters of the first machine learning model and the second model parameters of the second machine learning model, and optimize the value of the objective function to a preset value.

[0138] The target detection model determination module 911 is used to take the first machine learning model corresponding to the current first model parameters as the target detection model; the first machine learning model and the second machine learning model are established based on a generative adversarial network.

[0139] In one feasible embodiment, a heatmap generation module is used to obtain sample point cloud data of each of the multiple objects and the detection box label corresponding to each object;

[0140] Based on the preset target category and the corresponding preset point cloud range, the sample point cloud dataset of the target object and the sample point cloud dataset of the non-target object are determined from the sample point cloud data of each object among the multiple objects;

[0141] Based on the sample point cloud dataset of the non-target object, the detection box labels of each object among the multiple objects are filtered to obtain the detection box labels of the target object.

[0142] In one possible embodiment, the device further includes:

[0143] The voxel feature generation module is used to perform voxelization and feature extraction processing on the sample point cloud dataset of the target object in sequence to obtain voxel features.

[0144] In one feasible embodiment, the first machine learning model includes an image generation module, an encoder-decoder, a feature extraction network, and a detection module connected in sequence;

[0145] The predictive heatmap generation module is used to perform feature mapping on the voxel features using an image generation model to obtain a pseudo-two-dimensional image.

[0146] The pseudo-2D image is convolved and deconvolved using the encoder-decoder to obtain the first feature map.

[0147] Based on this feature extraction network, features are extracted from the first feature map to obtain the second feature map;

[0148] The detection module is used to detect and classify the second feature map to generate the predicted heatmap.

[0149] In one feasible embodiment, a reference heatmap generation module is used to determine the scale mapping ratio between the pseudo-2D image and the predicted heatmap based on the pseudo-2D image and the predicted heatmap.

[0150] The position of the center of the detection box of the target object in the reference heatmap is determined based on the position of the center of the detection box of the target object in the pseudo-2D image and the scale mapping ratio.

[0151] Based on the position of the center of the target object's detection box in the reference heatmap and the remaining labels of the target object's detection box, the target feature points of the pseudo-2D image are processed using a Gaussian radius assignment algorithm to generate the reference heatmap; the scale of the reference heatmap is the same as the scale of the predicted heatmap; the remaining labels of the target object's detection box are the detection box labels excluding the center position of the detection box; the target feature points are the pixels located within the detection box in the pseudo-2D image.

[0152] In one feasible embodiment, the objective function includes an adversarial loss function, an object classification loss function, and a bounding box parameter regression loss function;

[0153] The adversarial loss function Loss GAN It is determined according to the following formula:

[0154]

[0155] The target classification loss function Loss CLS It is determined according to the following formula:

[0156]

[0157] The detection box parameters regression loss function Loss REG It is determined according to the following formula:

[0158]

[0159] Where G is the first machine learning model; D is the second machine learning model; x and t represent the input point cloud data and reference heatmap labels, respectively; Px and Pt represent the probability distributions of the point cloud data and the reference heatmap, respectively; y represents the interpolation term for the bounding box labels of the predicted heatmap and the reference heatmap; α is a uniformly distributed random number from 0 to 1; and λ is the regularization coefficient of the gradient penalty term; HM CLS and These represent the category channels of the reference heatmap and the category channels of the predicted heatmap, respectively; HM REG and These represent the detection box parameter channels of the reference heatmap and the detection box parameter channels of the predicted heatmap, respectively; N Label The number of detection boxes for the target object; mask is the mask; the mask is determined based on the center position of the detection boxes for the target object.

[0160] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a target detection device provided in an embodiment of this application. In another aspect, the target detection device includes:

[0161] The second acquisition module 1001 is used to acquire a point cloud dataset around the vehicle;

[0162] The heatmap generation module 1003 is used to input the point cloud dataset into the object detection model for detection processing and output a heatmap corresponding to the point cloud dataset; the object detection model is trained based on the object detection model training method described above.

[0163] In one feasible embodiment, the target detection device further includes a detection box post-processing module, used to extract an initial target detection box from the heatmap obtained by the heatmap generation module, and to perform post-processing on the initial target detection box to obtain a target detection box result (see [reference]). Figure 3 ).

[0164] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0165] On the other hand, this application discloses a vehicle that includes the aforementioned target detection device.

[0166] Embodiments of this application also provide an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the target detection model training method described above.

[0167] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0168] Embodiments of this application also provide a computer storage medium, which can be disposed in a server to store at least one instruction, at least one program, code set, or instruction set related to implementing a target detection model training method in the method embodiment. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-described target detection model training method.

[0169] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0170] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0172] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0173] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a target detection model, the method comprising: include: Obtain the sample point cloud dataset of the target object and the detection bounding box labels of the target object; The sample point cloud dataset of the target object is sequentially subjected to voxelization and feature extraction processing to obtain voxel features; Generate a reference heatmap of the target object based on the detection box label of the target object; Based on the sample point cloud dataset of the target object, a first machine learning model is used to train the target detection to obtain a predicted heatmap. Based on the predicted heatmap and the reference heatmap, a second machine learning model is used for target matching training to obtain the objective function; Adjust the first model parameters of the first machine learning model and the second model parameters of the second machine learning model to optimize the value of the objective function to a preset value; The first machine learning model corresponding to the current first model parameters is used as the object detection model; the first machine learning model and the second machine learning model are established based on a generative adversarial network; The first machine learning model further includes an image generation module, an encoder-decoder, a feature extraction network, and a detection module connected in sequence. The step of training the first machine learning model for target detection based on the sample point cloud dataset of the target object to obtain a predicted heatmap includes: using the image generation module to perform feature mapping processing on the voxel features to obtain a pseudo-two-dimensional image; using the encoder-decoder to perform convolution and deconvolution processing on the pseudo-two-dimensional image to obtain a first feature map; performing feature extraction on the first feature map based on the feature extraction network to obtain a second feature map; and using the detection module to perform detection and classification processing on the second feature map to generate the predicted heatmap. The step of generating a reference heatmap of the target object based on the detection box labels of the target object includes: determining the scale mapping ratio between the pseudo-2D image and the predicted heatmap based on the pseudo-2D image and the predicted heatmap; determining the position of the center of the detection box of the target object in the reference heatmap based on the position of the center of the detection box of the target object in the pseudo-2D image and the scale mapping ratio; processing the target feature points of the pseudo-2D image using a Gaussian radius assignment algorithm based on the position of the center of the detection box of the target object in the reference heatmap and the remaining labels of the detection box of the target object to generate the reference heatmap; the scale of the reference heatmap is the same as the scale of the predicted heatmap; the remaining labels of the detection box of the target object are the detection box labels excluding the center position of the detection box; the target feature points are the pixels located within the detection box in the pseudo-2D image.

2. The target detection model training method of claim 1, wherein, The process of obtaining the sample point cloud dataset of the target object and the detection box labels of the target object includes: Obtain sample point cloud data for each of the multiple objects and the corresponding detection box label for each object; Based on the preset target category and the corresponding preset point cloud range, the sample point cloud dataset of the target object and the sample point cloud dataset of the non-target object are determined from the sample point cloud data of each of the multiple objects. Based on the sample point cloud dataset of the non-target objects, the detection box labels of each object in the plurality of objects are filtered to obtain the detection box labels of the target objects. 3.The target detection model training method of claim 1 or 2, wherein, The objective function includes an adversarial loss function, an object classification loss function, and a bounding box parameter regression loss function; The adversarial loss function Loss GAN is determined according to the following formula: ； The target classification loss function Loss CLS is determined according to the following formula: ； The detection frame parameter regression loss function Loss REG is determined according to the following formula: ； Where G is the first machine learning model; D is the second machine learning model; x and t represent the input point cloud data and the reference heatmap label, respectively; P x P t Let represent the probability distribution of the point cloud data and the probability distribution of the reference heatmap, respectively; y represents the interpolation term of the detection box labels of the predicted heatmap and the reference heatmap; α is a uniformly distributed random number between 0 and 1; and λ is the regularization coefficient of the gradient penalty term. and These represent the category channels of the reference heatmap and the category channels of the predicted heatmap, respectively. and These represent the detection box parameter channels of the reference heatmap and the detection box parameter channels of the predicted heatmap, respectively; N Label The number of detection boxes for the target object; mask is a mask; the mask is determined based on the center position of the detection boxes for the target object.

4. A target detection method characterized by, include: Obtain the point cloud dataset around the vehicle; The point cloud dataset is input into the target detection model for detection processing, and a heat map corresponding to the point cloud dataset is output. The target detection model is trained based on the target detection model training method described in any one of claims 1-3. 5.A device for training a target detection model, characterized in that, include: The first acquisition module is used to acquire the sample point cloud dataset of the target object and the detection box label of the target object; A reference heatmap generation module is used to generate a reference heatmap of the target object based on the detection box label of the target object; The predictive heatmap generation module is used to train a first machine learning model for target detection based on the sample point cloud dataset of the target object to obtain a predictive heatmap. The objective function determination module is used to perform target matching training using a second machine learning model based on the predicted heatmap and the reference heatmap to obtain the objective function; An optimization module is used to adjust the first model parameters of the first machine learning model and the second model parameters of the second machine learning model, and optimize the value of the objective function to a preset value; The target detection model determination module is used to use the first machine learning model corresponding to the current first model parameters as the target detection model; the first machine learning model and the second machine learning model are established based on a generative adversarial network. The voxel feature generation module is used to perform voxelization and feature extraction processing on the sample point cloud dataset of the target object in sequence to obtain voxel features; The first machine learning model further includes an image generation module, an encoder-decoder, a feature extraction network, and a detection module connected in sequence. The step of training the first machine learning model for target detection based on the sample point cloud dataset of the target object to obtain a predicted heatmap includes: using the image generation module to perform feature mapping processing on the voxel features to obtain a pseudo-two-dimensional image; using the encoder-decoder to perform convolution and deconvolution processing on the pseudo-two-dimensional image to obtain a first feature map; performing feature extraction on the first feature map based on the feature extraction network to obtain a second feature map; and using the detection module to perform detection and classification processing on the second feature map to generate the predicted heatmap. The step of generating a reference heatmap of the target object based on the detection box labels of the target object includes: determining the scale mapping ratio between the pseudo-2D image and the predicted heatmap based on the pseudo-2D image and the predicted heatmap; determining the position of the center of the detection box of the target object in the reference heatmap based on the position of the center of the detection box of the target object in the pseudo-2D image and the scale mapping ratio; processing the target feature points of the pseudo-2D image using a Gaussian radius assignment algorithm based on the position of the center of the detection box of the target object in the reference heatmap and the remaining labels of the detection box of the target object to generate the reference heatmap; the scale of the reference heatmap is the same as the scale of the predicted heatmap; the remaining labels of the detection box of the target object are the detection box labels excluding the center position of the detection box; the target feature points are the pixels located within the detection box in the pseudo-2D image.

6. A target detection apparatus characterized by comprising: include: The second acquisition module is used to acquire the point cloud dataset around the vehicle; The heatmap generation module is used to input the point cloud dataset into the target detection model for detection processing and output a heatmap corresponding to the point cloud dataset. The target detection model is trained based on the target detection model training method described in any one of claims 1-3.

7. A vehicle characterized by comprising: Includes the target detection device as described in claim 6.

Images