Target detection model construction method, target detection method and related device

Abstract

The present disclosure provides a method for constructing a target detection model, a target detection method and related devices, and relates to the fields of artificial intelligence technology such as deep learning, neural networks, target detection, and autonomous driving. The method includes: obtaining sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view, and the annotation information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data; extracting features from the sample three-dimensional point cloud data using a preset feature extraction model to obtain a three-dimensional point cloud feature map; training the training samples based on the three-dimensional point cloud feature map and the annotation information through the encoding network and the preset decoding network in turn to obtain an encoding network and a decoding network that meet the training end conditions; deleting the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end conditions to obtain a simplified decoding network; constructing a target detection model based on the feature extraction network, the encoding network that meets the training end conditions, and the simplified decoding network.

Description

Technical Field

[0001] The present disclosure relates to a data processing method, specifically to the fields of artificial intelligence technology such as deep learning, neural networks, target detection, and autonomous driving, and in particular to a target detection model construction method and a target detection method, as well as corresponding devices, electronic devices, computer-readable storage media, and computer program products. Background Art

[0002] Point cloud 3D (three-dimensional) object detection algorithm is a crucial perception algorithm in the field of autonomous driving. The input of the 3D object detection algorithm is the original point cloud, and the output is the location, length, width, height, and orientation angle information of the obstacles in the scene, which is used for subsequent obstacle tracking and downstream PNC (Planning and Control) tasks.

[0003] At present, the mainstream point cloud 3D object detection algorithms are mainly divided into two categories: anchor-based and anchor-free. Among them, the Anchor-Based algorithm requires pre-definition of anchors at each BEV (Bird's Eye View) grid position. This method has high accuracy, but the process is cumbersome and the generation of anchors increases time consumption; the Anchor-Free algorithm does not need to pre-define anchors, and predicts the heatmap to directly obtain the center of the target. Compared with the Anchor-Based algorithm, this method has a slightly lower effect but greatly improves the inference speed, that is, the two algorithms have their own advantages and disadvantages. Summary of the invention

[0004] The embodiments of the present disclosure provide a target detection model construction, a target detection method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] In a first aspect, an embodiment of the present disclosure proposes a method for constructing a target detection model, comprising: obtaining sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​a vehicle from a bird's-eye view, and annotation information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data; using a preset feature extraction model to extract features in the sample three-dimensional point cloud data to obtain a three-dimensional point cloud feature map; training training samples based on the three-dimensional point cloud feature map and the annotation information are sequentially trained through an encoding network and a preset decoding network to obtain an encoding network and a decoding network that meet the training end conditions; wherein the preset decoding network simultaneously uses an Anchor-Free algorithm and an Anchor-Based algorithm to calculate losses respectively, and simultaneously uses the respectively calculated losses as loss functions to update the network parameters of the decoding network; deleting the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end conditions to obtain a simplified decoding network; constructing a target detection model based on the feature extraction network, the encoding network that meets the training end conditions, and the simplified decoding network.

[0006] In a second aspect, an embodiment of the present disclosure proposes a target detection model construction device, comprising: a sample data acquisition unit, configured to acquire sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​a vehicle from a bird's-eye view, and annotation information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data; a feature extraction unit, configured to extract features from the sample three-dimensional point cloud data using a preset feature extraction model to obtain a three-dimensional point cloud feature map; a training unit, configured to train the training samples based on the three-dimensional point cloud feature map and the annotation information through an encoding network and a preset decoding network in sequence to obtain an encoding network and a decoding network that meet the training end conditions; wherein the preset decoding network simultaneously uses the Anchor-Free algorithm and the Anchor-Based algorithm to calculate the loss respectively, and simultaneously uses the respectively calculated losses as the loss function to update the network parameters of the decoding network; a decoding network simplification element, configured to delete the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end conditions to obtain a simplified decoding network; a target detection model construction unit, configured to construct a target detection model based on the feature extraction network, the encoding network that meets the training end conditions, and the simplified decoding network.

[0007] In a third aspect, an embodiment of the present disclosure proposes a target detection method, including: obtaining actual three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle by a three-dimensional laser scanner mounted on the target autonomous driving vehicle; inputting the actual three-dimensional point cloud data as input data into a target detection model, and obtaining the target detection model outputting annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data; wherein the target detection model is obtained according to the target detection model construction method described in the first aspect.

[0008] In the fourth aspect, an embodiment of the present disclosure proposes a target detection device, comprising: an actual three-dimensional point cloud data acquisition unit, configured to acquire actual three-dimensional point cloud data obtained by a three-dimensional laser scanner mounted on a target autonomous driving vehicle to scan the surrounding area of ​​the vehicle; a target detection unit, configured to input the actual three-dimensional point cloud data as input data into a target detection model, and obtain the target detection model outputs annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data; wherein the target detection model is obtained according to the target detection model construction device as described in the second aspect.

[0009] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it can implement the target detection model construction method described in the first aspect or the target detection method described in the third aspect.

[0010] In a sixth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the target detection model construction method described in the first aspect or the target detection method described in the third aspect when executed.

[0011] In the seventh aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the steps of the target detection model building method as described in the first aspect or the steps of the target detection method as described in the third aspect.

[0012] The target detection model construction scheme and target detection scheme provided by the embodiments of the present disclosure are for the three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view. Under the network framework of the encoding and decoding network, the Anchor-Free algorithm and the Anchor-Based algorithm are used simultaneously in the decoding network to calculate the loss respectively, and the respectively calculated losses are used as the loss function to update the network parameters of the decoding network. After obtaining the decoding network that meets the training end conditions, the network structure corresponding to the Anchor-Based algorithm is deleted to retain only the network structure corresponding to the Anchor-Free algorithm, so that the target detection model finally constructed retains the prediction accuracy provided by the Anchor-Based algorithm while having a faster prediction speed with the help of the Anchor-Free algorithm, thereby improving the detection effect of the targets contained in the three-dimensional point cloud data, and is more suitable for application in autonomous driving vehicles.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0015] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0016] Figure 2 A flowchart of a method for building a target detection model provided by an embodiment of the present disclosure;

[0017] Figure 3 A flowchart of another target detection model construction method provided in an embodiment of the present disclosure;

[0018] Figure 4 A flow chart of a target detection method provided by an embodiment of the present disclosure;

[0019] Figure 5 A flowchart of another target detection method provided by an embodiment of the present disclosure;

[0020] Figure 6-1 A schematic diagram of a feature map of different sizes provided in an embodiment of the present disclosure is used to predict the size of an object;

[0021] Figure 6-2 A specific data processing flow diagram provided for an embodiment of the present disclosure;

[0022] Figure 7 A structural block diagram of a target detection model building device provided in an embodiment of the present disclosure;

[0023] Figure 8 A structural block diagram of a target detection device provided in an embodiment of the present disclosure;

[0024] Fig. 9 A schematic diagram of the structure of an electronic device suitable for executing a target detection model building method and / or a target detection method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0026] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0027] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method, apparatus, electronic device, and computer-readable storage medium for training a face recognition model and recognizing a face of the present application can be applied.

[0028] like Figure 1 As shown, the system architecture 100 may include multiple autonomous driving vehicles 101, a network 102 and a server 103. A three-dimensional laser scanner is mounted on the autonomous driving vehicle 101 to facilitate scanning and obtaining three-dimensional point cloud data around the vehicle. The network 102 is used to provide a medium for a communication link between the autonomous driving vehicle 1013 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0029] The user can use the autonomous driving vehicle 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various applications for realizing information communication between the autonomous driving vehicle 101 and the server 103 can be installed, such as target detection applications, model calling applications, model building applications, etc.

[0030] The autonomous driving vehicle 101 and the server 103 are usually different types of hardware devices. In a special simulation scenario, they can also be software or virtual products of software.

[0031] The server 103 can provide various services through various built-in applications. Taking the target detection application that can provide real-time target detection services for the autonomous driving vehicle 101 while driving as an example, the server 103 can achieve the following effects when running the target detection application: first, the actual three-dimensional point cloud data obtained by scanning the surrounding area of ​​the target autonomous driving vehicle 101 through the three-dimensional laser scanner installed on the network 102; then, the actual three-dimensional point cloud data is input into the target detection model as input data, and the target detection model outputs the annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data.

[0032] Among them, the target detection model can be trained by the model building application built into the server 103 according to the following steps: first, obtain sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view through the network 102, and the annotation information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data; then, use the preset feature extraction model to extract features in the sample three-dimensional point cloud data to obtain a three-dimensional point cloud feature map; next, train the training samples based on the three-dimensional point cloud feature map and the annotation information through the encoding network and the preset decoding network in turn to obtain an encoding network and a decoding network that meet the training end conditions, and the preset decoding network simultaneously uses the Anchor-Free algorithm and the Anchor-Based algorithm to calculate the loss respectively, and simultaneously uses the respectively calculated losses as the loss function to update the network parameters of the decoding network; next, delete the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end conditions to obtain a simplified decoding network; finally, based on the feature extraction network, the encoding network that meets the training end conditions and the simplified decoding network, a target detection model is constructed.

[0033] Since training the target detection model requires more computing resources and stronger computing power, the target detection model construction method provided in the subsequent embodiments of this application is generally performed by a server 103 with stronger computing power and more computing resources. Correspondingly, the target detection model construction device is generally also arranged in the server 103. However, it should also be pointed out that when the vehicle-mounted terminal set on the autonomous driving vehicle 101 also has computing power and computing resources that meet the requirements, the vehicle-mounted terminal can also complete the above-mentioned various operations handed over to the server 103 through the target detection model construction application installed thereon, and then output the same results as the server 103. Correspondingly, the target detection model construction device can also be arranged in the vehicle-mounted terminal. In this case, the exemplary system architecture 100 may also not include the server 103 and the network 102.

[0034] Of course, the server used to train the target detection model may be different from the server that calls the trained target detection model for use. In particular, the target detection model trained by the server 103 can also be used to obtain a lightweight target detection model suitable for placement in the vehicle terminal through model distillation, that is, it can be flexibly selected to use the lightweight target detection model in the vehicle terminal or the more complex target detection model in the server 103 according to the actual recognition accuracy required. That is, in actual scenarios, when only the lightweight target detection model on the vehicle terminal is needed to meet the target detection requirements of the required accuracy and recognition speed, the exemplary system architecture may not include the server 103 and the network 102.

[0035] It should be understood that Figure 1 The number of autonomous driving vehicles, networks, and servers in the embodiment is only for illustration purposes. Any number of autonomous driving vehicles, networks, and servers may be provided as required.

[0036] Please refer to Figure 2 , Figure 2 A flowchart of a method for building a target detection model provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0037] Step 201: acquiring sample three-dimensional point cloud data obtained by scanning the surroundings of the vehicle from a bird's-eye view, and labeling information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data;

[0038] This step is intended to be performed by the execution subject of the target detection model construction method (e.g. Figure 1 The server 103 shown in the figure obtains sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view, as well as annotation information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data.

[0039] The sample 3D point cloud data can be obtained by scanning the surrounding area of ​​the autonomous vehicle by a 3D laser scanner mounted on the roof of the autonomous vehicle while the vehicle is driving, or by scanning the surrounding area of ​​the vehicle by a 3D laser scanner mounted on a low-altitude drone that moves synchronously with the ground vehicle, thereby making full use of the height information to obtain the 3D point cloud data from a bird's eye view. The annotation information is the annotation of the 3D positions of various types of targets contained in the sample 3D point cloud data by the annotation object (which can be an annotation person or an automatic annotation model that has been trained to have the required annotation capabilities), such as a 3D calibration frame.

[0040] Step 202: extracting features from the sample three-dimensional point cloud data using a preset feature extraction model to obtain a three-dimensional point cloud feature map;

[0041] Based on step 201, this step aims to extract features from the sample three-dimensional point cloud data using a preset feature extraction model to obtain a three-dimensional point cloud feature map, which can also be called a BEV feature map.

[0042] Step 203: training the training samples based on the three-dimensional point cloud feature map and the annotation information through the encoding network and the preset decoding network in sequence to obtain the encoding network and the decoding network that meet the training end conditions.

[0043] On the basis of step 202, this step aims to have the above-mentioned execution subject form a training sample based on the three-dimensional point cloud feature map and the annotation information, that is, the three-dimensional point cloud feature map is used as the sample input and the annotation information is used as the sample expected output, and then the training sample is trained in sequence through the encoding network and the preset decoding network to obtain the encoding network and the decoding network that meet the training end conditions.

[0044] It should be noted that the preset decoding network uses both the Anchor-Free algorithm and the Anchor-Based algorithm to calculate the loss respectively, and uses the respectively calculated losses as the loss function to update the network parameters of the decoding network. That is, compared with the conventional prior art of using the Anchor-Free algorithm or the Anchor-Based algorithm alone to calculate the loss and using the calculated loss as the loss function to update the network parameters of the decoding network, the solution provided in this embodiment in this step uses the two algorithms simultaneously to calculate the loss function respectively to update the network parameters, so that the model can learn the knowledge mined by the two algorithms as much as possible in the training stage, so as to have the advantages of both algorithms.

[0045] The following is an explanation of the two algorithms used in this step: Object detection is a key task in computer vision, which aims to detect and locate objects from images. In the field of object detection, there are two main algorithmic paradigms: Anchor-Free and Anchor-Based. Both methods have their own advantages and application scenarios:

[0046] Anchor-Free algorithm, which does not rely on anchor boxes, but performs target detection by directly regressing the position of the target or by generating dense target center points. The following are some common Anchor-Free algorithms:

[0047] CenterNet, which predicts the center point of the target and then regresses the bounding box of the target. It does not require an anchor box and uses a Gaussian heat map to represent the target center point. The predicted target center point can be obtained through regression, along with the size information of the target;

[0048] FCOS (Fully Convolutional One-Stage) is an anchor-free object detection algorithm that predicts the category, center, and bounding box size of the target at each pixel. It makes predictions at each position on the feature map instead of on a predefined anchor box.

[0049] Center-ness: This method uses the center-ness of the target to evaluate whether a location contains the target. The center-ness score can be used to filter out non-target areas.

[0050] Anchor-Based algorithms rely on anchor boxes, which are generated in the image, and the algorithm predicts the offsets and class probabilities of the targets associated with these anchor boxes through regression. The following are some common Anchor-Based algorithms:

[0051] Faster R-CNN, a classic object detection framework, uses anchor boxes to sample candidate regions on feature maps and then performs classification and regression on these regions.

[0052] SSD (Single Shot Multibox Detector), a single-stage object detection algorithm, also uses anchor boxes to detect on feature maps of different scales. Anchor boxes of different scales are used to handle objects of different sizes.

[0053] YOLO (You Only Look Once), another classic object detection algorithm, predicts bounding boxes by gridding the image. Each grid cell is responsible for detecting a specific area of ​​the object, while predicting multiple boxes and categories.

[0054] Compared with the Anchor-Based algorithm, the Anchor-Free algorithm is relatively simple, which reduces the complexity of designing Anchor Boxes, thereby significantly shortening the calculation time, and has better detection effect for small-sized targets.

[0055] Compared with Anchor-Free algorithms, Anchor-Based may be easier to handle various target shapes and sizes in some scenarios; in some cases, Anchor-Based methods may be more stable because they utilize the information of prior boxes and thus have relatively high recognition accuracy.

[0056] Step 204: Delete the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end condition to obtain a simplified decoding network;

[0057] On the basis of step 203, this step aims to delete the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end condition by the above-mentioned execution subject, that is, to obtain a simplified decoding network that only retains the network structure corresponding to the Anchor-Free algorithm.

[0058] Step 205: Based on the feature extraction network, the encoding network that meets the training end condition, and the simplified decoding network, a target detection model is constructed.

[0059] On the basis of step 204, this step aims to construct a target detection model for performing three-dimensional target detection on three-dimensional point cloud data from a bird's-eye view based on the three main parts of the feature extraction network, the encoding network that meets the training end conditions, and the streamlined decoding network by the above-mentioned execution subject.

[0060] The target detection model construction method provided by the embodiment of the present disclosure is for the three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view. Under the network framework of the encoding and decoding network, the Anchor-Free algorithm and the Anchor-Based algorithm are used simultaneously in the decoding network to calculate the loss respectively, and the respectively calculated losses are used as the loss function to update the network parameters of the decoding network. After obtaining the decoding network that meets the training end conditions, the network structure corresponding to the Anchor-Based algorithm is deleted to retain only the network structure corresponding to the Anchor-Free algorithm, so that the target detection model finally constructed retains the prediction accuracy provided by the Anchor-Based algorithm while having a faster prediction speed with the help of the Anchor-Free algorithm.

[0061] Please refer to Figure 3 , Figure 3 A flowchart of another target detection model construction method provided by an embodiment of the present disclosure. Process 300 includes the following steps:

[0062] Step 301: Obtain sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view, and labeling information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data;

[0063] Step 302: extracting multi-scale information from the sample 3D point cloud data through a preset multi-scale information extraction network to obtain feature maps of various sizes, and setting the feature maps of different sizes to be used only for predicting the only target with a matching size, and the matching sizes detected by the feature maps of different sizes are different from each other;

[0064] On the basis of step 301, this step aims to extract multi-scale information from the sample three-dimensional point cloud data through a preset multi-scale information extraction network by the above-mentioned execution subject to obtain feature maps of various sizes, and further set feature maps of different sizes to only predict targets with unique matching sizes, and the matching sizes detected by feature maps of different sizes are different.

[0065] The reason why multi-scale information is extracted in this step is that there are often multiple types of targets of different sizes around the vehicle, such as larger traffic signal settings, other vehicles, relatively small bicycles, pedestrians, and the smallest cone barrels. It is difficult for a fixed-scale feature map to take into account the recognition accuracy of targets of different sizes. Therefore, feature maps of multiple sizes are extracted to extract clear information about targets of various sizes as much as possible from feature maps of different sizes.

[0066] Extracting multi-scale information is an important consideration in computer vision tasks such as object detection, as objects of different sizes may appear in an image. The following are some commonly used networks or models that can effectively extract multi-scale information:

[0067] Feature Pyramid Network (FPN) is a network structure widely used in target detection tasks. It builds a pyramid-shaped structure on feature maps at different levels, allowing the network to focus on both low-level and high-level features, thereby extracting multi-scale information;

[0068] Spatial Pyramid Pooling (SPP) is a pooling layer structure that allows the network to use different sized pooling kernels for different parts of the input image. This enables the network to sample features at multiple scales, improving the ability to obtain multi-scale information;

[0069] DeepLabv3 is a semantic segmentation model that uses dilated convolution to expand the receptive field to capture information at different scales. This is also applicable to object detection tasks because both require a combination of global and local information.

[0070] YOLOv4, YOLOv5, and YOLO series object detection models usually use a single neural network to predict object bounding boxes at multiple scales at the same time. This design enables the model to process information of different scales in a single forward propagation;

[0071] EfficientNet is a series of efficient convolutional neural network structures that achieve the ability to extract information at multiple scales by scaling the width, depth, and resolution of the network;

[0072] HRNet (High-Resolution Network) focuses on maintaining high-resolution feature maps and effectively captures multi-scale information by jointly processing feature maps of different resolutions.

[0073] UNet,structure is common in semantic segmentation tasks. It transmits information at different levels through,the encoding and decoding structures, enabling the network to understand and restore features,of different scales.

[0074] The design principles of these networks or models aim to obtain multi-scale information of images through different mechanisms (pyramid structure, pooling, dilated convolution, etc.) to improve the detection ability of objects of different scales. In practical applications, the selection of a suitable model depends on the characteristics of the task, the availability of computing resources, and performance requirements.

[0075] In order to reduce unnecessary calculations as much as possible, the present embodiment further sets feature maps of different sizes to be used only for predicting a unique target with a matching size, and the matching sizes detected by feature maps of different sizes are different. For example, the feature map with the lowest resolution and the smallest size is set to be used only for predicting a target with the largest size (such as traffic lights and large vehicles); the feature map with the highest resolution and the largest size is set to be used only for predicting a target with the smallest size (such as cones set on the road); and the feature map with a resolution between the lowest and the highest and a size between the smallest and the largest is set to be used only for predicting a target with a size between the largest and the smallest (such as bicycles, pedestrians and small vehicles).

[0076] Step 303: training the training samples consisting of feature maps of various sizes and annotation information through the encoding network and the preset decoding network in sequence to obtain the encoding network and the decoding network that meet the training end condition;

[0077] On the basis of step 302, this step aims to train the training samples composed of feature maps and annotation information of various sizes through the encoding network and the preset decoding network in sequence by the above-mentioned execution subject to obtain the encoding network and the decoding network that meet the training end conditions.

[0078] The difference from step 203 is that the training samples input in the current step are composed of feature maps of multiple sizes and annotation information.

[0079] Step 304: Delete the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end condition to obtain a simplified decoding network;

[0080] Step 305: construct a target detection model based on the multi-scale information extraction network, the encoding network that meets the training end condition, and the simplified decoding network.

[0081] On the basis of step 304, this step aims to construct a target detection model for performing three-dimensional target detection on three-dimensional point cloud data from a bird's-eye view based on the three main parts of the multi-scale information extraction network, the encoding network that meets the training end conditions, and the streamlined decoding network by the above-mentioned execution subject.

[0082] Compared to Figure 2 In the process 200 shown, this embodiment additionally introduces a multi-scale information extraction scheme, in order to improve the prediction accuracy of the target detection model for targets of different sizes and reduce unnecessary operations as much as possible by using feature maps of various sizes and corresponding target prediction methods.

[0083] The above embodiments illustrate how to train a target detection model from various aspects. In order to highlight the effect of the trained target detection model from the actual usage scenario as much as possible, the present disclosure also specifically provides a solution for using the trained target detection model to solve practical problems. A target detection method can be found in Figure 4 , the process 400 includes the following steps:

[0084] Step 401: Acquire actual three-dimensional point cloud data obtained by scanning the surroundings of the target autonomous driving vehicle using a three-dimensional laser scanner mounted on the target autonomous driving vehicle;

[0085] This step is intended to be performed by a subject suitable for performing the target detection method (e.g. Figure 1 The on-board terminal 101 set on the autonomous driving vehicle shown in the figure or the combination with the server 105 obtains the actual three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from the three-dimensional laser scanner.

[0086] Step 402: Input the actual three-dimensional point cloud data as input data into the target detection model, and obtain the labeling information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data output by the target detection model.

[0087] That is, this embodiment inputs the acquired actual 3D point cloud data into the target detection model, and then obtains the annotation information of the 3D position of each target contained in the actual 3D point cloud data output by the target detection model, such as a 3D calibration frame.

[0088] The target detection method provided in this embodiment not only improves the detection accuracy of targets contained in three-dimensional point cloud data, but also shortens the detection time as much as possible, making the detection more real-time, by specifically applying the target detection model to an autonomous driving vehicle for target detection.

[0089] exist Figure 4 Based on the embodiment shown, this embodiment also Figure 5 Another target detection method is further provided, wherein the process 500 includes the following steps:

[0090] Step 501: Acquire actual three-dimensional point cloud data obtained by scanning the surrounding area of ​​the target autonomous driving vehicle using a three-dimensional laser scanner mounted on the vehicle;

[0091] Step 502: input the actual three-dimensional point cloud data as input data into the target detection model, and obtain the labeling information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data output by the target detection model;

[0092] Step 501 - Step 502 are consistent with step 401 - Step 402 and will not be repeated here.

[0093] Step 503: determining type information and attribute information of each target according to the annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data;

[0094] Based on step 502, this step aims to determine the type information and attribute information of each target based on the annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data by the above-mentioned execution subject. Among them, the type information is used to determine the type of the target (such as vehicle, pedestrian, traffic facility, etc.), and the attribute information can include: the three-dimensional position and motion state of the target.

[0095] Step 504: Adjust the expected driving trajectory of the target autonomous driving vehicle according to the type information and attribute information of each target.

[0096] Based on step 503, this step aims to cause the above-mentioned execution subject to adjust the originally set driving trajectory of the target autonomous driving vehicle according to the type information and attribute information of each target, so as to avoid various targets that may conflict with the original driving trajectory (such as other vehicles traveling in the opposite direction or suddenly appearing pedestrians, etc.).

[0097] To deepen understanding, the present disclosure also provides a specific implementation scheme in combination with a specific application scenario. Several key points involved in the implementation scheme are described in detail below:

[0098] 1. Process point clouds and generate multi-scale feature maps

[0099] For a given point cloud range and voxel size, the 3D space is divided into several pillars (point cloud pillars), then the point cloud is voxelized and projected into all pillars, and the features of each pillar are mapped using a fully connected layer, and then feature compression is performed to obtain 2DBEV features. ResNet101 (a residual network) and FPN are then used to further extract features to obtain BEV feature maps of multiple scales.

[0100] 2. Multi-scale perdiction

[0101] Traditional Centerpoint only predicts targets at a single scale. This approach cannot take into account targets of all sizes. At the same time, the detection effect is closely related to the size of the voxel. Another popular approach is to output feature maps of multiple scales, and then predict features of all sizes on each scale feature map. The disadvantage of this multi-scale approach is that it is difficult to force high-resolution feature maps to predict large targets, and it is also difficult to force low-resolution feature maps to predict small targets. To this end, this embodiment proposes a new multi-scale prediction algorithm, the core idea of ​​which is to let high-resolution feature maps only predict small targets, and let low-resolution feature maps only predict large targets.

[0102] Specifically, suppose that four categories, namely cars, bicycles, pedestrians, and cones, need to be detected. At the same time, the AM-Centerpoint of this embodiment outputs three scales of BEV feature maps, namely [c,h,w], [c,2h,2w], and [c,4h,4w]. h and w represent the size of the feature map, and the resolution of the three feature maps increases in sequence. Then, the feature map with the lowest resolution [c,h,w] is required to predict large targets such as cars, the feature map with moderate resolution [c,2h,2w] is required to predict pedestrians and bicycles, and finally the feature map with the highest resolution [c,4h,4w] is required to predict small targets such as cones. For a schematic diagram, see Figure 6-1 .

[0103] The multi-scale perdiction mechanism can simplify the learning process, reduce the learning difficulty, and take into account targets of different sizes.

[0104] 3. Anchor-Aided

[0105] The Anchor-Based algorithm has good results but slow inference speed, while the Anchor-Free algorithm has slightly worse results but fast inference speed. In order to obtain the advantages of both paradigms, this embodiment uses the Anchor-Aided mechanism, using both the Anchor-Based method and the Anchor-Free method to calculate the loss and optimize the network during training, and only using the Anchor-Free detection head during test inference.

[0106] The schematic diagram is shown below. Each feature map requires two heads: a classification head predicts the category and a regression head predicts the target attribute. In each head, two results, Anchor-Free and Anchor-Based, are predicted. For example, in the classification head, Anchor-Free predicts the category of each grid, and Anchor-Based predicts the category of each Anchor; in the regression head, Anchor-Free predicts the length, width, height and orientation angle of each grid, and Anchor-Based predicts the offset of the target relative to the Anchor.

[0107] Then, the Anchor-Free prediction results in the classification head and regression head are used to calculate the Anchor-Free loss to obtain the Anchor Free loss. Similarly, all the Anchor-Based prediction results are used for decoding, and then the Anchor-Based loss is calculated. During training, both losses are used to update the network at the same time, so that the network not only learns the Anchor-Free information, but also the Anchor-Based information, thereby improving the overall detection effect.

[0108] In other words, it is equivalent to that the first network head and the second network head are used to predict the feature map of each size in the preset decoding network mentioned in the above embodiment, the first network head is used to predict the target category, the Anchor-Free algorithm in the first network head is used to predict the type of each map grid, the Anchor-Based algorithm in the first network head is used to predict the category of each anchor box, and the second network head is used to predict the target attributes, the Anchor-Free algorithm in the second network head is used to predict the length, width, height and orientation angle of each map grid, and the Anchor-Based algorithm in the second network head is used to predict the offset of the target relative to the anchor box.

[0109] The overall plan can be found in Figure 6-2 Schematic diagram of .

[0110] During reasoning, the Anchor-Based related network is discarded and only the Anchor-Free branch is used for reasoning, thus ensuring that the reasoning speed meets the requirements.

[0111] The target detection solution proposed in this embodiment has the following advantages:

[0112] 1) In order to solve the problem that only a single-scale feature map can be used for detection, which makes it impossible to take into account objects of different sizes, a novel multi-scale prediction is proposed. Different from the traditional method of predicting objects of all sizes on feature maps of different scales, the multi-scale prediction of this embodiment predicts objects of different sizes on feature maps of different scales. This approach can effectively reduce the difficulty of network learning and improve the detection effect of objects of different sizes;

[0113] 2) For problems that are slightly worse than the Anchor-Based algorithm, the Anchor-Aided mechanism is used. During training, the losses of Anchor-Based and Anchor-Free are calculated simultaneously, and the model is trained together. During reasoning, only the Anchor-Free branch is used. This approach not only integrates the information of the Anchor-Based algorithm and improves the detection effect, but also maintains the reasoning advantage of Anchor-Free.

[0114] Further references Figure 7 and Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a target detection model construction device and an embodiment of a target detection device, respectively. Figure 2 The target detection model construction method embodiment shown in FIG. Figure 4 The above device can be applied to various electronic devices.

[0115] like Figure 7As shown, the target detection model construction device 700 of this embodiment may include: a sample data acquisition unit 701, a feature extraction unit 702, a training unit 703, a decoding network refinement element 704, and a target detection model construction unit 705. The sample data acquisition unit 701 is configured to acquire sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view, and the annotation information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data; the feature extraction unit 702 is configured to extract features from the sample three-dimensional point cloud data using a preset feature extraction model to obtain a three-dimensional point cloud feature map; the training unit 703 is configured to train the training samples based on the three-dimensional point cloud feature map and the annotation information through the encoding network and the preset decoding network in sequence to obtain an encoding network and a decoding network that meet the training end conditions; wherein the preset decoding network simultaneously uses the Anchor-Free algorithm and the Anchor-Based algorithm to calculate the loss respectively, and simultaneously uses the respectively calculated losses as the loss function to update the network parameters of the decoding network; the decoding network simplification element 704 is configured to delete the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end conditions to obtain a simplified decoding network; the target detection model construction unit 705 is configured to construct a target detection model based on the feature extraction network, the encoding network that meets the training end conditions, and the simplified decoding network.

[0116] In this embodiment, in the target detection model construction device 700, the specific processing of the sample data acquisition unit 701, the feature extraction unit 702, the training unit 703, the decoding network refinement unit 704, and the target detection model construction unit 705 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of steps 201 - 205 in the corresponding embodiment are not repeated here.

[0117] In some optional implementations of the present embodiment, the feature map of each size in the preset decoding network is predicted using the first network head and the second network head, wherein the first network head is used to predict the target category, the Anchor-Free algorithm in the first network head is used to predict the type of each map grid, and the Anchor-Based algorithm in the first network head is used to predict the category of each anchor box; the second network head is used to predict the target attributes, the Anchor-Free algorithm in the second network head is used to predict the length, width, height and orientation angle of each map grid, and the Anchor-Based algorithm in the second network head is used to predict the offset of the target relative to the anchor box.

[0118] In some optional implementations of this embodiment, the feature extraction unit 702 may include:

[0119] The multi-scale feature extraction subunit is configured to extract multi-scale information from the sample three-dimensional point cloud data through a preset multi-scale information extraction network to obtain feature maps of multiple sizes;

[0120] Correspondingly, the training unit 703 may be further configured as follows:

[0121] The training samples consisting of feature maps of various sizes and annotation information are trained in sequence through the encoding network and the preset decoding network.

[0122] In some optional implementations of this embodiment, the multi-scale feature extraction subunit is further configured as follows:

[0123] The sample three-dimensional point cloud data is input into a multi-scale information extraction network established based on the network architecture of the feature map pyramid network to obtain output feature maps of various sizes.

[0124] In some optional implementations of this embodiment, the target detection model building device 700 may further include:

[0125] The prediction setting unit for feature maps of different sizes is configured to set feature maps of different sizes only for predicting targets with unique matching sizes, and the matching sizes detected by the feature maps of different sizes are different from each other.

[0126] In some optional implementations of this embodiment, the different size feature map prediction setting unit is further configured to:

[0127] Set the feature map with the lowest resolution and smallest size to only predict the object with the largest size;

[0128] Set the feature map with the highest resolution and largest size to only predict the object with the smallest size;

[0129] Set feature maps with resolution between the lowest and highest and size between the smallest and largest to be used only to predict objects with size between the largest and smallest.

[0130] In some optional implementations of this embodiment, the sample data acquisition unit 701 may include a sample 3D point cloud data acquisition subunit configured to acquire sample 3D point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's eye view, and the sample 3D point cloud data acquisition subunit may be further configured to:

[0131] The sample three-dimensional point cloud data is obtained by scanning the surrounding area of ​​the vehicle using a three-dimensional laser scanner mounted on the roof of the autonomous vehicle.

[0132] like Figure 8As shown, the target detection device 800 of this embodiment may include: an actual three-dimensional point cloud data acquisition unit 801 and a target detection unit 802. The actual three-dimensional point cloud data acquisition unit 801 is configured to acquire the actual three-dimensional point cloud data obtained by scanning the surrounding area of ​​the target autonomous driving vehicle with a three-dimensional laser scanner mounted on the target autonomous driving vehicle; the target detection unit 802 is configured to input the actual three-dimensional point cloud data as input data into the target detection model, and obtain the labeling information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data output by the target detection model; wherein the target detection model is obtained according to the target detection model construction device 700.

[0133] In this embodiment, in the target detection device 800: the specific processing of the actual three-dimensional point cloud data acquisition unit 801 and the target detection unit 802 and the technical effects brought about by them can respectively correspond to the relevant descriptions in the method embodiment, and will not be repeated here.

[0134] In some optional implementations of this embodiment, the target detection device 800 further includes:

[0135] The type and attribute information determination unit is configured to determine the type information and attribute information of each target according to the annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data;

[0136] The expected driving trajectory adjustment unit is configured to adjust the expected driving trajectory of the target autonomous driving vehicle according to the type information and attribute information of each target.

[0137] This embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment. The target detection model construction apparatus and the target detection apparatus provided in this embodiment, for the three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view, under the network framework of the encoding and decoding network, specifically, in the decoding network, the Anchor-Free algorithm and the Anchor-Based algorithm are simultaneously used to calculate the loss respectively, and the respectively calculated losses are simultaneously used as the loss function to update the network parameters of the decoding network. After obtaining the decoding network that meets the training end conditions, the network structure corresponding to the Anchor-Based algorithm is deleted to retain only the network structure corresponding to the Anchor-Free algorithm, so that the target detection model finally constructed retains the prediction accuracy provided by the Anchor-Based algorithm while having a faster prediction speed with the help of the Anchor-Free algorithm, thereby improving the detection effect of the target contained in the three-dimensional point cloud data, and is more suitable for application in autonomous driving vehicles.

[0138] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the target detection model construction method and / or target detection method described in any of the above embodiments when executing.

[0139] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the target detection model construction method and / or target detection method described in any of the above embodiments when executed.

[0140] An embodiment of the present disclosure provides a computer program product, which, when executed by a processor, can implement the steps of the target detection model building method and / or the steps of the target detection method described in any of the above embodiments.

[0141] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0142] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0143] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0144] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a target detection model construction method and / or a target detection method. For example, in some embodiments, the target detection model construction method and / or the target detection method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the target detection model construction method and / or the target detection method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the target detection model building method and / or the target detection method in any other suitable manner (eg, by means of firmware).

[0145] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0147] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0149] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0150] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0151] The technical solution of the disclosed embodiment is to use the Anchor-Free algorithm and the Anchor-Based algorithm simultaneously in the decoding network to calculate the loss respectively under the network framework of the encoding and decoding network for the three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's-eye view, and use the respectively calculated losses as the loss function to update the network parameters of the decoding network. After obtaining the decoding network that meets the training end conditions, the network structure corresponding to the Anchor-Based algorithm is deleted to retain only the network structure corresponding to the Anchor-Free algorithm, so that the target detection model finally constructed retains the prediction accuracy provided by the Anchor-Based algorithm while having a faster prediction speed with the help of the Anchor-Free algorithm, thereby improving the detection effect of the target contained in the three-dimensional point cloud data, and is more suitable for application in autonomous driving vehicles.

[0152] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0153] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for building a target detection model, comprising: Acquire sample three-dimensional point cloud data obtained by scanning the surroundings of the vehicle from a bird's-eye view, and label information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data; Extracting multi-scale information from the sample three-dimensional point cloud data through a preset multi-scale information extraction network to obtain feature maps of various sizes; The training samples composed of feature maps based on multiple sizes and the annotation information are trained in sequence through an encoding network and a preset decoding network to obtain an encoding network and a decoding network that meet the training end condition, wherein the preset decoding network uses a first network head to perform category prediction and a second network head to perform attribute prediction for feature maps of each size, and the preset decoding network simultaneously uses an anchor-free Anchor-Free algorithm and an anchor-Based algorithm to calculate losses respectively, and simultaneously uses the respectively calculated losses as loss functions to update network parameters of the decoding network, the Anchor-Free algorithms in the first network head and the second network head are respectively used to predict the type of each map grid and the length, width, height and orientation angle of each map grid, and the Anchor-Based algorithms in the first network head and the second network head are respectively used to predict the category of each anchor box and the offset of the predicted target relative to the anchor box; Deleting the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end condition to obtain a simplified decoding network; Based on the multi-scale information extraction network, the encoding network that meets the training end condition and the simplified decoding network, a target detection model is constructed.

2. The method according to claim 1, wherein: The method of extracting multi-scale information from the sample three-dimensional point cloud data through a preset multi-scale information extraction network to obtain feature maps of various sizes includes: The sample three-dimensional point cloud data is input into a multi-scale information extraction network established based on a network architecture of a feature map pyramid network to obtain output feature maps of multiple sizes.

3. The method according to claim 1, further comprising: Feature maps of different sizes are set only for predicting the only target with a matching size, and the matching sizes detected by feature maps of different sizes are different from each other.

4. The method according to claim 3, wherein: The feature maps of different sizes are set only for predicting the only target with a matching size, and the matching sizes detected by the feature maps of different sizes are different from each other, including: Set the feature map with the lowest resolution and smallest size to only predict the object with the largest size; Set the feature map with the highest resolution and largest size to only predict the object with the smallest size; The feature maps with resolutions between the minimum and the maximum and sizes between the minimum and the maximum are set to be used only for predicting objects with sizes between the maximum and the minimum.

5. The method according to any one of claims 1 to 4, wherein: The method of obtaining sample three-dimensional point cloud data obtained by scanning the surrounding area of ​​the vehicle from a bird's eye view includes: The sample three-dimensional point cloud data is obtained by scanning the surrounding area of ​​the vehicle using a three-dimensional laser scanner mounted on the roof of the autonomous vehicle.

6. A target detection method, comprising: Acquire actual three-dimensional point cloud data obtained by scanning the surrounding area of ​​the target autonomous driving vehicle using a three-dimensional laser scanner mounted on the vehicle; The actual three-dimensional point cloud data is input as input data into a target detection model, and the target detection model outputs annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data; wherein the target detection model is obtained by the target detection model construction method according to any one of claims 1-5.

7. The method according to claim 6, further comprising: Determining type information and attribute information of each target according to the annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data; According to the type information and attribute information of each target, the expected driving trajectory of the target autonomous driving vehicle is adjusted.

8. A target detection model construction device, comprising: A sample data acquisition unit is configured to acquire sample three-dimensional point cloud data obtained by scanning the surroundings of the vehicle from a bird's-eye view, and annotation information of the three-dimensional position of the target contained in the sample three-dimensional point cloud data; A feature extraction unit is configured to extract multi-scale information from the sample three-dimensional point cloud data through a preset multi-scale information extraction network to obtain feature maps of multiple sizes; A training unit is configured to train training samples consisting of feature maps of multiple sizes and the annotation information through an encoding network and a preset decoding network in sequence to obtain an encoding network and a decoding network that meet the training end condition, wherein the preset decoding network uses a first network head for category prediction and a second network head for attribute prediction for feature maps of each size, and the preset decoding network simultaneously uses an anchor-free algorithm and an anchor-based algorithm to calculate losses respectively, and simultaneously uses the respectively calculated losses as loss functions to update network parameters of the decoding network, the Anchor-Free algorithms in the first network head and the second network head are respectively used to predict the type of each map grid and the length, width, height and orientation angle of each map grid, and the Anchor-Based algorithms in the first network head and the second network head are respectively used to predict the category of each anchor box and the offset of the predicted target relative to the anchor box; A decoding network simplification element is configured to delete the network structure corresponding to the Anchor-Based algorithm in the decoding network that meets the training end condition to obtain a simplified decoding network; The target detection model construction unit is configured to construct a target detection model based on the multi-scale information extraction network, the encoding network that meets the training end condition, and the simplified decoding network.

9. The device according to claim 8, wherein: The feature extraction unit is further configured to: The sample three-dimensional point cloud data is input into a multi-scale information extraction network established based on a network architecture of a feature map pyramid network to obtain output feature maps of multiple sizes.

10. The apparatus according to claim 8, further comprising: The prediction setting unit for feature maps of different sizes is configured to set feature maps of different sizes only for predicting targets with unique matching sizes, and the matching sizes detected by the feature maps of different sizes are different from each other.

11. The device according to claim 10, wherein: The different-size feature map prediction setting unit is further configured to: Set the feature map with the lowest resolution and smallest size to only predict the object with the largest size; Set the feature map with the highest resolution and largest size to only predict the object with the smallest size; The feature maps with resolutions between the minimum and the maximum and sizes between the minimum and the maximum are set to be used only for predicting objects with sizes between the maximum and the minimum.

12. The device according to any one of claims 8 to 11, wherein: The sample data acquisition unit includes a sample 3D point cloud data acquisition subunit configured to acquire sample 3D point cloud data obtained by scanning the surroundings of the vehicle from a bird's eye view, and the sample 3D point cloud data acquisition subunit is further configured to: The sample three-dimensional point cloud data is obtained by scanning the surrounding area of ​​the vehicle using a three-dimensional laser scanner mounted on the roof of the autonomous vehicle.

13. A target detection device, comprising: An actual three-dimensional point cloud data acquisition unit is configured to acquire actual three-dimensional point cloud data obtained by a three-dimensional laser scanner mounted on the target autonomous driving vehicle scanning the surrounding area of ​​the vehicle; The target detection unit is configured to input the actual three-dimensional point cloud data as input data into a target detection model, and obtain the labeling information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data output by the target detection model; wherein the target detection model is obtained according to the target detection model construction device according to any one of claims 8 to 12.

14. The apparatus according to claim 13, further comprising: a type and attribute information determination unit configured to determine type information and attribute information of each target according to the annotation information of the three-dimensional position of each target contained in the actual three-dimensional point cloud data; The expected driving trajectory adjustment unit is configured to adjust the expected driving trajectory of the target autonomous driving vehicle according to the type information and attribute information of each target.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target detection model building method described in any one of claims 1-5.

16. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target detection method described in claim 6 or 7.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the target detection model construction method according to any one of claims 1 to 5.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the target detection method according to claim 6 or 7.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the target detection model construction method according to any one of claims 1 to 5 and / or the steps of the target detection method according to claim 6 or 7.

Images