Target detection method, device, medium and vehicle for autonomous driving

Abstract

The present invention relates to the field of autonomous driving technology, and specifically provides a target detection method, device, medium and vehicle for autonomous driving, aiming to solve the problem of how to improve the computing speed of target detection to reduce the computing pressure of the vehicle-side chip under the premise of ensuring the autonomous driving function. To this end, the present invention can obtain the voxelized point cloud features within each voxel in the voxelization stage of the original point cloud data. Through the above configuration, the present invention does not need to use an additional model to extract features and calculate the average of the voxelized point cloud data. The features of the point cloud data within the voxel are integrated in the voxelization stage. Under the premise of ensuring the autonomous driving function, the preliminary extraction process of the original point cloud data is simplified, the computing speed of the target detection is improved, and the computing pressure of the vehicle-side chip is reduced.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and specifically provides a target detection method, device, medium and vehicle for autonomous driving. Background Art

[0002] With the continuous development of technology, autonomous driving and assisted driving have received increasing attention. At the same time, with the continuous upgrading of hardware and software technologies, the functions of autonomous driving have been further enhanced. However, because the object detection model based on autonomous driving scenarios is deployed on the vehicle, it is limited by the power consumption and computing power of the vehicle-side chip. Existing autonomous driving object detection methods often require high-computing chip designs, and the object detection models used to implement these methods also have large model sizes, which actually increases the computing pressure on the vehicle-side chip.

[0003] Accordingly, this field requires a new target detection solution for autonomous driving to solve the above problems. Summary of the Invention

[0004] In order to overcome the above-mentioned defects, the present invention is proposed to provide a solution or at least partially solve the problem of how to improve the computing speed of target detection to reduce the computing pressure of the vehicle-side chip while ensuring the autonomous driving function.

[0005] In a first aspect, the present invention provides a method for detecting an object in an autonomous driving system, comprising:

[0006] Get original point cloud data;

[0007] voxelize the original point cloud data to obtain voxelized point cloud features, wherein the voxelized point cloud features are M*C dimensional point cloud features, where M is the number of voxels and C is the dimension of the mean of the feature vector of the point cloud data contained in each voxel;

[0008] The voxelized point cloud features are input into a convolutional network for feature extraction, and the extracted features are input into a target detection head network to obtain target detection results.

[0009] In one technical solution of the above-mentioned object detection method for autonomous driving, voxelizing the original point cloud data to obtain voxelized point cloud features includes:

[0010] Set the voxel size and obtain the voxel coordinates (x, y, z) of the original point cloud data;

[0011] Deduplicating original point cloud data in each voxel according to the voxel coordinates to obtain deduplicated point cloud data;

[0012] Sampling the deduplicated point cloud data in each voxel according to the voxel coordinates to obtain sampled point cloud data;

[0013] The feature vector mean is calculated for the sampled point cloud data in each voxel to obtain the voxelized point cloud feature.

[0014] In one technical solution of the above-mentioned target detection method for autonomous driving,

[0015] Deduplicating the original point cloud data in each voxel according to the voxel coordinates to obtain deduplicated point cloud data includes:

[0016] For each voxel, according to the voxel coordinates, the voxel coordinates (x, y, z) of each original point cloud data in the voxel are weighted calculated to obtain the key code value of each original point cloud data;

[0017] Establishing a one-dimensional hash table according to the corresponding relationship between the key code value and the original point cloud data;

[0018] According to the one-dimensional hash table, the original point cloud data with the same key code value in each voxel is averaged to achieve deduplication of the original point cloud data in each voxel and obtain deduplicated point cloud data.

[0019] In one technical solution of the above-mentioned target detection method for autonomous driving,

[0020] The step of sampling the deduplicated point cloud data in each voxel according to the voxel coordinates to obtain the sampled point cloud data includes:

[0021] According to the one-dimensional hash table, the deduplicated point cloud data corresponding to the first N key code values ​​in each voxel are obtained as the sampled point cloud data.

[0022] In one technical solution of the above-mentioned target detection method for autonomous driving,

[0023] The step of inputting the voxelized point cloud features into a convolutional network for feature extraction, and inputting the extracted features into a target detection head network to obtain target detection results includes:

[0024] Inputting the voxelized point cloud features into a sparse convolutional network for downsampling to obtain sparse voxel features;

[0025] Inputting the sparse voxel features into a top-view convolutional network for feature extraction to obtain features on the top view;

[0026] The features on the top view are input into the target detection head network to obtain the target detection results.

[0027] In one technical solution of the above-mentioned object detection method for autonomous driving, inputting the voxelized point cloud features into a sparse convolutional network for downsampling to obtain sparse voxel features includes:

[0028] Downsampling is performed in the height direction of the voxelized point cloud features using a sparse convolutional network with a convolution kernel size of 3, a stride of 3, and zero padding.

[0029] In one technical solution of the above-mentioned target detection method for autonomous driving, the top-view convolutional network is an asymmetric network structure, and the activation function used is the h-swish activation function.

[0030] In a second aspect, a control device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the autonomous driving target detection method described in any one of the technical solutions of the above-mentioned autonomous driving target detection method.

[0031] In a third aspect, a computer-readable storage medium is provided, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the target detection method for autonomous driving described in any one of the technical solutions of the above-mentioned target detection method for autonomous driving.

[0032] In a fourth aspect, a vehicle is provided, comprising:

[0033] Vehicle-mounted laser radar, used to obtain raw point cloud data;

[0034] The control device described in the above-mentioned control device technical solution.

[0035] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0036] In implementing the technical solution of the present invention, the present invention can obtain voxelized point cloud features within each voxel of the original point cloud data during the voxelization stage, input the voxelized point cloud data into a convolutional network for feature extraction, and further input the extracted features into the target detection head network to obtain target detection results. Through the above configuration, the present invention does not need to use an additional model to extract features and calculate the average value of the voxelized point cloud data. The features of the point cloud data within the voxel are integrated during the voxelization stage. Under the premise of ensuring the autonomous driving function, the initial extraction process of the original point cloud data is simplified, the calculation speed of the target detection is improved, and the computing pressure of the vehicle-side chip is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. Those skilled in the art will readily appreciate that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Among them:

[0038] Figure 1 This is a flow chart showing the main steps of a method for detecting an object in an autonomous driving process according to an embodiment of the present invention;

[0039] Figure 2 1. A schematic diagram comparing the main process architecture of an object detection method for autonomous driving according to an embodiment of the present invention and an object detection method for autonomous driving in the prior art;

[0040] Figure 3 1 is a flow chart of main steps for improving the computing speed of a target detection method according to an embodiment of the present invention;

[0041] Figure 4 FIG. 1 is a flow chart of main steps for obtaining voxelized point cloud features according to an implementation of an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Some embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0043] In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module may include hardware circuitry, various suitable sensors, communication ports, and memory. It may also include software components, such as program code, or a combination of software and hardware. A processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor may be implemented in software, hardware, or a combination of both. Non-transitory computer-readable storage media include any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "A and / or B" refers to all possible combinations of A and B, such as only A, only B, or both A and B. The terms "at least one of A or B" or "at least one of A and B" have similar meanings to "A and / or B" and may include only A, only B, or both A and B. The singular forms "one" and "the" may also include the plural forms.

[0044] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart showing the main steps of target detection in autonomous driving according to an embodiment of the present invention. Figure 1 As shown, the target detection method for autonomous driving in the embodiment of the present invention mainly includes the following steps S101 to S103.

[0045] Step S101: Obtain original point cloud data.

[0046] In this embodiment, the original point cloud data collected by the vehicle side can be obtained.

[0047] In one embodiment, the original point cloud data may be point cloud data collected by a vehicle-mounted laser radar.

[0048] Step S102: Voxelize the raw point cloud data to obtain voxelized point cloud features. Voxelized point cloud features are M*C dimensional point cloud features, where M is the number of voxels and C is the dimension of the mean feature vector of the point cloud data contained in each voxel. Voxelization converts the disordered raw point cloud data into regular data that is easy to perform convolution operations on.

[0049] In this embodiment, please refer to the attached Figure 2 , Figure 2 1 is a schematic diagram comparing the main process architecture of the target detection method for autonomous driving according to an embodiment of the present invention and the target detection method for autonomous driving in the prior art. Figure 2 As shown, the original point cloud data is A*C dimensional point cloud data, where A is the number of original point cloud data and C is the dimension of the feature vector of the original point cloud data. During the voxelization process, the A*C dimensional original point cloud data can be converted into M*C dimensional voxelized point cloud features (M<A), that is, the feature vectors of the point cloud data contained in each voxel are averaged to obtain the feature vector mean, and the feature vector mean is used as the voxelized point cloud feature in the voxel. The method in the prior art is to first voxelize the A*C dimensional original point cloud data to obtain M*T*C dimensional voxel data (M represents the number of voxels and T represents the number of point clouds in each voxel), and then send the M*T*C dimensional voxel data to the VFE (Voxel Feature Extraction, point cloud feature extraction) network for feature extraction to obtain M*K dimensional voxel intra-features (K is the dimension of the extracted point cloud features), and then input the M*K dimensional voxel intra-features into the sparse convolutional network for feature extraction. Compared with the methods in the prior art, the method in the embodiment of the present invention does not require retraining and using the VFE network to extract the original point cloud data. It only requires taking the feature mean of the point cloud data within the voxel to obtain the voxelized point cloud features, which can simplify the processing process of the original point cloud data and improve the computing speed of the target detection process.

[0050] Step S103: Input the voxelized point cloud features into the convolutional network for feature extraction, and input the extracted features into the target detection head network to obtain the target detection results.

[0051] In this embodiment, the voxelized point cloud features obtained in step S102 can be input into a convolutional network to extract features from the voxelized point cloud features, and the extracted features can be input into a target detection head network to achieve target detection for autonomous driving.

[0052] In one embodiment, the object detection head network may be a region proposal network (RPN).

[0053] Based on the above steps S101-S103, the embodiment of the present invention can realize that each voxel of the original point cloud data contains the mean of the feature vector of the point cloud data during the voxelization stage, and input the voxelized point cloud data into the convolutional network for feature extraction, and further input the extracted features into the target detection head network to obtain the target detection results. Through the above configuration, the embodiment of the present invention does not need to use an additional model to extract features and calculate the mean of the voxelized point cloud data. The features of the point cloud data within the voxel are integrated during the voxelization stage. Under the premise of ensuring the autonomous driving function, the initial extraction process of the original point cloud data is simplified, the computing speed of the target detection is improved, and the computing pressure of the vehicle-side chip is reduced.

[0054] Step S102 and step S103 are further described below.

[0055] In one embodiment of the present invention, see the attached Figure 4 , Figure 4 FIG. 1 is a flow chart of main steps for obtaining voxelized point cloud features according to an implementation of an embodiment of the present invention.

[0056] like Figure 4 As shown, step S102 may include the following steps S1021 to S1024:

[0057] Step S1021: setting the voxel size and obtaining the voxel coordinates (x, y, z) of the original point cloud data;

[0058] In this embodiment, the voxel size may be set first, and the voxel coordinates of the original point cloud data may be obtained.

[0059] In one embodiment, the three-dimensional space where the original point cloud data is located may be divided according to a preset size, and the three-dimensional space may be divided into a plurality of voxels of uniform size.

[0060] In one embodiment, the voxel coordinates (x, y, z) of the original point cloud data may be obtained based on the relationship between the original point cloud data coordinate system and the voxel coordinate system.

[0061] In one implementation, a Voxelizer (voxel generator) may be used to set the voxel size.

[0062] Step S1022: Deduplication of the original point cloud data in each voxel according to the voxel coordinates to obtain deduplication point cloud data.

[0063] In this embodiment, step S1022 may further include the following steps S10221 to S10223:

[0064] Step S10221: For each voxel, perform weighted calculation on the voxel coordinates (x, y, z) of each original point cloud data in the voxel according to the voxel coordinates to obtain the key code value of each original point cloud data.

[0065] Step S10222: A one-dimensional hash table is created based on the corresponding relationship between the key value and the original point cloud data. A hash table is a data structure that is directly accessed based on the key value.

[0066] Step S10223: according to the one-dimensional hash table, the original point cloud data with the same key code value in each voxel is averaged to achieve deduplication of the original point cloud data in each voxel and obtain the deduplicated point cloud data.

[0067] In this embodiment, for each voxel, the voxel coordinates of each original point cloud data in the voxel can be weighted to obtain the key code value of each original point cloud data. A one-dimensional hash table is established based on the correspondence between the key code value and the original point cloud data, that is, the correspondence between the key code value and the original point cloud data can be queried through the one-dimensional hash table. And the original point cloud data is deduplicated according to the one-dimensional hash table. Specifically, the original point cloud data with the same key code value in each voxel is averaged, and the obtained average is used as the point cloud data corresponding to the key code value to obtain the deduplicated point cloud data. Those skilled in the art can set the weights of the voxel coordinates (x, y, z) according to the actual application needs.

[0068] Step S1023: sampling the deduplicated point cloud data in each voxel according to the voxel coordinates to obtain sampled point cloud data.

[0069] In this embodiment, step S1023 may further include the following steps:

[0070] According to the one-dimensional hash table, the deduplicated point cloud data corresponding to the first N key code values ​​in each voxel are obtained as the sampled point cloud data.

[0071] In this embodiment, based on the one-dimensional hash table established in step S1022, the deduplicated point cloud data corresponding to the first N key values ​​in each voxel can be used as the sampled point cloud data. Where N is a positive integer greater than 1. Those skilled in the art can set the value of N based on actual application needs.

[0072] Step S1024: Calculate the mean of the feature vectors of the sampled point cloud data in each voxel to obtain voxelized point cloud features.

[0073] In this embodiment, the feature vector mean value may be calculated for the sampled point cloud data in each voxel, and the feature vector mean value may be used as the voxelized point cloud feature in the voxel.

[0074] In one implementation of the embodiment of the present invention, step S103 may further include the following steps S1031 to S1033:

[0075] Step S1031: input the voxelized point cloud features into a sparse convolutional network for downsampling to obtain sparse voxel features.

[0076] Among them, the sparse convolutional network (Sparse Conv Network) is a 3D sparse convolutional network. The advantage of the 3D sparse convolutional network is that it not only considers the 3D spatial information of the 3D voxel features, but also utilizes the sparsity of the 3D voxel features, making the feature extraction process computationally less complex.

[0077] In one embodiment, the number of convolutional layers in the sparse convolutional network is 6. This can significantly speed up the operation of the sparse convolutional network.

[0078] In one embodiment, step S1031 may further include the following steps:

[0079] Downsampling is performed in the height direction of the voxelized point cloud features using a sparse convolutional network with a convolution kernel size of 3, a stride of 3, and zero padding.

[0080] In this embodiment, when downsampling in the height direction (Z axis), different from the setting method of the sparse convolution network in the prior art, the convolution kernel (kernelsize) of the sparse convolution network is set to 3, the stride (stride) is set to 3, and the padding (padding) is set to 0. This can achieve downsampling in the height direction faster and is also conducive to the extraction of sparse point cloud features.

[0081] Step S1032: Input the sparse voxel features into the top view convolutional network for feature extraction to obtain features on the top view.

[0082] In this embodiment, sparse voxel features can be input into a bird's eye view (BEV) convolutional network for feature extraction to obtain features on the bird's eye view. Inputting sparse voxel features into a bird's eye view convolutional network for feature extraction can increase the receptive field of the convolutional network.

[0083] In one embodiment, the top view convolutional network is an asymmetric network structure, in which the activation function used is the h-swish (hard swish) activation function.

[0084] In this embodiment, different from the commonly used VGG (Visual Geometry Group Network) structure or residual-block network structure, in this embodiment, the top view convolutional network is an asymmetric network structure, that is, the top view convolutional network is an asymmetric convolutional network. The application of asymmetric convolutional networks can effectively reduce the amount of computation of the convolutional network, thereby improving the computational efficiency of the top view convolutional network. At the same time, the activation function used by the top view convolutional network is the h-swish activation function. The application of the h-swish activation function can achieve better training effects in deep networks, and the amount of computation is small, which is more suitable for lightweight networks such as top view convolutional neural networks.

[0085] Step S1033: Input the features on the top view into the target detection head network to obtain the target detection results.

[0086] In this embodiment, the features on the top view can be input into the target detection head network, and the target detection result can be obtained by performing target detection.

[0087] In one embodiment, see the attached Figure 3 , Figure 3 FIG. 1 is a flow chart showing the main steps of improving the computational speed of a target detection method according to an embodiment of the present invention. Figure 3 As shown, the computational speed of the target detection method can be improved by accelerating the point cloud data flow, the network structure (convolutional network), and the CUDA (Compute Unified Device Architecture) operator. Specifically, point cloud data flow acceleration can be achieved through steps S1021 to S1024 described above; network structure acceleration is achieved by pruning the sparse convolutional network (reducing the number of convolutional layers, etc.); and CUDA operator acceleration is achieved by optimizing the computational flow of the Voxelizer module.

[0088] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0089] Those skilled in the art will appreciate that all or part of the processes in the method for implementing the above-mentioned embodiment of the present invention may also be accomplished by instructing the relevant hardware through a computer program. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, it may implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal, and software distribution medium capable of carrying the computer program code. It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0090] Furthermore, the present invention also provides a control device. In one embodiment of the control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the target detection method for autonomous driving according to the above-mentioned method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to a program for executing the target detection method for autonomous driving according to the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of the present invention. The control device can be a control device device formed by various electronic devices.

[0091] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of a computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the target detection method for autonomous driving according to the above-mentioned method embodiment. The program can be loaded and run by a processor to implement the target detection method for autonomous driving. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-transitory computer-readable storage medium.

[0092] Furthermore, the present invention also provides a vehicle. In one embodiment of the vehicle according to the present invention, the vehicle may include an onboard laser radar and the control device in the above control device embodiment. The onboard laser radar can be used to obtain raw point cloud data.

[0093] Furthermore, it should be understood that since the configuration of each module is merely for the purpose of illustrating the functional units of the apparatus of the present invention, the physical devices corresponding to these modules may be the processor itself, or a portion of the software in the processor, a portion of the hardware, or a combination of software and hardware. Therefore, the number of modules in the figure is merely illustrative.

[0094] Those skilled in the art will appreciate that the various modules in the device can be adaptively split or merged. Such splitting or merging of specific modules does not cause the technical solution to deviate from the principles of the present invention. Therefore, the technical solutions after splitting or merging will fall within the scope of protection of the present invention.

[0095] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A target detection method for autonomous driving, characterized in that: include: Get original point cloud data; Set the voxel size and get the voxel coordinates (x, y, z) of the original point cloud data; For each voxel, performing weighted calculation on the voxel coordinates (x, y, z) of each original point cloud data in the voxel according to the voxel coordinates to obtain a key code value of each original point cloud data; Establishing a one-dimensional hash table according to the corresponding relationship between the key code value and the original point cloud data; averaging the original point cloud data with the same key code value in each voxel according to the one-dimensional hash table to deduplicate the original point cloud data in each voxel and obtain deduplicated point cloud data; Sampling the deduplicated point cloud data in each voxel according to the voxel coordinates to obtain sampled point cloud data; The mean of the feature vectors of the sampled point cloud data in each voxel is calculated to obtain the voxelized point cloud features, where the voxelized point cloud features are M*C dimensional point cloud features, where M is the number of voxels and C is the dimension of the mean of the feature vectors of the point cloud data contained in each voxel. The voxelized point cloud features are input into a convolutional network for feature extraction, and the extracted features are input into a target detection head network to obtain target detection results.

2. The method according to claim 1, characterized in that The step of sampling the deduplicated point cloud data in each voxel according to the voxel coordinates to obtain the sampled point cloud data includes: According to the one-dimensional hash table, the deduplicated point cloud data corresponding to the first N key code values ​​in each voxel are obtained as the sampled point cloud data.

3. The method according to claim 1, characterized in that The step of inputting the voxelized point cloud features into a convolutional network for feature extraction, and inputting the extracted features into a target detection head network to obtain target detection results includes: Inputting the voxelized point cloud features into a sparse convolutional network for downsampling to obtain sparse voxel features; Inputting the sparse voxel features into a top-view convolutional network for feature extraction to obtain features on the top view; The features on the top view are input into the target detection head network to obtain the target detection results.

4. The method according to claim 3, characterized in that Inputting the voxelized point cloud features into a sparse convolutional network for downsampling to obtain sparse voxel features includes: Downsampling is performed in the height direction of the voxelized point cloud features using a sparse convolutional network with a convolution kernel size of 3, a stride of 3, and zero padding.

5. The method according to claim 3, characterized in that The top view convolutional network is an asymmetric network structure, in which the activation function used is the h-swish activation function.

6. A control device comprising a processor and a storage device, wherein the storage device is adapted to store a plurality of program codes, wherein: The program code is suitable for being loaded and run by the processor to execute the target detection method for autonomous driving according to any one of claims 1 to 5.

7. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the target detection method for autonomous driving according to any one of claims 1 to 5.

8. A vehicle, characterized in that: include: Vehicle-mounted laser radar, used to obtain raw point cloud data; The control device according to claim 6.

Images