Target detection method and device, vehicle, storage medium, chip and electronic equipment

Abstract

The present disclosure relates to the field of automatic driving, and relates to a target detection method and device, a vehicle, a storage medium, a chip and an electronic device. The method comprises: acquiring a plurality of point cloud data collected by a vehicle during driving; dividing the point cloud data into a plurality of point cloud data block sets, different point cloud data block sets corresponding to different height ranges in the vertical direction of the driving direction of the vehicle, each point cloud data block set comprising a plurality of point cloud data blocks, and each point cloud data block comprising a plurality of point cloud data; determining a pseudo image corresponding to each point cloud data block set; determining a target detection region according to a plurality of pseudo images; and determining a target object in the target detection region. In this way, objects whose point clouds are blocked can be effectively identified, and the accuracy of target detection is improved.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving, and more particularly to a target detection method, apparatus, vehicle, storage medium, chip, and electronic device. Background Technology

[0002] Point cloud data is a collection of point data on the surface of a target object detected by scanning devices such as LiDAR. In autonomous driving scenarios, point clouds acquired by LiDAR sensors are characterized by sparseness, close proximity to the road surface, low height, and small sample size. When using point cloud data for target detection, it is usually necessary to divide the point cloud data into multiple voxels, then use the average value of multiple points contained in each voxel as the feature information of each voxel, and finally use the feature information of each voxel for target detection.

[0003] However, because the number of voxels obtained from the segmentation is relatively large, while the number of point clouds in a large number of voxels is relatively small, the extracted feature information of the voxels is not accurate enough, resulting in a relatively low accuracy of target detection. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a target detection method, apparatus, vehicle, storage medium, chip, and electronic device.

[0005] According to a first aspect of the present disclosure, a target detection method is provided, comprising:

[0006] Acquire multiple point cloud data collected during vehicle operation;

[0007] The point cloud data is divided into multiple point cloud data block sets. Different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle's driving direction. Each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple point cloud data.

[0008] Determine the pseudo-image corresponding to each set of point cloud data blocks;

[0009] The target detection region is determined based on the multiple pseudo-images;

[0010] Identify the target object in the target detection area.

[0011] Optionally, dividing the point cloud data into multiple point cloud data block sets includes:

[0012] According to the horizontal plane where the vehicle's direction of travel is located, the multiple point cloud data are divided into multiple point cloud data blocks;

[0013] According to the direction perpendicular to the vehicle's driving direction, the multiple point cloud data blocks are divided into multiple point cloud data block sets.

[0014] Optionally, dividing the multiple point cloud data blocks into multiple point cloud data block sets according to the direction perpendicular to the vehicle's driving direction includes:

[0015] Based on the vertical position coordinates of the multiple point cloud data in the direction perpendicular to the vehicle's driving direction, determine the total height range corresponding to the multiple point cloud data.

[0016] Based on the preset number of sets and the total height range, multiple target height ranges are determined;

[0017] For each target height range, multiple target point cloud data with vertical position coordinates within the target height range are determined from multiple point cloud data, and the multiple target point cloud data are used as a set of point cloud data blocks corresponding to the target height range.

[0018] Optionally, determining multiple target height ranges based on the preset set size and the total height range includes:

[0019] Based on the preset set size, the total height range is divided into multiple target height ranges on an average basis; or...

[0020] Based on the vertical position coordinates of multiple point cloud data in the direction perpendicular to the vehicle's driving direction, the number of point cloud data at different heights is determined, and the total height range is divided into multiple target height ranges according to the preset set size and the number of point clouds.

[0021] Optionally, determining the target detection region based on the plurality of pseudo-images includes:

[0022] Determine the target feature map corresponding to each of the pseudo-images;

[0023] The target detection region is determined based on multiple target feature maps.

[0024] Optionally, determining the target feature map corresponding to each of the pseudo-images includes:

[0025] Multiple pseudo-images are input into a pre-trained feature extraction model to obtain multiple target feature maps output by the feature extraction model.

[0026] Optionally, the feature extraction model includes multiple feature extraction sub-models, with different pseudo-images corresponding to different feature extraction sub-models; the step of inputting multiple pseudo-images into the pre-trained feature extraction model to obtain multiple target feature maps output by the feature extraction model includes:

[0027] For each pseudo-image, based on the number of points in the point cloud data set corresponding to the pseudo-image, the number of convolutional kernels of the target feature extraction sub-model corresponding to the pseudo-image is determined from a pre-created convolutional kernel association relationship. The pseudo-image is then input into the target feature extraction sub-model, and convolution processing is performed on the pseudo-image according to the number of convolutional kernels of the target feature extraction sub-model to obtain the target feature map output by the target feature extraction sub-model. The convolutional kernel association relationship includes the correspondence between different point cloud numbers and the number of convolutional kernels.

[0028] Optionally, determining the target detection region based on the plurality of target feature maps includes:

[0029] Multiple target feature maps are input into a pre-trained region detection model to obtain the target detection region output by the region detection model.

[0030] According to a second aspect of the present disclosure, a target detection apparatus is provided, comprising:

[0031] The acquisition module is configured to acquire multiple point cloud data collected by the vehicle during its driving process;

[0032] The partitioning module is configured to divide the point cloud data into multiple point cloud data block sets. Different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle's driving direction. Each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple point cloud data.

[0033] The first determining module is configured to determine the pseudo image corresponding to each set of point cloud data blocks;

[0034] The second determining module is configured to determine the target detection region based on the plurality of said pseudo images;

[0035] The third determining module is configured to determine the target object in the target detection region.

[0036] Optionally, the partitioning module includes:

[0037] The first partitioning submodule is configured to divide the multiple point cloud data into multiple point cloud data blocks according to the horizontal plane where the vehicle travels.

[0038] The second partitioning submodule is configured to partition the multiple point cloud data blocks into multiple point cloud data block sets according to the vertical direction of the vehicle's driving direction.

[0039] Optionally, the second partitioning submodule is configured to first determine the total height range corresponding to the multiple point cloud data based on the vertical position coordinates of the multiple point cloud data in the direction perpendicular to the vehicle's driving direction; secondly, determine multiple target height ranges based on a preset set size and the total height range; and then, for each target height range, determine multiple target point cloud data whose vertical position coordinates are within the target height range from the multiple point cloud data, and use the multiple target point cloud data as a set of point cloud data blocks corresponding to the target height range.

[0040] Optionally, the second division submodule is configured to divide the total height range into multiple target height ranges on an average basis according to the preset set number; or, based on the vertical position coordinates of the multiple point cloud data in the direction perpendicular to the vehicle's driving direction, determine the number of point cloud data at different heights, and divide the total height range into multiple target height ranges according to the preset set number and the number of point cloud data.

[0041] Optionally, the second determining module includes:

[0042] The first determining submodule is configured to determine the target feature map corresponding to each of the pseudo-images;

[0043] The second determining submodule is configured to determine the target detection region based on multiple target feature maps.

[0044] Optionally, the first determining submodule is configured to input a plurality of the pseudo-images into a pre-trained feature extraction model to obtain a plurality of target feature maps output by the feature extraction model.

[0045] Optionally, the feature extraction model includes multiple feature extraction sub-models, with different pseudo-images corresponding to different feature extraction sub-models; the first determining sub-module is configured to, for each pseudo-image, determine the number of convolution kernels of the target feature extraction sub-model corresponding to the pseudo-image from a pre-created convolution kernel association relationship based on the number of points in the point cloud data block set corresponding to the pseudo-image, and input the pseudo-image into the target feature extraction sub-model, performing convolution processing on the pseudo-image according to the number of convolution kernels of the target feature extraction sub-model to obtain the target feature map output by the target feature extraction sub-model, wherein the convolution kernel association relationship includes the correspondence between different point cloud numbers and the number of convolution kernels.

[0046] Optionally, the second determining submodule is configured to input multiple target feature maps into a pre-trained region detection model to obtain the target detection region output by the region detection model.

[0047] According to a third aspect of the present disclosure, a vehicle is provided, comprising:

[0048] processor;

[0049] Memory used to store processor-executable instructions;

[0050] The processor is configured as follows:

[0051] Acquire multiple point cloud data collected during vehicle operation;

[0052] The point cloud data is divided into multiple point cloud data block sets. Different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle's driving direction. Each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple point cloud data.

[0053] Determine the pseudo-image corresponding to each set of point cloud data blocks;

[0054] The target detection region is determined based on the multiple pseudo-images;

[0055] Identify the target object in the target detection area.

[0056] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the target detection method provided in the first aspect of the present disclosure.

[0057] According to a fifth aspect of the present disclosure, a chip includes a processor and an interface; the processor is configured to read instructions to perform the steps of the target detection method provided in the first aspect of the present disclosure.

[0058] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising:

[0059] A memory on which computer programs are stored;

[0060] A processor; configured to execute the computer program in the memory to implement the steps of the target detection method provided in the first aspect of this disclosure.

[0061] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0062] By acquiring multiple point cloud data points collected during vehicle travel, the point cloud data is divided into multiple point cloud data block sets. Different point cloud data block sets correspond to different height ranges vertically to the vehicle's travel direction. Each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple point cloud data points. A pseudo-image is determined for each point cloud data block set. Based on the multiple pseudo-images, a target detection region is determined. The target object within the target detection region is then identified. In this way, by dividing the point cloud data according to different height ranges vertically to the vehicle's travel direction, different point cloud data block sets are obtained. Then, the point cloud data in these different point cloud data block sets is processed. For objects with dense vertical point clouds, more detailed features can be learned by dividing the data according to height, effectively identifying objects whose point clouds are occluded, and improving the accuracy of target detection.

[0063] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0064] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0065] Figure 1 This is a flowchart illustrating a target detection method according to an exemplary embodiment.

[0066] Figure 2 This is a flowchart illustrating another target detection method according to an exemplary embodiment.

[0067] Figure 3 This is a schematic diagram of a point cloud data block according to an exemplary embodiment.

[0068] Figure 4 This is a block diagram of a target detection device according to an exemplary embodiment.

[0069] Figure 5 It is based on Figure 4 The embodiment shown illustrates a block diagram of a partitioning module.

[0070] Figure 6 It is based on Figure 4 The illustrated embodiment shows a block diagram of a second determining module.

[0071] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0072] Figure 8This is a functional block diagram of a vehicle illustrating an exemplary embodiment. Detailed Implementation

[0073] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0074] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0075] Before detailing the specific implementation methods of this disclosure, the application scenarios of this disclosure will first be explained. Currently, point cloud target detection algorithms based on deep learning are the mainstream research direction. Existing point cloud-based 3D target detection methods typically employ voxel-based point cloud target detection methods. On the one hand, they use cubes of the same size as voxels to divide the point cloud data. However, setting a single voxel size can easily lose some local structures of the point cloud data, which is insufficient to represent all available information in the detection scene, thus reducing detection accuracy. On the other hand, feature encoding of voxelized point cloud data does not consider the positional relationships between points, resulting in a lack of rich feature representation of the point cloud data, which is not conducive to feature learning and further reduces detection accuracy.

[0076] To overcome the technical problems existing in the above-mentioned related technologies, this disclosure provides a target detection method, device, storage medium, and vehicle. By dividing point cloud data according to different height ranges in the vertical direction of the vehicle's driving direction, different sets of point cloud data blocks are obtained. Then, the point cloud data in the different sets of point cloud data blocks are processed. This enables the learning of more detailed features for objects with dense vertical point clouds according to height division, which can effectively identify objects whose point clouds are occluded and improve the accuracy of target detection.

[0077] The present disclosure will now be described in conjunction with specific embodiments.

[0078] Figure 1 This is a flowchart illustrating a target detection method according to an exemplary embodiment, such as... Figure 1 As shown, the method may include:

[0079] In step S101, multiple point cloud data collected during the vehicle's driving process are acquired.

[0080] The point cloud data can be a set of three-dimensional coordinate vectors recorded in the form of point clouds, obtained by the LiDAR during the scanning of the current scene. Each three-dimensional coordinate vector can be represented by [x, y, z]. In addition, the point cloud data can also include the reflected light intensity value r of each point cloud, and each point cloud data can be represented by [x, y, z, r]. The multiple point cloud data collected by the vehicle during its driving can be multiple point cloud data acquired by the vehicle through LiDAR scanning in the current scene.

[0081] Specifically, a fixed LiDAR on the vehicle can collect point cloud data of the surrounding environment and transmit the collected point cloud data to the control equipment for target detection. For example, a fixed LiDAR on an autonomous vehicle can collect point cloud data of the surrounding environment in real time while the vehicle is in motion and transmit it to the control equipment of the autonomous vehicle. The point cloud data transmitted by the LiDAR to the control equipment includes the three-dimensional coordinates of each point in the current scene and the reflected light intensity value at that point.

[0082] It should be noted that the point cloud data can be cropped according to the actual space dimensions to be perceived in the specified scene to obtain a target point cloud with dimensions of L×W×H. This removes redundant points in the point cloud data and improves data computation efficiency. Here, L, W, and H are the length, width, and height of the aforementioned actual space, respectively, and L, W, and H are the length dimensions corresponding to the x, y, and z axes of the three-dimensional coordinate system.

[0083] In step S102, the point cloud data is divided into multiple point cloud data block sets, and different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle's driving direction.

[0084] Each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple points cloud data. For example, the point cloud data block can be cubic in shape.

[0085] In this step, since the obtained point cloud data is spatially distributed, it can be gridded to facilitate analysis, ensuring that the spatially distributed point cloud data falls within the grid. When gridding the spatial distribution of the point cloud data, firstly, without considering the vertical height data, the horizontal plane where the vehicle's driving direction is located is taken as the scene plane. Several grids of the same size are divided on this scene plane, and then multiple point cloud data points are divided according to the size of these grids, resulting in several columnar point cloud grids. Then, according to the vertical direction of the vehicle's driving direction, each columnar point cloud grid is divided according to different height ranges. For each columnar point cloud grid, it is further divided vertically into multiple point cloud data blocks. Multiple point cloud data blocks within the same height range constitute a point cloud data layer. Each point cloud data layer includes multiple point cloud data blocks, and the multiple point cloud data blocks in a point cloud data layer constitute a set of point cloud data blocks.

[0086] In step S103, the pseudo-image corresponding to each point cloud data block set is determined.

[0087] Optionally, feature expansion can be performed on each data point included in each point cloud data block to obtain multiple point cloud data blocks with expanded features. One implementation involves expanding the feature [x,y,z,r] of each point cloud data point in the block into a 9-dimensional feature [x,y,z,r,x-Δx,y-Δy,z-Δz,x-xc,y-yc], where [r] represents the reflection intensity value of the point cloud data point itself, [x,y,z] represents the coordinates of the data point in three-dimensional space, [Δx,Δy,Δz] represents the average coordinates of all points within the point cloud data block, and [xc,yc] represents the center coordinates of the point cloud data point projected horizontally into the block. Another implementation involves expanding the feature [x,y,z,r] of each point cloud data point in the block into a 16-dimensional or higher feature.

[0088] Furthermore, due to the sparsity of point cloud data, multiple point cloud data blocks may contain no points or a very small number of points. Considering computational complexity, the number of points in each point cloud data block can be limited by the same amount, processing at most P non-empty point cloud data blocks. This can be achieved by imposing restrictions on the number of point cloud data blocks (P) in each point cloud data block set and the number of points (N) in each point cloud data block, creating a tensor of size (D, P, N), where D can refer to the point cloud feature dimension of each point cloud data. Specifically, if a point cloud data block contains too much data, a random sampling operation is performed to ensure that the number of points in that block is the preset number N; if the data is too little, it is padded with zeros for the preset number N. This allows for the creation of a tensor of size (D, P, N) for each point cloud data block set.

[0089] Furthermore, after obtaining a tensor of size (D, P, N), a simplified version of PointNet can be used to process and extract features from the tensorized point cloud data (i.e., applying a linear layer + BN layer + ReLU layer to each point) to generate a tensor of size (C, P, N). Then, max pooling is applied to each channel to output a tensor of size (C, P). Finally, the encoded features are scattered back to the original point cloud data block locations to create a pseudo-image of size (C, H, W).

[0090] It should be noted that after obtaining the (C, P) tensor, the (C, P) tensor generated in the previous step can be converted back to its original point cloud data block coordinates using the grid index value of each point, in order to create a pseudo-image of size (C, H, W). The height H and width W of the pseudo-image are determined by setting the range of point cloud coordinates on the XY plane and the size of each columnar point cloud grid when dividing multiple point cloud data into columnar point cloud grids. For example, assuming the X-axis range is [0, 69.12], the Y-axis range is [-39.68, 39.68], and the size of each columnar point cloud grid is 0.16*0.16, then if the X-axis represents the width, the Y-axis represents the height, and one columnar point cloud grid represents one pixel, then the width W = (69.12 - 0) / 0.16 = 432, and the height H = (39.68 - (-39.68)) / 0.16 = 496.

[0091] In step S104, the target detection region is determined based on the multiple pseudo-images.

[0092] In this step, the target feature map corresponding to each pseudo image can be determined first. For example, multiple pseudo images can be input into a pre-trained feature extraction model to obtain multiple target feature maps output by the feature extraction model. The feature extraction model includes multiple feature extraction sub-models, and different pseudo images correspond to different feature extraction sub-models.

[0093] Specifically, for each pseudo-image, based on the number of points in the point cloud data set corresponding to the pseudo-image, the number of convolutional kernels of the target feature extraction sub-model corresponding to the pseudo-image is determined from the pre-created convolutional kernel association relationship. The pseudo-image is then input into the target feature extraction sub-model, and convolution processing is performed on the pseudo-image according to the number of convolutional kernels of the target feature extraction sub-model to obtain the target feature map output by the target feature extraction sub-model. The convolutional kernel association relationship includes the correspondence between different point cloud numbers and the number of convolutional kernels.

[0094] Then, based on multiple feature maps of the target, the target detection region is determined. Specifically, multiple feature maps of the target can be input into a pre-trained region detection model to obtain the target detection region output by the region detection model.

[0095] In step S105, the target object in the target detection area is determined.

[0096] Using the above method, multiple point cloud data are acquired during vehicle movement. This point cloud data is then divided into multiple point cloud data block sets, each corresponding to a different height range in the vertical direction of the vehicle's travel direction. Each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple instances of that point cloud data. A pseudo-image is determined for each point cloud data block set. Based on these pseudo-images, a target detection region is determined. Finally, the target object within this target detection region is identified. By dividing the point cloud data according to different height ranges in the vertical direction of the vehicle's travel direction, different point cloud data block sets are obtained. Processing the point cloud data within these sets allows for the learning of more detailed features for objects with dense vertical point clouds, effectively identifying occluded objects and improving target detection accuracy. Furthermore, compared to using cubes of the same size as voxels to divide the point cloud data, this method improves detection efficiency.

[0097] Figure 2 This is a flowchart illustrating another target detection method according to an exemplary embodiment, such as... Figure 2 As shown, the method may include:

[0098] In step S201, multiple point cloud data collected during the vehicle's driving process are acquired.

[0099] In this step, the point cloud data can be a set of three-dimensional coordinate vectors recorded in the form of point clouds obtained by the LiDAR during the scanning of the current scene. Each three-dimensional coordinate vector can be represented by [x, y, z]. In addition, the point cloud data can also include the reflected light intensity value r of each point cloud, so each point cloud data can be represented by [x, y, z, r]. The multiple point cloud data collected by the vehicle during its movement can be multiple point cloud data acquired by the vehicle through LiDAR scanning in the current scene.

[0100] In step S202, the multiple point cloud data are divided into multiple point cloud data blocks according to the horizontal plane where the vehicle's driving direction is located.

[0101] For example, the horizontal plane where the vehicle's direction of travel is located can be used as the scene plane. With the vehicle's location as the origin, a two-dimensional coordinate system of a grid plane can be established on this scene plane. Without considering the vertical height data of multiple point cloud data sets, the multiple point cloud data sets are projected onto this two-dimensional coordinate system. According to the coordinates of this two-dimensional coordinate system, the data is divided into multiple grids, ultimately resulting in multiple cylindrical point cloud grids. For instance, if the preset coordinate system is a Cartesian coordinate system, the multiple target point clouds with a clipped space of L×W×H are divided at equal intervals along the x-axis and y-axis of the Cartesian coordinate system to obtain multiple cylindrical point cloud grids. Then, by dividing the grids according to different height ranges vertical to the vehicle's direction of travel, different point cloud data blocks are obtained. That is, the size of each point cloud data block is dL×dW×dH. Figure 3 As shown, Figure 3 This is a schematic diagram of a point cloud data block according to an exemplary embodiment.

[0102] In step S203, the multiple point cloud data blocks are divided into multiple point cloud data block sets according to the direction perpendicular to the vehicle's driving direction.

[0103] In this step, the total height range corresponding to multiple point cloud data can first be determined based on their vertical position coordinates in the direction perpendicular to the vehicle's travel direction. This total height range can refer to the range of the vertical coordinates of all point cloud data within the multiple point cloud datasets. Next, multiple target height ranges are determined based on a preset set size and the total height range. Finally, for each target height range, multiple target point cloud data with vertical coordinates within that target height range are identified from the multiple point cloud data datasets, and these target point cloud data are used as the point cloud data block set corresponding to that target height range. For example, based on the vertical height coordinate range of the target height range, multiple point cloud data within that vertical height coordinate range can be identified from the multiple point cloud data datasets, and the point cloud data within that vertical height coordinate range are considered the target point cloud data.

[0104] Specifically, during the process of acquiring multiple point cloud data by the vehicle, the total height range of the multiple point cloud data can be determined. Then, the total height range can be divided by a preset unit height to obtain the preset set size. The preset unit height can be determined by the staff during the feature extraction process based on the accuracy of the extracted feature information. In this case, the preset unit height can be directly used as multiple target height ranges. For example, if the total height range is -1m to 4m and the unit height is 1m, the preset set size can be determined as [4m-(-1m)] / 1m=5, and 1m can be directly used as multiple target height ranges. In another possible implementation, the preset set size can be determined by pre-setting a fixed unit point cloud size. During the process of acquiring multiple point cloud data by the vehicle, after the initial acquisition of point cloud data, the number of points in the original multiple point cloud data can be determined first. Then, based on the unit point cloud size, the original multiple point cloud data can be divided into multiple point cloud data layers. The preset set size can be determined based on the number of multiple point cloud data layers, where each point cloud data layer includes the same number of original point cloud data.

[0105] In one possible implementation, the total height range can be determined first based on the location information of multiple point cloud data. Then, the preset set size can be obtained, and the total height range can be divided into multiple target height ranges on an average basis according to the preset set size.

[0106] For example, the total height range can first be divided into multiple target height ranges according to the preset set number. Then, multiple point cloud data blocks can be divided into a preset set number of point cloud data block sets according to the target height range. The point cloud grid is a columnar point cloud grid obtained after dividing it without considering the vertical height. Specifically, each columnar point cloud grid can be divided into the preset set number of averages to obtain the preset set number of target height ranges. For example, if the vertical coordinate range of each point cloud data is -1m to 4m, the preset set number can be set to 5. This way, multiple point cloud grids can be divided into 5 target height ranges according to the preset set number, which are 5 target height ranges from 0m to 1m.

[0107] Alternatively, in another possible implementation, the number of point cloud data at different heights can be determined based on the vertical position coordinates of multiple point cloud data in the direction perpendicular to the vehicle's travel direction. Then, based on the preset set size and the number of point cloud data, the total height range is divided into multiple target height ranges. For example, if the vertical position coordinate of each point cloud data point is in the range of -1m to 4m, and the number of point cloud data points between 1m and 2m is the first size, and the number of point cloud data points between 2m and 3m is the second size, and the first size is much larger than the second size, a denser target height range can be divided within the 1m to 2m height range. This results in a smaller target height range within the 1m to 2m height range compared to the 2m to 3m height range. In other words, the denser the point cloud data, the smaller the target height range.

[0108] In step S204, the pseudo-image corresponding to each point cloud data block set is determined.

[0109] In this process, multiple points cloud data from each set of point cloud data blocks can be input into a preset feature extraction model to obtain a pseudo image corresponding to that set of point cloud data blocks.

[0110] In step S205, the target feature map corresponding to each pseudo-image is determined.

[0111] In this step, multiple pseudo-images can be input into a pre-trained feature extraction model to obtain multiple target feature maps output by the model. This feature extraction model includes multiple feature extraction sub-models, with different pseudo-images corresponding to different sub-models. This feature extraction model can be trained using existing model training methods, which will not be elaborated upon here.

[0112] Specifically, for each pseudo-image, based on the number of points in the point cloud data set corresponding to the pseudo-image, the number of convolutional kernels of the target feature extraction sub-model corresponding to the pseudo-image can be determined from the pre-created convolutional kernel association relationship. The pseudo-image is then input into the target feature extraction sub-model, and convolution processing is performed on the pseudo-image according to the number of convolutional kernels of the target feature extraction sub-model to obtain the target feature map output by the target feature extraction sub-model. The convolutional kernel association relationship includes the correspondence between different point cloud numbers and the number of convolutional kernels.

[0113] For example, for each pseudo-image, the set of point cloud data blocks corresponding to the pseudo-image is determined by the grid index value of the point cloud data. Then, based on the number of points in the point cloud data block set, and according to the preset correspondence between different point cloud numbers and the number of convolutional kernels, the number of convolutional kernels corresponding to the set of point cloud data blocks is determined. Then, the corresponding target feature extraction sub-model is determined based on the number of convolutional kernels. The pseudo-image can then be input into the target feature extraction sub-model to obtain the target feature map corresponding to each pseudo-image. The feature extraction model includes multiple target feature extraction sub-models. The target feature extraction sub-model can use an existing backbone network for target feature extraction. The backbone network can be the backbone network of RPN (Region Proposal Network). This part can be composed of a 2D convolutional neural network, which is used to extract high-dimensional features from the pseudo-image output by the first part of the network. The RPN backbone is divided into two sub-networks: a top-down sub-network is used to extract features on increasingly smaller spatial resolution feature maps, and the other sub-network is responsible for upsampling the features extracted from feature maps of different resolutions to the same dimension through deconvolution operations and then concatenating them.

[0114] Of course, those skilled in the art can flexibly choose other types of feature extraction models according to actual needs, and no specific limitations are made here.

[0115] In step S206, the target detection region is determined based on multiple target feature maps, and then the target object in the target detection region is determined.

[0116] In this step, multiple feature maps of the target can be input into a pre-trained region detection model to obtain the target detection region output by the model. This region detection model can be trained using existing model training methods, which will not be elaborated upon here.

[0117] Specifically, a pre-built region detection model can be obtained first, which can refer to a pre-built detection head network, including a classification branch network and a bounding box regression branch network. Then, the category information of the target detection region image in the target feature map is output based on the classification branch network, and the target detection box information corresponding to the target detection region image is generated through the bounding box regression branch network. Finally, the target object in the target detection region is determined based on the category information and the target detection box information.

[0118] In this way, by designing different numbers of convolution kernels for each layer of point cloud mesh according to the density of the point cloud, and by ensuring that the point cloud meshes of different point cloud data block sets do not share convolution parameters, it is possible to independently and meticulously learn the features of point clouds of different point cloud data block sets without increasing the number of parameters. This allows for the learning of more detailed features and improves the accuracy of target detection.

[0119] Figure 4 This is a block diagram of a target detection device 400 according to an exemplary embodiment. (Refer to...) Figure 4 The device includes an acquisition module 401, a division module 402, a first determination module 403, a second determination module 404, and a third determination module 405.

[0120] The acquisition module 401 is configured to acquire multiple point cloud data collected by the vehicle during its driving process;

[0121] The partitioning module 402 is configured to divide the point cloud data into multiple point cloud data block sets. Different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle's driving direction. Each point cloud data block set includes multiple point cloud grids, and each point cloud grid includes multiple points cloud data.

[0122] The first determining module 403 is configured to determine the pseudo image corresponding to each set of point cloud data blocks;

[0123] The second determining module 404 is configured to determine the target detection region based on multiple pseudo-images;

[0124] The third determination module 405 is configured to determine the target object in the target detection area.

[0125] Figure 5 It is based on Figure 4 The illustrated embodiment presents a block diagram of a modular division. (Refer to...) Figure 5 The partitioning module 402 includes:

[0126] The first partitioning submodule 4021 is configured to divide multiple point cloud data into multiple point cloud grids according to the horizontal plane where the vehicle's driving direction is located;

[0127] The second partitioning submodule 4022 is configured to divide the multiple point cloud grids into multiple point cloud data block sets according to the vertical direction of the vehicle's driving direction.

[0128] Optionally, the second partitioning submodule 4022 is configured to first determine the total height range corresponding to the multiple point cloud data based on the vertical position coordinates of the multiple point cloud data in the direction perpendicular to the vehicle's driving direction; secondly, determine multiple target height ranges based on the preset set size and the total height range; and then, for each target height range, determine multiple target point cloud data with vertical position coordinates within the target height range from the multiple point cloud data, and use the multiple target point cloud data as a set of point cloud data blocks corresponding to the target height range.

[0129] Optionally, the second division submodule 4022 is configured to divide the total height range into multiple target height ranges on an average basis according to the preset set number; or, based on the vertical position coordinates of the multiple point cloud data in the direction perpendicular to the vehicle's driving direction, determine the number of point cloud data at different heights, and divide the total height range into multiple target height ranges according to the preset set number and the number of point cloud data.

[0130] Figure 6 It is based on Figure 4 The illustrated embodiment shows a block diagram of a second determining module. (Refer to...) Figure 6 The second determining module 404 includes:

[0131] The first determining submodule 4041 is configured to determine the target feature map corresponding to each pseudo-image;

[0132] The second determining submodule 4042 is configured to determine the target detection region based on multiple target feature maps.

[0133] Optionally, the first determining submodule 4041 is configured to input multiple pseudo-images into a pre-trained feature extraction model to obtain multiple target feature maps output by the feature extraction model.

[0134] Optionally, the feature extraction model includes multiple feature extraction sub-models, with different pseudo-images corresponding to different feature extraction sub-models. The first determining sub-module is configured to, for each pseudo-image, determine the number of convolutional kernels of the target feature extraction sub-model corresponding to the pseudo-image from a pre-created convolutional kernel association relationship based on the number of points in the point cloud data block set corresponding to the pseudo-image, and input the pseudo-image into the target feature extraction sub-model, performing convolution processing on the pseudo-image according to the number of convolutional kernels of the target feature extraction sub-model to obtain the target feature map output by the target feature extraction sub-model. The convolutional kernel association relationship includes the correspondence between different point cloud numbers and the number of convolutional kernels.

[0135] Optionally, the second determining submodule 4042 is configured to input multiple target feature maps into a pre-trained region detection model to obtain the target detection region output by the region detection model.

[0136] The above scheme acquires multiple point cloud data points collected during vehicle travel. These point cloud data points are then divided into multiple point cloud data block sets, each corresponding to a different height range in the vertical direction of the vehicle's travel direction. Each point cloud data block set includes multiple point cloud grids, and each point cloud grid contains multiple points of that point cloud data. A pseudo-image is determined for each point cloud data block set. Based on these pseudo-images, a target detection region is determined. Finally, the target object within this target detection region is identified. By dividing the point cloud data according to different height ranges in the vertical direction of the vehicle's travel direction, different point cloud data block sets are obtained. Processing the point cloud data within these sets allows for the learning of more detailed features for objects with dense vertical point clouds, effectively identifying occluded objects and improving target detection accuracy. Furthermore, compared to using cubes of the same size as voxels to divide the point cloud data, this method improves detection efficiency.

[0137] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0138] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the target detection method provided in this disclosure.

[0139] Figure 7 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. For example, the electronic device 700 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0140] Reference Figure 7 The electronic device 700 may include one or more of the following components: processing component 702, memory 704, power supply component 706, multimedia component 708, audio component 710, input / output (I / O) interface 712, sensor component 714, and communication component 716.

[0141] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the target detection method described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.

[0142] Memory 704 is configured to store various types of data to support the operation of electronic device 700. Examples of this data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0143] Power supply component 706 provides power to various components of electronic device 700. Power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.

[0144] Multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0145] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0146] Input / output (I / O) interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0147] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of electronic device 700. For example, sensor assembly 714 can detect the on / off state of electronic device 700, the relative positioning of components such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0148] Communication component 716 is configured to facilitate wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0149] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the target detection method described above.

[0150] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of an electronic device 700 to complete the target detection method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0151] The aforementioned electronic device can be a standalone electronic device or a part of a standalone electronic device. For example, in one embodiment, the electronic device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be a single IC or a collection of multiple ICs. The chip can include, but is not limited to, the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), and SoC (System on Chip). The aforementioned integrated circuit or chip can be used to execute executable instructions (or code) to implement the aforementioned target detection method. The executable instructions can be stored in the integrated circuit or chip or obtained from other devices or equipment. For example, the integrated circuit or chip includes a processor, memory, and an interface for communicating with other devices. The executable instructions can be stored in the processor, and when the executable instructions are executed by the processor, the target detection method described above is implemented; or, the integrated circuit or chip can receive the executable instructions through the interface and transmit them to the processor for execution to implement the target detection method described above.

[0152] See Figure 8 , Figure 8This is a functional block diagram illustrating a vehicle 800 as an exemplary embodiment. The vehicle 800 can be configured for fully or partially autonomous driving modes. For example, the vehicle 800 can acquire environmental information about its surroundings through a perception system 820, and based on the analysis of the surrounding environmental information, derive an autonomous driving strategy to achieve fully autonomous driving, or present the analysis results to the user to achieve partial autonomous driving.

[0153] Vehicle 800 may include various subsystems, such as an infotainment system 810, a perception system 820, a decision control system 830, a drive system 840, and a computing platform 850. Optionally, vehicle 800 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 800 may be interconnected via wired or wireless means.

[0154] In some embodiments, the infotainment system 810 may include a communication system 811, an entertainment system 812, and a navigation system 813.

[0155] Communication system 811 may include a wireless communication system that can communicate wirelessly with one or more devices, either directly or via a communication network. For example, the wireless communication system may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system may utilize WiFi or a wireless local area network (WLAN) to communicate. In some embodiments, the wireless communication system may utilize an infrared link, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system may include one or more dedicated short-range communications (DSRC) devices that can enable public and / or private data communication between vehicles and / or roadside stations.

[0156] The entertainment system 812 may include a display device, a microphone, and speakers, allowing users to listen to the radio and play music in the vehicle; or connect their mobile phones to the vehicle and project their screens onto the display device, which may be touch-sensitive, allowing users to operate the system by touching the screen.

[0157] In some cases, the user's voice signal can be acquired through a microphone, and based on the analysis of the voice signal, the user can control certain aspects of the vehicle 800, such as adjusting the interior temperature. In other cases, music can be played to the user through the audio system.

[0158] The navigation system 813 may include map services provided by a map provider to provide navigation for the vehicle 800. The navigation system 813 can be used in conjunction with the vehicle's global positioning system 821 and inertial measurement unit 822. The map services provided by the map provider can be two-dimensional maps or high-precision maps.

[0159] The perception system 820 may include several sensors for sensing information about the environment surrounding the vehicle 800. For example, the perception system 820 may include a global positioning system 821 (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU) 822, a lidar 823, a millimeter-wave radar 824, an ultrasonic radar 825, and a camera device 826. The perception system 820 may also include sensors for the internal systems of the monitored vehicle 800 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a critical function for the safe operation of the vehicle 800.

[0160] The Global Positioning System 821 is used to estimate the geographical location of vehicle 800.

[0161] The inertial measurement unit 822 is used to sense changes in the pose of the vehicle 800 based on inertial acceleration. In some embodiments, the inertial measurement unit 822 may be a combination of an accelerometer and a gyroscope.

[0162] The lidar 823 uses lasers to sense objects in the environment in which the vehicle 800 is located. In some embodiments, the lidar 823 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0163] The millimeter-wave radar 824 uses radio signals to sense objects in the surrounding environment of the vehicle 800. In some embodiments, in addition to sensing objects, the millimeter-wave radar 824 can also be used to sense the speed and / or direction of travel of objects.

[0164] The ultrasonic radar 825 can use ultrasonic signals to sense objects around the vehicle 800.

[0165] The camera device 826 is used to capture image information of the surrounding environment of the vehicle 800. The camera device 826 may include a monocular camera, a binocular camera, a structured light camera, and a panoramic camera, etc. The image information acquired by the camera device 826 may include still images or video stream information.

[0166] The decision control system 830 includes a computing system 831 that analyzes and makes decisions based on information acquired by the sensing system 820. The decision control system 830 also includes a vehicle controller 832 that controls the power system of the vehicle 800, as well as a steering system 833, a throttle 834, and a braking system 835 for controlling the vehicle 800.

[0167] The computing system 831 is operable to process and analyze various information acquired by the perception system 820 to identify targets, objects, and / or features in the environment surrounding the vehicle 800. Targets may include pedestrians or animals, and objects and / or features may include traffic signals, road boundaries, and obstacles. The computing system 831 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other techniques. In some embodiments, the computing system 831 may be used to map the environment, track objects, estimate object speeds, etc. The computing system 831 can analyze the acquired information and derive a control strategy for the vehicle.

[0168] The vehicle controller 832 can be used to coordinate the control of the vehicle's power battery and engine 841 to improve the power performance of the vehicle 800.

[0169] The steering system 833 is operable to adjust the forward direction of the vehicle 800. For example, in one embodiment, it can be a steering wheel system.

[0170] Throttle 834 is used to control the operating speed of engine 841 and thus the speed of vehicle 800.

[0171] Braking system 835 is used to control the deceleration of vehicle 800. Braking system 835 can use friction to slow down wheel 844. In some embodiments, braking system 835 can convert the kinetic energy of wheel 844 into electric current. Braking system 835 may also take other forms to slow down the rotational speed of wheel 844 to control the speed of vehicle 800.

[0172] The drive system 840 may include components that provide powered motion to the vehicle 800. In one embodiment, the drive system 840 may include an engine 841, an energy source 842, a transmission system 843, and wheels 844. The engine 841 may be an internal combustion engine, an electric motor, an air-compressed engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine 841 converts the energy source 842 into mechanical energy.

[0173] Examples of energy sources 842 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 842 can also provide energy to other systems of vehicle 800.

[0174] The drivetrain 843 transmits mechanical power from the engine 841 to the wheels 844. The drivetrain 843 may include a gearbox, a differential, and a drive shaft. In one embodiment, the drivetrain 843 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 844.

[0175] Some or all of the functions of vehicle 800 are controlled by computing platform 850. Computing platform 850 may include at least one processor 851, which can execute instructions 853 stored in a non-transitory computer-readable medium such as memory 852. In some embodiments, computing platform 850 may also be multiple computing devices that control individual components or subsystems of vehicle 800 in a distributed manner.

[0176] Processor 851 can be any conventional processor, such as a commercially available CPU. Alternatively, processor 851 may also include a Graphic Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System-on-Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof. Although Figure 8 The processor, memory, and other components of a computer within the same block are functionally illustrated; however, those skilled in the art will understand that the processor, computer, or memory may actually include multiple processors, computers, or memories that may or may not be stored in the same physical enclosure. For example, memory may be a hard disk drive or other storage media located in an enclosure different from that of the computer. Therefore, references to a processor or computer will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor, which performs calculations only related to the component's specific function.

[0177] In this embodiment of the disclosure, the processor 851 can execute the target detection method described above.

[0178] In various aspects described herein, processor 851 may be located remotely from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor located within the vehicle, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0179] In some embodiments, memory 852 may contain instructions 853 (e.g., program logic) that can be executed by processor 851 to perform various functions of vehicle 800. Memory 852 may also contain additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the infotainment system 810, perception system 820, decision control system 830, and drive system 840.

[0180] In addition to instruction 853, memory 852 may also store data such as road maps, route information, vehicle position, direction, speed, and other vehicle data, as well as other information. This information can be used by vehicle 800 and computing platform 850 during operation of vehicle 800 in autonomous, semi-autonomous, and / or manual modes.

[0181] The computing platform 850 can control the functions of the vehicle 800 based on inputs received from various subsystems, such as the drive system 840, the perception system 820, and the decision control system 830. For example, the computing platform 850 can utilize inputs from the decision control system 830 to control the steering system 833 to avoid obstacles detected by the perception system 820. In some embodiments, the computing platform 850 is operable to provide control over many aspects of the vehicle 800 and its subsystems.

[0182] Optionally, one or more of these components may be installed separately from or associated with the vehicle 800. For example, the memory 852 may exist partially or completely separately from the vehicle 800. The components may be communicatively coupled together in a wired and / or wireless manner.

[0183] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 8 This should not be construed as a limitation on the embodiments disclosed herein.

[0184] Autonomous vehicles traveling on roads, such as vehicle 800 above, can identify objects in their surroundings to determine adjustments to their current speed. These objects can be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment the autonomous vehicle should make can be determined.

[0185] Optionally, vehicle 800 or its associated perception and computing devices (e.g., computing system 831, computing platform 850) can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of the others, so all identified objects can be considered together to predict the behavior of a single identified object. Vehicle 800 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what steady state the vehicle needs to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed of vehicle 800, such as the lateral position of vehicle 800 in the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0186] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 800 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., vehicles in adjacent lanes on the road).

[0187] The vehicle 800 described above can be any type of vehicle, such as a car, truck, motorcycle, bus, boat, airplane, helicopter, recreational vehicle, train, etc. This disclosure does not impose any particular limitation.

[0188] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the target detection method described above when executed by the programmable device.

[0189] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0190] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A target detection method, characterized in that, include: Acquire multiple point cloud data collected during vehicle operation; The point cloud data is divided into multiple point cloud data blocks according to the horizontal plane where the vehicle travels; According to the vertical direction of the vehicle's driving direction, the multiple point cloud data blocks are divided into multiple point cloud data block sets; different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle's driving direction, each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple point cloud data. Determine the pseudo-image corresponding to each set of point cloud data blocks; The target detection region is determined based on the multiple pseudo-images; Identify the target object within the target detection area; The step of determining the target detection region based on the multiple pseudo-images includes: Multiple pseudo-images are input into a pre-trained feature extraction model to obtain a target feature map corresponding to each pseudo-image output by the feature extraction model. The feature extraction model includes multiple feature extraction sub-models, and different pseudo-images correspond to different feature extraction sub-models; The target detection region is determined based on multiple target feature maps.

2. The method according to claim 1, characterized in that, The step of dividing the multiple point cloud data blocks into multiple point cloud data block sets according to the vertical direction of the vehicle's driving direction includes: Based on the vertical position coordinates of the multiple point cloud data in the direction perpendicular to the vehicle's driving direction, determine the total height range corresponding to the multiple point cloud data. Based on the preset number of sets and the total height range, multiple target height ranges are determined; For each target height range, multiple target point cloud data with vertical position coordinates within the target height range are determined from multiple point cloud data, and the multiple target point cloud data are used as a set of point cloud data blocks corresponding to the target height range.

3. The method according to claim 2, characterized in that, The step of determining multiple target height ranges based on the preset set size and the total height range includes: Based on the preset set size, the total height range is divided into multiple target height ranges on an average basis; or... Based on the vertical position coordinates of multiple point cloud data in the direction perpendicular to the vehicle's driving direction, the number of point cloud data at different heights is determined, and the total height range is divided into multiple target height ranges according to the preset set size and the number of point clouds.

4. The method according to any one of claims 1-3, characterized in that, The step of inputting multiple pseudo-images into a pre-trained feature extraction model to obtain a target feature map corresponding to each pseudo-image output by the feature extraction model includes: For each pseudo-image, based on the number of points in the point cloud data set corresponding to the pseudo-image, the number of convolutional kernels of the target feature extraction sub-model corresponding to the pseudo-image is determined from a pre-created convolutional kernel association relationship. The pseudo-image is then input into the target feature extraction sub-model, and convolution processing is performed on the pseudo-image according to the number of convolutional kernels of the target feature extraction sub-model to obtain the target feature map output by the target feature extraction sub-model. The convolutional kernel association relationship includes the correspondence between different point cloud numbers and the number of convolutional kernels.

5. The method according to claim 1, characterized in that, Determining the target detection region based on multiple target feature maps includes: Multiple target feature maps are input into a pre-trained region detection model to obtain the target detection region output by the region detection model.

6. A target detection device, characterized in that, include: The acquisition module is configured to acquire multiple point cloud data collected by the vehicle during its driving process; The partitioning module is configured to divide the multiple point cloud data into multiple point cloud data blocks according to the horizontal plane where the vehicle travels; and to divide the multiple point cloud data blocks into multiple point cloud data block sets according to the vertical direction of the vehicle travels; different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle travels, and each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple point cloud data. The first determining module is configured to determine the pseudo image corresponding to each set of point cloud data blocks; The second determining module is configured to determine the target detection region based on the plurality of said pseudo images; The third determining module is configured to determine the target object in the target detection region; The step of determining the target detection region based on the multiple pseudo-images includes: Multiple pseudo-images are input into a pre-trained feature extraction model to obtain a target feature map corresponding to each pseudo-image output by the feature extraction model. The feature extraction model includes multiple feature extraction sub-models, and different pseudo-images correspond to different feature extraction sub-models; The target detection region is determined based on multiple target feature maps.

7. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Acquire multiple point cloud data collected during vehicle operation; According to the horizontal plane where the vehicle travels, the multiple point cloud data are divided into multiple point cloud data blocks; according to the vertical direction of the vehicle travels, the multiple point cloud data blocks are divided into multiple point cloud data block sets; different point cloud data block sets correspond to different height ranges in the vertical direction of the vehicle travels, each point cloud data block set includes multiple point cloud data blocks, and each point cloud data block includes multiple point cloud data. Determine the pseudo-image corresponding to each set of point cloud data blocks; The target detection region is determined based on the multiple pseudo-images; Identify the target object within the target detection area; The step of determining the target detection region based on the multiple pseudo-images includes: Multiple pseudo-images are input into a pre-trained feature extraction model to obtain a target feature map corresponding to each pseudo-image output by the feature extraction model; the feature extraction model includes multiple feature extraction sub-models, and different pseudo-images correspond to different feature extraction sub-models. The target detection region is determined based on multiple target feature maps.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1-5.

9. A chip, characterized in that, It includes a processor and an interface; the processor is used to read instructions to execute the method of any one of claims 1-5.

10. An electronic device, characterized in that, include: A memory on which computer programs are stored; processor; For executing the computer program in the memory to implement the steps of the method according to any one of claims 1-5.

Images