Target detection method based on clustering optimization

Abstract

The application provides a target detection method based on clustering optimization, which comprises the following steps: receiving target point cloud data, wherein the target point cloud data corresponds to a target image, and the target point cloud data is point cloud data of a predetermined region obtained by a laser radar, and the predetermined region is a region spaced apart from a vehicle by a predetermined distance; performing filtering processing and clustering processing on the target point cloud data to obtain a plurality of clustering results, and performing merging processing on the plurality of clustering results to obtain at least one target independent region; determining the size of a target voxel based on at least the at least one target independent region, and constructing a target pseudo image based on the size of the target voxel, and performing target detection based on the target pseudo image. In this way, the division of the target voxel is reasonable, thereby solving the problem that the division of the voxel is unreasonable in the prior art.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and more specifically, to a target detection method based on clustering optimization. Background Technology

[0002] LiDAR is one of the most important sensors in the field of autonomous driving, and it is indispensable for high-level autonomous vehicles. Due to its high accuracy, short detection time, and ability to acquire depth information, LiDAR-based autonomous driving algorithms are constantly emerging, especially in the field of object detection.

[0003] In existing technologies, target detection algorithms based on LiDAR can be mainly divided into two directions: voxel-based and point-based. Among them, voxel-based algorithms have gradually become the mainstream due to their performance advantages. However, voxel-based algorithms also have some related drawbacks. For example, the accuracy and performance of voxel-based algorithms are highly dependent on the size of the voxels. If the voxels are large, there will be more information loss in a frame of point cloud data; if the voxels are small, the computational load of the algorithm will be large and the memory consumption will also increase accordingly.

[0004] Currently, in voxel-based algorithms, the division of voxels in a frame of point cloud data often relies on human experience. This division of voxels in a frame of point cloud data is rather unreasonable, resulting in inaccurate target detection results.

[0005] Therefore, there is an urgent need for a more reasonable method to automatically divide voxels. Summary of the Invention

[0006] The main purpose of this application is to provide a target detection method based on cluster optimization to solve the problem that the classification of voxels in the prior art is unreasonable.

[0007] According to one aspect of the present invention, a target detection method based on clustering optimization is provided, comprising: receiving target point cloud data, wherein the target point cloud data corresponds to a frame of target image, and the target point cloud data is point cloud data of a predetermined region acquired by a lidar, wherein the predetermined region is a region at a predetermined distance from a vehicle; performing filtering and clustering processing on the target point cloud data to obtain multiple clustering results, and merging the multiple clustering results to obtain at least one target independent region; determining the size of a target voxel based on at least one target independent region, constructing a target pseudo-image based on the size of the target voxel, and performing target detection based on the target pseudo-image.

[0008] Optionally, determining the size of a target voxel based on at least one of the target independent regions includes: performing a view transformation on the at least one target independent region to obtain the total area of ​​the at least one target independent region from the BEV viewpoint; performing a view transformation on the target point cloud data to obtain the total area of ​​the target point cloud data from the BEV viewpoint; calculating the ratio of the total area of ​​the at least one target independent region from the BEV viewpoint to the total area of ​​the target point cloud data from the BEV viewpoint to obtain a first target ratio, and determining the size of the target voxel based at least on the first target ratio.

[0009] Optionally, determining the size of a target voxel based on at least one of the target independent regions includes: determining the smallest rectangular frame surrounding the target independent region to obtain at least one first rectangular frame; performing a view transformation on the at least one target independent region to obtain the area of ​​the at least one target independent region from a BEV viewpoint, and performing a view transformation on the at least one first rectangular frame to obtain the area corresponding to the at least one first rectangular frame from the BEV viewpoint; calculating the ratio of the area of ​​the at least one target independent region from the BEV viewpoint to the area of ​​the at least one first rectangular frame from the BEV viewpoint to obtain at least one second target ratio, and determining the size of the target voxel of the corresponding target independent region based at least on the second target ratio.

[0010] Optionally, constructing a target pseudo-image based on the size of the target voxel includes: determining the smallest rectangular box surrounding the independent region of the target, obtaining at least one second rectangular box and the position information of the second rectangular box in the target image; cropping the target image based on the at least one second rectangular box and the position information of the second rectangular box in the target image to obtain at least one predetermined pseudo-image; and dividing the at least one predetermined pseudo-image according to the size of the target voxel to obtain at least one target pseudo-image.

[0011] Optionally, target detection based on the target pseudo-image includes: adjusting the size of the target pseudo-image to obtain an adjusted target pseudo-image; performing target detection on the adjusted target pseudo-image to obtain a detection result, wherein the detection result includes the category probability of obstacles in the target pseudo-image and a third bounding box, wherein the category probability is the confidence level of the obstacle belonging to each obstacle category, and the third bounding box is the smallest bounding box surrounding the obstacle.

[0012] Optionally, after performing target detection on the adjusted target pseudo-image and obtaining the detection result, the target detection method further includes: if the target image in the target point cloud data is a first frame image, determining the target category probability of the obstacle in the target pseudo-image based on the category probability of the target pseudo-image and the first category probability map corresponding to the target image, and constructing a second category probability map corresponding to the target image; if the target image in the target point cloud data is not the first frame image, determining the target category probability of the obstacle in the target pseudo-image based on the category probability of the target pseudo-image and the second category probability map corresponding to the previous frame target image.

[0013] Optionally, the process of determining the first category probability map corresponding to the target image includes: determining a first region of the target pseudo-image in the target image based on the size of the target pseudo-image and the position information of the center point of the target pseudo-image in the target image; and setting the probability of each voxel point in the first region to the clustering probability based on the clustering probability of the clustering result corresponding to the target pseudo-image, thereby obtaining the first category probability map.

[0014] Optionally, the process of constructing the second category probability map corresponding to the target image includes: determining a second region of the third rectangle in the target image based at least on the size of the third rectangle and the position information of the center point of the third rectangle in the target image; and setting the probability of each voxel point in the second region to the target category probability based on the target category probability corresponding to the third rectangle, thereby obtaining the second category probability map.

[0015] Optionally, after performing target detection on the adjusted target pseudo-image and obtaining the detection result, the target detection method further includes: performing coordinate transformation on the information of the obstacle in the third rectangular frame based on the parameter information of the third rectangular frame and the lidar to obtain the position information of the obstacle in the real environment.

[0016] Optionally, merging multiple clustering results to obtain at least one target independent region includes: performing connectivity detection on multiple clustering results, merging intersecting clustering results, and obtaining at least one target independent region.

[0017] In this embodiment of the invention, the target detection method based on clustering optimization first receives target point cloud data of a vehicle within a predetermined area; then, the received target point cloud data is filtered and clustered to obtain multiple clustering results, and these multiple clustering results are merged to obtain at least one target-independent region; finally, based on at least one target-independent region, the size of the target voxel corresponding to the target-independent region is determined, and a target pseudo-image is constructed based on the target voxel size, and target detection is performed based on the constructed target pseudo-image. Compared to existing technologies that determine the target voxel size based on manual experience, this application filters and clusters the target point cloud data to obtain multiple clustering results, and merges these multiple clustering results to obtain one or more target-independent regions. Then, based on one or more target-independent regions, the size of the target voxel corresponding to the target-independent region is determined, thus ensuring a more reasonable division of the target voxel and solving the problem of unreasonable voxel division in existing technologies. The target detection method of this application constructs a target pseudo-image corresponding to the independent region of the target based on the size of the target voxel. This ensures that the point cloud information in the target pseudo-image is all valid information, and ensures that the target detection algorithm based on the target pseudo-image has good performance and high detection accuracy, thereby balancing the relationship between the size of the target voxel and the target detection algorithm. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 A flowchart of a cluster-optimized target detection method according to an embodiment of this application is shown;

[0020] Figure 2 A schematic diagram of the structure of a target detection device based on clustering optimization according to an embodiment of this application is shown. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] As mentioned in the background section, the classification of voxels in the prior art is rather unreasonable. In order to solve the above problems, a typical implementation of this application provides a target detection method based on cluster optimization.

[0025] According to an embodiment of this application, a target detection method based on clustering optimization is provided.

[0026] Figure 1 This is a flowchart of a cluster-based target detection method according to an embodiment of this application. Figure 1 As shown, the target detection method includes the following steps:

[0027] Step S101: Receive target point cloud data. The target point cloud data corresponds to a frame of target image. The target point cloud data is point cloud data of a predetermined area obtained by the lidar. The predetermined area is an area at a predetermined distance from the vehicle.

[0028] Step S102: Filter and cluster the target point cloud data to obtain multiple clustering results, and merge the multiple clustering results to obtain at least one independent target region.

[0029] Step S103: Determine the size of the target voxel based on at least one of the aforementioned target independent regions, construct a target pseudo-image based on the size of the target voxel, and perform target detection based on the aforementioned target pseudo-image.

[0030] In the aforementioned clustering-based target detection method, firstly, target point cloud data of the vehicle within a predetermined area is received; then, the received target point cloud data is filtered and clustered to obtain multiple clustering results, which are then merged to obtain at least one target-independent region; finally, based on at least one of the aforementioned target-independent regions, the size of the target voxel corresponding to each target-independent region is determined, and a target pseudo-image is constructed based on the target voxel size. Target detection is then performed based on the constructed target pseudo-image. Compared to existing technologies that determine the target voxel size based on manual experience, this application filters and clusters the target point cloud data to obtain multiple clustering results, which are then merged to obtain one or more target-independent regions. The size of the target voxel corresponding to each target-independent region is then determined based on these one or more target-independent regions. This ensures a more reasonable division of the target voxel, thus solving the problem of unreasonable voxel division in existing technologies. The target detection method of this application constructs a target pseudo-image corresponding to the independent region of the target based on the size of the target voxel. This ensures that the point cloud information in the target pseudo-image is all valid information, and ensures that the target detection algorithm based on the target pseudo-image has good performance and high detection accuracy, thereby balancing the relationship between the size of the target voxel and the target detection algorithm.

[0031] In one specific embodiment of this application, the target point cloud data is filtered to obtain filtered target point cloud data. Then, the filtered target point cloud data is clustered to obtain multiple clustering results (i.e., clusters), and the clustering probability corresponding to each clustering result can also be obtained. Filtering the target point cloud data reduces the impact of isolated noise in the target point cloud data on the clustering results.

[0032] In another specific embodiment of this application, the target point cloud data is clustered to obtain multiple clustering results (i.e., clusters) and the clustering probability corresponding to each clustering result. Then, the multiple clustering results are filtered to obtain filtered clustering results. In this embodiment, filtering is performed on multiple clustering results to ensure that the obtained clustering results are relatively accurate.

[0033] In practical applications, any feasible filtering method in the existing technology can be used to filter the target point cloud data or to filter multiple clustering results. This application does not limit the specific clustering method. For example, the specific filtering method can be a voxel filter, bilateral filtering, spatial distribution-based denoising algorithm (Statistical Outlier Removal, SOR), etc.

[0034] In practical applications, any feasible clustering method in the existing technology can be used to cluster the target point cloud data. This application does not limit the specific clustering method. For example, Spectral Clustering (SC), Hierarchical Clustering (HC), and Mean-shift clustering are all acceptable methods.

[0035] Specifically, in this application, when a single target independent region is obtained, the size of the target voxel of that target independent region is determined, at least based on that target independent region. When multiple target independent regions are obtained, the size of the target voxel of each target independent region is determined, at least based on each target independent region.

[0036] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0037] To simplify the determination of the target voxel size, in one embodiment of this application, the determination of the target voxel size is based on at least one of the aforementioned target independent regions, comprising: performing a viewpoint transformation on the at least one of the aforementioned target independent regions to obtain the total area of ​​the at least one of the aforementioned target independent regions from the BEV viewpoint; performing a viewpoint transformation on the aforementioned target point cloud data to obtain the total area of ​​the aforementioned target point cloud data from the aforementioned BEV viewpoint; calculating the ratio of the total area of ​​the at least one of the aforementioned target independent regions from the aforementioned BEV viewpoint to the total area of ​​the aforementioned target point cloud data from the aforementioned BEV viewpoint to obtain a first target ratio; and determining the size of the target voxel based at least on the aforementioned first target ratio.

[0038] Specifically, in the above embodiments, when a single target independent region is obtained, the viewpoint of the target independent region is transformed to obtain the target independent region from the BEV viewpoint, and the total area of ​​the target independent region from the BEV viewpoint is calculated; then, the viewpoint of the target point cloud data is transformed to obtain the target point cloud data from the BEV viewpoint, and the total area of ​​the target point cloud data from the BEV viewpoint is calculated; finally, a first target ratio is obtained based on the ratio of the total area of ​​the target independent region from the BEV viewpoint to the total area of ​​the target point cloud data from the BEV viewpoint. When multiple target independent regions are obtained, the viewpoint of the multiple target independent regions is transformed to obtain multiple target independent regions from the BEV viewpoint, and the sum of the areas of the multiple target independent regions from the BEV viewpoint is calculated, i.e., the total area; then, the viewpoint of the target point cloud data is transformed to obtain the target point cloud data from the BEV viewpoint, and the total area of ​​the target point cloud data from the BEV viewpoint is calculated; finally, a first target ratio is obtained based on the ratio of the total area of ​​the multiple target independent regions from the BEV viewpoint to the total area of ​​the target point cloud data from the BEV viewpoint. Based on the first target ratio, the size of the target voxel is determined, so that each target independent region has a target voxel of the same size. Subsequently, a target pseudo-image of the corresponding target independent region is constructed based on the target voxel, so that the target voxel in each target pseudo-image is the same.

[0039] In one specific embodiment of this application, the size of the target voxel can be determined based on a first target ratio and a predefined predetermined table. Specifically, the process involves determining the size of the target voxel corresponding to the first target ratio from the first predetermined table. This allows for a more straightforward determination of the target voxel size.

[0040] In another embodiment of this application, determining the size of a target voxel based on at least one of the aforementioned target independent regions includes: determining the smallest rectangular frame surrounding the target independent region to obtain at least one first rectangular frame; performing a view transformation on the at least one of the aforementioned target independent regions to obtain the area of ​​the at least one of the aforementioned target independent regions from a BEV viewpoint, and performing a view transformation on the at least one of the aforementioned first rectangular frames to obtain the area corresponding to the at least one of the aforementioned first rectangular frames from a BEV viewpoint; calculating the ratio of the area of ​​the at least one of the aforementioned target independent regions from a BEV viewpoint to the area of ​​the at least one of the aforementioned first rectangular frames from a BEV viewpoint to obtain at least one second target ratio, and determining the size of the target voxel corresponding to the aforementioned target independent region based at least on the aforementioned second target ratio. This allows for a simpler way to obtain the size of the target voxel and ensures that the determined size of the target voxel is more reasonable.

[0041] Specifically, in the above embodiments, when a single target independent region is obtained, the smallest rectangle surrounding the target independent region is determined to obtain a corresponding first rectangle; then, the viewpoints of the target independent region and the corresponding first rectangle are transformed to obtain the target independent region and the first rectangle from the BEV viewpoint, and the area of ​​the target independent region and the area of ​​the corresponding first rectangle from the BEV viewpoint are calculated; finally, based on the ratio of the area of ​​the target independent region to the area of ​​the first rectangle, a second target ratio is determined, and the size of the target voxel of the corresponding target independent region is determined at least based on the second target ratio. When multiple target independent regions are obtained, the smallest rectangle surrounding the target independent region is determined to obtain a first rectangle corresponding to each target independent region; then, the viewpoints of each target independent region and the corresponding first rectangle are transformed to obtain each target independent region and the corresponding first rectangle from the BEV viewpoint, and the areas of each target independent region and the corresponding first rectangle from the BEV viewpoint are calculated; finally, the ratio of the area of ​​each target independent region to the area of ​​the corresponding first rectangle is calculated to obtain a second target ratio, and the size of the target voxel of the corresponding target independent region is determined at least based on the second target ratio. This ensures that the target voxel size is different for each independent target region, making the determined target voxel more reasonable. This further ensures that the subsequent construction of the target pseudo-image corresponding to each independent target region based on the target voxel size is more reasonable, thus further ensuring better target detection performance and accuracy.

[0042] In one specific embodiment of this application, the size of the target voxel can be determined based on a second target ratio and a predefined predetermined table. Specifically, the size of the target voxel corresponding to the second target ratio is determined from the second predetermined table. This allows for a more straightforward determination of the target voxel size.

[0043] To obtain the target pseudo image more easily, in another embodiment of this application, the target pseudo image is constructed based on the size of the target voxel, including: determining the smallest rectangular box surrounding the target independent region, obtaining at least one second rectangular box and the position information of the second rectangular box in the target image; cropping the target image based on the at least one second rectangular box and the position information of the second rectangular box in the target image to obtain at least one predetermined pseudo image; and dividing the at least one predetermined pseudo image according to the size of the target voxel to obtain at least one target pseudo image.

[0044] Specifically, when there are multiple independent target regions, multiple target pseudo-images with different target voxel sizes can be obtained, which can also be called pseudo-image groups.

[0045] Specifically, in this application, the first rectangle and the second rectangle can be the same rectangle.

[0046] In another embodiment of this application, target detection based on the aforementioned target pseudo-image includes: adjusting the size of the target pseudo-image to obtain an adjusted target pseudo-image; performing target detection on the adjusted target pseudo-image to obtain a detection result, wherein the detection result includes the category probability of obstacles in the target pseudo-image and a third bounding box, wherein the category probability is the confidence level that the obstacle belongs to each obstacle category, and the third bounding box is the smallest bounding box surrounding the obstacle. In this embodiment, target detection is performed based on the adjusted target pseudo-image. Since the point cloud data in the target pseudo-image are all valid point cloud data, this further ensures that the target detection algorithm has good performance and high detection accuracy.

[0047] In practical applications, any feasible method in the existing technology can be used for size adjustment to obtain the adjusted target pseudo-image. This application does not limit the specific method of size adjustment. In one specific embodiment of this application, when multiple target pseudo-images are obtained, an average size can be determined based on the size corresponding to each target pseudo-image. Then, based on the average size and the size of each target pseudo-image, the size of the corresponding target pseudo-image is adjusted. For example, if the size of a target pseudo-image is smaller than the average size, the corresponding target pseudo-image can be expanded using the padding method; if the size of a target pseudo-image is larger than the average size, the corresponding target pseudo-image can be reduced using the ROI pooling method. This makes it relatively easy to adjust the size of each target pseudo-image to the average size.

[0048] Specifically, this application does not limit the specific algorithm for object detection; the algorithm can be flexibly adjusted according to the actual object detection scenario.

[0049] To more accurately determine the category of an obstacle, in one embodiment of this application, target detection is performed on the adjusted target pseudo-image. After obtaining the detection result, the target detection method further includes: when the target image of the target point cloud data is a first frame image, determining the target category probability of the obstacle in the target pseudo-image based on the category probability of the target pseudo-image and the first category probability map corresponding to the target image, and constructing a second category probability map corresponding to the target image; when the target image of the target point cloud data is not the first frame image, determining the target category probability of the obstacle in the target pseudo-image based on the category probability of the target pseudo-image and the second category probability map corresponding to the previous frame target image.

[0050] Specifically, in the above embodiments, when the target image corresponding to the target point cloud data is the first frame, since the target image does not have a second category probability map, the target category probability of the obstacle can be determined based on the obstacle category probability obtained by target detection and the first category probability map corresponding to the target image; when the target image corresponding to the target point cloud data is not the first frame, since the previous frame target image corresponding to the target image has been constructed as a second category probability map, the target category probability of the obstacle can be determined based on the obstacle category probability obtained by target detection and the second category probability map corresponding to the previous frame target image. This ensures that the determined obstacle target category probability is relatively accurate, and ensures that the obstacle category is accurately determined based on the target category probability.

[0051] Specifically, in the above embodiments, the process of determining the target category probability of obstacles in the target pseudo-image based on the category probability of the target pseudo-image and the first category probability map corresponding to the target image can be as follows:

[0052] Step 1: Assuming that the target detection method of this application can detect obstacles in five categories, namely Category 1, Category 2, Category 3, Category 4 and Category 5;

[0053] Step 2: For a target pseudo-image, after target detection, the probability that the obstacle in the target pseudo-image belongs to category 1 is P1, the probability that it belongs to category 2 is P2, the probability that it belongs to category 3 is P3, the probability that it belongs to category 4 is P4, and the probability that it belongs to category 5 is P5. Thus, a category probability group (P1, P2, P3, P4, P5) can be obtained.

[0054] Step 3: Based on the target pseudo-image, determine the cluster probability of the obstacles in the target pseudo-image belonging to a certain category in the first category probability map;

[0055] Step 4: Combine the clustering probability of obstacles in the target pseudo-image belonging to a certain category with the above category probability groups to obtain a target category probability group, namely (P1, P2, P3, P4, P5, P...). 聚类 );

[0056] Step 5: Assign probability groups to the target category (P1, P2, P3, P4, P5, P...). 聚类 Normalization is performed to obtain the normalized (P1', P2', P3', P4', P5', P... 聚类 '), and according to P 聚类 'The category to which P belongs,' 聚类 Add to the same category, for example, P 聚类 'P3' belongs to the same category, therefore P... 聚类 Adding P1' to P3', we get (P1', P2', P3' + P 聚类 P1', P4', P5'). Then (P1', P2', P3' + P 聚类 The probability of the largest value among P', P4', and P5' is determined as the target category probability.

[0057] Specifically, since the target class probability of the obstacle in the target pseudo-image is determined based on the class probability of the target pseudo-image and the second class probability map corresponding to the target image in the previous frame, it is consistent with the target class probability of the obstacle in the target pseudo-image determined based on the class probability of the target pseudo-image and the first class probability map corresponding to the target image, so it will not be elaborated further.

[0058] To further ensure the accuracy of the target category probability of obstacles in the subsequently determined target pseudo-image, in another embodiment of this application, the process of determining the first category probability map corresponding to the target image includes: determining the first region of the target pseudo-image in the target image based on the size of the target pseudo-image and the position information of the center point of the target pseudo-image in the target image; and setting the probability of each voxel point in the first region to the clustering probability based on the clustering probability of the clustering result corresponding to the target pseudo-image, thereby obtaining the first category probability map.

[0059] In another embodiment of this application, the process of constructing the second category probability map corresponding to the target image includes: determining a second region of the third rectangle in the target image based at least on the size of the third rectangle and the position information of the center point of the third rectangle in the target image; setting the probability of each voxel point in the second region to the target category probability based on the target category probability corresponding to the third rectangle, thereby obtaining the second category probability map. In this embodiment, after determining the target category probability corresponding to the obstacle in each target pseudo-image, the second region corresponding to the third rectangle in the target image is determined based on the position information of the third rectangle and the center point of the third rectangle in the target image; and the probability of each voxel point in the corresponding second region is set to the target category probability of the obstacle corresponding to the third rectangle, thereby obtaining the second category probability map. This further ensures that the target category probability of the target pseudo-image of the target image in the next frame is relatively accurate.

[0060] In order to obtain the location information of obstacles in the real environment more easily, and to further ensure that the decision-making system can control the vehicle more accurately based on the location information of obstacles, in another embodiment of this application, the adjusted target pseudo image is subjected to target detection. After obtaining the detection result, the target detection method further includes: performing coordinate transformation on the information of the obstacle in the third rectangle based on the parameter information of the third rectangle and the lidar to obtain the location information of the obstacle in the real environment.

[0061] Specifically, in the above embodiments, the parameter information of the lidar may include the lidar's intrinsic and extrinsic parameters.

[0062] Specifically, in the above embodiments, the real environment refers to the actual environment.

[0063] In one embodiment of this application, merging multiple clustering results to obtain at least one target independent region includes: performing connectivity detection on the multiple clustering results, merging intersecting clustering results, and obtaining at least one target independent region. In this embodiment, by performing connectivity detection on each clustering result to merge identical clustering results, at least one target independent region is obtained, thus achieving a relatively simple way to obtain at least one target independent region.

[0064] This application also provides a target detection device based on clustering optimization. It should be noted that the target detection device based on clustering optimization provided in this application can be used to execute the target detection method based on clustering optimization provided in this application. The target detection device based on clustering optimization provided in this application will be described below.

[0065] Figure 2 This is a schematic diagram of the target detection device based on clustering optimization according to an embodiment of this application. Figure 2 As shown, the target detection device includes:

[0066] The receiving unit 10 is used to receive target point cloud data, which corresponds to a frame of target image, and the target point cloud data is point cloud data of a predetermined area obtained by the lidar, which is an area at a predetermined distance from the vehicle.

[0067] The processing unit 20 is used to perform filtering and clustering processing on the target point cloud data to obtain multiple clustering results, and to merge the multiple clustering results to obtain at least one target independent region.

[0068] The first determining unit 30 is used to determine the size of the target voxel based on at least one of the aforementioned target independent regions, construct a target pseudo image based on the size of the target voxel, and perform target detection based on the aforementioned target pseudo image.

[0069] In the aforementioned target detection device based on clustering optimization, the receiving unit receives target point cloud data of the vehicle within a predetermined area; the processing unit performs filtering and clustering processing on the received target point cloud data to obtain multiple clustering results, and merges these multiple clustering results to obtain at least one target-independent region; the first determining unit determines the size of the target voxel corresponding to at least one of the aforementioned target-independent regions, constructs a target pseudo-image based on the target voxel size, and performs target detection based on the constructed target pseudo-image. Compared to existing technologies that determine the target voxel size based on manual experience, this application performs filtering and clustering processing on the target point cloud data to obtain multiple clustering results, and merges these multiple clustering results to obtain one or more target-independent regions. Then, based on one or more target-independent regions, the size of the target voxel corresponding to the target-independent region is determined. This ensures a more reasonable division of the target voxel, thereby solving the problem of unreasonable voxel division in existing technologies. The target detection method of this application constructs a target pseudo-image corresponding to the independent region of the target based on the size of the target voxel. This ensures that the point cloud information in the target pseudo-image is all valid information, and ensures that the target detection algorithm based on the target pseudo-image has good performance and high detection accuracy, thereby balancing the relationship between the size of the target voxel and the target detection algorithm.

[0070] In one specific embodiment of this application, the target point cloud data is filtered to obtain filtered target point cloud data. Then, the filtered target point cloud data is clustered to obtain multiple clustering results (i.e., clusters), and the clustering probability corresponding to each clustering result can also be obtained. Filtering the target point cloud data reduces the impact of isolated noise in the target point cloud data on the clustering results.

[0071] In another specific embodiment of this application, the target point cloud data is clustered to obtain multiple clustering results (i.e., clusters) and the clustering probability corresponding to each clustering result. Then, the multiple clustering results are filtered to obtain filtered clustering results. In this embodiment, filtering is performed on multiple clustering results to ensure that the obtained clustering results are relatively accurate.

[0072] In practical applications, any feasible filtering method in the existing technology can be used to filter the target point cloud data or to filter multiple clustering results. This application does not limit the specific clustering method. For example, the specific filtering method can be a voxel filter, bilateral filtering, spatial distribution-based denoising algorithm (Statistical Outlier Removal, SOR), etc.

[0073] In practical applications, any feasible clustering method in the existing technology can be used to cluster the target point cloud data. This application does not limit the specific clustering method. For example, Spectral Clustering (SC), Hierarchical Clustering (HC), and Mean-shift clustering are all acceptable methods.

[0074] Specifically, in this application, when a single target independent region is obtained, the size of the target voxel of that target independent region is determined, at least based on that target independent region. When multiple target independent regions are obtained, the size of the target voxel of each target independent region is determined, at least based on each target independent region.

[0075] To simplify the determination of the target voxel size, in one embodiment of this application, the first determining unit includes a first conversion module, a second conversion module, and a first calculation module. The first conversion module performs a viewpoint conversion on at least one of the target independent regions to obtain the total area of ​​at least one of the target independent regions from a BEV viewpoint. The second conversion module performs a viewpoint conversion on the target point cloud data to obtain the total area of ​​the target point cloud data from a BEV viewpoint. The first calculation module calculates the ratio of the total area of ​​at least one of the target independent regions from a BEV viewpoint to the total area of ​​the target point cloud data from a BEV viewpoint, obtaining a first target ratio, and determines the size of the target voxel based at least on the first target ratio.

[0076] Specifically, in the above embodiments, when a single target independent region is obtained, the viewpoint of the target independent region is transformed to obtain the target independent region from the BEV viewpoint, and the total area of ​​the target independent region from the BEV viewpoint is calculated; then, the viewpoint of the target point cloud data is transformed to obtain the target point cloud data from the BEV viewpoint, and the total area of ​​the target point cloud data from the BEV viewpoint is calculated; finally, a first target ratio is obtained based on the ratio of the total area of ​​the target independent region from the BEV viewpoint to the total area of ​​the target point cloud data from the BEV viewpoint. When multiple target independent regions are obtained, the viewpoint of the multiple target independent regions is transformed to obtain multiple target independent regions from the BEV viewpoint, and the sum of the areas of the multiple target independent regions from the BEV viewpoint is calculated, i.e., the total area; then, the viewpoint of the target point cloud data is transformed to obtain the target point cloud data from the BEV viewpoint, and the total area of ​​the target point cloud data from the BEV viewpoint is calculated; finally, a first target ratio is obtained based on the ratio of the total area of ​​the multiple target independent regions from the BEV viewpoint to the total area of ​​the target point cloud data from the BEV viewpoint. Based on the first target ratio, the size of the target voxel is determined, so that each target independent region has a target voxel of the same size. Subsequently, a target pseudo-image of the corresponding target independent region is constructed based on the target voxel, so that the target voxel in each target pseudo-image is the same.

[0077] In one specific embodiment of this application, the size of the target voxel can be determined based on a first target ratio and a predefined predetermined table. Specifically, the process involves determining the size of the target voxel corresponding to the first target ratio from the first predetermined table. This allows for a more straightforward determination of the target voxel size.

[0078] In another embodiment of this application, the first determining unit further includes a first determining module, a third conversion module, and a second calculation module. The first determining module is used to determine the smallest rectangular frame surrounding the target independent region, obtaining at least one first rectangular frame. The third conversion module is used to perform a viewpoint conversion on the at least one target independent region to obtain the area of ​​the at least one target independent region from a BEV viewpoint, and to perform a viewpoint conversion on the at least one first rectangular frame to obtain the area corresponding to the at least one first rectangular frame from a BEV viewpoint. The second calculation module is used to calculate the ratio of the area of ​​the at least one target independent region from a BEV viewpoint to the area of ​​the at least one first rectangular frame from a BEV viewpoint, obtaining at least one second target ratio, and, based at least on the second target ratio, to determine the size of the target voxel corresponding to the target independent region. This allows for a simpler determination of the target voxel size and ensures that the determined target voxel size is more reasonable.

[0079] Specifically, in the above embodiments, when a single target independent region is obtained, the smallest rectangle surrounding the target independent region is determined to obtain a corresponding first rectangle; then, the viewpoints of the target independent region and the corresponding first rectangle are transformed to obtain the target independent region and the first rectangle from the BEV viewpoint, and the area of ​​the target independent region and the area of ​​the corresponding first rectangle from the BEV viewpoint are calculated; finally, based on the ratio of the area of ​​the target independent region to the area of ​​the first rectangle, a second target ratio is determined, and the size of the target voxel of the corresponding target independent region is determined at least based on the second target ratio. When multiple target independent regions are obtained, the smallest rectangle surrounding the target independent region is determined to obtain a first rectangle corresponding to each target independent region; then, the viewpoints of each target independent region and the corresponding first rectangle are transformed to obtain each target independent region and the corresponding first rectangle from the BEV viewpoint, and the areas of each target independent region and the corresponding first rectangle from the BEV viewpoint are calculated; finally, the ratio of the area of ​​each target independent region to the area of ​​the corresponding first rectangle is calculated to obtain a second target ratio, and the size of the target voxel of the corresponding target independent region is determined at least based on the second target ratio. This ensures that the target voxel size is different for each independent target region, making the determined target voxel more reasonable. This further ensures that the subsequent construction of the target pseudo-image corresponding to each independent target region based on the target voxel size is more reasonable, thus further ensuring better target detection performance and accuracy.

[0080] In one specific embodiment of this application, the size of the target voxel can be determined based on a second target ratio and a predefined predetermined table. Specifically, the size of the target voxel corresponding to the second target ratio is determined from the second predetermined table. This allows for a more straightforward determination of the target voxel size.

[0081] To obtain the target pseudo-image more easily, in another embodiment of this application, the first determining unit further includes a second determining module, a cropping module, and a segmentation module. The second determining module is used to determine the smallest rectangular frame surrounding the independent region of the target, obtaining at least one second rectangular frame and its position information in the target image. The cropping module is used to crop the target image based on the at least one second rectangular frame and its position information in the target image, obtaining at least one predetermined pseudo-image. The segmentation module is used to segment the at least one predetermined pseudo-image according to the size of the target voxel, obtaining at least one target pseudo-image.

[0082] Specifically, when there are multiple independent target regions, multiple target pseudo-images with different target voxel sizes can be obtained, which can also be called pseudo-image groups.

[0083] Specifically, in this application, the first rectangle and the second rectangle can be the same rectangle.

[0084] In another embodiment of this application, the first determining unit further includes an adjustment module and a target detection module. The adjustment module adjusts the size of the target pseudo-image to obtain an adjusted target pseudo-image. The target detection module performs target detection on the adjusted target pseudo-image to obtain a detection result. The detection result includes the category probability of obstacles in the target pseudo-image and a third bounding box. The category probability is the confidence level that the obstacle belongs to each obstacle category, and the third bounding box is the smallest bounding box surrounding the obstacle. In this embodiment, target detection is performed based on the adjusted target pseudo-image. Since the point cloud data in the target pseudo-image are all valid point cloud data, this further ensures good performance and high detection accuracy of the target detection algorithm.

[0085] In practical applications, any feasible method in the existing technology can be used for size adjustment to obtain the adjusted target pseudo-image. This application does not limit the specific method of size adjustment. In one specific embodiment of this application, when multiple target pseudo-images are obtained, an average size can be determined based on the size corresponding to each target pseudo-image. Then, based on the average size and the size of each target pseudo-image, the size of the corresponding target pseudo-image is adjusted. For example, if the size of a target pseudo-image is smaller than the average size, the corresponding target pseudo-image can be expanded using the padding method; if the size of a target pseudo-image is larger than the average size, the corresponding target pseudo-image can be reduced using the ROI pooling method. This makes it relatively easy to adjust the size of each target pseudo-image to the average size.

[0086] Specifically, this application does not limit the specific algorithm for object detection; the algorithm can be flexibly adjusted according to the actual object detection scenario.

[0087] To more accurately determine the category of an obstacle, in one embodiment of this application, the target detection device further includes a second determining unit and a third determining unit. The second determining unit is used to perform target detection on the adjusted target pseudo-image. After obtaining the detection result, if the target image in the target point cloud data is a first frame image, it determines the target category probability of the obstacle in the target pseudo-image based on the category probability of the target pseudo-image and the first category probability map corresponding to the target image, and constructs a second category probability map corresponding to the target image. The third determining unit is used to determine the target category probability of the obstacle in the target pseudo-image based on the category probability of the target pseudo-image and the second category probability map corresponding to the previous frame target image if the target image in the target point cloud data is not the first frame image.

[0088] Specifically, in the above embodiments, when the target image corresponding to the target point cloud data is the first frame, since the target image does not have a second category probability map, the target category probability of the obstacle can be determined based on the obstacle category probability obtained by target detection and the first category probability map corresponding to the target image; when the target image corresponding to the target point cloud data is not the first frame, since the previous frame target image corresponding to the target image has been constructed as a second category probability map, the target category probability of the obstacle can be determined based on the obstacle category probability obtained by target detection and the second category probability map corresponding to the previous frame target image. This ensures that the determined obstacle target category probability is relatively accurate, and ensures that the obstacle category is accurately determined based on the target category probability.

[0089] Specifically, in the above embodiments, the process of determining the target category probability of obstacles in the target pseudo-image based on the category probability of the target pseudo-image and the first category probability map corresponding to the target image can be as follows:

[0090] Step 1: Assuming that the target detection method of this application can detect obstacles in five categories, namely Category 1, Category 2, Category 3, Category 4 and Category 5;

[0091] Step 2: For a target pseudo-image, after target detection, the probability that the obstacle in the target pseudo-image belongs to category 1 is P1, the probability that it belongs to category 2 is P2, the probability that it belongs to category 3 is P3, the probability that it belongs to category 4 is P4, and the probability that it belongs to category 5 is P5. Thus, a category probability group (P1, P2, P3, P4, P5) can be obtained.

[0092] Step 3: Based on the target pseudo-image, determine the cluster probability of the obstacles in the target pseudo-image belonging to a certain category in the first category probability map;

[0093] Step 4: Combine the clustering probability of obstacles in the target pseudo-image belonging to a certain category with the above category probability groups to obtain a target category probability group, namely (P1, P2, P3, P4, P5, P...). 聚类 );

[0094] Step 5: Assign probability groups to the target category (P1, P2, P3, P4, P5, P...). 聚类 Normalization is performed to obtain the normalized (P1', P2', P3', P4', P5', P... 聚类 '), and according to P 聚类 'The category to which P belongs,' 聚类 Add to the same category, for example, P 聚类 'P3' belongs to the same category, therefore P... 聚类 Adding P1' to P3', we get (P1', P2', P3' + P 聚类 P1', P4', P5'). Then (P1', P2', P3' + P 聚类 The probability of the largest value among P', P4', and P5' is determined as the target category probability.

[0095] Specifically, since the target class probability of the obstacle in the target pseudo-image is determined based on the class probability of the target pseudo-image and the second class probability map corresponding to the target image in the previous frame, it is consistent with the target class probability of the obstacle in the target pseudo-image determined based on the class probability of the target pseudo-image and the first class probability map corresponding to the target image, so it will not be elaborated further.

[0096] To further ensure the accuracy of the target category probabilities of obstacles in the subsequently determined target pseudo-image, in another embodiment of this application, the second determining unit includes a third determining module and a first execution module. The third determining module is used to determine a first region of the target pseudo-image in the target image based on the size of the target pseudo-image and the position information of the center point of the target pseudo-image in the target image. The first execution module is used to set the probability of each voxel point in the first region to the clustering probability based on the clustering probability of the clustering result corresponding to the target pseudo-image, thereby obtaining the first category probability map.

[0097] In another embodiment of this application, the second determining unit further includes a fourth determining module and a second execution module. The fourth determining module is used to determine a second region of the third rectangle in the target image based at least on the size of the third rectangle and the position information of the center point of the third rectangle in the target image. The second execution module is used to set the probability of each voxel point in the second region to the target category probability based on the target category probability corresponding to the third rectangle, thereby obtaining the second category probability map. In this embodiment, after determining the target category probability corresponding to the obstacle in each target pseudo-image, a second region corresponding to the third rectangle is determined in the target image based on the third rectangle and the position information of the center point of the third rectangle in the target image. The probability of each voxel point in the corresponding second region is then set to the target category probability of the obstacle corresponding to the third rectangle, resulting in a second category probability map. This further ensures that the target category probability of the target pseudo-image of the target image in the next frame is relatively accurate.

[0098] In order to obtain the location information of obstacles in the real environment more easily, and to further ensure that the decision-making system can control the vehicle more accurately based on the location information of obstacles, in another embodiment of this application, the target detection device further includes a conversion unit, which is used to perform target detection on the adjusted target pseudo image, and after obtaining the detection result, perform coordinate transformation on the information of the obstacle in the third rectangular frame based on the parameter information of the third rectangular frame and the lidar to obtain the location information of the obstacle in the real environment.

[0099] Specifically, in the above embodiments, the parameter information of the lidar may include the lidar's intrinsic and extrinsic parameters.

[0100] Specifically, in the above embodiments, the real environment refers to the actual environment.

[0101] In one embodiment of this application, the processing unit includes a merging module for performing connectivity detection on multiple clustering results, merging intersecting clustering results, and obtaining at least one target independent region. In this embodiment, by performing connectivity detection on each clustering result to merge identical clustering results and thereby obtaining at least one target independent region, at least one target independent region is obtained relatively easily.

[0102] The aforementioned target detection device based on clustering optimization includes a processor and a memory. The aforementioned receiving unit, processing unit, and first determining unit are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0103] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of unreasonable voxel partitioning in existing technologies.

[0104] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0105] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the above-described target detection method based on clustering optimization.

[0106] This invention provides a processor for running a program, wherein the program executes the target detection method based on clustering optimization.

[0107] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0108] Step S101: Receive target point cloud data. The target point cloud data corresponds to a frame of target image. The target point cloud data is point cloud data of a predetermined area obtained by the lidar. The predetermined area is an area at a predetermined distance from the vehicle.

[0109] Step S102: Filter and cluster the target point cloud data to obtain multiple clustering results, and merge the multiple clustering results to obtain at least one independent target region.

[0110] Step S103: Determine the size of the target voxel based on at least one of the aforementioned target independent regions, construct a target pseudo-image based on the size of the target voxel, and perform target detection based on the aforementioned target pseudo-image.

[0111] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0112] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0113] Step S101: Receive target point cloud data. The target point cloud data corresponds to a frame of target image. The target point cloud data is point cloud data of a predetermined area obtained by the lidar. The predetermined area is an area at a predetermined distance from the vehicle.

[0114] Step S102: Filter and cluster the target point cloud data to obtain multiple clustering results, and merge the multiple clustering results to obtain at least one independent target region.

[0115] Step S103: Determine the size of the target voxel based on at least one of the aforementioned target independent regions, construct a target pseudo-image based on the size of the target voxel, and perform target detection based on the aforementioned target pseudo-image.

[0116] To enable those skilled in the art to better understand the technical solution of this application, the technical solution and technical effects of this application will be described below in conjunction with specific embodiments.

[0117] Example

[0118] The target detection method in this application incorporates the concept of clustering. The clustering results are used for filtering, focusing on the regions where the clusters are located to obtain the target voxel size, i.e., the quantization parameter of the target voxel. The specific steps are as follows:

[0119] Step 1: Receive the target point cloud data (where the target image corresponding to the target point cloud data is the first frame), and filter the target point cloud data to remove isolated noise in the target point cloud data and reduce the impact of isolated noise on subsequent clustering processing steps.

[0120] Step 2: Perform clustering on the target point cloud data to obtain multiple clustering results (clusters).

[0121] Step 3: Perform connectivity monitoring on multiple clustering results, merge intersecting clustering results to obtain one or more independent target regions; in this embodiment, the specific target detection steps are described by obtaining multiple independent target regions.

[0122] Step 4: Determine the size of the target voxel based on multiple independent target regions. There are two specific approaches:

[0123] 1) Perform viewpoint transformation on multiple independent target regions to obtain multiple independent target regions from the BEV perspective; perform viewpoint transformation on the target point cloud data to obtain target point cloud data from the BEV perspective; calculate the sum of the areas of the multiple independent target regions from the BEV perspective, and calculate the total area of ​​the target point cloud data from the BEV perspective; finally, calculate the ratio of the sum of the areas of the multiple independent target regions from the BEV perspective to the total area of ​​the target point cloud data from the BEV perspective to obtain the first target ratio. The size of the target voxel is determined based at least on the first target ratio and a first predetermined table.

[0124] 2) Perform viewpoint transformation on multiple independent target regions to obtain multiple independent target regions from the BEV perspective; perform viewpoint transformation on each first rectangle surrounding the independent target regions to obtain multiple first rectangles from the BEV perspective; calculate the area of ​​each independent target region from the BEV perspective, and calculate the total area of ​​each first rectangle from the BEV perspective; finally, calculate the ratio of the area of ​​each independent target region to the area of ​​its corresponding first rectangle from the BEV perspective to obtain multiple second target ratios. The size of the target voxel is determined based at least on the second target ratios and a second predetermined table.

[0125] Step 5: Construct a pseudo-image of the target based on the size of the target voxel mentioned above. There are two specific methods:

[0126] 1) Corresponding to the first scheme in step four, determine the smallest bounding box surrounding each independent target region, obtaining multiple second bounding boxes and their position information in the target image; based on the position information of each second bounding box in the target image, crop the target image to obtain multiple predetermined pseudo-images. Then, using the determined target voxel size, perform voxel division on the multiple predetermined pseudo-images to obtain multiple target pseudo-images. In this case, the target voxel size in the multiple target pseudo-images is consistent.

[0127] 2) Corresponding to the second scheme in step four, determine the smallest bounding box surrounding each independent target region, obtaining multiple second bounding boxes and their position information in the target image; based on the position information of each second bounding box in the target image, crop the target image to obtain multiple predetermined pseudo-images. Based on the size of the target voxel corresponding to each predetermined pseudo-image, perform voxel division on the corresponding predetermined pseudo-image to obtain multiple target pseudo-images. In this case, the sizes of the target voxels in the multiple target bitmaps obtained are inconsistent.

[0128] At the same time, based on the operation in step 5, k (i.e., multiple) target pseudo-images of different sizes can be obtained.

[0129] Step 6: Construct a first-class probability map on the target image. The specific process is as follows: Based on the size (w) of the target pseudo-image... i h i ) and the location information of the center point of the target pseudo-image in the target image ((x i y i The first region of each target pseudo-image in the target image is determined; based on the clustering probability of the clustering result corresponding to the target pseudo-image, the probability of each voxel point in the corresponding first region is set as the clustering probability to obtain the first category probability map.

[0130] Step 7: Calculate the dimensions of each target pseudo-image and their average size. If a target pseudo-image's size is smaller than the average size, padding is used to expand it; if a target pseudo-image's size is larger than the average size, ROI pooling is used to shrink it. This step yields multiple resized target pseudo-images, which can then form a pseudo-image batch.

[0131] Step 8: Pass the pseudo-image batch obtained in Step 7 to the neural network for detection and obtain the detection results. The detection results include the obstacle class probability and the third bounding box (bbox size) in each target pseudo-image. The obstacle class probability is the confidence level that the obstacle belongs to each obstacle class. If there are five detected classes, the obstacle class probabilities can be a class probability set (P1, P2, P3, P4, P5). Simultaneously, based on the corresponding target pseudo-image, determine the cluster probability corresponding to the target pseudo-image in the first class probability map, thus obtaining the target class probability set (P1, P2, P3, P4, P5, P...). 聚类 Again, for the target category probability groups (P1, P2, P3, P4, P5, P...),... 聚类Normalization is performed to obtain the normalized (P1', P2', P3', P4', P5', P... 聚类 '), and according to P 聚类 'The category to which P belongs,' 聚类 Add to the same category, for example, P 聚类 'P3' belongs to the same category, therefore P... 聚类 Adding P1' to P3', we get (P1', P2', P3' + P 聚类 P1', P4', P5'). Then (P1', P2', P3' + P 聚类 The probability of the largest value among P', P4', and P5' is determined as the target category probability P.

[0132] Step 9: Based on the dimensions of each third bounding box (i.e., the bbox obtained in step 8) and the position information of the center point of each third bounding box in the target image, determine the second region of the corresponding third bounding box in the target image; based on the target class probability corresponding to each third bounding box, set the probability of each voxel point in the second region to the corresponding target class probability, and obtain the second class probability map. In this way, when calculating the target class probability of the target point cloud data in the second frame, the target class probability is determined based on the second class probability map of the first frame.

[0133] Step 10: Based on the parameter information of the third rectangle and the LiDAR, perform coordinate transformation on the obstacle information in the third rectangle to obtain the obstacle's position information in the real environment.

[0134] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0136] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0138] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0139] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0140] 1) In the target detection method based on clustering optimization of this application, firstly, target point cloud data of the vehicle in a predetermined area is received; then, the received target point cloud data is filtered and clustered to obtain multiple clustering results, and the multiple clustering results are merged to obtain at least one target independent region; finally, based on at least one of the above-mentioned target independent regions, the size of the target voxel corresponding to the target independent region is determined, and a target pseudo-image is constructed based on the size of the target voxel, and target detection is performed based on the constructed target pseudo-image. Compared with the prior art, which determines the size of the target voxel based on manual experience, this application filters and clusters the target point cloud data to obtain multiple clustering results, and merges the multiple clustering results to obtain one or more target independent regions. Then, based on one or more target independent regions, the size of the target voxel corresponding to the target independent region is determined, which ensures that the division of the target voxel is more reasonable, thereby solving the problem of unreasonable voxel division in the prior art. The target detection method of this application constructs a target pseudo-image corresponding to the independent region of the target based on the size of the target voxel. This ensures that the point cloud information in the target pseudo-image is all valid information, and ensures that the target detection algorithm based on the target pseudo-image has good performance and high detection accuracy, thereby balancing the relationship between the size of the target voxel and the target detection algorithm.

[0141] 2) In the target detection device based on clustering optimization of this application, the receiving unit is used to receive target point cloud data of the vehicle in a predetermined area; the processing unit is used to perform filtering and clustering processing on the received target point cloud data to obtain multiple clustering results, and to merge the multiple clustering results to obtain at least one target independent region; the first determining unit is used to determine the size of the target voxel corresponding to the target independent region based on at least one of the above-mentioned target independent regions, and to construct a target pseudo-image based on the size of the target voxel, and to perform target detection based on the constructed target pseudo-image. Compared with the prior art, which determines the size of the target voxel based on manual experience, this application performs filtering and clustering processing on the target point cloud data to obtain multiple clustering results, and merges the multiple clustering results to obtain one or more target independent regions. Then, based on one or more target independent regions, the size of the target voxel corresponding to the target independent region is determined, which ensures that the division of the target voxel is more reasonable, thereby solving the problem of unreasonable voxel division in the prior art. The target detection method of this application constructs a target pseudo-image corresponding to the independent region of the target based on the size of the target voxel. This ensures that the point cloud information in the target pseudo-image is all valid information, and ensures that the target detection algorithm based on the target pseudo-image has good performance and high detection accuracy, thereby balancing the relationship between the size of the target voxel and the target detection algorithm.

[0142] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A target detection method based on clustering optimization, characterized in that, include: Receive target point cloud data, the target point cloud data corresponds to a frame of target image, and the target point cloud data is point cloud data of a predetermined area obtained by lidar, the predetermined area is an area at a predetermined distance from the vehicle. The target point cloud data is filtered and clustered to obtain multiple clustering results, and the multiple clustering results are merged to obtain at least one independent target region. The size of the target voxel is determined based on at least one of the target independent regions, and a target pseudo-image is constructed based on the size of the target voxel, and target detection is performed based on the target pseudo-image; Determining the size of a target voxel based on at least one of the target independent regions includes: performing a view transformation on at least one of the target independent regions to obtain the total area of ​​at least one of the target independent regions from a BEV viewpoint; performing a view transformation on the target point cloud data to obtain the total area of ​​the target point cloud data from the BEV viewpoint; calculating the ratio of the total area of ​​at least one of the target independent regions from the BEV viewpoint to the total area of ​​the target point cloud data from the BEV viewpoint to obtain a first target ratio; and determining the size of the target voxel based at least on the first target ratio. Alternatively, determining the size of a target voxel based on at least one of the target independent regions includes: determining the smallest rectangular frame surrounding the target independent region to obtain at least one first rectangular frame; performing a view transformation on the at least one target independent region to obtain the area of ​​the at least one target independent region from a BEV viewpoint, and performing a view transformation on the at least one first rectangular frame to obtain the area corresponding to the at least one first rectangular frame from the BEV viewpoint; calculating the ratio of the area of ​​the at least one target independent region from the BEV viewpoint to the area of ​​the at least one first rectangular frame from the BEV viewpoint to obtain at least one second target ratio, and determining the size of the target voxel of the corresponding target independent region based at least on the second target ratio.

2. The target detection method according to claim 1, characterized in that, Constructing a target pseudo-image based on the size of the target voxel includes: Determine the smallest bounding box that surrounds the target independent region, and obtain at least one second bounding box and the position information of the second bounding box in the target image; Based on at least one second bounding box and the position information of the second bounding box in the target image, a cropping is performed in the target image to obtain at least one predetermined pseudo image; According to the size of the target voxel, at least one of the predetermined pseudo images is divided to obtain at least one target pseudo image.

3. The target detection method according to claim 1 or 2, characterized in that, Target detection based on the target pseudo-image includes: The size of the target pseudo-image is adjusted to obtain the adjusted target pseudo-image; The adjusted target pseudo-image is subjected to target detection to obtain detection results. The detection results include the category probability of obstacles in the target pseudo-image and a third bounding box. The category probability is the confidence level of the obstacle belonging to each obstacle category, and the third bounding box is the smallest bounding box that surrounds the obstacle.

4. The target detection method according to claim 3, characterized in that, After performing target detection on the adjusted target pseudo-image and obtaining the detection result, the target detection method further includes: When the target image in the target point cloud data is a first frame image, the target category probability of the obstacle in the target pseudo image is determined based on the category probability of the target pseudo image and the first category probability map corresponding to the target image, and a second category probability map corresponding to the target image is constructed. If the target image in the target point cloud data is not the first frame image, the target category probability of the obstacle in the target pseudo image is determined based on the category probability of the target pseudo image and the second category probability map corresponding to the target image in the previous frame.

5. The target detection method according to claim 4, characterized in that, The process of determining the first category probability map corresponding to the target image includes: Based on the size of the target pseudo-image and the position information of the center point of the target pseudo-image in the target image, a first region of the target pseudo-image in the target image is determined; Based on the clustering probability of the clustering result corresponding to the target pseudo-image, the probability of each voxel point in the first region is set to the clustering probability to obtain the first category probability map.

6. The target detection method according to claim 4, characterized in that, The process of constructing the second category probability map corresponding to the target image includes: The second region of the third rectangle in the target image is determined based at least on the size of the third rectangle and the position information of the center point of the third rectangle in the target image; Based on the target category probability corresponding to the third rectangle, the probability of each voxel point in the second region is set to the target category probability to obtain the second category probability map.

7. The target detection method according to claim 3, characterized in that, After performing target detection on the adjusted target pseudo-image and obtaining the detection result, the target detection method further includes: Based on the parameter information of the third rectangle and the lidar, coordinate transformation is performed on the information of the obstacle in the third rectangle to obtain the position information of the obstacle in the real environment.

8. The target detection method according to claim 1 or 2, characterized in that, The clustering results are merged to obtain at least one target independent region, including: Connectivity detection is performed on multiple clustering results, and intersecting clustering results are merged to obtain at least one target independent region.

Images