Target detection method and device

Abstract

This application discloses a target detection method and apparatus, relating to the field of information processing technology, particularly to the fields of autonomous driving or intelligent transportation. This application optimizes and updates the clustering results of 3D point clouds by combining image detection information from target scene images, thereby avoiding under-segmentation and over-segmentation problems in the target recognition process and improving the accuracy of target recognition. Specifically, by comparing the overlap between the initially clustered 3D point cloud and the target detection boxes after image detection, under-segmentation problems in the initial clustering process can be identified; by comprehensively analyzing the overlap between the initially clustered 3D point cloud and the target detection boxes after image detection, as well as the preset target size corresponding to each target detection box, over-segmentation problems in the initial clustering process can be identified and corrected.

Description

Technical Field

[0001] This application relates to the field of information processing technology, and in particular to target detection methods and apparatus in the fields of autonomous driving or intelligent transportation. Background Technology

[0002] Advanced driver assistance systems (ADAS) or autonomous driving systems typically incorporate various 3D point perception modules, such as laser sensors, radar, sonar, or ultrasonic sensors, to perceive the surrounding environment and target information. By perceiving the surrounding environment and targets, target recognition can be achieved, thereby effectively avoiding obstacles and increasing the comfort and safety of autonomous driving. These targets include vehicles, pedestrians, guardrails, bollards, bushes, lampposts, vehicle access barriers / stop barriers, overpasses, culverts, or buildings. Typically, these 3D point perception modules, based on acquired 3D point clouds, use clustering to obtain information such as the position, orientation, speed, and shape of one or more targets.

[0003] However, in scenarios where multiple different targets occlude each other, the 3D point sensing module cannot accurately distinguish between different targets, resulting in undersegmentation. Furthermore, for the same target, the discontinuity of the data acquired by the 3D point sensing module may lead to oversegmentation. Undersegmentation refers to multiple targets being mistaken for a single target, while oversegmentation refers to a single target being mistaken for multiple targets. For example, assuming the 3D point sensing module is a laser sensor, in... Figure 1 In the scene shown in (a) on the left, because the bushes and the car next to them obstruct each other, the reflected light from the laser emitted by the laser sensor is difficult for the laser sensor to distinguish. In this case, the laser sensor will mistakenly identify the bushes and the car next to them as a single target 1, resulting in undersegmentation. Similarly, in Figure 1 In the scenario shown on the right side of (a), due to the mutual occlusion between the person and the vehicle next to them, the reflected light from the laser emitted by the laser sensor on the person and the vehicle next to them is difficult for the laser sensor to distinguish. In this case, the laser sensor will mistakenly treat the person and the vehicle next to them as a single target 2, resulting in under-segmentation. Figure 1 In the scenario shown in (b), the position and speed of the truck and its cargo are synchronized, forming a single unit for the ADAS or AD system. However, due to the discontinuity of the reflected light from the laser sensor on the truck and cargo, the laser sensor may incorrectly classify them as multiple targets (e.g., ...). Figure 1As shown in (b) (targets 3-9), oversegmentation occurs. The aforementioned undersegmentation and oversegmentation phenomena can lead to inaccurate target recognition, resulting in misjudgments of obstacles and affecting the safety of autonomous driving. Furthermore, oversegmentation increases unnecessary computational load. Summary of the Invention

[0004] This application provides a target detection method and apparatus that can solve the problems of undersegmentation and oversegmentation in the target recognition process and improve the accuracy of target recognition.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, a target detection method is provided, comprising: acquiring image detection information of a target scene image and clustering information of a 3D point cloud of the target scene; wherein, the image detection information includes: the position and size of multiple target detection boxes, the target category and identification information corresponding to the multiple target detection boxes; the clustering information of the 3D point cloud includes at least one first cluster target and the 3D points corresponding to the at least one first cluster target; updating the clustering information of the 3D point cloud based on the overlap between the projection point of the 3D point corresponding to the at least one first cluster target in the image coordinate system and the corresponding target detection box in the multiple target detection boxes, and in combination with the target preset size corresponding to each target detection box; wherein the image coordinate system is the coordinate system of the target scene image; wherein, the updated clustering information of the 3D point cloud includes at least one second cluster target and the 3D points corresponding to the at least one second cluster target; the number of second cluster targets is less than or equal to the number of first cluster targets.

[0007] The technical solution provided in the first aspect above optimizes and updates the clustering results of the 3D point cloud by combining image detection information from the target scene image, thereby avoiding undersegmentation and oversegmentation problems in the target recognition process and improving the accuracy of target recognition. Specifically, by comparing the overlap between the pre-clustered 3D point cloud and the target detection boxes after image detection, undersegmentation problems in the pre-clustering process can be identified; by comprehensively analyzing the overlap between the pre-clustered 3D point cloud and the target detection boxes after image detection, as well as the preset target size corresponding to each target detection box, oversegmentation problems in the pre-clustering process can be identified and the results corrected.

[0008] In one possible implementation, obtaining the clustering information of the 3D point cloud of the target scene includes: projecting the 3D point cloud onto a 2D raster image to obtain multiple 2D points and the correspondence between the 3D point cloud and the multiple 2D points; clustering the multiple 2D points into at least one first clustering target; and determining the 3D point corresponding to the at least one first clustering target based on the correspondence between the 3D point cloud and the multiple 2D points. Through the above process, the 3D point cloud can be initially clustered to obtain preliminary target recognition results.

[0009] In one possible implementation, updating the clustering information of the 3D point cloud based on the overlap between the projection points of the 3D points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes, combined with the preset target size corresponding to each target detection box, includes: projecting the 3D point cloud onto the image coordinate system; determining the correspondence between each target detection box and the 3D point in the plurality of target detection boxes based on the overlap between the projection points of the 3D points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes; and updating the clustering information of the 3D point cloud based on the correspondence between each target detection box and the 3D point in the plurality of target detection boxes, combined with the preset target size corresponding to each target detection box. By combining the image detection information of the target scene image, the clustering results of the 3D point cloud are optimized and updated to avoid undersegmentation and oversegmentation problems in the target recognition process, thereby improving the accuracy of target recognition.

[0010] In one possible implementation, determining the correspondence between each target detection box and the 3D point in the plurality of target detection boxes based on the overlap between the projection point of the 3D point corresponding to the at least one first clustering target in the image coordinate system and the corresponding target detection box in the plurality of target detection boxes includes: determining whether the first projection point is in the first target detection box; if the first projection point is in the first target detection box, then determining that the 3D point corresponding to the first projection point matches the first target detection box; the first projection point is the projection point of any 3D point among the 3D points corresponding to the at least one first clustering target in the image coordinate system, and the first target detection box is any one of the plurality of target detection boxes. Matching 3D points and target detection boxes using the above method can achieve high matching accuracy.

[0011] In one possible implementation, the method further includes: if the first projection point is in the first target detection box and the second target detection box, then d1 and d2 are calculated; wherein, the second target detection box is one of the multiple target detection boxes, and the second target detection box overlaps with the first target detection box; d1 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the first target detection box other than the overlapping region; d2 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the second target detection box other than the overlapping region; if d2 is less than d1, then the 3D point corresponding to the first projection point is determined to match the second target detection box; if d2 is greater than or equal to d1, then the 3D point corresponding to the first projection point is determined to match the first target detection box. Through this method, areas prone to target classification misclassification can be analyzed in detail to obtain more accurate target recognition results.

[0012] In one possible implementation, the above-mentioned updating of the clustering information of the 3D point cloud based on the correspondence between each target detection box and a 3D point in multiple target detection boxes, combined with the preset target size corresponding to each target detection box, includes: determining the number of first cluster targets matching each target detection box; if a first detection box matches multiple first cluster targets in at least one of the above first cluster targets, and the maximum distance between any two of the multiple first cluster targets is less than a preset distance, then determining whether to merge the multiple first cluster targets based on the preset target size corresponding to the first detection box; specifically, if the difference between the overall size of the multiple first cluster targets and the preset size of the first cluster targets of a preset category in the above multiple first cluster targets is less than a preset threshold, then merging the multiple first cluster targets into a second cluster target. Through the above method, regions prone to oversegmentation can be analyzed in detail, and when necessary, multiple adjacent first cluster targets can be merged into a second cluster target to obtain more accurate target recognition results.

[0013] In one possible implementation, the image detection information further includes: the confidence score of the target category corresponding to each of the plurality of target detection boxes, where the confidence score characterizes the credibility of the target category; the method further includes: labeling a first three-dimensional point with the confidence score of the target category corresponding to a third target detection box; the first three-dimensional point is a three-dimensional point whose projection point is in the third target detection box; the third target detection box is any one of the plurality of target detection boxes; labeling a second three-dimensional point with a pre-set confidence score; the second three-dimensional point is a three-dimensional point whose projection point is not in any of the plurality of target detection boxes. The confidence scores of the three-dimensional points are used for subsequent reference. For example, driver assistance systems or autonomous driving systems may take measures such as braking, deceleration, or lane changing based on the confidence score of the target category. Alternatively, driver assistance systems or autonomous driving systems may decide whether to track the target based on the confidence score of the target category, for example, to further determine the target category.

[0014] In one possible implementation, the above-mentioned acquisition of image detection information of the target scene image includes: performing feature recognition and feature analysis on the target scene image to acquire the above-mentioned image detection information.

[0015] In one possible implementation, the target scene image and the 3D point cloud of the target scene are acquired simultaneously. By acquiring the target scene image and the 3D point cloud of the target scene simultaneously, the accuracy of the second clustering target obtained based on both can be guaranteed.

[0016] Secondly, a target detection device is provided, comprising: a processing unit, configured to acquire image detection information of a target scene image and clustering information of a three-dimensional point cloud of the target scene; wherein the image detection information includes: the position and size of multiple target detection boxes, the target category and identification information corresponding to the multiple target detection boxes; the clustering information of the three-dimensional point cloud includes at least one first cluster target and three-dimensional points corresponding to the at least one first cluster target; and updating the clustering information of the three-dimensional point cloud based on the overlap between the projection point of the three-dimensional point corresponding to the at least one first cluster target in the image coordinate system and the corresponding target detection box in the multiple target detection boxes, combined with the target preset size corresponding to each target detection box; wherein the image coordinate system is the coordinate system of the target scene image; wherein the updated clustering information of the three-dimensional point cloud includes at least one second cluster target and three-dimensional points corresponding to the at least one second cluster target; the number of second cluster targets is less than or equal to the number of first cluster targets.

[0017] The technical solution provided in the second aspect above optimizes and updates the clustering results of the 3D point cloud by combining image detection information from the target scene image, thereby avoiding undersegmentation and oversegmentation problems in the target recognition process and improving the accuracy of target recognition. Specifically, by comparing the overlap between the initially clustered 3D point cloud and the target detection boxes after image detection, undersegmentation problems in the initial clustering process can be identified; by comprehensively analyzing the overlap between the initially clustered 3D point cloud and the target detection boxes after image detection, as well as the preset target size corresponding to each target detection box, oversegmentation problems in the initial clustering process can be identified and corrected.

[0018] In one possible implementation, the processing unit acquires clustering information of the 3D point cloud of the target scene, including: projecting the 3D point cloud onto a 2D raster image to obtain multiple 2D points and the correspondence between the 3D point cloud and the multiple 2D points; clustering the multiple 2D points into at least one first clustering target; and determining the 3D point corresponding to the at least one first clustering target based on the correspondence between the 3D point cloud and the multiple 2D points. Through the above process, the 3D point cloud can be initially clustered to obtain preliminary target recognition results.

[0019] In one possible implementation, the processing unit updates the clustering information of the 3D point cloud based on the overlap between the projection points of the 3D points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes, combined with the preset target size corresponding to each target detection box. This includes: the processing unit projects the 3D point cloud onto the image coordinate system; determines the correspondence between each target detection box and the 3D point in the plurality of target detection boxes based on the overlap between the projection points of the 3D points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes; and updates the clustering information of the 3D point cloud based on the correspondence between each target detection box and the 3D point in the plurality of target detection boxes, combined with the preset target size corresponding to each target detection box. By combining image detection information from the target scene image, the clustering results of the 3D point cloud are optimized and updated to avoid undersegmentation and oversegmentation problems in the target recognition process, thereby improving the accuracy of target recognition.

[0020] In one possible implementation, the processing unit determines the correspondence between each target detection box and a 3D point in the plurality of target detection boxes based on the overlap between the projection points of the 3D points corresponding to the at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes. This includes: the processing unit determining whether a first projection point is in a first target detection box; if the first projection point is in a first target detection box, then determining that the 3D point corresponding to the first projection point matches the first target detection box; the first projection point is the projection point of any one of the 3D points corresponding to the at least one first clustering target in the image coordinate system, and the first target detection box is any one of the plurality of target detection boxes. By matching the 3D points and target detection boxes using the above method, high matching accuracy can be obtained.

[0021] In one possible implementation, the processing unit is further configured to calculate d1 and d2 if the first projection point is in the first target detection box and the second target detection box; wherein the second target detection box is one of the plurality of target detection boxes, and the second target detection box overlaps with the first target detection box; d1 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the first target detection box other than the overlapping region; d2 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the second target detection box other than the overlapping region; if d2 is less than d1, it is determined that the 3D point corresponding to the first projection point matches the second target detection box; if d2 is greater than or equal to d1, it is determined that the 3D point corresponding to the first projection point matches the first target detection box. Through the above method, areas prone to target classification misclassification can be analyzed in detail to obtain more accurate target recognition results.

[0022] In one possible implementation, the processing unit updates the clustering information of the 3D point cloud based on the correspondence between each target detection box and a 3D point in multiple target detection boxes, combined with the preset target size corresponding to each target detection box. This includes: the processing unit determining the number of first cluster targets matching each target detection box; if a first detection box matches multiple first cluster targets from at least one of the first cluster targets, and the maximum distance between any two of these multiple first cluster targets is less than a preset distance, then based on the preset target size corresponding to the first detection box, it determines whether to merge the multiple first cluster targets; specifically, if the difference between the overall size of the multiple first cluster targets and the preset size of the first cluster targets of a preset category in the multiple first cluster targets is less than a preset threshold, then the multiple first cluster targets are merged into a second cluster target. Through this method, regions prone to oversegmentation can be analyzed in detail, and when necessary, multiple adjacent first cluster targets can be merged into a second cluster target, resulting in more accurate target recognition results.

[0023] In one possible implementation, the image detection information further includes: the confidence score of the target category corresponding to each of the plurality of target detection boxes, whereby the confidence score characterizes the credibility of the target category; the processing unit is further configured to: label a first three-dimensional point using the confidence score of the target category corresponding to a third target detection box; the first three-dimensional point is a three-dimensional point whose projection point is in the third target detection box; the third target detection box is any one of the plurality of target detection boxes; and label a second three-dimensional point using a pre-set confidence score; the second three-dimensional point is a three-dimensional point whose projection point is not in any of the plurality of target detection boxes. The confidence scores of the three-dimensional points are used for subsequent reference. For example, driver assistance systems or autonomous driving systems may take measures such as braking, deceleration, or lane changing based on the confidence score of the target category. Alternatively, driver assistance systems or autonomous driving systems may decide whether to track the target based on the confidence score of the target category, for example, by further determining the target category.

[0024] In one possible implementation, the above-mentioned acquisition of image detection information of the target scene image includes: performing feature recognition and feature analysis on the target scene image to acquire the above-mentioned image detection information.

[0025] In one possible implementation, the target detection device further includes a data acquisition unit for simultaneously acquiring a target scene image and a 3D point cloud of the target scene. By simultaneously acquiring the target scene image and the 3D point cloud of the target scene, the accuracy of the second clustered target obtained based on both can be guaranteed.

[0026] Thirdly, a target detection device is provided, comprising: a memory for storing a computer program; and a processor for executing the computer program to support the target detection device in acquiring image detection information of a target scene image and clustering information of a three-dimensional point cloud of the target scene; wherein the image detection information includes: the position and size of multiple target detection boxes, the target category and identification information corresponding to the multiple target detection boxes; the clustering information of the three-dimensional point cloud includes at least one first cluster target and three-dimensional points corresponding to the at least one first cluster target; and updating the clustering information of the three-dimensional point cloud based on the overlap between the projection point of the three-dimensional point corresponding to the at least one first cluster target in the image coordinate system and the corresponding target detection box in the multiple target detection boxes, combined with the target preset size corresponding to each target detection box; wherein the image coordinate system is the coordinate system of the target scene image; wherein the updated clustering information of the three-dimensional point cloud includes at least one second cluster target and three-dimensional points corresponding to the at least one second cluster target; the number of second cluster targets is less than or equal to the number of first cluster targets.

[0027] The technical solution provided in the third aspect above optimizes and updates the clustering results of the 3D point cloud by combining image detection information from the target scene image, thereby avoiding undersegmentation and oversegmentation problems in the target recognition process and improving the accuracy of target recognition. Specifically, by comparing the overlap between the pre-clustered 3D point cloud and the target detection boxes after image detection, undersegmentation problems in the pre-clustering process can be identified; by comprehensively analyzing the overlap between the pre-clustered 3D point cloud and the target detection boxes after image detection, as well as the preset target size corresponding to each target detection box, oversegmentation problems in the pre-clustering process can be identified and corrected.

[0028] In one possible implementation, obtaining the clustering information of the 3D point cloud of the target scene includes: projecting the 3D point cloud onto a 2D raster image to obtain multiple 2D points and the correspondence between the 3D point cloud and the multiple 2D points; clustering the multiple 2D points into at least one first clustering target; and determining the 3D point corresponding to the at least one first clustering target based on the correspondence between the 3D point cloud and the multiple 2D points. Through the above process, the 3D point cloud can be initially clustered to obtain preliminary target recognition results.

[0029] In one possible implementation, updating the clustering information of the 3D point cloud based on the overlap between the projection points of the 3D points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes, combined with the preset target size corresponding to each target detection box, includes: projecting the 3D point cloud onto the image coordinate system; determining the correspondence between each target detection box and the 3D point in the plurality of target detection boxes based on the overlap between the projection points of the 3D points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes; and updating the clustering information of the 3D point cloud based on the correspondence between each target detection box and the 3D point in the plurality of target detection boxes, combined with the preset target size corresponding to each target detection box. By combining the image detection information of the target scene image, the clustering results of the 3D point cloud are optimized and updated to avoid undersegmentation and oversegmentation problems in the target recognition process, thereby improving the accuracy of target recognition.

[0030] In one possible implementation, determining the correspondence between each target detection box and the 3D point in the plurality of target detection boxes based on the overlap between the projection point of the 3D point corresponding to the at least one first clustering target in the image coordinate system and the corresponding target detection box in the plurality of target detection boxes includes: determining whether the first projection point is in the first target detection box; if the first projection point is in the first target detection box, then determining that the 3D point corresponding to the first projection point matches the first target detection box; the first projection point is the projection point of any 3D point among the 3D points corresponding to the at least one first clustering target in the image coordinate system, and the first target detection box is any one of the plurality of target detection boxes. Matching 3D points and target detection boxes using the above method can achieve high matching accuracy.

[0031] In one possible implementation, the processor is further configured to execute the computer program to support the target detection device in calculating d1 and d2 when the first projection point is in the first target detection box and the second target detection box; wherein, the second target detection box is one of the plurality of target detection boxes, and the second target detection box overlaps with the first target detection box; d1 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the first target detection box other than the overlapping region; d2 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the second target detection box other than the overlapping region; if d2 is less than d1, it is determined that the 3D point corresponding to the first projection point matches the second target detection box; if d2 is greater than or equal to d1, it is determined that the 3D point corresponding to the first projection point matches the first target detection box. Through the above method, areas prone to target classification misjudgment can be analyzed in detail to obtain more accurate target recognition results.

[0032] In one possible implementation, the above-mentioned updating of the clustering information of the 3D point cloud based on the correspondence between each target detection box and a 3D point in multiple target detection boxes, combined with the preset target size corresponding to each target detection box, includes: determining the number of first cluster targets matching each target detection box; if a first detection box matches multiple first cluster targets in at least one of the above first cluster targets, and the maximum distance between any two of the multiple first cluster targets is less than a preset distance, then determining whether to merge the multiple first cluster targets based on the preset target size corresponding to the first detection box; specifically, if the difference between the overall size of the multiple first cluster targets and the preset size of the first cluster targets of a preset category in the above multiple first cluster targets is less than a preset threshold, then merging the multiple first cluster targets into a second cluster target. Through the above method, regions prone to oversegmentation can be analyzed in detail, and when necessary, multiple adjacent first cluster targets can be merged into a second cluster target to obtain more accurate target recognition results.

[0033] In one possible implementation, the image detection information further includes: the confidence score of the target category corresponding to each of the plurality of target detection boxes, whereby the confidence score characterizes the credibility of the target category; the processor is further configured to execute the computer program to support the target detection device in labeling a first three-dimensional point with the confidence score of the target category corresponding to a third target detection box; the first three-dimensional point is a three-dimensional point whose projection point is in the third target detection box; the third target detection box is any one of the plurality of target detection boxes; and labeling a second three-dimensional point with a pre-set confidence score; the second three-dimensional point is a three-dimensional point whose projection point is not in any of the plurality of target detection boxes. The confidence score labeling of the three-dimensional points is used for subsequent reference. For example, driver assistance systems or autonomous driving systems may take measures such as braking, deceleration, or lane changing based on the confidence score of the target category. Alternatively, driver assistance systems or autonomous driving systems may decide whether to track the target based on the confidence score of the target category, for example, to further determine the target category.

[0034] In one possible implementation, the above-mentioned acquisition of image detection information of the target scene image includes: performing feature recognition and feature analysis on the target scene image to acquire the above-mentioned image detection information.

[0035] In one possible implementation, the target detection device further includes a visual perception module and a 3D point perception module, used to simultaneously acquire a target scene image and a 3D point cloud of the target scene. By simultaneously acquiring the target scene image and the 3D point cloud of the target scene, the accuracy of the second clustered target obtained based on both can be guaranteed.

[0036] Fourthly, a computer-readable storage medium is provided that stores computer program code, which, when executed by a processor, implements the method as described in any possible implementation of the first aspect.

[0037] Fifthly, a chip system is provided, comprising a processor and a memory, wherein computer program code is stored in the memory; when the computer program code is executed by the processor, it implements the method as described in any possible implementation of the first aspect. The chip system may be composed of chips or may include chips and other discrete devices.

[0038] Sixthly, a computer program product is provided that, when run on a computer, enables the implementation of the method as described in any possible implementation of the first aspect. Attached Figure Description

[0039] Figure 1 Examples of two common target recognition methods are shown in the image.

[0040] Figure 2 Two system architecture diagrams provided for embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the hardware structure of a target detection device provided in an embodiment of this application;

[0042] Figure 4 A schematic flowchart of a target detection method provided in an embodiment of this application;

[0043] Figure 5 This application provides an example diagram of obtaining a target detection box by performing image detection on a target scene image;

[0044] Figure 6 An example diagram illustrating the projection of a three-dimensional point cloud onto a two-dimensional raster image, provided as an embodiment of this application;

[0045] Figure 7 An example diagram of clustering results based on the DFS algorithm provided in this application embodiment;

[0046] Figure 8 An example diagram showing the projection of a 3D point cloud of a target scene onto the image coordinate system of the target scene, provided in an embodiment of this application;

[0047] Figure 9 A flowchart for determining the correspondence between three-dimensional points and target detection boxes is provided in an embodiment of this application;

[0048] Figure 10 An example diagram illustrating the matching of a 3D point with multiple target detection boxes, provided as an embodiment of this application;

[0049] Figure 11 Two example diagrams for target recognition provided in the embodiments of this application;

[0050] Figure 12 A schematic diagram illustrating the identification of oversegmentation problems provided in an embodiment of this application;

[0051] Figure 13 A schematic diagram illustrating the calculation of target size based on the L-shape algorithm, provided for an embodiment of this application;

[0052] Figure 14 A structural block diagram of a target detection device provided in an embodiment of this application;

[0053] Figure 15 This is a structural block diagram of another target detection device provided in an embodiment of this application. Detailed Implementation

[0054] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0055] This application provides a target detection method and apparatus that can be applied to three-dimensional (3D) environmental perception scenarios. For example, the method can be applied to ADAS systems for automobiles, drones, or satellites. The target detection method provided in this application can solve the problems of under-segmentation and over-segmentation in target detection, thereby improving the accuracy of target detection.

[0056] To better understand the embodiments of this application, the system architecture to which the embodiments of this application can be applied is described below.

[0057] The system architecture applicable to the embodiments of this application may include a target detection device. This target detection device deploys a perception module. The perception module may include a visual perception module and a 3D point perception module. For example, the visual perception module may include, but is not limited to, image sensors or imaging sensors, such as cameras, video cameras, or imaging radars (such as synthetic aperture radar). The 3D point perception module may include, but is not limited to, laser sensors (such as lidar or lidar scanners), millimeter-wave radar, sonar, or ultrasonic sensors. For specific descriptions of image sensors, imaging sensors, laser sensors, millimeter-wave radar, sonar, or ultrasonic sensors, please refer to the explanations and descriptions in conventional technology; this application will not elaborate further.

[0058] In this application, the target detection device is used to measure the surrounding environment through a sensing module to obtain measurement data of the surrounding environment. For example, the measurement data may be an image of the surrounding environment (e.g., an image of the target scene) and a three-dimensional point cloud of the surrounding environment (e.g., a three-dimensional point cloud of the target scene). Furthermore, the target detection device is also used to identify targets in the surrounding environment based on the aforementioned measurement data. For example, the target may be a vehicle, pedestrian, guardrail, bollard, bushes, lamppost, vehicle access barrier / stop barrier, overpass culvert, or building, etc.

[0059] For example, in ADAS applications for automobiles, drones, or satellites, target detection devices can sense the surrounding environment during the movement of the vehicle, drone, or satellite using visual perception modules and 3D point perception modules. Then, by sensing the surrounding environment, target recognition is achieved, thereby effectively avoiding obstacles and increasing the comfort and safety of autonomous driving.

[0060] Alternatively, the system architecture applicable to this application embodiment may include a target detection device and a perception module. The perception module is independent of the target detection device. The perception module may include a visual perception module and a 3D point perception module. The perception module is used to measure the surrounding environment and obtain measurement data of the surrounding environment. The target detection device is used to identify targets in the surrounding environment based on the measurement data obtained by the perception module. The perception module can communicate with the target detection device via a wireless network or a wired network.

[0061] Please refer to Figure 2 , Figure 2 The diagram illustrates two system architectures provided in embodiments of this application. For example... Figure 2 As shown in (a) of the diagram, the system architecture 20 includes a target detection device 200. The target detection device 200 includes a perception module 210. The perception module 210 includes a visual perception module 211 and a 3D point perception module 212. (The text repeats itself here.) Figure 2 As shown in (b), the system architecture 20 includes a target detection device 200 and a perception module 210. The perception module 210 is deployed outside of and independent of the target detection device 200. The perception module 210 includes a visual perception module 211 and a three-dimensional point perception module 212.

[0062] in, Figure 2The sensing module 210 shown can be a mobile device. For example, the sensing module 210 can be a vehicle-mounted platform, such as one mounted on a car, motorcycle, or bicycle. Alternatively, the sensing module 210 can be a ship-mounted platform, such as one mounted on a boat, ship, or motorboat. Alternatively, the sensing module 210 can be an airborne platform, such as one mounted on a drone, helicopter, jet aircraft, or balloon. Alternatively, the sensing module 210 can be a spaceborne platform, such as one mounted on a satellite. Alternatively, the sensing module 210 can be an intelligent agent device, such as one mounted on robots in various application scenarios, such as home robots or industrial robots.

[0063] Please refer to Figure 3 , Figure 3 A schematic diagram of the hardware structure of a target detection device provided in an embodiment of this application is shown. Figure 3 As shown, the target detection device 200 may include components such as a processor 301, a detection device 302, a memory 303, a display 304, a communication device 305, a positioning device 306, a peripheral interface 307, and a power supply 308. These components can communicate via one or more communication buses or signal lines, such as a controller area network (CAN) bus. Figure 3 (Not shown in the image). Those skilled in the art will understand that... Figure 3 The hardware structure shown does not constitute a limitation on the target detection device. The target detection device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0064] In some embodiments, for example for Figure 2 In the system architecture shown in (a), the target detection device 200 and the detection device 302 may include, but are not limited to, the aforementioned visual perception module 211 and 3D point perception module 212. The detection device 302 is used to measure the target scene and obtain measurement data of the target scene. The measurement data of the target scene includes the target scene image and information such as the position, azimuth, or velocity of multiple points of at least one target in the target scene relative to the target detection device 200 (or more specifically, relative to the 3D point perception module 212).

[0065] For example, the detection device 302 may include a radar. The target detection device 200 can emit radar waves (or radar detection signals) and receive the reflected waves of the radar signals after they have passed through the target to detect information such as the target's position or structure. For example, the radar may be a lidar or millimeter-wave radar.

[0066] The communication device 305 may include radio frequency (RF) circuitry, Bluetooth, or Wi-Fi. The RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, and a duplexer. In some embodiments, for example, for... Figure 2 In the system architecture shown in (b), the target detection device 200 and the communication device 305 can be used to receive measurement data of the target scene from the perception module 210.

[0067] The memory 303 is used to store computer programs or data, etc.

[0068] The processor 301 can perform various functions and data processing by running computer programs stored in the memory 303. For example, it can obtain information such as the position, azimuth, or radial velocity of at least one target relative to the target detection device 200 (or more specifically, relative to the sensor platform 110) based on the measurement data of the target scene acquired by the sensing module 210.

[0069] The sensor platform acquires measurement data of surrounding targets to obtain image detection information of the target scene image, and obtains information such as the position, azimuth, or velocity of at least one target relative to the target detection device or module 100 (or more specifically, relative to the 3D point perception module 212), thereby obtaining a 3D point cloud of the target scene. Furthermore, based on the image detection information of the target scene image and the 3D point cloud of the target scene, targets in the target scene are accurately identified.

[0070] It should be understood that the above Figure 3 The hardware modules included in the target detection device 200 shown are merely illustrative and do not constitute a limitation of this application.

[0071] The target detection method provided in this application can be applied to objects with... Figure 3 This invention relates to target detection devices with a structure or similar structure. Alternatively, it can be applied to target detection devices with other structures; this application is not limited to these specific applications.

[0072] The following provides a detailed description of a target detection method provided in the embodiments of this application.

[0073] It should be noted that the target scene image in this embodiment is acquired by a visual perception module. For example, the target scene image is obtained by... Figure 2 (a) or Figure 2The visual perception module 211 shown in (b) acquires the data. For example, the visual perception module can be a camera, video camera, or imaging radar (such as synthetic aperture radar). For instance, multiple cameras can be mounted on the top of an autonomous vehicle, periodically acquiring images of the target scene from multiple directions at a frequency of 30 FPS. Multiple cameras capture images of the road surface from multiple directions. The 3D point cloud of the target scene is acquired by the 3D point perception module. For example, the 3D point cloud of the target scene is... Figure 2 (a) or Figure 2 The information is acquired by the three-dimensional point perception module 212 (e.g., a laser sensor, millimeter-wave radar, sonar, or ultrasonic sensor) shown in (b). For example, a laser sensor may be mounted on the top of an autonomous vehicle, which acquires information returned from the target scene at a frequency of 10 Hz.

[0074] Furthermore, in this embodiment of the application, the target scene image and the 3D point cloud of the target scene are acquired synchronously. Figure 2 Taking the system architecture shown in (a) as an example, the target detection device 200 can call the visual perception module 211 at the first moment to acquire the target scene image. For example, the visual perception module 211 includes a camera, and the target detection device 200 can call the camera application at the first moment to call the camera of the visual perception module 211 and acquire the target scene image. At the same time, the target detection device 200 can call the 3D point perception module 212 at the first moment to acquire the 3D point cloud of the target scene.

[0075] The visual perception module 211 and the 3D point perception module 212 can synchronously acquire the target scene image and the 3D point cloud of the target scene under the control of a synchronization switch. Alternatively, the visual perception module 211 and the 3D point perception module 212 can synchronously acquire the target scene image and the 3D point cloud of the target scene based on the same timestamp. The timestamp is typically a character sequence. A timestamp can uniquely identify a specific moment. For example, the timestamp can be generated by a digital timestamp service (DTS) module. For instance, the DTS module can be located in the processor 301. Alternatively, the visual perception module 211 and the 3D point perception module 212 can synchronously acquire the target scene image and the 3D point cloud of the target scene based on the same random number. The random number can uniquely identify a specific moment. This application embodiment does not limit the synchronization method.

[0076] Please refer to Figure 4 , Figure 4This is a flowchart illustrating a target detection method provided in an embodiment of this application. The execution entity of this method can be a perception system, a fused perception system, or a planning / control system integrating the above systems, such as an assisted driving or autonomous driving system. Alternatively, the execution entity of this method can also be software or hardware (such as a target detection device connected or integrated with the corresponding perception module via wireless or wired connection). The following different execution steps can be implemented centrally or in a distributed manner.

[0077] like Figure 4 As shown, an embodiment of this application provides a target detection method including the following steps S401-S403:

[0078] S401. Obtain image detection information of the target scene image.

[0079] The image detection information includes, but is not limited to: the positions and sizes of multiple target detection boxes, the target categories corresponding to the multiple target detection boxes, and identification information. Each target detection box includes a target image identified by the target detection device 200. The position of each target detection box indicates the location of the target image within it. The size of each target detection box indicates the size of the target image within it. The target category corresponding to each target detection box indicates the target category corresponding to the target image within it, such as vehicles, pedestrians, lane lines, zebra crossings, traffic lights, streetlights, guardrails, bollards, bushes, lampposts, vehicle access barriers / stops, overpasses, culverts, or buildings, etc. The identification information of each target detection box is used to uniquely identify the target image within it. For example, this identification information can be an ID (identification).

[0080] For example, image detection information can be represented as {R1, ..., R...} s , ..., R k} represents. Where R s ={x s y s w s h s cls s}, where K is the number of target detection boxes obtained by the target detection device 200, and s∈(1,K). s and y s These are the coordinates of the bottom left, top left, bottom right, or top right corner of the s-th object detection box; w s h is the width of the s-th object detection box; s cls is the height of the s-th object detection bounding box. s It is the target category corresponding to the target image in the s-th target detection box.

[0081] Where, x s and ys Whether the coordinates of the s-th object detection box are the bottom left, top left, bottom right, or top right corner depends on the image coordinate system settings of the target scene image. For example, if the origin of the image coordinate system is at the bottom left corner of the target scene image, the x-axis is to the right, and the y-axis is perpendicular to the x-axis and upwards, then x... s and y s These are the coordinates of the bottom left corner of the s-th object detection box. If the origin of the image coordinate system is at the top left corner of the target scene image, the x-axis is to the right, and the y-axis is perpendicular to the x-axis and pointing downwards, then x... s and y s These are the coordinates of the top-left corner of the s-th object detection box. If the origin of the image coordinate system is at the bottom-right corner of the target scene image, the x-axis is to the left, and the y-axis is perpendicular to the x-axis and upwards, then x... s and y s These are the coordinates of the bottom right corner of the s-th object detection box. If the origin of the image coordinate system is at the top right corner of the target scene image, the x-axis is to the left, and the y-axis is perpendicular to the x-axis and pointing downwards, then x... s and y s These are the coordinates of the upper right corner of the s-th object detection box. This application does not limit the setting of the image coordinate system of the target scene image.

[0082] In some embodiments, the target detection device 200 can obtain the aforementioned image detection information by recognizing preset features in the target scene image. These preset features are image features of targets that may affect the passage of the visual perception module carrier (such as a vehicle). It is understood that for autonomous vehicles, targets within their line of sight, or further, targets within their line of sight that are relatively close and may affect their passage, are often the targets of greater concern to the vehicle. For example, targets such as vehicles, pedestrians, guardrails, bollards, lampposts, vehicle access barriers / stop barriers, overpasses / culverts, or nearby buildings receive a higher degree of attention. Targets such as roads do not affect vehicle passage and therefore receive less attention.

[0083] For example, Figure 5 An example of a target detection bounding box obtained by image detection of a target scene image is shown. For example... Figure 5 As shown, the target detection device 200 is a vehicle-mounted device. This target detection device 200 obtains multiple target detection boxes by performing feature recognition and analysis on the acquired target scene image. For example... Figure 5 The bounding boxes shown have IDs of 1, 2, 3, ..., 16. Figure 5The target category corresponding to the object detection box with ID 1 is electric vehicle; the target categories corresponding to IDs 2 and 5 are buses; the target categories corresponding to IDs 3, 4, 6, ..., 13 are cars; the target categories corresponding to IDs 14 and 15 are bollards; the target category corresponding to ID 16 is a lamppost; the target category corresponding to ID 17 is a red light; and the target category corresponding to ID 18 is a zebra crossing. For Figure 5 For the vehicle shown, the targets corresponding to the target detection boxes with IDs 1, 2, 3, ..., 18 are highly regarded by the vehicle.

[0084] In some embodiments, the target detection device 200 may employ a neural network algorithm to perform image detection on the target scene image, including feature recognition and feature analysis, to obtain the aforementioned image detection information. For example, the processor 301 may include a network processing unit (NPU) chip, which may integrate a convolutional neural network algorithm. The target detection device 200 can perform image detection on the target scene image through the NPU chip to obtain the aforementioned image detection information. For methods and processes of image detection using neural network algorithms, please refer to the descriptions in conventional techniques; they will not be elaborated upon here.

[0085] In some embodiments, lightweight neural networks (e.g., MobileNets or MobileNets V2) can be used to improve the speed of image detection by the NPU chip. Alternatively, an accelerator, such as the Cambricon DianNao accelerator, can be integrated into the NPU chip.

[0086] In other embodiments, the target detection device 200 may also perform image detection on the target scene image based on conventional algorithms. For example, algorithms based on traditional SIFT features or Haar features. This application does not limit the specific method of image detection.

[0087] S402. Obtain clustering information of the 3D point cloud of the target scene.

[0088] The clustering information of the 3D point cloud includes at least one first clustering target and the 3D points corresponding to the at least one first clustering target.

[0089] In some embodiments, a clustering algorithm can be used to cluster the 3D point cloud of the target scene to obtain at least one first clustering target and the 3D points corresponding to each first clustering target.

[0090] In this application, the purpose of clustering is to determine the 3D points corresponding to different categories of targets from the 3D point cloud of the target scene. For example, from... Figure 1 In the 3D point cloud of the target scene shown in (a), the 3D points corresponding to the bushes, vehicles, and people are determined respectively. Figure 1 In the 3D point cloud of the target scene shown in (b), the 3D points corresponding to the truck and the cargo are determined.

[0091] For example, the clustering algorithm mentioned above could be a depth-first search (DFS) algorithm. Alternatively, it could be a density-based clustering algorithm, such as density-based spatial clustering of applications with noise (DBSCAN) or its extensions, such as hierarchical DBSCAN (HDBSCAN). Another example is the ordering points to identify the clustering structure (OPTICS) algorithm. It can be understood that clustering algorithms such as DFS, DBSCAN, HDBSCAN, or OPTICS can separate different targets or the 3D points corresponding to different targets from the 3D point cloud of a target scene.

[0092] The Depth-First Search (DFS) algorithm is used to traverse or search a tree or graph. This method searches the tree's branches as deeply as possible by traversing its nodes along its depth. When all edges containing a given node have been explored, the search backtracks to the starting node of the edge that discovered that node. This process continues until all nodes reachable from the source node have been found. If any undiscovered nodes remain, one is selected as the source node, and the above process is repeated until all nodes have been visited. For detailed explanations of clustering algorithms such as DFS, DBSCAN, HDBSCAN, or OPTICS, please refer to the explanations and descriptions in general technical documentation; they will not be elaborated upon here.

[0093] In one possible implementation, step S402 above may include steps 1, 2 and 3:

[0094] Step 1: Project the 3D point cloud of the target scene onto a 2D raster map to obtain multiple 2D points, and the correspondence between the 3D point cloud and these multiple 2D points.

[0095] The aforementioned two-dimensional raster map can be an occupation grid map (OGM), for example, Figure 6 The OGM is shown in (b) above. Alternatively, it can be a 2D raster image parallel to the ground or other 2D raster images. The following example illustrates one implementation method of step 1 above, using the projection of the 3D point cloud of the target scene onto the OGM as an example:

[0096] Assume the 3D point cloud of the target scene is represented by {p1, ..., p...} i , ..., p N} represents. Where p i ={x wi y wi , z wi}, where N is the number of 3D points in the 3D point cloud of the target scene, p i x is the i-th 3D point in the 3D point cloud of the target scene, i∈(1,N), x wi y wi and z wi These are the x-axis, y-axis, and z-axis coordinates of the i-th 3D point in a preset coordinate system (e.g., the carrier coordinate system of the 3D point perception module). After obtaining the 3D point cloud of the target scene, the 3D point cloud of the target scene is projected top-down into the OGM. That is, the height information of each 3D point (e.g., the information in the z-axis direction in the preset coordinate system) is ignored, and the information of the other two dimensions (e.g., the information in the x-axis and y-axis directions in the preset coordinate system) is scaled into the OGM according to a certain ratio. This results in N 2D points, which are represented by {p1′, ..., p...} i ′,…,p N '} represents the 3D point p. i ={x wi y wi , z wi} Corresponding to the i-th two-dimensional point p i ′={x i y i}, x i and y i These are the x-axis and y-axis coordinates of the i-th two-dimensional point in the OGM, respectively.

[0097] like Figure 6 As shown, Figure 6 Image (a) shows a top-down view of a 3D point cloud of a target scene. Figure 6 The white dots shown in (a) represent the laser points of the target scene captured by the laser sensor. These laser points can be mapped to a set resolution (e.g., 300x300). Figure 6 In the OGM shown in (b), specifically, using the grid frame in the OGM corresponding to the origin of the 3D point perception module's carrier coordinate system as a reference, the laser points of the target scene are mapped to... Figure 6 In the OGM shown in (b) of the diagram.

[0098] Step 2: Cluster the obtained two-dimensional points into at least one first clustering target.

[0099] For example, clustering algorithms such as DFS, DBSCAN, HDBSCAN, or OPTICs can separate different targets from multiple two-dimensional points, thus obtaining at least one first cluster target. The clustering results can be represented as {V1, ..., V...} j , ..., V m} represents the number of first clustering targets obtained. j This is the j-th first clustering target obtained, j∈(1,m). Each first clustering target corresponds to one or more two-dimensional points. For example, V j ={p j ′},p j ′ is the two-dimensional point corresponding to the j-th first clustering target.

[0100] Step 3: Based on the correspondence between the obtained 3D point cloud and multiple 2D points, determine the 3D point corresponding to at least one of the first clustering targets.

[0101] As mentioned above, there is a one-to-one correspondence between 3D point clouds and 2D points, and the correspondence between the two is known. Therefore, based on the correspondence between 3D point clouds and multiple 2D points, we can determine the 3D points corresponding to each first cluster target in at least one first cluster target obtained in step 2 above.

[0102] like Figure 7 The image shows an example of clustering results based on the DFS algorithm. More specifically... Figure 7 It shows the Figure 6 Example of clustering results for the 3D point cloud corresponding to the top view shown in (a) is shown in the figure. Figure 7 The target boxes a, b, ..., y in the image correspond to the y first cluster targets in the target scene, and the 3D points in each target box are the 3D points corresponding to the first cluster target.

[0103] S403. Based on the overlap between the projection points of the three-dimensional points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the multiple target detection boxes, and combined with the target preset size corresponding to each target detection box, update the clustering information of the three-dimensional point cloud.

[0104] Specifically, step S403 above may include steps A, B and C:

[0105] Step A: Project the 3D point cloud of the target scene onto the image coordinate system of the target scene.

[0106] Specifically, a transformation matrix can be used to project the 3D point cloud of the target scene onto the image coordinate system of the target scene from the carrier coordinate system (such as the vehicle body coordinate system) of the 3D point perception module.

[0107] For example, suppose the transformation matrix from the vehicle coordinate system to the image coordinate system is R. 3×4 Then, the coordinates of the 3D point cloud of the target scene in the image coordinate system can be obtained according to the following formulas 1 and 2:

[0108]

[0109]

[0110] Where, x wi y wi and z wi These are three-dimensional points p. i The x-axis, y-axis, and z-axis coordinates in the vehicle coordinate system. ci and y ci These are three-dimensional points p. i The x- and y-coordinates of the projection point in the image coordinate system. That is, the coordinates of the i-th 3D point p. i ={x wi y wi , z wi The projection point in the image coordinate system is a two-dimensional point p. i 2D ={x ci y ci For the other N-1 3D points, the same method is used to map them to the other N-1 projection points in the image coordinate system. Please refer to [reference needed]. Figure 8 , Figure 8 An example diagram is shown where a 3D point cloud of a target scene is projected onto the image coordinate system of the target scene. For example... Figure 8 As shown, the 3D points corresponding to targets such as roads, vans, gantry frames, bushes, trees, and lampposts collected by the 3D point sensing device can be accurately projected onto the targets such as roads, vans, gantry frames, bushes, trees, and lampposts in the target scene image collected by the visual perception module.

[0111] For example, Table 1 below shows a set of experimental data on the error of projecting a 3D point cloud of a target scene onto the image coordinate system of the target scene:

[0112] Table 1

[0113]

[0114] Where ID is the identifier of the target detection box in the target scene image, Delta_U is the horizontal pixel error, and Delta_V is the vertical pixel error. As shown in Table 1, the average pixel error of the 3D point cloud of the target scene projected onto the image coordinate system of the target scene is about 1 pixel, which is relatively small.

[0115] Step B: Determine the correspondence between the three-dimensional points in the multiple target detection boxes and the three-dimensional points based on the overlap between the projection points of the three-dimensional points corresponding to at least one first cluster target in the image coordinate system and the corresponding target detection boxes in the multiple target detection boxes.

[0116] It can be understood that the overlap between the projection point of the 3D point corresponding to each first cluster target in the image coordinate system and each target detection box in the multiple target detection boxes can be reflected by the positional relationship between the projection point of the 3D point corresponding to each first cluster target in the image coordinate system and each target detection box in the multiple target detection boxes. In step A above, after the 3D point cloud of the target scene is projected onto the image coordinate system of the target scene, the positional relationship between the projection point of at least one 3D point corresponding to the first cluster target in the image coordinate system and each target detection box in the target detection box can be known.

[0117] One implementation method is to determine the correspondence between each target detection box and a 3D point in multiple target detection boxes by judging whether the first projection point is in the first target detection box. Specifically, if the first projection point is in the first target detection box, then the 3D point corresponding to the first projection point is determined to correspond to the first target detection box. Here, the first projection point is the projection point in the image coordinate system of any one of the 3D points corresponding to at least one first clustering target. The first target detection box is any one of the multiple target detection boxes.

[0118] For example, such as Figure 9 As shown, if the three-dimensional point p i If the projection point of point p falls within the first target detection box, then the 3D point p i It matches the first target detection box; the 3D point p can be considered to be... i This is the 3D point corresponding to the target within the first target detection box. If the 3D point p i If the projection point of a 3D point does not fall within any target detection box, then it does not match any target detection box; it can be considered that the 3D point p i It is less likely to attract attention, for example, the 3D point is the 3D point corresponding to the road.

[0119] For example, for each 3D point of each first cluster target, the matching of the 3D point with the image detection result is determined by whether its coordinates projected onto the image coordinate system are within the target detection box. The specific judgment criteria are as follows:

[0120]

[0121] Where, p i 2D ={x ci y ci} is the i-th 3D point p in the 3D point cloud of the target scene. i (p i ={x wi y wi , z wi The projection point of}) in the image coordinate system. i∈(1,N). R s ={x s y s w s h s cls s} represents the image detection information corresponding to the s-th target detection box, where s∈(1,K).

[0122] In some embodiments, a 3D point may match multiple target detection boxes. For example, a first projected point may be in both a first target detection box and a second target detection box. The second target detection box is one of multiple target detection boxes, and it overlaps with the first target detection box.

[0123] like Figure 10 As shown, the projection point falling in region A geometrically matches both target detection boxes B and C. Similarly, the projection point falling in region D geometrically matches both target detection boxes C and E. In this case, it is necessary to further determine which target detection box the 3D point corresponding to the projection points falling in regions A and D matches.

[0124] In this case, the target detection method provided in this application embodiment further includes: calculating d1 and d2. Wherein, d1 is the nearest distance between the 3D point corresponding to the first projection point and the 3D point corresponding to the region of the first target detection box other than the above-mentioned overlapping region; d2 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point to the 3D point corresponding to the first projection point in the region of the second target detection box other than the above-mentioned overlapping region.

[0125] If d2 is less than d1, then the 3D point corresponding to the first projection point is determined to match the second target detection box; if d2 is greater than or equal to d1, then the 3D point corresponding to the first projection point is determined to match the first target detection box. Alternatively, if d2 is greater than d1, then the 3D point corresponding to the first projection point is determined to match the first target detection box; if d2 is less than or equal to d1, then the 3D point corresponding to the first projection point is determined to match the second target detection box. This application is not limited.

[0126] like Figure 9 As shown, if the three-dimensional point p i If the projection point falls within the first target detection box and the second target detection box, then the 3D point p is calculated. i In the region of the first target detection box, excluding the overlapping area mentioned above, and the 3D point p i The distance d1 between the nearest 3D points; and the 3D point p i In the region of the second target detection box other than the overlapping area mentioned above, and the 3D point p i The distance d2 between the nearest 3D points is used, and then the 3D point p is determined based on the relationship between the magnitudes of d1 and d2. i Which object detection box is it matched with? For example... Figure 9 As shown, assuming d2 < d1, then the three-dimensional point p is determined. i It matches the second target detection box.

[0127] In one possible implementation, d1 could be a three-dimensional point p. i In the region of the first target detection box, excluding the overlapping area mentioned above, and the 3D point p i The Eulerian distance between the nearest 3D points, d2 can be the Eulerian distance between 3D points p. i In the region of the second target detection box other than the overlapping area mentioned above, and the 3D point p i The Eulerian distance between the nearest three-dimensional points.

[0128] For example, the three-dimensional point p can be obtained according to the following formula. i In the region of the first target detection box, excluding the overlapping area mentioned above, and the 3D point p i The Eulerian distance between the nearest 3D points; and the Eulerian distance between 3D points p. i In the region of the second target detection box other than the overlapping area mentioned above, and the 3D point p i The Eulerian distance between the nearest 3D points:

[0129] d1 = min 0＜s＜u (p k1 -p s ) 2 ;

[0130] d2=min 0＜s＜u (p k2 -p s ) 2 .

[0131] Where u is the number of 3D points in the overlapping area of ​​the first and second target detection boxes where the projection point is located. s It represents the coordinates of any 3D point within the overlapping area of ​​the first and second target detection boxes. k1It is the coordinate of any three-dimensional point within the first target detection bounding box, excluding the aforementioned overlapping area. k2 The coordinates of any three-dimensional point within the second target detection bounding box, excluding the aforementioned overlapping area.

[0132] By using step B above, we can avoid the situation where the laser light reflected from multiple targets due to mutual occlusion is difficult for the laser sensor to distinguish, causing the laser sensor to mistakenly treat multiple targets as a single target, resulting in undersegmentation. Figure 11 As shown in (a) in the figure.

[0133] Step C: Based on the correspondence between each target detection box and a 3D point in the multiple target detection boxes, and combined with the preset target size corresponding to each target detection box, update the clustering information of the 3D point cloud.

[0134] The updated 3D point cloud clustering information includes at least one second clustering target and the 3D points corresponding to that second clustering target. The number of second clustering targets is less than or equal to the number of first clustering targets.

[0135] As one implementation, step C above may include performing the following steps for each target detection box:

[0136] Step a: Determine the number of first cluster targets that match each target detection box. If a first detection box matches multiple first cluster targets from at least one of the above first cluster targets, and the maximum distance between any two of these multiple first cluster targets is less than a preset distance, then proceed to step b.

[0137] Understandably, over-segmentation is typically addressed by clustering clusters that are close to each other and have a dependency relationship. For example, Figure 1 In the scenario shown in (b), the goods (such as...) Figure 1 In (b) shown, the motion of targets 4-9 depends on target 3 (i.e., the truck). Specifically, targets 4-9 depend on the power of target 3, and maintain synchronized motion orientation and speed with target 3. However, for ADAS or AD systems, targets 3-9 can be treated as a whole, such as... Figure 11 As shown, the truck and the cargo on it are treated as a single target (e.g. Figure 11 For target 5 shown in (b) in the figure, obtain information such as the position, orientation, speed of movement and shape of the overall target.

[0138] Therefore, step C above is mainly used to address the oversegmentation phenomenon in the object detection process. First, it is necessary to identify the scene targets where this phenomenon may occur. That is, multiple first-cluster targets (e.g., objects matching the same object detection box (e.g., object detection box K)) Figure 1(Targets 3-9 are shown in (b)).

[0139] In some embodiments, each first cluster target can be used as the center, with a preset distance as the radius r, to find the first cluster target that is closest to it. For example... Figure 12 As shown, with cluster target 1201 as the center and a preset distance as the radius r, the first cluster target closest to it is found. It can be seen that cluster target 1202 is likely to be dependent on cluster target 1201, while cluster targets 1203, 1204, 1205, and 1206 are unlikely to be dependent on cluster target 1201. Here, r can be set based on empirical values. For example, a batch of training data can be selected, and the average length, width, and height of each target category can be calculated, with the average length used as the radius r.

[0140] For example, the size of the target can be calculated based on the L-shape algorithm, for example, Figure 13 A schematic diagram illustrating the calculation of target dimensions based on the L-shape algorithm is shown. Figure 13 As shown, for each target, the point where the laser hits the car forms an L-shape. The L-shape algorithm uses a rectangle with orientation and size (e.g., ...) Figure 13 The rectangles r1, r2, r3, and r4 shown are matched with 3D points. The matching criterion is that the sum of the distances from each 3D point to the nearest edge of the rectangle is minimized. The orientation of the rectangle matched according to this criterion is considered to be the orientation of the target corresponding to the 3D point, and the size of the rectangle is considered to be the size of the target corresponding to the 3D point. Alternatively, other size estimation algorithms can be used, and specific details can be found in conventional techniques, which will not be elaborated upon in this application.

[0141] It should be noted that, Figure 12 As an example only, the search area is not limited to a circular area; it can also be a rectangle or a square, etc., without application restrictions.

[0142] Step b: Based on the preset size of the target corresponding to the first detection box, determine whether to merge multiple first cluster targets corresponding to the first detection box.

[0143] Specifically, if the difference between the overall size of the aforementioned multiple first cluster targets and the preset size of the first cluster target of the preset category among the aforementioned multiple first cluster targets is less than a preset threshold, then the aforementioned multiple first cluster targets are merged into a second cluster target.

[0144] Assume the pre-set truck dimensions are 8 meters in length and 3 meters in height. Figure 11 In the scenario shown in (b), then according to w cls =8,h cls =3 Determine whether to merge the truck and its cargo. For example, if 0 < w Kmax<w cls And 0 < h Kmax <h cls Determine the truck and the cargo on it to be combined. Among them, w Kmax yes Figure 11 The overall length h of the truck and its cargo in the target detection box K shown in (b) is... Kmax yes Figure 11 The overall height of the truck and the cargo on the truck in the target detection box K shown in (b) is shown.

[0145] In some embodiments, the image detection information obtained in step S401 above further includes: the confidence scores of the target categories corresponding to multiple target detection boxes. For example, the image detection information can be represented as {R1, ..., R...} s , ..., R k} represents. Where R s ={x s y s w s h s cls s score s}, score s It is the confidence score of the target category corresponding to the target image in the s-th target detection box.

[0146] score s This is used to represent the reliability of the category corresponding to the target in the s-th object detection box. For example, score s ∈[0, 1]. Where, score s The higher the value, the more likely the target in the s-th bounding box is cls. s The greater the likelihood of the corresponding category, the lower the likelihood of false detection (also known as "false detection") by the target detection device 200. s The lower the value, the more likely the target in the s-th object detection box is cls. s The lower the probability of a corresponding category, the greater the likelihood of a false detection by the target detection device 200. For example, if the confidence level of a target matching a category is below 0.3, it can be considered a false detection.

[0147] In some embodiments, the target detection method provided in this application may further include: marking the confidence level of the target category corresponding to each three-dimensional point.

[0148] One implementation approach is to use the confidence score of the target category corresponding to the third target detection box to label the first 3D point. Here, the first 3D point is the 3D point of the projection point within the third target detection box, and the third target detection box is any one of the aforementioned target detection boxes. For example, the confidence score of the target category corresponding to the target image in the s-th target detection box is 'score'.s Then the confidence score of the 3D point corresponding to the projection point of the s-th target detection box is the score. s .

[0149] As one implementation, the second 3D point is a 3D point whose projection point is not in any of the multiple target detection boxes mentioned above. In this case, a pre-set confidence level is used to label the second 3D point. For example, if the projection point corresponding to the second 3D point is not in any of the target detection boxes corresponding to the image detection information, then a confidence value of 0.01 is used to label the confidence level of the 3D point corresponding to the second 3D point.

[0150] By marking the confidence level of 3D points, the reliability of target detection results can be assessed for future reference. For example, driver assistance systems (ADAS) or autonomous driving systems may take measures such as braking, deceleration, or lane changing based on the confidence level of the target category. Alternatively, ADAS or autonomous driving systems may determine whether to track the target based on the confidence level of the target category, for example, to further determine the target category. Or, they may decide whether to integrate data collected by other sensing devices to further determine the target category, etc. (No application or limitation).

[0151] It should be understood that the various solutions in the embodiments of this application can be used in a reasonable combination, and the explanations or descriptions of the various terms appearing in the embodiments can be referenced or explained to each other in the various embodiments, without limitation.

[0152] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0153] It is understood that, in order to achieve the functions of any of the above embodiments, the target detection device 200 includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0154] This application embodiment can divide the target detection device 200 into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0155] For example, when dividing functional modules in an integrated manner, such as Figure 14 The diagram shown is a structural block diagram of a target detection device provided in an embodiment of this application. The target detection device 200 may include a processing unit 1410.

[0156] The processing unit 200 is configured to support the target detection device 200 in executing the aforementioned steps S401, S402, and S403, acquiring image detection information of the target scene image and clustering information of the 3D point cloud of the target scene; and updating the clustering information of the 3D point cloud based on the overlap between the projection points of at least one first clustering target's 3D points in the image coordinate system and the aforementioned plurality of target detection boxes, combined with the target preset size corresponding to each target detection box. And / or, other processes used in the techniques described herein.

[0157] In one possible structure, such as Figure 15 As shown, the target detection device 200 may also include a data acquisition unit 1420 for simultaneously acquiring target scene images and three-dimensional point clouds of the target scene.

[0158] In an alternative approach, when data transmission is implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disk (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0159] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, portable hard disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can be located in an ASIC. Alternatively, the ASIC can be located in a probing device. Of course, the processor and storage medium can also exist as discrete components in the probing device.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0161] In one alternative embodiment, this application provides a chip system including a processor and a memory, wherein the memory stores computer program code; when the computer program code is executed by the processor, it implements the method in any possible implementation of this application. The chip system may be composed of chips or may include chips and other discrete devices.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0163] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0166] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A target detection method, characterized in that, The method includes: The method acquires image detection information of a target scene image and clustering information of a 3D point cloud of the target scene; the image detection information includes: the position and size of multiple target detection boxes, the target category and identification information corresponding to the multiple target detection boxes; the clustering information of the 3D point cloud includes at least one first cluster target and the 3D points corresponding to the at least one first cluster target. Based on the overlap between the projection points of the 3D points corresponding to the at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes, the correspondence between each target detection box and the 3D point is determined. Based on the correspondence and the target preset size corresponding to each target detection box, the clustering information of the 3D point cloud is updated; wherein the image coordinate system is the coordinate system of the target scene image. The updated clustering information of the 3D point cloud includes at least one second clustering target and the 3D points corresponding to the at least one second clustering target; the number of the second clustering targets is less than or equal to the number of the first clustering targets.

2. The method according to claim 1, characterized in that, The step of obtaining the clustering information of the 3D point cloud of the target scene includes: The three-dimensional point cloud is projected onto a two-dimensional raster image to obtain multiple two-dimensional points, and the correspondence between the three-dimensional point cloud and the multiple two-dimensional points. Cluster the plurality of two-dimensional points into at least one first clustering target; Based on the correspondence between the three-dimensional point cloud and the plurality of two-dimensional points, the three-dimensional point corresponding to the at least one first clustering target is determined.

3. The method according to claim 1 or 2, characterized in that, The step of determining the correspondence between each target detection box and the three-dimensional point based on the overlap between the projection point of the at least one first clustering target corresponding to the three-dimensional point in the image coordinate system and the corresponding target detection box of each of the plurality of target detection boxes includes: Project the 3D point cloud onto the image coordinate system; The correspondence between each target detection box and the three-dimensional point is determined based on the overlap between the projection point of the three-dimensional point corresponding to the at least one first clustering target in the image coordinate system and the target detection box corresponding to each of the plurality of target detection boxes.

4. The method according to claim 3, characterized in that, The step of determining the correspondence between each target detection box and the three-dimensional point based on the overlap between the projection point of the at least one first clustering target corresponding to the three-dimensional point in the image coordinate system and the corresponding target detection box of each of the plurality of target detection boxes includes: Determine whether the first projection point is within the first target detection box; If the first projection point is in the first target detection box, then the three-dimensional point corresponding to the first projection point is determined to match the first target detection box; the first projection point is the projection point of any three-dimensional point among the three-dimensional points corresponding to the at least one first clustered target in the image coordinate system, and the first target detection box is any one of the plurality of target detection boxes.

5. The method according to claim 4, characterized in that, The method further includes: If the first projection point is within the first target detection box and the second target detection box, then d1 and d2 are calculated; wherein, the second target detection box is one of the plurality of target detection boxes, and the second target detection box overlaps with the first target detection box; d1 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the first target detection box other than the overlapping region; d2 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the second target detection box other than the overlapping region. If d2 is less than d1, then the three-dimensional point corresponding to the first projection point is determined to match the second target detection box; If d2 is greater than or equal to d1, then the three-dimensional point corresponding to the first projection point is determined to match the first target detection box.

6. The method according to any one of claims 1-2 and 4-5, characterized in that, The step of updating the clustering information of the 3D point cloud based on the correspondence and the preset size of the target corresponding to each target detection box includes: Determine the number of first clustered targets that match each target detection box; If the first detection box matches multiple first cluster targets among the at least one first cluster target, and the maximum distance between any two of the multiple first cluster targets is less than a preset distance, then based on the preset target size corresponding to the first detection box, it is determined whether to merge the multiple first cluster targets; specifically including: If the difference between the overall size of the plurality of first cluster targets and the preset size of the first cluster target of the preset category among the plurality of first cluster targets is less than a preset threshold, then the plurality of first cluster targets are merged into a second cluster target.

7. The method according to any one of claims 1-2 and 4-5, characterized in that, The image detection information further includes: the confidence score of the target category corresponding to each target detection box in the plurality of target detection boxes, wherein the confidence score is used to characterize the credibility of the target category; the method further includes: The first 3D point is labeled with the confidence score of the target category corresponding to the third target detection box; the first 3D point is the 3D point of the projection point in the third target detection box; the third target detection box is any one of the plurality of target detection boxes; A second three-dimensional point is marked with a pre-set confidence level; the second three-dimensional point is a three-dimensional point whose projection point is not in any of the multiple target detection boxes.

8. The method according to any one of claims 1-2 and 4-5, characterized in that, The image detection information obtained from the target scene image includes: The target scene image is subjected to feature recognition and feature analysis to obtain the image detection information.

9. The method according to any one of claims 1-2 and 4-5, characterized in that, The target scene image and the 3D point cloud of the target scene are acquired synchronously.

10. A target detection device, characterized in that, The target detection device includes: The processing unit is configured to acquire image detection information of a target scene image and clustering information of a 3D point cloud of the target scene; the image detection information includes: the position and size of multiple target detection boxes, the target category and identification information corresponding to the multiple target detection boxes; the clustering information of the 3D point cloud includes at least one first clustering target and the 3D points corresponding to the at least one first clustering target; and, Based on the overlap between the projection points of the 3D points corresponding to at least one first clustering target in the image coordinate system and the corresponding target detection boxes in the plurality of target detection boxes, the correspondence between each target detection box and the 3D point is determined. Based on the correspondence and the target preset size corresponding to each target detection box, the clustering information of the 3D point cloud is updated; wherein the image coordinate system is the coordinate system of the target scene image. The updated clustering information of the 3D point cloud includes at least one second clustering target and the 3D points corresponding to the at least one second clustering target; the number of the second clustering targets is less than or equal to the number of the first clustering targets.

11. The apparatus according to claim 10, characterized in that, The processing unit acquires clustering information of the 3D point cloud of the target scene, including: The processing unit projects the three-dimensional point cloud onto a two-dimensional raster image to obtain multiple two-dimensional points and the correspondence between the three-dimensional point cloud and the multiple two-dimensional points; clusters the multiple two-dimensional points into at least one first clustering target; and determines the three-dimensional point corresponding to the at least one first clustering target based on the correspondence between the three-dimensional point cloud and the multiple two-dimensional points.

12. The apparatus according to claim 10 or 11, characterized in that, The processing unit determines the correspondence between each target detection box and the three-dimensional point based on the overlap between the projection point of the at least one first clustering target corresponding to the three-dimensional point in the image coordinate system and the corresponding target detection box in each of the plurality of target detection boxes, including: The processing unit projects the three-dimensional point cloud onto the image coordinate system; and determines the correspondence between each target detection box and the three-dimensional point based on the overlap between the projection point of the three-dimensional point corresponding to the at least one first clustering target in the image coordinate system and the target detection box corresponding to each of the plurality of target detection boxes.

13. The apparatus according to claim 12, characterized in that, The processing unit determines the correspondence between each target detection box and the three-dimensional point based on the overlap between the projection point of the at least one first clustering target corresponding to the three-dimensional point in the image coordinate system and the corresponding target detection box in each of the plurality of target detection boxes, including: The processing unit determines whether the first projection point is in the first target detection frame; If the first projection point is in the first target detection box, then the three-dimensional point corresponding to the first projection point is determined to match the first target detection box; the first projection point is the projection point of any three-dimensional point among the three-dimensional points corresponding to the at least one first clustered target in the image coordinate system, and the first target detection box is any one of the plurality of target detection boxes.

14. The apparatus according to claim 13, characterized in that, The processing unit is also used for, If the first projection point is within the first target detection box and the second target detection box, then d1 and d2 are calculated; wherein, the second target detection box is one of the plurality of target detection boxes, and the second target detection box overlaps with the first target detection box; d1 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the first target detection box other than the overlapping region; d2 is the distance between the 3D point corresponding to the first projection point and the nearest 3D point in the region of the second target detection box other than the overlapping region. If d2 is less than d1, then the three-dimensional point corresponding to the first projection point is determined to match the second target detection box; If d2 is greater than or equal to d1, then the three-dimensional point corresponding to the first projection point is determined to match the first target detection box.

15. The apparatus according to any one of claims 10-11 and 13-14, characterized in that, The processing unit updates the clustering information of the 3D point cloud based on the correspondence and the preset size of the target corresponding to each target detection box, including: The processing unit determines the number of first cluster targets that match each target detection box; If the first detection box matches multiple first cluster targets among the at least one first cluster target, and the maximum distance between any two of the multiple first cluster targets is less than a preset distance, then based on the preset target size corresponding to the first detection box, it is determined whether to merge the multiple first cluster targets; specifically including: If the difference between the overall size of the plurality of first cluster targets and the preset size of the first cluster target of the preset category among the plurality of first cluster targets is less than a preset threshold, then the plurality of first cluster targets are merged into a second cluster target.

16. The apparatus according to any one of claims 10-11 and 13-14, characterized in that, The image detection information also includes: the confidence level of the target category corresponding to each target detection box in the plurality of target detection boxes, wherein the confidence level is used to characterize the credibility of the target category; The processing unit is further configured to: label a first three-dimensional point with the confidence level of the target category corresponding to the third target detection box; the first three-dimensional point is a three-dimensional point whose projection point is located in the third target detection box; the third target detection box is any one of the plurality of target detection boxes; and, A second three-dimensional point is marked with a pre-set confidence level; the second three-dimensional point is a three-dimensional point whose projection point is not in any of the multiple target detection boxes.

17. The apparatus according to any one of claims 10-11 and 13-14, characterized in that, The processing unit acquires image detection information of the target scene image, including: The processing unit performs feature recognition and feature analysis on the target scene image to obtain the image detection information.

18. The apparatus according to any one of claims 10-11 and 13-14, characterized in that, The target detection device further includes: The data acquisition unit is used to simultaneously acquire the target scene image and the three-dimensional point cloud of the target scene.

19. A target detection device, characterized in that, The target detection device includes: Memory, used to store computer programs; A processor for executing the computer program to implement the target detection method as described in any one of claims 1-9.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program code, which, when executed by the processing circuit, implements the target detection method as described in any one of claims 1-9.

21. A chip system, characterized in that, The chip system includes a processing circuit and a storage medium, wherein the storage medium stores computer program code; when the computer program code is executed by the processing circuit, it implements the target detection method as described in any one of claims 1-9.

Images