A method, device and computer readable storage medium for multiple object detection

Abstract

The present disclosure relates to a method, device and computer readable storage medium for multiple target detection. The method comprises: acquiring point cloud data containing the multiple targets; performing feature extraction on the point cloud data to obtain point cloud features; clustering the point cloud features; performing confidence evaluation on the clustering results of the clustering by using point cloud evaluation features to obtain evaluation results of the clustering results, wherein the point cloud evaluation features are used to represent the quality of the point cloud data; determining high-confidence targets and low-confidence targets in the clustering results according to the evaluation results; and calibrating the detection information of the low-confidence targets by using the detection information of the high-confidence targets to output detection results. The present disclosure calibrates low-confidence targets by high-confidence targets, which not only improves the detection rate of targets, but also enhances the confidence of target detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the technical field of target detection. More particularly, the present disclosure relates to a method, device and computer readable storage medium for multiple target detection. BACKGROUND

[0002] This section is intended to provide background information to the embodiments of the present disclosure recited in the claims. The description herein can include concepts that can be pursued, but are not necessarily ones that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, that which is in this section is not prior art to the claims and is not admitted to be prior art by its inclusion in this section.

[0003] In the current field of intelligent transportation, through various sensors (such as a laser radar at the roadside) and edge computing, targets such as traffic participants and road conditions can be instantaneously intelligently perceived or detected. However, when perceiving or detecting, the confidence of the detection result of the target may be low due to ranging errors, light scattering or occlusion and the like. This low confidence is usually manifested as a small number of three-dimensional points of the target, poor point cloud effect, large signal noise, incomplete contour and the like. In view of this, how to effectively enhance the confidence of the detected target becomes a problem to be solved. SUMMARY

[0004] To solve at least one or more of the above technical problems, the present disclosure provides a method, device and computer readable storage medium for multiple target detection. The present disclosure calibrates low-confidence targets by using high-confidence targets, not only improving the detection rate of the target, but also enhancing the confidence of the target detection. In view of this, the present disclosure provides corresponding solutions in the following aspects.

[0005] In a first aspect, the present disclosure provides a method for multiple target detection, comprising: acquiring point cloud data containing the multiple targets; performing feature extraction on the point cloud data to obtain point cloud features; clustering the point cloud features; performing confidence evaluation on the clustering results of the clustering by using point cloud evaluation features to obtain evaluation results of each clustering result, wherein the point cloud evaluation features are used to characterize the quality of the point cloud data; determining high-confidence targets and low-confidence targets in the clustering results according to the evaluation results; and calibrating the detection information of the low-confidence targets by using the detection information of the high-confidence targets to output a detection result.

[0006] In one embodiment, the point cloud evaluation features include density features, height features and / or line sequence features of the point cloud data.

[0007] In another embodiment, the confidence evaluation of the clustering results of the point cloud clustering includes a weighted operation on the evaluation results of multiple point cloud evaluation features to obtain the evaluation results of each clustering result.

[0008] In another embodiment, determining the high-confidence targets and low-confidence targets in the clustering results based on the evaluation results includes: comparing the evaluation results of each clustering result, and taking the target corresponding to the relatively better target evaluation result in the evaluation results as the high-confidence target, and taking the targets other than the high-confidence targets as low-confidence targets; or comparing the evaluation results of each clustering result, and taking the target corresponding to the relatively better target evaluation result in the evaluation results that meets the preset standard as the high-confidence target, and taking the targets other than the high-confidence targets as low-confidence targets.

[0009] In another embodiment, calibrating the detection information of the low-confidence target using the detection information of the high-confidence target to output a detection result includes: constructing a multilateral network containing the low-confidence target based on the high-confidence target; calibrating the detection information of the low-confidence target in each grid using the detection information of the high-confidence target in each grid of the multilateral network to output a detection result.

[0010] In another embodiment, constructing a multilateral network containing the low-confidence target based on the high-confidence target includes: using the high-confidence target as the corner point of each grid in the multilateral network, and the low-confidence target being contained within each grid; and using a greedy triangle algorithm to construct a multilateral network from the corner points, wherein the grids in the multilateral network are triangular grids.

[0011] In another embodiment, the detection information of low-confidence targets within each grid in the multilateral network is calibrated using the detection information of high-confidence targets in each grid to output the detection result. This includes: determining the triangular grid where the low-confidence target is located based on the target point of the triangular grid where the high-confidence target is located, wherein the target point includes the centroid and / or center of the triangle of the grid where the high-confidence target is located; and calibrating the detection information of the low-confidence target using the detection information of high-confidence targets on the triangular grid where the low-confidence target is located to output the detection result.

[0012] In yet another embodiment, determining the triangular mesh in which the low-confidence target is located based on the target point of the mesh in which the high-confidence target is located comprises: constructing a search tree of triangle centers and / or triangle barycenters of the mesh in which the high-confidence target is located; traversing the search tree to search for the triangle center and / or triangle barycenter closest to the low-confidence target; and taking the triangular mesh corresponding to the closest triangle center and / or triangle barycenter as the triangular mesh in which the low-confidence target is located.

[0013] In yet another embodiment, calibrating the detection information of the low-confidence target using the detection information of the high-confidence target on the triangular mesh in which the low-confidence target is located to output the detection result comprises: taking the detection information of the high-confidence target on the triangular mesh in which the low-confidence target is located as a constraint on the detection information of the low-confidence target to output the detection result.

[0014] In yet another embodiment, calibrating the detection information of the low-confidence target using the detection information of the high-confidence target on the triangular mesh in which the low-confidence target is located to output the detection result further comprises: obtaining a target region in the road scene, the target region comprising a drivable region of the current road or a lane region corresponding to the current driving direction; and taking the target region and the detection information of the high-confidence target on the triangular mesh in which the low-confidence target is located as a constraint on the detection information of the low-confidence target to output the detection result.

[0015] In yet another embodiment, the detection information comprises one or more of target type information, speed information, heading angle, and position information.

[0016] In yet another embodiment, calibrating the detection information of the low-confidence target in each of the groups of meshes using the detection information of the high-confidence target on each of the groups of meshes to output the detection result comprises: taking the extracted point cloud features, the detection result of the previous frame of point cloud, and the mesh information of each of the groups of meshes as inputs of a Kalman filter to output the detection result, wherein the mesh information comprises the high-confidence target and the low-confidence target corresponding to each of the groups of meshes, and the mesh information is used as a constraint when outputting the detection information of the corresponding low-confidence target.

[0017] In a second aspect, the present disclosure further provides a device for multiple target detection, comprising: a processor; and a memory connected to the processor, the memory storing computer program code, when the computer program code is executed by the processor, causing the device to execute the foregoing multiple embodiments.

[0018] In a third aspect, the present disclosure also provides a computer-readable storage medium comprising program instructions for multiple target detection, which when executed by one or more processors cause implementation of the foregoing multiple embodiments.

[0019] According to embodiments of the present disclosure, by clustering point cloud data of multiple targets and using point cloud evaluation features to evaluate the confidence of the clustering results, high-confidence targets and low-confidence targets in the multiple targets are determined. Further, the low-confidence targets are calibrated using the high-confidence targets, thereby improving the detection rate of targets and enhancing the confidence of target detection. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other objects, features and advantages of the present disclosure exemplary embodiments will be readily understood through reading the detailed description below taken in conjunction with the accompanying drawings. In the drawings, several embodiments of the present disclosure are illustrated by way of example and not limitation in the figures, like reference numerals refer to like or corresponding portions throughout:

[0021] Figure 1 is an exemplary schematic diagram illustrating laser radar detection;

[0022] Figure 2 is an exemplary flowchart illustrating a method of multiple target detection according to embodiments of the present disclosure;

[0023] Figure 3 is an exemplary schematic diagram illustrating polar coordinate partitioned grid cells according to embodiments of the present disclosure;

[0024] Figure 4 is an exemplary flowchart illustrating a method of multiple edge net construction according to embodiments of the present disclosure;

[0025] Figures 5a-5d is an exemplary schematic diagram illustrating multiple edge net construction according to embodiments of the present disclosure;

[0026] Figure 6 is an exemplary flowchart illustrating a method of determining a triangular grid in which a low-confidence target is located according to embodiments of the present disclosure;

[0027] Figure 7 is an exemplary schematic diagram illustrating surrounding targets of a low-confidence target and a road scene in which the low-confidence target is located according to embodiments of the present disclosure; and

[0028] Figure 8 is an exemplary schematic diagram illustrating a method of multiple target detection according to embodiments of the present disclosure. DETAILED DESCRIPTION

[0029] The principles and spirits of the present disclosure will be described below with reference to a number of exemplary embodiments. It should be understood that these embodiments are given only so that those skilled in the art can better understand and implement the present disclosure, and do not limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided so that the present disclosure is more thorough and complete, and the scope of the present disclosure is fully conveyed to those skilled in the art.

[0030] As described in the section of background art, in the current field of intelligent transportation, through various sensors and edge computing, instantaneous intelligent perception or detection can be performed on targets such as traffic participants and road conditions. The aforementioned sensors may, for example, be lidar sensors. As known by those skilled in the art, lidar is a signal source using laser. When pulsed laser emitted by a laser is irradiated onto a target, scattering occurs, and part of the light wave is reflected to the receiver of the lidar, so that target information is obtained according to the principle of laser ranging.

[0031] However, when detecting targets by using lidar, the detection rate of targets may be low and the confidence of the detected targets may be low due to problems such as ranging error, light scattering or occlusion. For example, in one scenario, when a target is far away from the lidar, the point cloud of the detected target is sparse. In another scenario, for example Figure 1 As shown, when there are many targets, there may be occlusion problems between targets, resulting in a small number of three-dimensional points of the target and an incomplete contour of the detected target. The above scenarios will be described in detail below. Figure 1

[0032] Figure 1 is an exemplary schematic diagram showing lidar detection. As shown, it is assumed that target 1, target 2 and target 3 shown as rectangular boxes are detected within the detection range of lidar P, wherein the size of the rectangular box can represent the number of target point clouds. That is, the larger the rectangular box, the more the number of point clouds; on the contrary, the smaller the rectangular box, the fewer the number of point clouds. For target 1, it is far away from lidar P, so the detected target point cloud is sparse, and thus the target contour is incomplete. For target 2, it is close to lidar P, so the detected target point cloud is dense, and thus the target contour is more complete. For target 3, the pulsed laser emitted by lidar P is occluded by target 2, so that the reflected echo cannot be received by lidar P, so that the number of detected target 3 point clouds is small, and the contour of target 3 is also incomplete.

[0033] As described above, when detecting targets by using lidar, the number of target point clouds may be small and the target contour may be incomplete due to problems such as ranging error, light scattering or occlusion, so that the detection rate and confidence of the target are low.​

[0034] To overcome the defects of one or more of the above aspects, the present disclosure provides a method, device and computer readable storage medium for multiple target detection. Through the scheme of the present disclosure, the detection rate of targets and the confidence of detection results can be improved.

[0035] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by a person skilled in the art without creative labor fall within the scope of protection of the present disclosure.

[0036] Figure 2 FIG. 2 is an exemplary flowchart illustrating a method 200 for multiple target detection according to an embodiment of the present disclosure. As shown in the figure, at step S202, point cloud data containing multiple targets is acquired. In one embodiment, the aforementioned multiple targets can be traffic participants and / or road facilities. In some scenarios, the traffic participants can be, for example, motor vehicles, non-motor vehicles, or pedestrians, etc., and the road facilities can be, for example, traffic signal lights or road barrier facilities, etc. Based on the foregoing description, it can be known that the point cloud data of the aforementioned multiple targets can be acquired by a laser radar. In some embodiments, the point cloud data of the multiple targets can also be acquired by a 3D scanning device such as a stereo camera or a time-of-flight camera, etc.

[0037] Based on the aforementioned acquired point cloud data, at step S204, feature extraction is performed on the point cloud data to obtain point cloud features. It can be understood that the point cloud features can be the original features of the point cloud data directly acquired by the laser radar or the other aforementioned 3D scanning devices, and the point cloud features obtained after feature extraction based on the point cloud data collected by different scanning devices will be different. For example, when a laser radar is used to acquire point cloud data and extract point cloud features therefrom, the point cloud features can generally include three-dimensional coordinates, velocity, and laser reflection intensity of the point cloud, etc. When a stereo camera or a time-of-flight camera is used to acquire point cloud data, the point cloud features obtained after feature extraction therefrom can include three-dimensional coordinates and color information of the point cloud, etc. In addition, the aforementioned point cloud features can also include features such as normal or principal curvature of the point cloud, etc.

[0038] Based on the extracted point cloud features, the process proceeds to step S206. At this step, the point cloud features are clustered. As known to those skilled in the art, clustering is a process of partitioning a set of physical or abstract objects into multiple classes of similar objects, and a cluster generated by clustering is a collection of data objects, which are similar to each other in the same cluster and dissimilar to objects in other clusters. In embodiments of the present disclosure, for example, a motor vehicle in a plurality of targets can be regarded as an object in the same cluster, and a non-motor vehicle can be regarded as an object in the same cluster after clustering the point cloud features. In some embodiments, different clustering algorithms can be selected based on different analysis methods. For example, based on a partitioning method, a K-Means algorithm can be selected to implement the clustering process of the point cloud features, based on a density-based method, a DBSCAN algorithm can be selected, and based on a grid-based method, a STING algorithm can be selected, and the present disclosure does not make any limitation in this regard.

[0039] After obtaining the clustering results through the above clustering operation, at step S208, the clustering results are evaluated for confidence using point cloud evaluation features to obtain evaluation results of each clustering result. The aforementioned point cloud evaluation features can be used to represent the quality of the point cloud data. In an embodiment, the point cloud evaluation features can include but are not limited to density features, height features, and / or line sequence features. For example, in some embodiments, the aforementioned point cloud evaluation features can also be a Fast Point Feature Histogram (“FPFH”) or a Signature of Histograms of Orientations (“SHOT”). Those skilled in the art can arbitrarily select one or more point cloud evaluation features for confidence evaluation of the clustering results, and then obtain evaluation results for subsequent use in determining high-confidence targets and low-confidence targets in the clustering results. The density features, height features, and line sequence features will be described in detail below.

[0040] For the density features, the greater the density of the clustered point cloud data, the easier it is to detect small targets, and the more complete the detected target outline. That is, the greater the density of the point cloud data, the better the quality of the point cloud data. In one implementation scenario, a grid cell based on polar coordinate partitioning can be drawn for each target after clustering. Further, by calculating the number of point clouds in each grid cell, the density features can be obtained. It can be understood that the aforementioned density features are feature descriptions based on the distribution of target point clouds in the polar coordinate system, and are related to the scanning angle of the detection device (e.g., a laser radar) and the distance from the detection device. The grid cell based on polar coordinate partitioning will be described in detail below. Figure 3 Detailed description.

[0041] For the height feature, since the point cloud data contains three-dimensional information, the point cloud data also has a corresponding distribution feature in the z-axis of the three-dimensional space coordinates, i.e., the height feature. In an embodiment, the z-axis direction can be further divided into layers based on the grid cells divided based on the polar coordinates described above. The height feature of the point cloud can be obtained by normalizing the number of target point clouds in each layer after the z-axis direction layer division. Similar to the density feature described above, the greater the height feature, the better the quality of the point cloud data.

[0042] For the line sequence feature, the number of point clouds on each laser line can be counted to obtain the line sequence feature. In an implementation scenario, the description of the laser line sequence number (i.e., the line sequence feature) can also be added when calculating the height feature to jointly constitute the height line sequence feature.

[0043] The point cloud evaluation features (density feature, height feature, and / or line sequence feature) obtained above can be further used to evaluate the confidence of the clustering results. Specifically, the greater the density feature, height feature, and / or line sequence feature, the higher the score of the clustering results, and the higher the confidence. In an implementation scenario, any one or more of the density feature, height feature, and / or line sequence feature can be used to evaluate the confidence. It should be understood that when multiple point cloud evaluation features are used, the multiple point cloud evaluation features can be first fused to obtain a unified point cloud evaluation feature. Further, the unified point cloud evaluation feature is used to evaluate the confidence. In an embodiment, the fusion operation described above can be a weighting operation. In some embodiments, different point cloud evaluation features can be used to first evaluate the confidence, and then the evaluation results of the multiple point cloud evaluation features are weighted to obtain the evaluation results of the clustering results.

[0044] After the evaluation results of the clustering results are obtained as described above, the process proceeds to step S210. In this step, the high-confidence targets and low-confidence targets in the clustering results are determined according to the evaluation results. More specifically, the high-confidence targets and low-confidence targets can be determined by comparing the evaluation results of the clustering results. In an embodiment, the target corresponding to the relatively optimal target evaluation result in the evaluation results can be directly used as the high-confidence target, and the targets other than the high-confidence target can be used as the low-confidence targets. In other words, the present disclosure proposes that the target with a higher score result is directly used as the high-confidence target, and the remaining targets are used as the low-confidence targets.

[0045] In another embodiment, the target corresponding to the target evaluation result in the evaluation results that is relatively optimal and meets the preset standard can also be taken as a high-confidence target. The aforementioned preset standard can be set based on the scanning characteristics in the detection region of the laser radar, and the preset standards under different detection regions can be different. For example, for a region close to the laser radar (e.g., the region where target 2 in Figure 1 is located), the quality of the point cloud data obtained is high, and thus the preset standard is high; and for a region far from the laser radar (e.g., the region where target 1 in Figure 1 is located), the point cloud data obtained by scanning is sparse, and thus the preset standard is low. That is, even if the evaluation result (in terms of score) of a target in a region far from the laser radar is lower than that of a target in a region close to the laser radar, if the evaluation result of the target far from the laser radar meets the preset standard set for the far region, the target can still be taken as a high-confidence target.

[0046] According to the determined high-confidence target and low-confidence target, at step S212, the detection information of the low-confidence target is calibrated by using the detection information of the high-confidence target to output a detection result. Specifically, first, a multi-edge network containing the low-confidence target can be constructed based on the high-confidence target, and then the detection information of the low-confidence target in each grid can be calibrated by using the detection information of the high-confidence target on the grid in the multi-edge network, so as to output the detection result.

[0047] Figure 3 is an exemplary schematic diagram showing a grid unit of polar coordinate partitioning according to an embodiment of the present disclosure. As shown in the figure, a fixed point O in the plane is taken, which is called a polar point, then a ray Ox is drawn, which is called a polar axis, and a length unit and a positive direction of angle (e.g., counterclockwise direction) are selected, thereby establishing a polar coordinate system. The coordinates of a point in the polar coordinate system are called polar coordinates, and the polar coordinates are two-dimensional. For example, for any point M in the plane, the length of the line segment OM can be represented as , and the angle from Ox to OM can be represented as In the polar coordinate system, can be called the polar radius of point M, can be called the polar angle of point M, and the ordered number pair ( , ) is called the polar coordinates of point M. For a target point cloud, the polar coordinates can be used to describe the distance of the point cloud from the laser radar and the scanning angle of the laser radar, thereby describing the distribution of the point cloud.

[0048] Further shown in the figure are a plurality of concentric circles and a plurality of rays, forming a grid cell based on polar coordinate partitioning. The grid cell can be partitioned based on the clustered target point cloud. Assuming the plurality of solid dots shown in the figure represent the target point cloud in the clustering result, the density feature can be calculated based on the rays and the rays and the grid cell H partitioned by the concentric circle a and the concentric circle b. That is, the density feature can be obtained by counting the number of target points (shown as solid dots in the figure) within the grid cell H.

[0049] As described above, the grid cell based on polar coordinate partitioning can obtain the point cloud evaluation feature. Further, the clustering result can be evaluated for confidence by the point cloud evaluation feature, so that the high-confidence targets and the low-confidence targets can be determined.

[0050] In one embodiment, the high-confidence targets can be used as the corner points of the grid in the multi-lateral network, and the high-confidence targets can be used to enclose the low-confidence targets, i.e., the low-confidence targets are contained in the grid. Further, the greedy triangle algorithm can be used to construct the multi-lateral network with the corner points. In some embodiments, the grid in the multi-lateral network can be a polygon, such as but not limited to a triangle or a quadrilateral. In the embodiments of the present disclosure, the grid in the constructed multi-lateral network is an exemplary triangular grid. The construction process of the multi-lateral network will be described in detail below. Figure 4

[0051] Figure 4 is an exemplary flowchart showing a method 400 of constructing a multi-lateral network according to an embodiment of the present disclosure. As shown, at step S402, the discrete points on the known plane are connected to each other two by two to form a planar graph covering the entire set of discrete points. It should be understood that in the planar graph, the connecting edges in the plane do not contain any discrete points except the end points, and the connecting edges in the plane do not intersect each other. Additionally, each face in the aforementioned planar graph can be a triangular face, and the union of each triangular face constitutes the convex hull of the set of discrete points (for example Figure 5a Next, at step S404, a large triangle is constructed using the input set of high-confidence target points, and a triangle list is created. The aforementioned large triangle contains all the discrete points.

[0052] Based on the established triangle list, at step S406, a point in the set of high-confidence target points is inserted into the triangle list, and an inscribed circle containing the point is found in the triangle list (for example Figure 5b Further, at step S408, the common edges of the inscribed circle are deleted (for example Figure 5c After deleting the common edges, at step S410, the inserted point is connected to the vertices of the existing triangle to form a new triangular partition (for example​Figure 5d The flow returns to step S406, and the aforementioned steps S406 to S410 are repeatedly executed until all points in the high-confidence target point set are inserted, so as to complete the construction of the multi-edge mesh of the high-confidence target in the entire scene. In one embodiment, the triangular meshes in the constructed multi-edge mesh can be sequentially arranged in order of triangular number, three vertex coordinates, and link order number of three edges, and stored in the memory, so as to be called when the triangular mesh corresponding to the low-confidence target is determined by subsequent traversal search tree.

[0053] Figures 5a-5d is an exemplary schematic diagram illustrating the construction of the multi-edge mesh according to an embodiment of the present disclosure. Figure 5a illustrates an exemplary planar graph in which discrete points on a known plane are connected to each other in pairs to cover the entire discrete point set. As described above, the connecting edges of the planar graph do not contain discrete points and do not contain intersecting edges. It is shown in the figure that each face of the planar graph is a triangular face. Figure 5b illustrates a schematic diagram of inserting a point v in the high-confidence target point set into the triangular list, based on which the circumscribed circle c1 and the circumscribed circle c2 containing the point v are found in the triangular list, wherein the circumscribed circle c1 and the circumscribed circle c2 have a common edge e. Figure 5c is a schematic diagram illustrating the deletion of the common edge e. Figure 5d illustrates a schematic diagram of connecting the inserted point v with the vertices of the existing triangle, thereby constituting a new triangular partition.

[0054] According to the constructed multi-edge mesh, the detection information of the low-confidence target in each mesh can be calibrated by using the detection information of the high-confidence target on each mesh, so as to output the detection result. To this end, the triangular mesh in which the low-confidence target is located can be first determined based on the target point of the mesh in which the high-confidence target is located. In one embodiment, the target point of the mesh in which the high-confidence target is located can be the triangular barycenter and / or the triangular center. For an equilateral triangle, the triangular barycenter and the triangular center coincide. Then, the detection information of the low-confidence target is calibrated by using the detection information of the high-confidence target on the triangular mesh in which the low-confidence target is located, so as to output the detection result. For ease of understanding, the following will first combine Figure 6 describe how to determine the triangular mesh in which the low-confidence target is located.

[0055] Figure 6is an exemplary flowchart illustrating a method 600 of determining the triangular mesh where the low-confidence target is located according to an embodiment of the present disclosure. As shown, at step S602, a search tree of the triangle centers and / or triangle barycenters of the mesh where the high-confidence targets are located is constructed. In one embodiment, the search tree can be a binary search tree. Based on the constructed search tree, at step S604, the search tree is traversed to search for the triangle centers and / or triangle barycenters that are the closest neighbors of the low-confidence targets. That is, the triangle mesh from the multi-lateral network constructed based on the high-confidence targets is queried to find the triangle centers and / or triangle barycenters that are the closest to each of the low-confidence targets. At step S606, the triangular mesh corresponding to the closest triangle centers and / or triangle barycenters is determined as the triangular mesh where the low-confidence targets are located. According to the foregoing steps, the triangular mesh where each of the low-confidence targets is located can be determined.

[0056] It needs to be understood that the low-confidence target is not only affected by its surrounding targets during the course of motion, but also affected by the road scene where it is located. Therefore, when calibrating the detection information of the low-confidence target, the constraints of its surrounding targets and the road scene can be considered simultaneously. As an example, the detection information of the high-confidence targets on the triangular mesh where the low-confidence target is located and the target region in the road scene can be used as the constraints of the detection information of the low-confidence target. In some embodiments, the foregoing target region can include the drivable region of the current road or the lane region corresponding to the current driving direction. The constraints of the surrounding targets and the road scene of the low-confidence target will be described in detail below. Figure 7 The constraints of the surrounding targets and the road scene of the low-confidence target are described in detail.

[0057] Figure 7 is an exemplary schematic diagram illustrating the surrounding targets and the road scene of the low-confidence target according to an embodiment of the present disclosure. As shown, it is assumed that the high-confidence target A, the high-confidence target B, the high-confidence target C and the low-confidence target D are detected by the lidar P. In combination with the foregoing description, a triangular mesh with the high-confidence target A, the high-confidence target B and the high-confidence target C as the corner points can be constructed, and the low-confidence target D is included in the triangular mesh. It is further shown in the diagram that the high-confidence target A drives on lane 1, the high-confidence target C and the low-confidence target D drive on lane 2, and the high-confidence target B drives on other lanes (not shown in the diagram). In this scenario, the low-confidence target D is constrained by its surrounding targets (i.e. the high-confidence target A, the high-confidence target B and the high-confidence target C) and the drivable region (lane 2). Thus, the detection information of the low-confidence target D can be calibrated by using the detection information of the high-confidence target A, the high-confidence target B and the high-confidence target C and the lane 2. In one embodiment, the detection information can include one or more of the target type information, the speed information, the heading angle and the position information.

[0058] In one implementation scenario, when calibrating the detection information of the low-confidence target, the extracted point cloud features (such as three-dimensional coordinates, speed, etc.), the detection result of the previous frame of point cloud (i.e., the historical detection information of the target), and the grid information of each grid in the multi-edge networking can be specifically taken as the input of the Kalman filter, and then the detection result is output, so as to complete the calibration of the detection information of the low-confidence target. In one embodiment, the grid information is the high-confidence target and the low-confidence target corresponding to each grid, and the grid information is used to constrain when outputting the detection information of the corresponding low-confidence target.

[0059] As for the Kalman filter described above, it is an algorithm for optimal estimation of system state by using linear system state equation and observing data of system input and output. According to the Markov assumption of target driving, i.e., the current state of the target is related to the state of the target in the previous frame. Therefore, the historical state information (such as position information, speed information and / or heading angle information) of the target is input into the Kalman filter, and the error in the detection process is corrected through the Kalman filter, so that the optimal state information of the current frame of the target can be obtained. In one embodiment, the mean and variance of the state quantity (such as the position, speed and / or heading angle of the target) of the target in the previous frame are denoted as 、 respectively, and the mean and variance of the state quantity and variance of the current frame of the target predicted by using the mean and variance of the state quantity of the previous frame can be denoted as the following formula:

[0060] = (1)

[0061] = (2)

[0062] wherein, 、 respectively represent the coefficient matrix of the change of the state of the target, 、 respectively represent the mean and variance of the predicted state quantity of the current frame, represents the error variance matrix introduced in the prediction process.

[0063] Based on the mean and variance of the predicted state quantity of the current frame, the gain coefficient of the Kalman filter can be further calculated, which can be expressed by the following formula:

[0064] (3)

[0065] wherein, represents the gain coefficient of the Kalman filter, a variance of the predicted current frame, a coefficient matrix representing a target state change, an error covariance matrix between a predicted value and an actual value of the current frame.

[0066] the gain coefficient obtained above , the final result of the state information of the target current frame can be output, and the specific calculation formula is as follows:

[0067] (4)

[0068] ) (5)

[0069] wherein, , respectively represent the final result of the mean and variance of the state quantity of the current frame, , respectively represent the mean and variance of the predicted state quantity of the current frame, represents a measurement value of the current frame, represents a coefficient matrix of a target state change, represents a unit matrix.

[0070] Based on the Kalman filtering principle described above, in the embodiment of the disclosure, in addition to inputting the historical detection information of the low-confidence target into the Kalman filter, the high-confidence targets around the low-confidence target and the target area of the road scene are also added to constrain the low-confidence target. The constraint is specifically reflected in the difference of the error variance matrix in the formula (2) of the Kalman filtering described above. In one implementation scenario, assuming that the state of the target obeys a Gaussian distribution, the probability distribution of the position of the low-confidence target obeys a two-dimensional Gaussian distribution with the detected position as the mean point and the variance . In addition, the area of the triangular mesh where the low-confidence target is located is related to the position where the low-confidence target is located, for example, when the area of the triangle is small, the variance value of the low-confidence target distribution is also small. Based on this, the relationship formula of the variance of the triangular mesh where the low-confidence target is located can be established as , wherein s represents the area of the triangular mesh, represents the variance.

[0071] As for the constraint of the high-confidence target on the low-confidence target, the position information of the high-confidence target on the triangular mesh where the low-confidence target is located is constrained in the front and back positions and the left and right positions of the low-confidence target. In the scenario, the error variance matrix , wherein represents the variance of the front and back positions, This represents the variance of the left and right positions. In a given implementation scenario, the variances of both the front / back position and the left / right position do not exceed half the distance between them.

[0072] Regarding the constraint of the target region on low-confidence targets, the motion of low-confidence targets in a road scene can be decomposed into motion perpendicular to the road and motion parallel to the road. In this scenario, the error variance matrix of the low-confidence target during its motion can be set as follows: ,in This represents the variance parallel to the road direction. This represents the variance perpendicular to the road direction.

[0073] Combining the variance distributions under the two constraints mentioned above, the final error variance distribution can be generated, i.e. It's understandable that targets are typically located within the road and mostly travel parallel to it. Therefore, the values ​​for the error variance matrix can be determined by... and The smaller of the two values ​​is sufficient. Next, the final error variance matrix is ​​used. Replace the error variance matrix in the traditional Kalman filter with the error variance matrix in formula (2) above. Output the final detection results.

[0074] The solution of this embodiment extracts features from point cloud data containing multiple targets, thereby clustering the point cloud features. The clustering results are then evaluated using point cloud evaluation features, and high-confidence targets and low-confidence targets are determined based on the evaluation results of each cluster. Furthermore, the detection information of low-confidence targets is calibrated using the detection information of high-confidence targets to improve the target detection rate and enhance the confidence of target detection.

[0075] Figure 8 This is an overall exemplary schematic diagram illustrating a method 800 for detecting multiple targets according to an embodiment of the present disclosure. As shown, in step S802, scanning is performed using a lidar. After lidar scanning, in step S804, the detected targets can be obtained. In one embodiment, the detected targets are typically represented as point clouds. In step S806, for the aforementioned detected targets, the confidence level of the detection results is graded, thereby classifying the targets into high-confidence targets and low-confidence targets. Specific confidence level grading results can be referred to the above... Figure 1The description is not repeated here. Then, at step S808, according to the ranked high-confidence targets, a region is constructed with high-confidence targets. That is, a polygonal network with high-confidence targets as corners is constructed. Based on the constructed polygonal network, at step S810, the low-confidence targets in the region are calibrated with the high-confidence targets. In calibrating the low-confidence targets, in addition to being constrained by the high-confidence targets, historical information and input vehicle distribution and road information can also be input, that is, steps S812 and S814. After the foregoing calibration, at step S816, the low-confidence target information is enriched. Finally, at step S818, the final detection result is output.

[0076] According to the foregoing description in combination with the drawings, those skilled in the art can also understand that the embodiments of the present disclosure can also be implemented by a software program, and thus the present disclosure also provides a device and a computer product, which can be used to implement the method of multiple target detection described in the present disclosure in combination with the drawings.

[0077] It should be noted that although the operations of the method of the present disclosure are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all of the shown operations must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can change the order of execution. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps.

[0078] It should be understood that when the terms "first", "second", "third", and "fourth" and the like are used in the claims, the specification, and the drawings of the present disclosure, these terms are used only to distinguish different objects, and are not used to describe a particular order. The terms "include" and "contain" used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or sets thereof.

[0079] It should also be understood that the terms used in the present disclosure specification are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. As used in the present disclosure specification and claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms. It should be further understood that the term "and / or" used in the present disclosure specification and claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0080] As used in the specification and claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "once it is determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]," depending on the context.

[0081] While various embodiments of the disclosure have been shown and described herein, it will be apparent to those skilled in the art that more modifications, changes, and substitutions are possible without departing from the disclosure. It should be understood that various alternatives to the present disclosure described herein can be employed in practicing the present disclosure. It is intended that the following claims define the scope of the disclosure and that methods equivalent to those claims recited herein are within the scope of the disclosure.

Claims

1. A method for detecting multiple targets, characterized in that, include: Obtain point cloud data containing the multiple targets; Point cloud features are obtained by extracting features from the point cloud data; Cluster the point cloud features; The clustering results of the clustering are evaluated for confidence using point cloud evaluation features to obtain the evaluation results of each clustering result. The point cloud evaluation features are used to characterize the quality of the point cloud data. The point cloud evaluation features include the density features, height features, and / or line sequence features of the point cloud data. Based on the evaluation results, determine the high-confidence targets and low-confidence targets in the clustering results; as well as The detection information of the low-confidence target is calibrated using the detection information of the high-confidence target to output the detection result; The process of calibrating the detection information of the low-confidence target using the detection information of the high-confidence target to output the detection result includes: Construct a multilateral network that includes the low-confidence targets based on the high-confidence targets; The detection information of low-confidence targets within each grid in the multilateral network is calibrated using the detection information of high-confidence targets in each grid to output the detection results; Constructing a multilateral network that includes the low-confidence target based on the high-confidence target includes: The high-confidence targets are used as the corner points of each grid in the multilateral network, and the low-confidence targets are included in each of the grids. The corner points are constructed into a polygon network using a greedy triangle algorithm, wherein the mesh in the polygon network is a triangular mesh. The detection information of low-confidence targets within each grid in the multilateral network is calibrated using the detection information of high-confidence targets in each grid to output detection results including: The triangular grid in which the low-confidence target is located is determined based on the target point of the triangular grid in which the high-confidence target is located, wherein the target point includes the centroid and / or center of the triangle in which the high-confidence target is located; The detection information of the low-confidence target is calibrated using the detection information of the high-confidence target on the triangular grid where the low-confidence target is located, so as to output the detection result; Determining the triangular grid containing the low-confidence target based on the target points in the grid containing the high-confidence target includes: Construct a search tree for the centroid and / or center of the triangle in the grid containing the high-confidence target; Traverse the search tree to search for the centroid and / or center of the triangle closest to the low-confidence target; and The triangular mesh corresponding to the nearest centroid and / or center of the triangle is taken as the triangular mesh where the low confidence target is located; The detection information of the low-confidence target is calibrated using the detection information of high-confidence targets on the triangular mesh where the low-confidence target is located, to output the detection results, including: The detection information of high-confidence targets on the triangular mesh where the low-confidence target is located is used as a constraint on the detection information of the low-confidence target to output the detection result.

2. The method according to claim 1, characterized in that, Optimizing the confidence of the clustering results using point cloud evaluation features involves weighting the evaluation results of multiple point cloud evaluation features to obtain the evaluation results of each clustering result.

3. The method according to claim 1, characterized in that, Based on the evaluation results, the high-confidence and low-confidence targets in the clustering results are determined as follows: The evaluation results of each clustering are compared, and the target corresponding to the relatively better evaluation result in the evaluation results is taken as the high confidence target, and the targets other than the high confidence target are taken as the low confidence target. or The evaluation results of each clustering are compared. The targets corresponding to the relatively better evaluation results that meet the preset standards are taken as high-confidence targets, and the targets other than the high-confidence targets are taken as low-confidence targets.

4. The method according to claim 1, characterized in that, The calibration of the detection information of the low-confidence target using the detection information of the high-confidence target on the triangular mesh where the low-confidence target is located, in order to output the detection result, further includes: Obtain the target area in the road scene, the target area including the drivable area of ​​the current road or the lane area corresponding to the current driving direction; The detection information of the target region and the detection information of high-confidence targets on the triangular grid where the low-confidence targets are located are used as constraints for the detection information of the low-confidence targets, so as to output the detection results.

5. The method according to any one of claims 1-4, characterized in that, The detection information includes one or more of the following: target type information, speed information, heading angle, and position information.

6. The method according to claim 1, characterized in that, Using the detection information of high-confidence targets on each of the aforementioned network grids, the detection information of low-confidence targets within the corresponding grids is calibrated, and the detection results are output, including: The extracted point cloud features, the detection results of the previous frame's point cloud, and the grid information of each grid in the multi-sided network are used as inputs to the Kalman filter to output detection results. The grid information refers to the high-confidence targets and low-confidence targets corresponding to each grid, and the grid information is used to constrain the output of the detection information of the corresponding low-confidence targets.

7. An apparatus for detecting multiple targets, comprising: processor; as well as A memory connected to the processor, the memory storing computer program code, which, when executed by the processor, causes the device to perform the method as described in any one of claims 1-6.

8. A computer-readable storage medium comprising program instructions for detecting a plurality of targets, wherein when the program instructions are executed by one or more processors, the method according to any one of claims 1-6 is implemented.

Images