Post-processing method of target detection result and computer program product

Abstract

The invention provides a target detection result post-processing method and a computer program product, and relates to the technical field of target detection, and the method comprises the steps: obtaining a target detection result detected by a sensor; wherein the target detection result comprises bounding boxes and attribute information of a plurality of targets; dividing the plurality of targets into one or more target clusters based on a preset combination condition and the bounding boxes of the plurality of targets; wherein the merging condition comprises that the distance between the bounding boxes is smaller than a preset distance threshold value, or an inclusion relation exists between the bounding boxes; the two targets meeting the combination condition belong to the same target cluster; screening out the bounding box of one target from the bounding boxes of the targets in the target cluster as the bounding box of the combined targets; based on the attribute information of the targets in the target cluster, determining the attribute information of the combined targets; wherein the target clusters are in one-to-one correspondence with the combined targets. The calculation amount can be reduced, and the accuracy of the merging result is improved.

Description

Technical Field

[0001] The present application relates to the technical field of object detection, and particularly to a post-processing method for object detection results and a computer program product. Background Art

[0002] The track splitting phenomenon refers to the situation where, during the process of tracking multiple objects, the track of an object is wrongly split into multiple independent parts, resulting in the object being misidentified as multiple different objects. The track splitting phenomenon may occur in various scenarios such as autonomous vehicles, intelligent robots, sea ships, submarines, etc. Especially in complex traffic scenarios, the track splitting phenomenon will seriously affect the object detection effect, and in severe cases, even cause major traffic accidents. Therefore, it is necessary to post-process the object detection results to solve the track splitting problem.

[0003] The existing technology generally judges whether two objects are merged by the intersection over union. However, this method may require calculating the area of an irregular convex polygon, resulting in a large amount of calculation. In addition, if the size difference between two bounding boxes is relatively large, the merging result obtained by calculating the intersection over union may be incorrect, that is, two bounding boxes that are actually the same object are judged not to be merged. Summary of the Invention

[0004] In view of at least one of the above technical problems existing in the prior art, the present application is proposed. The present application can reduce the amount of calculation and improve the accuracy of the merging result.

[0005] In a first aspect, an embodiment of the present application provides a post-processing method for object detection results, including:

[0006] Obtaining the object detection results detected by a sensor; wherein, the object detection results include: bounding boxes and attribute information of multiple objects;

[0007] Based on a preset merging condition and the bounding boxes of the multiple objects, dividing the multiple objects into one or more object clusters; wherein, the merging condition includes: the distance between the bounding boxes is less than a preset distance threshold, or, there is an inclusion relationship between the bounding boxes; two objects that meet the merging condition belong to the same object cluster;

[0008] Selecting a bounding box of one object from the bounding boxes of the objects in the object cluster as the bounding box of the merged object;

[0009] Based on the attribute information of the objects in the object cluster, determining the attribute information of the merged object;

[0010] Wherein, there is a one-to-one correspondence between the object cluster and the merged object.

[0011] Optionally,

[0012] Among them, the merging condition is that there is an inclusion relationship between the bounding boxes;

[0013] Based on the preset merging condition and the bounding boxes of the multiple targets, dividing the multiple targets into one or more target clusters includes:

[0014] Based on the corner coordinates of the bounding boxes of the two targets, determine whether there is an inclusion relationship between the bounding boxes of the two targets. If so, determine that the two targets belong to the same target cluster; otherwise, determine that the two targets belong to different target clusters.

[0015] Optionally,

[0016] Among them, the merging condition is that the distance between the bounding boxes is less than a preset distance threshold;

[0017] Based on the preset merging condition and the bounding boxes of the multiple targets, dividing the multiple targets into one or more target clusters includes:

[0018] Calculate the Euclidean distance between the main points of the bounding boxes of the two targets; among them, the main points include: corner points, centroids, and midpoints of sides;

[0019] Determine whether the minimum Euclidean distance between the main points is less than the distance threshold. If so, determine that the two targets belong to the same target cluster; otherwise, determine that the two targets belong to different target clusters.

[0020] Optionally,

[0021] Selecting a bounding box of a target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target includes:

[0022] When the number of targets with the target type in the target cluster is 1, determine the bounding box of the target with the target type as the bounding box of the merged target.

[0023] Optionally,

[0024] Selecting a bounding box of a target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target includes:

[0025] When the number of targets with the target type in the target cluster is greater than 1, determine the bounding box of the target with the largest size among the targets with the target type as the bounding box of the merged target.

[0026] Optionally,

[0027] Selecting a bounding box of a target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target includes:

[0028] In the case where there is no target of the target type in the target cluster, determine the bounding box of the target closest to the host vehicle as the bounding box of the merged target.

[0029] Optionally,

[0030] Based on the attribute information of the targets in the target cluster, determine the attribute information of the merged target, including:

[0031] Determine that the life cycle of the merged target is the maximum value of the life cycles of the targets in the target cluster;

[0032] Determine that the confidence level of the merged target is the maximum value of the confidence levels of the targets in the target cluster;

[0033] Determine whether there are moving targets in the target cluster. If so, determine that the merged target is a moving target; otherwise, determine that the merged target is a stationary target.

[0034] Optionally,

[0035] Based on the attribute information of the targets in the target cluster, determine the attribute information of the merged target, including:

[0036] Determine that the lateral position, longitudinal position, lateral speed, longitudinal speed, target size, target serial number, and target category of the merged target are respectively the same as those of the selected target.

[0037] Optionally,

[0038] Based on the preset merging conditions and the bounding boxes of the multiple targets, divide the multiple targets into one or more target clusters, including:

[0039] Determine whether there is a target among the unlabeled targets that satisfies the merging conditions with the current target. If so, add the target that satisfies the merging conditions with the current target to the cache queue, and loop to execute to determine whether there is a target among the unlabeled targets that have not been added to the cache queue that satisfies the merging conditions with the targets in the cache queue. If so, add the target that satisfies the merging conditions with the targets in the cache queue to the cache queue until there is no target among the unlabeled targets that have not been added to the cache queue that satisfies the merging conditions with the targets in the cache queue. Mark the current target and the targets in the cache queue as the same target cluster, and clear the cache queue;

[0040] Update the current target to one of the unlabeled targets, and repeat the above process until all the multiple targets are marked.

[0041] In a second aspect, an embodiment of the present application provides a computer program product, and when the computer program / instructions are executed by a processor, the methods described in any of the above embodiments are implemented.

[0042] The post-processing method for target detection results and the computer program product provided by the present application determine whether targets are merged by calculating the distance between bounding boxes or whether there is an inclusion relationship between bounding boxes, that is, multiple targets in the same cluster are obtained by track splitting of the same target and will be merged into one target. The bounding box of the merged target is selected from the bounding boxes of one of the targets in the cluster, and the attribute information of the merged target is determined based on the attribute information of the targets in the cluster. The present application can reduce the calculation amount and improve the accuracy of the merging result. Description of the Drawings

[0043] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0044] Figure 1 is a flowchart of a post-processing method for target detection results provided by an embodiment of the present application;

[0045] Figure 2 is a schematic diagram of the spatial relationship between targets provided by an embodiment of the present application;

[0046] Figure 3 is a schematic diagram of the corner points of a bounding box provided by an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of the main point of a bounding box provided by an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of the host vehicle and the target provided by an embodiment of the present application;

[0049] Figure 6 is a flowchart of marking target clusters provided by an embodiment of the present application;

[0050] Figure 7 is a flowchart of marking targets that meet the merging conditions provided by an embodiment of the present application;

[0051] Figure 8 is a schematic diagram of a post-processing device for target detection results provided by an embodiment of the present application. Detailed Embodiments

[0052] To enable those skilled in the art to better understand the technical solutions of the embodiments of the present application, the following will, in conjunction with the accompanying drawings in the embodiments of the present application, clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the scope of protection of the present application.

[0053] As Figure 1 shown, the embodiments of the present application provide a post-processing method for target detection results, including the following steps:

[0054] Step S101: Obtain the bounding boxes and attribute information of multiple targets detected by a sensor.

[0055] The sensor includes but is not limited to lidar, millimeter-wave radar, camera, ultrasonic radar, etc. The attribute information includes: target type, life cycle, confidence level, lateral position, longitudinal position, lateral speed, longitudinal speed, target size, target serial number, and many others.

[0056] Step S102: Based on preset merging conditions and the bounding boxes of multiple targets, divide the multiple targets into one or more target clusters.

[0057] As Figure 2 shown, when the phenomenon of track splitting occurs, the spatial relationships between targets can be roughly divided into the following categories: The first category, the target sizes are similar and the centroid distances are relatively close; the second category, the target sizes are quite different but the centroid distances are relatively close; the third category, the target sizes are quite different and the centroid distances are relatively far; the fourth category, there is an inclusion relationship between the bounding boxes.

[0058] In view of the spatial relationships between targets, the merging conditions provided by the present application include: the distance between the bounding boxes is less than a preset distance threshold, or there is an inclusion relationship between the bounding boxes.

[0059] Among them, two targets that meet the merging conditions belong to the same target cluster.

[0060] Step S103: Select a bounding box of one target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target.

[0061] Multiple targets in a target cluster are obtained by track splitting of one target. Therefore, it is necessary to merge the targets in the same target cluster, and a bounding box of one target can be selected as the bounding box of the merged target. The specific screening logic will be described in subsequent embodiments.

[0062] Step S104: Determine the attribute information of the merged target based on the attribute information of the targets in the target cluster.

[0063] Wherein, the target cluster corresponds to the merged target one by one.

[0064] In the embodiment of the present application, by calculating the distance between the bounding boxes or whether there is an inclusion relationship between the bounding boxes, it is judged whether the targets are merged, that is, multiple targets in the same cluster are obtained by track splitting of the same target and will be merged into one target. The bounding box of the merged target is selected from the bounding boxes of one of the targets in the cluster, and the attribute information of the merged target is determined based on the attribute information of the targets in the cluster. The present application can reduce the calculation amount and improve the accuracy of the merging result.

[0065] In an embodiment of the present application, the merging condition is that there is an inclusion relationship between the bounding boxes.

[0066] Based on the preset merging condition and the bounding boxes of multiple targets, dividing the multiple targets into one or more target clusters includes:

[0067] Based on the corner point coordinates of the bounding boxes of two targets, determine whether there is an inclusion relationship between the bounding boxes of the two targets. If so, determine that the two targets belong to the same target cluster; otherwise, determine that the two targets belong to different target clusters.

[0068] Assume that in the vehicle coordinate system, the centroid coordinate of the bounding box is p0=(x0,y0), the length and width of the bounding box are represented by L and W respectively, and the heading angle of the bounding box is θ. Then the four corner point coordinates of the bounding box can be expressed as formulas (1)-(4):

[0069]

[0070] As Figure 3 shown, the bounding box is a rectangular box, and the four corner points are the four vertices. In the embodiment of the present application, the set composed of all the corner points of the bounding box is called the "corner point set".

[0071] To judge whether there is an inclusion relationship between two bounding boxes, it can be judged point by point whether the four corner points of one bounding box are completely included in the other bounding box, or whether the four corner points of the other bounding box are completely included in one of them.

[0072] As Figure 3 shown, for the bounding box EFGH, the necessary and sufficient condition for determining that the corner point F is included in the bounding box ABCD is formula (5):

[0073]

[0074] Similarly, it is possible to determine point by point whether the corner points E, G, and H are within the bounding box ABCD. If all four corner points E, F, G, and H of the bounding box EFGH are included in the bounding box ABCD, it indicates that the bounding box EFGH is included in the bounding box ABCD, that is, there is an inclusion relationship between the two bounding boxes; vice versa.

[0075] In the embodiment of the present application, by calculating the corner point set to determine whether there is an inclusion relationship between two bounding boxes, and further determining whether two targets can be merged. Compared with calculating the intersection over union of the bounding boxes, the computational complexity can be greatly reduced. Even if the size difference between the two bounding boxes is relatively large, it is possible to accurately determine whether merging is required through the corner point set. Compared with calculating the intersection over union of the bounding boxes, it is possible to more accurately determine whether two targets can be merged.

[0076] In one embodiment of the present application, the merging condition is that the distance between the bounding boxes is less than a preset distance threshold.

[0077] Based on the preset merging condition and the bounding boxes of multiple targets, dividing the multiple targets into one or more target clusters includes:

[0078] Calculating the Euclidean distance between the principal points of the bounding boxes of two targets; where the principal points include: corner points, centroids, and midpoints of sides;

[0079] Determining whether the minimum Euclidean distance between the principal points is less than the distance threshold. If so, determining that the two targets belong to the same target cluster; otherwise, determining that the two targets belong to different target clusters.

[0080] Continuing with the above example, the midpoint coordinates of the four sides can be expressed as follows:

[0081]

[0082] In the embodiment of the present application, the set composed of all the corner points of the bounding box, the midpoint of each side, and the centroid of the bounding box can be referred to as the "principal point set", and the principal point set can refer to Figure 4 each point shown in.

[0083] In the embodiment of the present application, the distance between the bounding boxes is calculated based on the principal point sets of the two bounding boxes.

[0084] Assume that the principal point sets of two bounding boxes are respectively expressed as P = {p i | i = 0, 1, 2,..., 8} and Q = {q j | j = 0, 1, 2,..., 8}, then the minimum Euclidean distance between two principal points can be expressed as:

[0085] d min = min{ED(p i , q j ) | i, j = 0, 1, 2,..., 8} (10)

[0086] Among them, ED(p i , q j ) represents the Euclidean distance between the i-th principal point of one bounding box and the j-th principal point of another bounding box. For a rectangle, there are 8 points including the vertices and the midpoints of each side, and 1 centroid point. Therefore, the set of principal points of a rectangle has a total of 9 points. Thus, the value ranges of i and j are 0 - 8.

[0087] Among them, the calculation formula of ED(p i , q j ) is as follows:

[0088]

[0089] Among them, p ix represents the x-axis coordinate value of the i-th principal point of one bounding box; q jx represents the x-axis coordinate value of the j-th principal point of another bounding box; p iy represents the y-axis coordinate value of the i-th principal point of one bounding box; q ky represents the y-axis coordinate value of the j-th principal point of another bounding box.

[0090] In summary, if the minimum Euclidean distance d min between two principal points is less than the distance threshold, it indicates that these two targets belong to the same target cluster, and the two targets can be merged into one target.

[0091] In the embodiment of the present application, by calculating the Euclidean distance between principal points to determine whether two targets belong to the same target cluster, that is, to determine whether the two targets can be merged. Compared with the method of calculating the intersection over union of two bounding boxes, the calculation amount of the embodiment of the present application is less, and it can avoid the problem of low accuracy of the merging result caused by large differences in bounding boxes. In actual application scenarios, the principal point set can only include corner points and midpoints, and the Euclidean distance can be replaced by the cosine distance, which is not limited to the foregoing implementation manner.

[0092] The target merging principle of the present application is: preferentially retain the bounding box with a target type, secondly retain the bounding box with a larger size, and finally retain the bounding box closer to the vehicle itself.

[0093] Based on this, when the number of targets with a target type in the target cluster is 1, determine the bounding box of the target with the target type as the bounding box of the merged target;

[0094] When the number of targets with a target type in the target cluster is greater than 1, determine the bounding box of the target with the largest size among the targets with the target type as the bounding box of the merged target;

[0095] In the case where there is no target with the target type in the target cluster, determine the bounding box of the target closest to the host vehicle as the bounding box of the merged target.

[0096] For example, in the target cluster, there are Target 1, Target 2, and Target 3, and none of these three targets have the attribute of target type. Then, use the bounding box of the target closest to the host vehicle among these three targets as the bounding box of the merged target. If Target 1 and Target 2 do not have the attribute of target type, while Target 3 has the target type, then use the bounding box of Target 3 as the bounding box of the merged target. If Target 1 and Target 2 have the attribute of target type, while Target 3 does not have the target type, then use the bounding box of the target with the larger size among Target 1 and Target 2 as the bounding box of the merged target.

[0097] In one embodiment, the target type includes other vehicles, pedestrians, cyclists, cones, or square columns, etc.

[0098] In the embodiments of the present application, the target type is given the highest priority, followed by the target size, and finally the distance from the host vehicle. In actual application scenarios, the screening priority of the attribute items can also be adjusted based on business needs. For example, preferentially select the bounding box of the target closest to the host vehicle as the bounding box of the merged target, and then the bounding box of the target with the largest size. In the case where the number of targets with the target type in the target cluster is greater than 1, it is also possible to determine the bounding box of the target closest to the host vehicle among the targets with the target type as the bounding box of the merged target.

[0099] The attribute information of the merged target determined through the embodiments of the present application is more in line with the actual driving scenario, further improving the driving safety of the host vehicle.

[0100] In one embodiment of the present application, based on the attribute information of the targets in the target cluster, determining the attribute information of the merged target includes:

[0101] Determine that the life cycle of the merged target is the maximum value of the life cycles of the targets in the target cluster;

[0102] Determine that the confidence level of the merged target is the maximum value of the confidence levels of the targets in the target cluster;

[0103] Determine whether there are moving targets in the target cluster. If so, determine that the merged target is a moving target; otherwise, determine that the merged target is a stationary target.

[0104] In actual application scenarios, the life cycle and confidence level of the merged target can also be determined in other ways. For example, determine that the life cycle of the merged target is the median of the life cycles of the targets in the target cluster, and determine that the confidence level of the merged target is the median of the confidence levels of the targets in the target cluster.

[0105] When determining the life cycle and confidence level in this application, the life cycle and confidence level of each target in the target cluster are considered respectively, which can improve the accuracy of the merging result.

[0106] In an embodiment of this application, based on the attribute information of the targets in the target cluster, the attribute information of the merged target is determined, including:

[0107] It is determined that the horizontal position, vertical position, horizontal speed, vertical speed, target size, target serial number, and target category of the merged target are respectively the same as those of the selected targets.

[0108] For example, if the bounding box of target 1 is selected from the target cluster as the bounding box of the merged target, then the horizontal position of the merged target is the same as that of target 1, and the vertical position of the merged target is the same as that of target 1, and so on. The embodiment of this application can improve the accuracy of the merging result.

[0109] In an embodiment of this application, based on the preset merging conditions and the bounding boxes of multiple targets, the multiple targets are divided into one or more target clusters, including:

[0110] Determine whether there is a target among the unlabeled targets that satisfies the merging condition with the current target. If so, add the target that satisfies the merging condition with the current target to the cache queue. Loop to execute to determine whether there is a target among the unlabeled targets that have not been added to the cache queue that satisfies the merging condition with the targets in the cache queue. If so, add the target that satisfies the merging condition with the targets in the cache queue to the cache queue until there is no target among the unlabeled targets that have not been added to the cache queue that satisfies the merging condition with the targets in the cache queue. Mark the current target and the targets in the cache queue as the same target cluster, and clear the cache queue; update the current target to one of the unlabeled targets, and repeat the above process until all multiple targets are marked.

[0111] Determine whether there is a target among the unlabeled targets that satisfies the merging condition with the current target. If not, mark the target cluster to which the current target belongs, update the current target to one of the unlabeled targets, and repeat to execute to determine whether there is a target among the unlabeled targets that satisfies the merging condition with the current target until all multiple targets are marked.

[0112] The embodiment of this application draws on the density-based clustering algorithm and uses a recursive algorithm to divide the targets. One target cluster corresponds to one merged target.

[0113] Considering that microprocessing units generally do not support dynamic memory allocation and management, the embodiments of the present application use arrays to label the target clusters to which the targets belong, which can avoid the operation of allocating dynamic memory.

[0114] As Figure 5 shown, assume that a certain sensor subsystem detects 8 targets existing in front of the host vehicle. For the convenience of description, the targets can be numbered in the order from left to right. For example, target 1, target 2, target 3... target 8.

[0115] If target 2, target 3, target 5, and target 6 meet the merging conditions, these four targets are marked as the same target cluster. Here, a recursive method can be used to label all targets. The process of labeling the target clusters is as follows:

[0116] The first step: Assume that the number of targets is N. The target cluster category flag bits of each target are stored in a one-dimensional array of 1*N, denoted as labels. First, perform an initialization operation on the target cluster category flag bit array labels of each target. Specifically, when performing the initialization operation on the target cluster category flag bit array labels of each target, it is necessary to initialize the number c of the current target cluster and the cache queue que. Set the initial value of each element in the 1*N array to 0, c = 0, and the cache queue que is an empty queue.

[0117] The second step: Assign values to the target cluster category flag bits of each target. Specifically, traverse each target one by one. First, assign the target cluster category flag bit of the target to c + 1. Then, according to the merging conditions, determine whether the current target can be merged with other targets (that is, whether they are in the same target cluster). If there are no other targets that can be merged with the current target, the cache queue que is an empty queue, and proceed to the next step; otherwise, place the other targets that can be merged with the current target into the cache queue que one by one. Then, determine whether there are other targets among the other targets that have not been added to the cache queue and can be merged with the targets in the cache queue; if so, add the other targets that can be merged with the targets in the cache queue to the cache queue as well; until no other targets can be added to the cache queue, mark the target cluster category flag bits of all targets in the cache queue as c + 1, and empty the cache queue que; The third step: When the cache queue que is empty, traverse the next unlabeled target in the same way and mark the target cluster category flag bits of the targets in the cache queue corresponding to the target until the target cluster category flag bits of all targets are marked, and then this process can be ended.

[0118] As Figure 6As shown, a schematic flowchart of a process 600 for marking target clusters is presented. The process 600 for marking target clusters may include the following steps S601, S602, S603, S604, S605, S606, S607, S608, S609, S610, and S611:

[0119] In step S601, an initialization operation is performed on the target cluster category flag bit array for each target. For example, the initialization operation may include creating a new empty cache queue que and setting parameters c = 1 and i = 1, where c represents the number of the current target cluster and i is the index of the currently traversed target.

[0120] In step S602, it is determined whether the parameter i is less than the number of targets N; if so, step S603 is executed, otherwise, this process is ended.

[0121] In step S603, the index index of the target is assigned the value of i.

[0122] In step S604, labels(index) is assigned the value of c.

[0123] In step S605, it is determined whether there are other targets that can be merged with the index-th target; if so, step S606 is executed; otherwise, step S607 is executed.

[0124] In step S606, the index index corresponding to the unlabeled and mergable target is placed in the cache queue que.

[0125] In step S607, it is determined whether the cache queue que is an empty queue; if so, step S608 is executed; otherwise, step S611 is executed.

[0126] In step S608, c is assigned the value of c + 1.

[0127] In step S609, i is assigned the value of i + 1.

[0128] In step S610, it is determined whether the value of labels(i) is 0; if so, return to execute step S602, otherwise, return to execute step S609.

[0129] In step S611, an element is popped from the cache queue que to determine the index index of the next target; then return to execute step S604.

[0130] A number of targets meet the merging conditions, and subsequently, these targets that meet the merging conditions will be merged into one target cluster. In other words, in Figure 7In the shown flow chart, the c value of the target that meets the merging condition can be set to the same value, so as to facilitate the subsequent merging of the targets with the same c value.

[0131] Furthermore, the idea of using a one-dimensional array to mark whether the merging condition is met is as follows: Use a one-dimensional array with a length of N to describe which other targets can be merged with the currently traversed target, and mark the array value corresponding to the index of the other target that can be merged with the currently traversed target as 1, otherwise, mark it as 0.

[0132] As Figure 7 shown, a schematic flow chart of the process 700 for marking the targets that meet the merging condition is shown. The process 700 for marking the targets that meet the merging condition includes the following steps S701, step S702, step S703, step S704, step S705, step S706, step S707, step S708, step S709, step S710, step S711 and step S712:

[0133] In step S701, establish an array labels, and set parameters c = 1, i = 1, where c represents the number of the current target cluster, and i is the index of the currently traversed target.

[0134] In step S702, determine whether the parameter i is less than the number of targets N; if so, execute step S703, otherwise, end this process;

[0135] In step S703, assign the target index index to i, establish a one-dimensional array array with a fixed length of 1*N, and use it to mark other targets that meet the merging condition.

[0136] In step S704, assign labels(index) to c.

[0137] In step S705, determine whether there are other targets that can be merged with the index-th target; if so, execute step S706; otherwise, execute step S707.

[0138] In step S706, mark the corresponding positions of the other targets that can be merged with the index-th target in the array array as 1.

[0139] In step S707, determine whether all elements of array are 0; if so, execute step S708; otherwise, execute step S711.

[0140] In step S708, assign c to c + 1, c = c + 1

[0141] In step S709, assign i to i + 1.

[0142] In step S710, it is determined whether the value of labels(i) is 0; if so, return to execute step S702, otherwise, return to execute step S709.

[0143] In step S711, traverse the non-zero elements in array. Assume the index of the non-zero element is j, then assign j to index.

[0144] In step S712, assign 0 to array(j), and then return to execute step S704.

[0145] In another example of the present application, an adjacency matrix can be used to implement the process of merging targets. The idea of merging targets is as follows: use an N*N matrix to describe the relationship between targets, and the value of each element a i,j , i, j ∈ [1, N] is used to represent whether the merging condition is satisfied between two targets (for example, target i and target j). For example, if a i,j ≠0, it means that target i and target j satisfy the merging condition, otherwise, it means that target i and target j do not satisfy the merging condition. When a i,j is not 0, the specific value of a i,j depends on which type of cluster the current target cluster belongs to.

[0146] For example, assume there are 10 targets, then a 10*10 matrix can be defined and the 10*10 matrix is initialized so that each element in the matrix is 0. Then traverse the targets one by one.

[0147] For target 1, if target 1 can be merged with target 3 and target 4, then the corresponding elements a 1,3 , a 1,4 , a 3,1 , a 4,1 in the 10*10 matrix can be marked as 1, indicating that target 1, target 3, and target 4 all belong to the first cluster.

[0148] For target 2, if target 2 can be merged with target 8, then mark the elements a 2,8 , a 8,2 as 2, indicating that target 2 and target 8 belong to the second cluster.

[0149] For target 3, if the third target can be merged with target 6, then mark the elements a 3,6 , a 6,3 as the cluster where target 3 is located, that is, 1.

[0150] And so on, repeat the above steps until all targets are traversed.

[0151] Among them, Target 1, Target 3, Target 4, and Target 6 belong to the same cluster and can therefore be merged into one target; Target 2 and Target 8 belong to the same cluster and can be merged into one target. The above clustering process uses an adjacency matrix of a fixed size to describe the clustering process and clustering results, avoiding the occupation of dynamic memory.

[0152] In an embodiment of the present application, after post-processing the target detection results obtained by each sensor, the post-processing results corresponding to each sensor can be fused. Among them, the post-processing results include the bounding boxes and attribute information of the merged targets. Specifically, the post-processing results can be merged according to the type of sensor. For example, for a 4V5R12U perception fusion system, among them, the target detection results of cameras and ultrasonic radars for stationary targets are relatively accurate, while the detection results for moving targets, especially high-speed moving targets, are not very satisfactory. Millimeter-wave radars have better detection results for moving targets, but poor detection effects for stationary targets; ultrasonic radars and millimeter-wave radars have weak recognition capabilities for target types and target sizes, while cameras can better detect target types and target sizes; the positioning accuracy of ultrasonic radars and millimeter-wave radars is generally higher than that of cameras; the detection distance of ultrasonic radars is relatively close, and the detection limit is usually only 2-5m; millimeter-wave radars usually have detection blind spots at close ranges.

[0153] The embodiment of the present application combines the different working characteristics of each sensor to fuse the post-processing results. It preferentially uses the target type and target size determined based on the camera after post-processing as the output type and output size of the fusion target; for stationary targets, it preferentially uses the target position determined based on the ultrasonic radar after post-processing as the output position of the fusion target; for moving targets, it preferentially uses the target position determined based on the millimeter-wave radar after post-processing as the output position of the fusion target; the life cycle and confidence level of the fusion target are selected as the maximum values of the life cycle and confidence level determined based on various sensors after post-processing.

[0154] As Figure 8 shown, an apparatus for post-processing target detection results in an embodiment of the present application includes:

[0155] An acquisition module 801, configured to acquire target detection results detected by a sensor; among them, the target detection results include: the bounding boxes and attribute information of multiple targets;

[0156] A partitioning module 802, configured to partition multiple targets into one or more target clusters based on a preset merging condition and the bounding boxes of the multiple targets; among them, the merging condition includes: the distance between the bounding boxes is less than a preset distance threshold, or there is an inclusion relationship between the bounding boxes; two targets that meet the merging condition belong to the same target cluster;

[0157] The screening module 803 is configured to screen out the bounding box of one target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target.

[0158] The determination module 804 is configured to determine the attribute information of the merged target based on the attribute information of the targets in the target cluster.

[0159] Wherein, the target cluster corresponds to the merged target one by one.

[0160] An embodiment of the present application provides a computer program product, characterized in that when the computer program / instructions are executed by a processor, the methods of any of the above embodiments are implemented.

[0161] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (Central Processing Unit, abbreviated as CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processing, abbreviated as DSP), application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), field-programmable gate arrays (Field-Programmable Gate Array, abbreviated as FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0162] It should also be understood that the memory mentioned in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0163] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) is integrated in the processor.

[0164] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0165] In addition to the data bus, this bus may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, all kinds of buses are labeled as buses in the figure.

[0166] It should also be understood that the first, second, third, fourth, and various numerical numbers involved herein are only for the convenience of description and are not used to limit the scope of this application.

[0167] It should be understood that the term "and / or" in this text is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the character " / " in this text generally represents an "or" relationship between the associated objects before and after.

[0168] In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware processor, or executed and completed by the combination of the hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0169] In various embodiments of the present application, the magnitude of the sequence numbers of the above processes does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0170] Those of ordinary skill in the art can realize that the various illustrative logical blocks (ILB) and steps described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0171] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical, or other form.

[0172] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, in each embodiment of the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0174] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive), etc.

[0175] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A post-processing method for object detection results, characterized in that Including: Obtaining a target detection result detected by a sensor; wherein, the target detection result includes: bounding boxes and attribute information of multiple targets; Based on a preset merging condition and the bounding boxes of the multiple targets, dividing the multiple targets into one or more target clusters; wherein, the merging condition includes: the distance between the bounding boxes is less than a preset distance threshold, or, there is an inclusion relationship between the bounding boxes; two targets satisfying the merging condition belong to the same target cluster; Selecting a bounding box of one target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target; Based on the attribute information of the targets in the target cluster, determining the attribute information of the merged target; Wherein, there is a one-to-one correspondence between the target cluster and the merged target.

2. The method according to claim 1, Characterized in that Wherein, the merging condition is that there is an inclusion relationship between the bounding boxes; Based on a preset merging condition and the bounding boxes of the multiple targets, dividing the multiple targets into one or more target clusters includes: Based on the corner coordinates of the bounding boxes of the two targets, determining whether there is an inclusion relationship between the bounding boxes of the two targets. If so, determining that the two targets belong to the same target cluster, otherwise determining that the two targets belong to different target clusters.

3. The method according to claim 1, Characterized in that Wherein, the merging condition is that the distance between the bounding boxes is less than a preset distance threshold; Based on a preset merging condition and the bounding boxes of the multiple targets, dividing the multiple targets into one or more target clusters includes: Calculating the Euclidean distance between the main points of the bounding boxes of the two targets; wherein, the main points include: corner points, centroids, and midpoints of sides; Determining whether the minimum Euclidean distance between the main points is less than the distance threshold. If so, determining that the two targets belong to the same target cluster, otherwise, determining that the two targets belong to different target clusters.

4. The method according to claim 1, characterized in that Selecting a bounding box of one target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target includes: When the number of targets with a target type in the target cluster is 1, determining the bounding box of the target with the target type as the bounding box of the merged target.

5. The method according to claim 1, characterized in that Selecting a bounding box of one target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target includes: When the number of targets with a target type in the target cluster is greater than 1, determining the bounding box of the target with the largest size among the targets with the target type as the bounding box of the merged target.

6. The method according to claim 1, characterized in that Selecting a bounding box of one target from the bounding boxes of the targets in the target cluster as the bounding box of the merged target includes: When there is no target with a target type in the target cluster, determining the bounding box of the target closest to the vehicle itself as the bounding box of the merged target.

7. The method according to claim 1, characterized in that Based on the attribute information of the targets in the target cluster, determining the attribute information of the merged target includes: Determine that the life cycle of the merged target is the maximum value of the life cycles of the targets in the target cluster; Determine that the confidence level of the merged target is the maximum value of the confidence levels of the targets in the target cluster; Determine whether there is a moving target in the target cluster. If so, determine that the merged target is a moving target; otherwise, determine that the merged target is a stationary target.

8. The method according to claim 1, wherein: Based on the attribute information of the targets in the target cluster, determine the attribute information of the merged target, including: Determine that the horizontal position, vertical position, horizontal speed, vertical speed, target size, target serial number, and target category of the merged target are respectively the same as those of the selected target.

9. The method according to claim 1, wherein: Based on a preset merging condition and the bounding boxes of the multiple targets, divide the multiple targets into one or more target clusters, including: Determine whether there is a target among the unlabeled targets that satisfies the merging condition with the current target. If so, add the target that satisfies the merging condition with the current target to the cache queue, and loop to execute to determine whether there is a target among the unlabeled targets that have not been added to the cache queue and that satisfies the merging condition with the targets in the cache queue. If so, add the target that satisfies the merging condition with the targets in the cache queue to the cache queue until there is no target among the unlabeled targets that have not been added to the cache queue and that satisfies the merging condition with the targets in the cache queue. Then, label the current target and the targets in the cache queue as the same target cluster, and empty the cache queue; Update the current target to one of the unlabeled targets, and repeat the above process until all the multiple targets are labeled.

10. A computer program product, characterized in that, When the computer program / instruction is executed by a processor, it implements the method according to any one of claims 1 to 9.

Images