Method and device for eliminating redundant bounding boxes in object detection

Abstract

The application provides a method and device for eliminating redundant detection boxes in target detection, and relates to the technical field of data processing. The method comprises the following steps: projecting a plurality of three-dimensional detection boxes detected by a target detection model onto a two-dimensional bird's eye plane, determining polar coordinates of at least two vertices of a two-dimensional detection box corresponding to the three-dimensional detection box in a polar coordinate system, and performing conversion processing on the polar coordinates to determine coordinates of the vertices in an orthogonal coordinate system; the three-dimensional detection box has a confidence level, and the at least two vertices are diagonal points of each other; determining an intersection-union ratio of a reference detection box and a candidate detection box according to the coordinates of the at least two vertices of the reference detection box in the orthogonal coordinate system and the coordinates of the at least two vertices of the candidate detection box in the orthogonal coordinate system; and deleting a three-dimensional detection box corresponding to a candidate detection box with an intersection-union ratio greater than a preset threshold value from the plurality of three-dimensional detection boxes. The application improves the elimination effect of the redundant detection boxes and also improves the accuracy of the final detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a method and device for eliminating redundant detection boxes in target detection. BACKGROUND

[0002] In recent years, camera 3D target detection algorithms based on BEV gradually become mainstream. In the target detection algorithm, non maximum suppression (nms) is a necessary post-processing process, and the purpose is to eliminate redundant prediction boxes on the same object.

[0003] However, the current non maximum suppression scheme does not consider that it is difficult to make accurate depth estimation of the target based on the image (camera) only, so it is more likely to have the characteristic that the same target is repeatedly detected in the direction of the ray between the ego vehicle and the target vehicle. It also does not consider that the depth estimation error of the target based on the image (camera) is linearly related to the actual distance, and the Euclidean distance difference of the detection boxes of the same real target is large at a long distance, which easily leads to the characteristic that the non maximum suppression nms is invalid. Then, the current elimination effect of the redundant detection boxes is not ideal, and the final target detection result is not accurate enough.

[0004] Therefore, how to improve the elimination effect of the redundant detection boxes and then improve the accuracy of the target detection result has become a problem to be solved at present. SUMMARY

[0005] The present application provides a method and device for eliminating redundant detection boxes in target detection, to solve the problem of how to improve the elimination effect of the redundant detection boxes and then improve the accuracy of the target detection result.

[0006] In a first aspect, the present application provides a method for eliminating redundant detection boxes in target detection, which comprises:

[0007] projecting a plurality of three-dimensional detection boxes detected by a target detection model into a two-dimensional bird's eye plane, determining the polar coordinates of at least two vertices of a two-dimensional detection box corresponding to the three-dimensional detection box in a polar coordinate system, and performing conversion processing on the polar coordinates to determine the coordinates of the vertices in an orthogonal coordinate system; wherein the three-dimensional detection box has a confidence, and the at least two vertices are opposite corner points;

[0008] determining the intersection over union of the reference detection box and the candidate detection box according to the coordinates of at least two vertices of the reference detection box in the orthogonal coordinate system and the coordinates of at least two vertices of the candidate detection box in the orthogonal coordinate system; wherein the reference detection box is a two-dimensional detection box corresponding to the three-dimensional detection box with the highest confidence, and the candidate detection box is a two-dimensional detection box other than the reference detection box;

[0009] delete the three-dimensional bounding box corresponding to the candidate bounding box whose intersection over union is greater than the preset threshold in the plurality of three-dimensional bounding boxes.

[0010] Optionally, the polar coordinates are converted to determine the coordinates of the vertex in the orthogonal coordinate system, comprising:

[0011] The product of the polar angle and a preset polar angle coefficient in the polar coordinates of the vertex is determined, and the product is determined as the horizontal coordinate of the vertex in the orthogonal coordinate system.

[0012] The relative distance of the vertex is determined, and the product of the relative distance and a preset polar radius coefficient is determined, and the product is determined as the vertical coordinate of the vertex in the orthogonal coordinate system; wherein the relative distance is the ratio of the polar radius of the vertex to the minimum polar radius, and the minimum polar radius is the polar radius with the minimum value among the polar radii of the vertices.

[0013] Optionally, the plurality of three-dimensional bounding boxes detected by the target detection model are projected into a two-dimensional bird's eye plane, and the polar coordinates of at least two vertices of a two-dimensional bounding box corresponding to the three-dimensional bounding box in the polar coordinate system are determined, comprising:

[0014] The plurality of three-dimensional bounding boxes detected by the target detection model are projected into a two-dimensional bird's eye plane, and a polar coordinate system is established in the bird's eye plane;

[0015] Based on the position of the two-dimensional bounding box corresponding to the three-dimensional bounding box in the polar coordinate system, at least two vertices of the two-dimensional bounding box corresponding to the three-dimensional bounding box are determined, and the polar coordinates of the vertices are determined.

[0016] Optionally, based on the position of the two-dimensional bounding box corresponding to the three-dimensional bounding box in the polar coordinate system, at least two vertices of the two-dimensional bounding box corresponding to the three-dimensional bounding box are determined, comprising:

[0017] With the origin of the polar coordinate system as the vertex of a sector, based on the position of the two-dimensional bounding box corresponding to the three-dimensional bounding box in the polar coordinate system, the smallest sector capable of completely covering the two-dimensional bounding box corresponding to the three-dimensional bounding box is found.

[0018] The intersection point of the radius of the smallest sector and the two-dimensional bounding box is determined to be the vertex of the two-dimensional bounding box.

[0019] Optionally, the polar coordinates of the vertex are determined, comprising:

[0020] The length of the connecting line between the vertex and the origin of the polar coordinate system is determined as the polar radius, and the included angle between the connecting line and the polar axis is determined as the polar angle.

[0021] Optionally, before determining the intersection-union ratio of the reference detection frame and the candidate detection frame according to the coordinates of at least two vertices of the reference detection frame and the coordinates of at least two vertices of the candidate detection frame in the orthogonal coordinate system, the method further comprises:

[0022] By comparing the confidence degrees of the plurality of three-dimensional detection frames, the two-dimensional detection frame corresponding to the three-dimensional detection frame with the highest confidence degree is determined as the reference detection frame, and the two-dimensional detection frames other than the reference detection frame are determined as candidate detection frames.

[0023] Optionally, determining the intersection-union ratio of the reference detection frame and the candidate detection frame according to the coordinates of at least two vertices of the reference detection frame and the coordinates of at least two vertices of the candidate detection frame in the orthogonal coordinate system comprises:

[0024] Based on the coordinates of at least two vertices of the reference detection frame and the coordinates of at least two vertices of the candidate detection frame in the orthogonal coordinate system, the intersection area and the union area of the reference detection frame and the candidate detection frame are calculated.

[0025] The ratio of the intersection area to the union area is determined, and the ratio is determined as the intersection-union ratio.

[0026] In a second aspect, the present application provides a device for eliminating redundant detection frames in target detection, the device comprising:

[0027] A coordinate processing unit is configured to project a plurality of three-dimensional detection frames detected by a target detection model to a two-dimensional bird's eye plane, determine the polar coordinates of at least two vertices of a two-dimensional detection frame corresponding to the three-dimensional detection frames in a polar coordinate system, and perform conversion processing on the polar coordinates to determine the coordinates of the vertices in an orthogonal coordinate system; wherein the three-dimensional detection frames have confidence degrees, and the at least two vertices are opposite corner points of each other.

[0028] A calculation unit is configured to determine the intersection-union ratio of a reference detection frame and a candidate detection frame according to the coordinates of at least two vertices of the reference detection frame and the coordinates of at least two vertices of the candidate detection frame in the orthogonal coordinate system; wherein the reference detection frame is a two-dimensional detection frame corresponding to a three-dimensional detection frame with the highest confidence degree, and the candidate detection frame is a two-dimensional detection frame other than the reference detection frame.

[0029] An elimination unit is configured to delete the three-dimensional detection frames corresponding to the candidate detection frames whose intersection-union ratios are greater than a preset threshold value in the plurality of three-dimensional detection frames.

[0030] In a third aspect, the present application provides an electronic device, which comprises a processor and a memory connected in communication with the processor.

[0031] The memory stores computer-executable instructions;

[0032] The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of the preceding.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method according to any one of the preceding.

[0034] In a fifth aspect, the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the method according to any one of the preceding.

[0035] The method and device for eliminating redundant detection boxes in target detection provided by the present application comprise the following steps: projecting a plurality of three-dimensional detection boxes detected by a target detection model to a two-dimensional bird's eye plane, determining polar coordinates of at least two vertices of a two-dimensional detection box corresponding to the three-dimensional detection box in a polar coordinate system, and performing conversion processing on the polar coordinates to determine coordinates of the vertices in an orthogonal coordinate system; wherein the three-dimensional detection box has a confidence degree, and the at least two vertices are diagonal points of each other; determining an intersection-over-union of a reference detection box and a candidate detection box according to coordinates of at least two vertices of the reference detection box in the orthogonal coordinate system and coordinates of at least two vertices of the candidate detection box in the orthogonal coordinate system; wherein the reference detection box is a two-dimensional detection box corresponding to a three-dimensional detection box with the highest confidence degree, and the candidate detection box is a two-dimensional detection box other than the reference detection box; and deleting a three-dimensional detection box corresponding to a candidate detection box with an intersection-over-union greater than a preset threshold value in the plurality of three-dimensional detection boxes. The scheme of the present application fully considers the distribution characteristics of each detection box in the scene based on the camera 3D target detection algorithm, introduces polar coordinates and relative distances when performing non-maximum suppression processing, thereby improving the elimination effect of redundant detection boxes and improving the accuracy of the final detection result. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0037] Figure 1 A flowchart of a method for eliminating redundant detection boxes in target detection provided by an embodiment of the present application;

[0038] Figure 2 An example diagram of a two-dimensional detection box in a polar coordinate system provided by an embodiment of the present application;

[0039] Figure 3A schematic diagram of a position relationship between a reference detection frame and a candidate detection frame on an orthogonal coordinate system is provided for an embodiment of the present application.

[0040] Figure 4 A structural schematic diagram of a device for eliminating redundant detection frames in target detection is provided for an embodiment of the present application.

[0041] Figure 5 A structural schematic diagram of a device for eliminating redundant detection frames in target detection is provided for an embodiment of the present application.

[0042] Figure 6 A structural schematic diagram of an electronic device is provided for an embodiment of the present application.

[0043] The above drawings have shown the specific embodiments of the present application, which will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application concept in any way, but to illustrate the present application concept for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0044] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same reference numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims.

[0045] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, if any, are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the use of these terms herein is to be construed as interchangeable unless the context clearly dictates otherwise. Furthermore, the terms "comprise", "comprising", "comprises" and "comprising" and the like in the description and the claims of the present application, if any, are used from the meaning associating with the term "including" and vice versa, and are intended not to exclude, for example, additional steps or elements, but to "consist essentially of to particularly point out the essential characteristics of the present application to be used by those skilled in the art, unless the context clearly indicates otherwise.

[0046] In recent years, camera 3D target detection algorithms based on BEV have gradually become mainstream. Such algorithms usually represent features in the BEV space and learn target categories and bounding boxes in this space. As a dense detection scheme, camera 3D target detection usually needs to perform dense grid division in the BEV space, then extract corresponding features on image features according to the projection relationship, and finally obtain the target detection result.

[0047] Currently, the conventional division method of the BEV space is usually to perform equal-interval division in the x and y directions. However, due to the same size of the grid in the BEV space, the actual imaging area corresponding to the projection of the far grid to the image is small, while the imaging area of the near grid is large. This equal-interval division method will lead to a large number of redundant features extracted by adjacent grids far away for camera 3D target detection, that is, many redundant detection boxes will be included in the detected 3D detection boxes.

[0048] In addition, the current redundant detection box elimination scheme does not consider that it is difficult to make accurate depth estimation of the target based on the image (camera) only, so it is more likely to have the feature that the same target vehicle is repeatedly detected in the ray direction of the ego vehicle and the target vehicle. It also does not consider that the depth estimation error of the target based on the image (camera) is close to linearly related to the actual distance, and the Euclidean distance difference of the detection boxes of the same real target is large at a long distance, which is likely to cause the feature that the non-maximum suppression (nms) is invalid. In turn, it leads to the effect of eliminating the current redundant detection box not being ideal, and the final target detection result not being accurate enough.

[0049] To solve the above problems, the application provides a new method for eliminating redundant detection boxes in target detection. Considering that the repeated detection of the ray direction between the ego vehicle and the target vehicle has very strong geometric characteristics, it is more suitable to use the polar coordinate mode to represent it. Therefore, when eliminating redundant detection boxes, the application projects the obtained three-dimensional detection box on the bird's eye plane, first represents the coordinates of the points in the form of polar coordinates, and then maps them back to the orthogonal coordinate system. In addition, considering that the depth estimation error of the target is linearly related to the target distance, the Euclidean distance difference of the detection boxes of the same real target at a far distance is large enough to easily cause the nms to fail, so the application uses relative distance in the radial coordinate to measure. Finally, the necessary intersection over union is calculated through the coordinates in the orthogonal coordinate system, and then the intersection over union calculated in this way is used to eliminate the redundant prediction box. The application fully considers the distribution characteristics of each detection box in the current camera 3D target detection algorithm scene, introduces polar coordinates and relative distance when performing non-maximum suppression processing, so as to improve the elimination effect of redundant detection boxes and improve the accuracy of the final detection result.

[0050] The technical solutions of the application and how the technical solutions of the application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described again in some examples. The embodiments of the application will be described below with reference to the accompanying drawings.

[0051] Figure 1 A flowchart of a method for eliminating redundant detection boxes in target detection provided by an embodiment of the application. The execution subject of the embodiment of the application can be a redundant detection box elimination device in target detection, which can be located on an electronic device. The electronic device can be a mobile terminal such as a mobile phone, a tablet computer, a computer, etc., and the application does not make any limitation. The embodiment of the application will be described in detail taking the execution subject as an example.

[0052] As shown in Figure 1 The method for eliminating redundant detection boxes in target detection provided by the embodiment includes the following steps.

[0053] S101, project the plurality of three-dimensional detection boxes detected by the target detection model onto the two-dimensional bird's eye plane, determine the polar coordinates of at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system, and perform conversion processing on the polar coordinates to determine the coordinates of the vertices in the orthogonal coordinate system; wherein the three-dimensional detection box has a confidence, and the at least two vertices are opposite corner points.

[0054] Exemplarily, in camera-based 3D target detection, each detected object is usually labeled with a three-dimensional bounding box (3D Bounding Box) to represent the position and size of the target object; at the same time, a confidence (Confidence) is also labeled on each three-dimensional bounding box to represent the measure of whether the target detected by the three-dimensional bounding box is reliable. The confidence is usually a real number between 0 and 1, representing the reliability of the target detected by the model. The higher the confidence, the more confident the model is about the detection of the target, and vice versa.

[0055] Among the detected three-dimensional bounding boxes that have not been subjected to non-maximum suppression processing, there may be many redundant detection boxes, and the redundant detection boxes need to be eliminated to obtain more accurate final target detection results. In particular, the current conventional division method of BEV space is still to divide the x and y directions at equal intervals, which will lead to a large amount of redundant calculation in the far space. In order to reduce the overhead of such redundancy in memory resources and computing resources, the present application can consider using a non-uniform BEV space division method, that is, using more sparse division in the far space and more dense division in the near space. In addition, in order to facilitate convolution calculation, a space division method similar to polar coordinates and a ring convolution calculation method can also be designed. The space division method similar to polar coordinates is more related to the depth error distribution of the camera, which can reduce the redundancy of feature extraction in the BEV space. Using a ring convolution layer, that is, still using a 3x3 kernel for convolution operation, and moving the convolution kernel along the tangential and radial directions from the origin of the polar coordinates, which is very similar to the conventional convolution operation, the main difference is that the padding at the tangential boundary can include repeated areas. In this way, the redundancy of the three-dimensional bounding box can be reduced, the efficiency of eliminating redundant detection boxes can be improved, and the computing resources can also be saved.

[0056] The plurality of three-dimensional bounding boxes detected by the target detection model in the present application can be processed in the above manner, or can not be processed, which is not limited by the present application, and they are all applicable to the redundant detection box elimination scheme of the present application.

[0057] Exemplarily, when performing the elimination of the redundant detection boxes in the present application, the plurality of three-dimensional detection boxes detected by the target detection model are projected into the two-dimensional bird's eye plane, and the polar coordinates of at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system are determined. Correspondingly, each three-dimensional detection box has a corresponding confidence. In addition, based on the processing principle of non-maximum suppression, it is necessary to calculate the Intersection over Union (IoU) corresponding to each detection box, and the calculation of the Intersection over Union involves the area. Understandably, when calculating the area of the rectangular box, it is necessary to determine the length of each side, and at least two vertices of the opposite corners are needed to determine the length of each side. Therefore, in the present application, it is necessary to determine the polar coordinates of at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system. Then, the polar coordinates are converted to determine the coordinates of the vertices in the orthogonal coordinate system for subsequent calculation of the Intersection over Union.

[0058] Exemplarily, projecting the plurality of three-dimensional detection boxes detected by the target detection model into the two-dimensional bird's eye plane, and determining the polar coordinates of at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system can include:

[0059] S1, projecting the plurality of three-dimensional detection boxes detected by the target detection model into the two-dimensional bird's eye plane, and establishing a polar coordinate system in the bird's eye plane.

[0060] S2, based on the position of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system, determining at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box which are opposite corners, and determining the polar coordinates of the vertices.

[0061] Specifically, after projecting the plurality of three-dimensional detection boxes detected by the target detection model into the two-dimensional bird's eye plane, each three-dimensional detection box has a corresponding two-dimensional detection box in the bird's eye plane. Then, the polar coordinate system is directly established in the bird's eye plane, so that the position of each two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system is determined. Then, based on the position of the two-dimensional detection box in the polar coordinate system, at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box which are opposite corners can be determined, as well as the polar coordinates thereof.

[0062] Exemplarily, based on the position of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system, at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box which are opposite corners can be determined.

[0063] S21, taking the origin of the polar coordinate system as the vertex of the sector, based on the position of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system, finding the smallest sector that can completely cover the two-dimensional detection box corresponding to the three-dimensional detection box.

[0064] S22, determine that the intersection of the radius of the minimum sector and the two-dimensional detection frame is the vertex of the two-dimensional detection frame.

[0065] Exemplarily, taking the origin of the polar coordinate system as the vertex of the sector, the minimum sector capable of completely covering the two-dimensional detection frame corresponding to the three-dimensional detection frame is found in the bird's eye plane where the polar coordinate system is established, and the radius of the minimum sector has at least two intersection points with the covered two-dimensional detection frame, and the two points are the vertices opposite to each other. It can be understood that when there are multiple intersection points, the two vertices opposite to each other are also included.

[0066] Exemplarily, after at least two vertices opposite to each other of the two-dimensional detection frame corresponding to the three-dimensional detection frame are found in the bird's eye plane, the polar coordinates of the vertices can be determined based on the established polar coordinate system.

[0067] In one example, determining the polar coordinates of the vertex can include:

[0068] Connecting the vertex and the origin of the polar coordinate system, determining the length of the connecting line as the polar radius, and determining the included angle between the connecting line and the polar axis as the polar angle.

[0069] Exemplarily, Figure 2 An example diagram of a two-dimensional detection frame in a polar coordinate system is provided for the embodiments of the present application. After the three-dimensional detection frame is projected onto the two-dimensional bird's eye plane, it appears as a corresponding two-dimensional detection frame. In actual scenarios, there are multiple three-dimensional detection frames, and there are also multiple corresponding two-dimensional detection frames, Figure 2 only one of which is shown in the example. As shown in Figure 2 the polar coordinate system is established in the bird's eye plane, O is the origin of the polar coordinate system, the origin O of the polar coordinate system is taken as the vertex of the sector, and the radius of the minimum sector capable of completely covering the two-dimensional detection frame has two intersection points A and B with the two-dimensional detection frame. Connecting OA, the length of the connecting line OA is the polar radius of the vertex A, which can be denoted as p1, and the included angle θ1 between the connecting line OA and the polar axis is the polar angle of the vertex A, and then the polar coordinates of the vertex A can be denoted as (θ1, p1). Similarly, connecting OB, the length of the connecting line OB is the polar radius of the vertex B, which can be denoted as p2, and the included angle θ2 between the connecting line OB and the polar axis is the polar angle of the vertex B, and then the polar coordinates of the vertex B can be denoted as (θ2, p2).

[0070] Exemplarily, after determining the polar coordinates of at least two vertices of the two-dimensional detection frame corresponding to the three-dimensional detection frame in the polar coordinate system, the present application continues to convert the obtained polar coordinates to determine the coordinates of the vertex in the orthogonal coordinate system.

[0071] In one example, converting the polar coordinates to determine the coordinates of the vertex in the orthogonal coordinate system can include:

[0072] S10, determine the product of the polar angle of the vertex and the preset polar angle coefficient in the polar coordinates, and determine the product as the horizontal coordinate of the vertex in the orthogonal coordinate system.

[0073] S20, determine the relative distance of the vertex, and determine the product of the relative distance and the preset polar radius coefficient, and determine the product as the vertical coordinate of the vertex in the orthogonal coordinate system; wherein the relative distance is the ratio of the polar radius of the vertex to the minimum polar radius, and the minimum polar radius is the polar radius with the minimum value among the polar radii of the vertices.

[0074] Exemplarily, when the polar coordinates are converted, the product of the polar angle and the preset polar angle coefficient can be determined as the horizontal coordinate of the vertex in the orthogonal coordinate system. The preset polar angle coefficient is a coefficient set in advance, which can be denoted as W θ . The specific value of the preset polar angle coefficient W θ is not limited in the present application. Accordingly, the horizontal coordinate of the vertex A in the orthogonal coordinate system can be determined as W θ *θ1, and the horizontal coordinate of the vertex B in the orthogonal coordinate system can be determined as W θ *θ2.

[0075] The vertical coordinate can be measured by using the relative distance, which is normalized according to the closer target for subsequent calculation. Specifically, first, determine the polar radius with the minimum value among the polar radii of all the vertices as the minimum polar radius, which can be denoted as ρ near . Then, determine the ratio of the polar radius of the vertex to the minimum polar radius to obtain the relative distance of the vertex, and then determine the product of the relative distance and the preset polar radius coefficient as the vertical coordinate of the vertex in the orthogonal coordinate system. The preset polar radius coefficient is a coefficient set in advance, which can be denoted as W ρ . The specific value of the preset polar radius coefficient W ρ is not limited in the present application. Accordingly, the vertical coordinate of the vertex A in the orthogonal coordinate system can be determined as W ρ *ρ1 / ρ near , and the vertical coordinate of the vertex B in the orthogonal coordinate system can be determined as W ρ *ρ2 / ρ near .

[0076] Exemplarily, if the coordinates of the vertex A in the orthogonal coordinate system are denoted as (x1, y1), and the coordinates of the vertex B in the orthogonal coordinate system are denoted as (x2, y2), then: x1=W θ *θ1; y1=W ρ *ρ1 / ρ near ; x2=W θ *θ2; y2=W ρ *ρ2 / ρ near . Based on the two vertices which are opposite angles, a rectangular frame can be determined, that is, based on the coordinates of the vertices A and B in the orthogonal coordinate system, the position of the rectangular frame is uniquely determined.

[0077] All detected three-dimensional detection boxes are processed in the above manner to determine the coordinates of at least two vertices of each two-dimensional detection box corresponding to the three-dimensional detection box in the orthogonal coordinate system, that is, the position of the two-dimensional detection box corresponding to the three-dimensional detection box in the orthogonal coordinate system after coordinate conversion.

[0078] S102, according to the coordinates of at least two vertices of the reference detection box in the orthogonal coordinate system and the coordinates of at least two vertices of the candidate detection box in the orthogonal coordinate system, determine the intersection and union ratio of the reference detection box and the candidate detection box; wherein the reference detection box is the two-dimensional detection box corresponding to the three-dimensional detection box with the highest confidence, and the candidate detection box is the two-dimensional detection box other than the reference detection box.

[0079] Exemplarily, after determining the coordinates of at least two vertices of each two-dimensional detection box corresponding to the three-dimensional detection box in the orthogonal coordinate system, the intersection and union ratio of the reference detection box and each candidate detection box can be determined according to the obtained coordinates. Wherein, the reference detection box is the two-dimensional detection box corresponding to the three-dimensional detection box with the highest confidence, and the candidate detection box is the two-dimensional detection box other than the reference detection box.

[0080] In some examples, the reference detection box and the candidate detection box are not predetermined, and the reference detection box and the candidate detection box need to be determined before calculating the intersection and union ratio. In one example, determining the reference detection box and the candidate detection box can include:

[0081] Comparing the confidence of multiple three-dimensional detection boxes, determining the two-dimensional detection box corresponding to the three-dimensional detection box with the highest confidence as the reference detection box, and determining the two-dimensional detection box other than the reference detection box as the candidate detection box.

[0082] Exemplarily, each three-dimensional detection box has a corresponding confidence, comparing the confidence of all three-dimensional detection boxes, finding out the two-dimensional detection box corresponding to the three-dimensional detection box with the highest confidence to determine as the reference detection box, and the remaining two-dimensional detection boxes are candidate detection boxes.

[0083] Exemplarily, the intersection and union ratio of the reference detection box and the candidate detection box can be calculated according to the coordinates of the reference detection box and the candidate detection box in the orthogonal coordinate system determined in the foregoing. In one example, according to the coordinates of at least two vertices of the reference detection box in the orthogonal coordinate system and the coordinates of at least two vertices of the candidate detection box in the orthogonal coordinate system, determine the intersection and union ratio of the reference detection box and the candidate detection box, can include:

[0084] S100, based on the coordinates of at least two vertices of the reference detection frame on the orthogonal coordinate system and the coordinates of at least two vertices of the candidate detection frame on the orthogonal coordinate system, calculating the intersection area and the union area of the reference detection frame and the candidate detection frame.

[0085] S200, determining the ratio of the intersection area to the union area, and determining the ratio as the intersection-union ratio.

[0086] Exemplarily, the intersection area and the union area of the reference detection frame and the candidate detection frame are calculated according to the coordinate values respectively, and then the ratio of the intersection area to the union area is calculated, and the obtained ratio is the intersection-union ratio of the reference detection frame and the candidate detection frame.

[0087] Exemplarily, Figure 3 A schematic diagram of the positional relationship between the reference detection frame and the candidate detection frame on the orthogonal coordinate system is provided for the embodiments of the present application. As shown in the orthogonal coordinate system, Figure 3 if the rectangular frame S is the reference detection frame, the coordinates of the two opposite vertexes S1 and S2 thereof have been determined as (X1, Y1) and (X2, Y2) according to the scheme of the present application; if the rectangular frame C is the candidate detection frame, the coordinates of the two opposite vertexes C1 and C2 thereof have also been determined as (X3, Y3) and (X4, Y4) according to the scheme of the present application; then, the intersection area and the union area of the reference detection frame S and the candidate detection frame C can be calculated according to the coordinates of the vertexes S1 and S2 and the coordinates of the vertexes C1 and C2, and then the intersection-union ratio of the reference detection frame S and the candidate detection frame C can be obtained. Wherein, the intersection area of the reference detection frame S and the candidate detection frame C can be denoted as S∩C, the union area of the reference detection frame S and the candidate detection frame C can be denoted as S∪C, and the intersection-union ratio of the reference detection frame S and the candidate detection frame C can be denoted as IOU, then:

[0088] S∩C = (X2-X3)(Y2-Y3)

[0089] S∪C = (X2-X1)(Y2-Y1) + (X4-X3)(Y4-Y3) - (X2-X3)(Y2-Y3)

[0090] IOU = S∩C / S∪C

[0091] Accordingly, the intersection-union ratio of each candidate detection frame and the reference detection frame can be calculated in the above manner, and then the conventional nms process can be continued.

[0092] In addition, if the coordinates of the other vertexes in Figure 3 can also be converted to the vertexes shown in Figure 3 and then calculated, and the implementation principle is similar to the present application, which will not be described here.

[0093] S103, delete the three-dimensional detection frame corresponding to the candidate detection frame whose intersection-over-union is greater than the preset threshold in the plurality of three-dimensional detection frames.

[0094] Exemplarily, after calculating the intersection-over-union of each candidate detection frame and the reference detection frame, the obtained intersection-over-union is compared with the preset threshold, and the three-dimensional detection frame corresponding to the candidate detection frame whose intersection-over-union is greater than the preset threshold is deleted. The above steps are repeated until all three-dimensional detection frames are processed, and the elimination of redundant detection frames is completed.

[0095] The method for eliminating redundant detection frames in target detection provided by the embodiment of the application comprises the following steps: projecting a plurality of three-dimensional detection frames detected by a target detection model into a two-dimensional bird's eye plane, determining the polar coordinates of at least two vertices of a two-dimensional detection frame corresponding to the three-dimensional detection frame in a polar coordinate system, and performing conversion processing on the polar coordinates to determine the coordinates of the vertices in an orthogonal coordinate system; wherein the three-dimensional detection frame has a confidence, and the at least two vertices are opposite corner points of each other; determining the intersection-over-union of a reference detection frame and a candidate detection frame according to the coordinates of at least two vertices of the reference detection frame in the orthogonal coordinate system and the coordinates of at least two vertices of the candidate detection frame in the orthogonal coordinate system; wherein the reference detection frame is a two-dimensional detection frame corresponding to a three-dimensional detection frame with the highest confidence, and the candidate detection frame is a two-dimensional detection frame other than the reference detection frame; and deleting the three-dimensional detection frame corresponding to the candidate detection frame whose intersection-over-union is greater than a preset threshold in the plurality of three-dimensional detection frames. The scheme of the application fully considers the distribution characteristics of each detection frame in the current camera 3D target detection algorithm scene, introduces polar coordinates and relative distances when performing non-maximum suppression processing, thereby improving the elimination effect of redundant detection frames and improving the accuracy of the final detection result.

[0096] The following is an apparatus embodiment of the application, which can be used to execute the method embodiments of the application. For details not disclosed in the apparatus embodiments of the application, refer to the method embodiments of the application.

[0097] Figure 4 FIG. 1 is a structural schematic diagram of a device for eliminating redundant detection frames in target detection provided by an embodiment of the application. As shown in FIG. 1, the device for eliminating redundant detection frames in target detection 40 provided by the embodiment of the application comprises a coordinate processing unit 401, a calculation unit 402, and an elimination unit 403. Figure 4

[0098] The coordinate processing unit 401 is configured to project a plurality of three-dimensional detection frames detected by a target detection model into a two-dimensional bird's eye plane, determine the polar coordinates of at least two vertices of a two-dimensional detection frame corresponding to the three-dimensional detection frame in a polar coordinate system, and perform conversion processing on the polar coordinates to determine the coordinates of the vertices in an orthogonal coordinate system; wherein the three-dimensional detection frame has a confidence, and the at least two vertices are opposite corner points of each other.​

[0099] The computing unit 402 is configured to determine an intersection union ratio of the reference detection frame and the candidate detection frame according to the coordinates of the at least two vertices of the reference detection frame in the orthogonal coordinate system and the coordinates of the at least two vertices of the candidate detection frame in the orthogonal coordinate system; wherein the reference detection frame is a two-dimensional detection frame corresponding to a three-dimensional detection frame with the highest confidence, and the candidate detection frame is a two-dimensional detection frame other than the reference detection frame.

[0100] The eliminating unit 403 is configured to delete the three-dimensional detection frame corresponding to the candidate detection frame with the intersection union ratio greater than the preset threshold value from the plurality of three-dimensional detection frames.

[0101] The device provided in the embodiment can be used to execute the method of the above-mentioned embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0102] Figure 5 FIG. 5 is a structural schematic diagram of another target detection redundant detection frame elimination device provided in the embodiments of the present application. Figure 5 As shown in the figure, the target detection redundant detection frame elimination device 50 provided in the embodiments of the present application includes a coordinate processing unit 501, a computing unit 502, and an eliminating unit 503.

[0103] The coordinate processing unit 501 is configured to project the plurality of three-dimensional detection frames detected by the target detection model into a two-dimensional bird's eye plane, determine the polar coordinates of at least two vertices of the two-dimensional detection frame corresponding to the three-dimensional detection frame in the polar coordinate system, and perform conversion processing on the polar coordinates to determine the coordinates of the vertices in the orthogonal coordinate system; wherein the three-dimensional detection frame has a confidence, and the at least two vertices are opposite corner points.

[0104] The computing unit 502 is configured to determine an intersection union ratio of the reference detection frame and the candidate detection frame according to the coordinates of the at least two vertices of the reference detection frame in the orthogonal coordinate system and the coordinates of the at least two vertices of the candidate detection frame in the orthogonal coordinate system; wherein the reference detection frame is a two-dimensional detection frame corresponding to a three-dimensional detection frame with the highest confidence, and the candidate detection frame is a two-dimensional detection frame other than the reference detection frame.

[0105] The eliminating unit 503 is configured to delete the three-dimensional detection frame corresponding to the candidate detection frame with the intersection union ratio greater than the preset threshold value from the plurality of three-dimensional detection frames.

[0106] In one example, the coordinate processing unit 501 includes a horizontal coordinate conversion module 5011 and a vertical coordinate conversion module 5012.

[0107] The horizontal coordinate conversion module 5011 is configured to determine the product of the polar angle in the polar coordinates of the vertex and a preset polar angle coefficient, and determine the product as the horizontal coordinate of the vertex in the orthogonal coordinate system.

[0108] The ordinate conversion module 5012 is configured to determine a relative distance of the vertex, and determine a product of the relative distance and a preset polar radius coefficient as the ordinate of the vertex in the orthogonal coordinate system, where the relative distance is a ratio of a polar radius of the vertex to a minimum polar radius, and the minimum polar radius is a polar radius with the minimum value among the polar radii of the vertices.

[0109] In one example, the coordinate processing unit 501 further includes a polar coordinate system establishing module 5013 and a polar coordinate determining module 5014.

[0110] The polar coordinate system establishing module 5013 is configured to project the plurality of three-dimensional bounding boxes detected by the target detection model into a two-dimensional bird's eye plane, and establish a polar coordinate system in the bird's eye plane.

[0111] The polar coordinate determining module 5014 is configured to determine at least two vertices of the two-dimensional bounding box corresponding to the three-dimensional bounding box which are opposite-angle points based on the position of the two-dimensional bounding box corresponding to the three-dimensional bounding box in the polar coordinate system, and determine polar coordinates of the vertices.

[0112] In one example, the polar coordinate determining module 5014 includes a finding module 50141 and a vertex determining module 50142.

[0113] The finding module 50141 is configured to take the origin of the polar coordinate system as a vertex of a sector, find a minimum sector capable of completely covering the two-dimensional bounding box corresponding to the three-dimensional bounding box based on the position of the two-dimensional bounding box corresponding to the three-dimensional bounding box in the polar coordinate system.

[0114] The vertex determining module 50142 is configured to determine that the intersection of the radius of the minimum sector and the two-dimensional bounding box is a vertex of the two-dimensional bounding box.

[0115] In one example, the polar coordinate determining module 5014 further includes a polar coordinate value determining module 50143.

[0116] The polar coordinate value determining module 50143 is configured to connect the vertex and the origin of the polar coordinate system, determine that the length of the connecting line is a polar radius, and determine that the included angle between the connecting line and the polar axis is a polar angle.

[0117] In one example, the device 50 further includes a comparison unit 504.

[0118] The comparison unit 504 is configured to, before determining the intersection ratio of the reference bounding box and the candidate bounding box according to the coordinates of the at least two vertices of the reference bounding box in the orthogonal coordinate system and the coordinates of the at least two vertices of the candidate bounding box in the orthogonal coordinate system, compare the confidence degrees of the plurality of three-dimensional bounding boxes, determine the two-dimensional bounding box corresponding to the three-dimensional bounding box with the highest confidence degree as the reference bounding box, and determine the two-dimensional bounding boxes other than the reference bounding box as the candidate bounding boxes.

[0119] In one example, the computing unit 502 includes a first computing module 5021 and a second computing module 5022.

[0120] The first computing module 5021 is configured to calculate the intersection area and the union area of the reference detection frame and the candidate detection frame based on the coordinates of the at least two vertices of the reference detection frame in the orthogonal coordinate system and the coordinates of the at least two vertices of the candidate detection frame in the orthogonal coordinate system.

[0121] The second computing module 5022 is configured to determine the ratio of the intersection area to the union area, and determine the ratio as the intersection-union ratio.

[0122] The apparatus provided in the embodiment can be used to execute the method of the above-described embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0123] It should be noted that the division of each module of the above apparatus is only a logical functional division, and all or part of the modules can be integrated into one physical entity, or can be physically separated. The modules can all be implemented in the form of software invoked by a processing element; or all be implemented in the form of hardware; or part of the modules are implemented in the form of software invoked by a processing element, and part of the modules are implemented in the form of hardware. In addition, the functions of the above data processing modules can be stored in the memory of the apparatus in the form of program code, and be invoked and executed by a processing element of the apparatus. The implementation of other modules is similar. In addition, all or part of the modules can be integrated together, or can be independently implemented. The processing element herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each of the above modules can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the processing element.

[0124] Figure 6 A structural schematic diagram of an electronic device provided in the embodiment of the present application is shown in FIG. 6. As shown in FIG. 6, the electronic device 60 includes a processor 601 and a memory 602 in communication with the processor. Figure 6

[0125] The memory 602 stores computer execution instructions, and the processor 601 executes the computer execution instructions stored in the memory 602 to implement the method of any one of the preceding embodiments.

[0126] ​In the specific implementation of the electronic device, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0127] The embodiments of the present application further provide a computer readable storage medium, which stores computer execution instructions. The computer execution instructions are executed by a processor to implement the method according to any one of the preceding embodiments.

[0128] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by computer instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program is executed to perform the steps of the above-mentioned method embodiments; and the foregoing storage medium includes ROM, RAM, magnetic disc or optical disc and various storage medium capable of storing program codes.

[0129] The embodiments of the present application further provide a computer program product, which includes a computer program. The computer program is executed by a processor to implement the method according to any one of the preceding embodiments.

[0130] Those skilled in the art will readily conceive other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present application, which follow the general principles of the present application and include common knowledge or conventional technical means in the art that are not disclosed by the present application. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present application are indicated by the following claims.

[0131] It should be understood that the present application is not limited to the precise construction that has been described above and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for eliminating redundant bounding boxes in object detection, characterized in that, The method comprises: projecting a plurality of three-dimensional detection boxes detected by a target detection model into a two-dimensional bird's eye plane, and establishing a polar coordinate system in the bird's eye plane; based on the position of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system, determining at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box which are opposite corners, and determining the polar coordinates of the vertices; performing conversion processing on the polar coordinates to determine the coordinates of the vertices in the orthogonal coordinate system; wherein the three-dimensional detection box has a confidence level, and the at least two vertices are opposite corners; determining the intersection-over-union of the reference detection box and the candidate detection box according to the coordinates of at least two vertices of the reference detection box and the coordinates of at least two vertices of the candidate detection box in the orthogonal coordinate system; wherein the reference detection box is the two-dimensional detection box corresponding to the three-dimensional detection box with the highest confidence level, and the candidate detection box is the two-dimensional detection box other than the reference detection box; deleting the three-dimensional detection box corresponding to the candidate detection box whose intersection-over-union is greater than a preset threshold from the plurality of three-dimensional detection boxes.

2. The method of claim 1, wherein, The conversion processing on the polar coordinates to determine the coordinates of the vertices in the orthogonal coordinate system comprises: determining the product of the polar angle in the polar coordinates of the vertex and a preset polar angle coefficient as the horizontal coordinate of the vertex in the orthogonal coordinate system; determining the relative distance of the vertex, and determining the product of the relative distance and a preset polar radius coefficient as the vertical coordinate of the vertex in the orthogonal coordinate system; wherein the relative distance is the ratio of the polar radius of the vertex to the minimum polar radius, and the minimum polar radius is the polar radius with the minimum value among the polar radii of the vertices.

3. The method of claim 1, wherein, Based on the position of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system, determining at least two vertices of the two-dimensional detection box corresponding to the three-dimensional detection box which are opposite corners, comprises: taking the origin of the polar coordinate system as the vertex of a sector, based on the position of the two-dimensional detection box corresponding to the three-dimensional detection box in the polar coordinate system, finding the smallest sector that can completely cover the two-dimensional detection box corresponding to the three-dimensional detection box; determining that the intersection of the radius of the smallest sector and the two-dimensional detection box is the vertex of the two-dimensional detection box.

4. The method of claim 3, wherein, Determining the polar coordinates of the vertex comprises: connecting the vertex and the origin of the polar coordinate system, determining the length of the connecting line as the polar radius, and determining the included angle between the connecting line and the polar axis as the polar angle.

5. The method of claim 1, wherein, Before determining the intersection-over-union of the reference detection box and the candidate detection box according to the coordinates of at least two vertices of the reference detection box and the coordinates of at least two vertices of the candidate detection box in the orthogonal coordinate system, the method further comprises: comparing the confidence levels of the plurality of three-dimensional detection boxes, determining the two-dimensional detection box corresponding to the three-dimensional detection box with the highest confidence level as the reference detection box, and determining the two-dimensional detection boxes other than the reference detection box as the candidate detection boxes.

6. The method of claim 1, wherein, Determining the intersection-over-union of the reference detection box and the candidate detection box according to the coordinates of at least two vertices of the reference detection box and the coordinates of at least two vertices of the candidate detection box in the orthogonal coordinate system comprises: The intersection area and the union area of the reference bounding box and the candidate bounding box are calculated based on coordinates of at least two vertices of the reference bounding box on an orthogonal coordinate system and coordinates of at least two vertices of the candidate bounding box on the orthogonal coordinate system; A ratio of the intersection area to the union area is determined, and the ratio is determined as the intersection-union ratio.

7. A device for eliminating redundant detection boxes in target detection, characterized in that, The apparatus comprises: A coordinate processing unit is configured to project a plurality of three-dimensional bounding boxes detected by a target detection model into a two-dimensional bird's eye plane, determine polar coordinates of at least two vertices of a two-dimensional bounding box corresponding to the three-dimensional bounding box in a polar coordinate system, and perform conversion processing on the polar coordinates to determine coordinates of the vertices on an orthogonal coordinate system; wherein the three-dimensional bounding box has a confidence level, and the at least two vertices are opposite corner points of each other; A calculation unit is configured to determine an intersection-union ratio of a reference bounding box and a candidate bounding box based on coordinates of at least two vertices of the reference bounding box on an orthogonal coordinate system and coordinates of at least two vertices of the candidate bounding box on the orthogonal coordinate system; wherein the reference bounding box is a two-dimensional bounding box corresponding to a three-dimensional bounding box with the highest confidence level, and the candidate bounding box is a two-dimensional bounding box other than the reference bounding box; An elimination unit is configured to delete a three-dimensional bounding box corresponding to a candidate bounding box whose intersection-union ratio is greater than a preset threshold from the plurality of three-dimensional bounding boxes. The coordinate processing unit is specifically configured to project a plurality of three-dimensional bounding boxes detected by a target detection model into a two-dimensional bird's eye plane, and establish a polar coordinate system in the bird's eye plane; determine at least two vertices of a two-dimensional bounding box corresponding to the three-dimensional bounding box, which are opposite corner points of each other, based on a position of the two-dimensional bounding box in the polar coordinate system, and determine polar coordinates of the vertices.

8. An electronic device, comprising: The electronic device comprises a processor and a memory connected in communication with the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1-6.

Images