Millimeter wave radar-based target box detection optimization method, medium and electronic device

Abstract

The application provides a target frame detection optimization method based on a millimeter wave radar, a medium and an electronic device. The method comprises: obtaining an initial target frame set generated based on a millimeter wave radar; calculating an overlap area ratio of any two target frames in the initial target frame set; determining an output target frame according to the overlap area ratio and updating the initial target frame set. The application effectively solves the problem of frame size estimation deviation caused by point cloud sparsity in the traditional method, improves the accuracy and reliability of target detection, realizes adaptive processing of complex road scenes, effectively reduces the false detection rate, improves the accuracy and stability of the detection result, and can effectively improve the perception performance of the automatic driving system.

Description

Technical Field

[0001] This application belongs to the field of intelligent driving technology, and in particular relates to the field of target box detection technology in autonomous driving. Background Technology

[0002] With the rapid development of autonomous driving technology, onboard sensor systems play a crucial role in intelligent driving environmental perception. Among them, millimeter-wave radar, as an important environmental perception sensor, has been widely used in autonomous driving technology. However, traditional millimeter-wave radar point clustering generation methods still have some shortcomings in target detection and bounding box estimation.

[0003] First, due to the sparsity of point cloud data, traditional methods are prone to errors in bounding box size estimation during target detection, affecting detection accuracy. Second, when targets have an orientation, non-axially aligned bounding boxes may overlap, and existing methods using fixed thresholds to determine overlap are ill-suited for complex road scenarios. These problems severely impact the detection and recognition of surrounding targets during autonomous driving, reducing the overall performance of the system.

[0004] To address these issues, the industry has been exploring new technological solutions. For example, some research has attempted to use multi-frame point traces and target length, width, and height information from 4D millimeter-wave radar for probability calculation and classification. However, these methods often rely on empirically adjusted thresholds, introducing additional instability factors and still carrying the risk of misclassification in terms of accuracy. Therefore, designing a method capable of dynamically optimizing target box detection has become particularly important to improve the perception capabilities and overall performance of autonomous driving systems. Summary of the Invention

[0005] This application provides a target box detection optimization method, medium, and electronic device based on millimeter-wave radar to improve the perception performance of autonomous driving systems.

[0006] In a first aspect, embodiments of this application provide a target bounding box detection optimization method based on millimeter-wave radar, comprising: obtaining an initial target bounding box set generated based on millimeter-wave radar; calculating the overlap area ratio of any two target bounding boxes in the initial target bounding box set; determining the output target bounding box based on the overlap area ratio and updating the initial target bounding box set.

[0007] In one implementation of the first aspect, calculating the percentage of overlapping area between two target boxes in the initial target box set includes: calculating the intersection area of ​​the two target boxes; calculating the area of ​​the minimum bounding rectangle of the two target boxes; and obtaining the percentage of overlapping area between the two target boxes based on the ratio of the intersection area to the area of ​​the minimum bounding rectangle.

[0008] In one implementation of the first aspect, calculating the intersection area of ​​the two target boxes includes: determining the normal directions corresponding to the four sides of the rectangle of each target box as the separation axis; obtaining the coordinates of the four vertices of the rectangle of each target box, and projecting all vertices of the rectangle of each target box onto the corresponding separation axis to obtain their respective projection intervals; determining whether the two target boxes overlap based on the projection intervals on the separation axis, and calculating the intersection area of ​​the two target boxes in the overlap state.

[0009] In one implementation of the first aspect, the step of determining whether two target boxes overlap based on the projection intervals on the separation axis and calculating the intersection area of ​​the two target boxes in the overlap state includes: if the projection intervals on any separation axis do not overlap, then the intersection area is determined to be 0; if the projection intervals on all separation axes overlap, then the two target boxes are determined to overlap; if the intersection area is a convex polygon, the vertices of the convex polygon are composed of the intersection points of the two rectangular sides and the vertices contained by the other, and the area of ​​the convex polygon is obtained as the intersection area.

[0010] In one implementation of the first aspect, determining the intersection area of ​​the two target boxes based on the overlapping area includes: if the overlapping area is 0, then the intersection area is 0; if the overlapping area is not 0, then the intersection area is equal to the overlapping area.

[0011] In one implementation of the first aspect, determining the output target box and updating the initial target box set based on the overlap area ratio includes: detecting whether the overlap area ratio is within a preset threshold range; if yes, retaining the target box with higher confidence as the output target box; if no, merging the two target boxes to form a single target box when the overlap area ratio is greater than the maximum threshold of the preset threshold range, and using this target box as the output target box; and using the two target boxes as output target boxes when the overlap area ratio is less than the minimum threshold of the preset threshold range.

[0012] In one implementation of the first aspect, the method further includes: comparing the overlapping data of the target box detected in the current frame with the overlapping data of the target box detected in at least one previous frame; when a preset triggering condition is met, merging the target box of the current frame with the target box of at least one previous frame, and using the merged target box as the output target box.

[0013] In one implementation of the first aspect, the preset triggering conditions include: consistent overlap state, consistent overlap region shape, and consistent overlap area.

[0014] In a second aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target box detection optimization method based on millimeter-wave radar as described in any one of the first aspects of this application.

[0015] Thirdly, embodiments of this application provide an electronic device, the electronic device including a processor and a memory; the memory stores program instructions; the processor is used to run the program instructions to execute the target box detection optimization method based on millimeter-wave radar as described in any one of the first aspects of this application.

[0016] The target bounding box detection optimization method based on millimeter-wave radar provided in this application has the following beneficial effects:

[0017] This application effectively solves the problem of bounding box size estimation bias caused by point cloud sparsity in traditional methods, improves the accuracy and reliability of target detection, realizes adaptive processing of complex road scenes, effectively reduces the false detection rate, improves the accuracy and stability of detection results, and can effectively improve the perception performance of autonomous driving systems. Attached Figure Description

[0018] Figure 1 The diagram shown is an overall flowchart of a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application.

[0019] Figure 2 The flowchart shown is a process for calculating the percentage of overlapping area between two target boxes in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application.

[0020] Figure 3 The flowchart shown is a process for calculating the intersection area of ​​two target boxes in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application.

[0021] Figure 4 The diagram shows the principle of calculating the overlap area of ​​two target boxes in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application.

[0022] Figure 5 The diagram shows the principle of calculating the overlap area of ​​two target boxes in the projection interval on the separation axis in the target box detection optimization method based on millimeter-wave radar according to an embodiment of this application.

[0023] Figure 6 The flowchart shown is a process for determining the output target box based on the overlap area ratio in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application.

[0024] Figure 7The flowchart shown is a process for updating the initial target box set in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application.

[0025] Figure 8 The diagram shown is a schematic representation of the structure of an electronic device according to an embodiment of this application.

[0026] Component designation explanation

[0027] 100 electronic devices 101 memory 102 processor 103 monitor S100~S300 step S210~S230 step S211~S213 step S310~S370 step Detailed Implementation

[0028] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0029] Existing target bounding box detection technologies have the following drawbacks:

[0030] Traditional millimeter-wave radar point clustering generates target bounding boxes, but the sparsity of the point cloud leads to inaccurate bounding box size estimation, affecting detection accuracy. When targets have orientations, non-axially aligned bounding boxes may overlap. Existing methods, which use fixed thresholds to determine overlap, struggle to adapt to complex road scenarios, resulting in inaccurate and unstable detection results. Current target detection methods lack effective self-calibration mechanisms when multiple calibration objects are present, hindering accurate target identification and classification. Current target detection algorithms lack flexible target ID switching mechanisms when handling complex interchange scenarios, failing to effectively cope with changing traffic environments. Furthermore, existing technologies often struggle to balance accuracy and speed in real-time target detection and tracking, failing to meet the target classification and detection requirements in vehicle-side scenarios. Therefore, existing technologies suffer from problems such as biased bounding box size estimation, inaccurate bounding box overlap detection, and a lack of flexible target ID switching mechanisms in millimeter-wave radar target bounding box detection.

[0031] The purpose of this embodiment is to dynamically optimize the target box detection method in order to improve the perception capability and overall performance of the autonomous driving system.

[0032] The following will refer to the appendices in the embodiments of this application. Figure 1 To be continued Figure 8 This application provides a detailed description of the technical solutions in its embodiments. This allows those skilled in the art to understand and implement the millimeter-wave radar-based target box detection optimization method of this embodiment without inventive effort.

[0033] This embodiment provides an optimized target bounding box detection method based on millimeter-wave radar. Figure 1 The flowchart shown is a representation of the target bounding box detection optimization method based on millimeter-wave radar in this application. Figure 1 As shown, the target box detection optimization method based on millimeter-wave radar provided in this application includes the following steps S100 to S300.

[0034] Step S100: Obtain the initial target bounding box set generated based on millimeter-wave radar;

[0035] Step S200: Calculate the percentage of overlapping area between any two target boxes in the initial target box set;

[0036] Step S300: Determine the output target box based on the overlap area ratio and update the initial target box set.

[0037] In this embodiment, the output target box is determined based on the percentage of the overlapping area of ​​the two target boxes. This can effectively solve the problem of bounding box size estimation deviation caused by point cloud sparsity in traditional methods, and improve the accuracy and reliability of target detection.

[0038] The following is in conjunction with the appendix Figure 2 To be continued Figure 7 The above steps S100 to S300 of the target box detection optimization method based on millimeter-wave radar in this embodiment will be described in detail.

[0039] Step S100: Obtain the initial target bounding box set generated based on millimeter-wave radar.

[0040] In this embodiment, an initial set of target bounding boxes generated by point clustering is obtained based on millimeter-wave radar. Specifically, obtaining the initial set of target bounding boxes generated based on millimeter-wave radar includes:

[0041] 1) Collect environmental point cloud data using millimeter-wave radar.

[0042] For example, configure the millimeter-wave radar operating mode, such as selecting "point cloud output mode," and set the detection range, such as the range threshold, angle range, and data output frequency. Environmental point cloud data is acquired via a 77GHz forward-facing radar, with the point cloud data ranging from -5° to 35° elevation angle and 0° to 360° azimuth angle, and a frequency sweep period of 13Hz. Real-time reception of raw point cloud data output from the millimeter-wave radar is also possible, including the coordinates (X, Y, Z), reflection intensity, velocity, and other information for each point.

[0043] 2) Preprocess the point cloud data to obtain the initial target box set generated by point clustering.

[0044] For example, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm can be used for clustering to obtain an initial set of bounding boxes generated from point cloud clusters. DBSCAN is a density-based clustering algorithm that does not rely on a preset number of clusters but identifies target point clusters through "density connectivity," making it highly suitable for the sparse characteristics of millimeter-wave radar point clouds. In target detection or point cloud clustering, DBSCAN can be used to generate initial bounding boxes from point clouds or 2D point clouds.

[0045] The DBSCAN algorithm parameters, neighborhood radius ε and minimum number of neighbors MinPts for core points, are set as follows: The DBSCAN clustering process for generating initial bounding boxes is as follows:

[0046] Traverse all points and mark "core points" (number of points in their ε-neighborhood ≥ MinPts), "boundary points" (points within the neighborhood of core points but not core points themselves), and "noise points" (neither core nor boundary points). Starting from a core point, continuously merge all connected points within its ε-neighborhood to form a complete point cluster; noise points are directly discarded and do not participate in subsequent bounding box generation. For each valid point cluster, extract its extreme coordinates in 3D space (X, Y, Z axes), X... min X max Y min Y max Z min Z max Each point cluster corresponds to a potential target. For each cluster, calculate the axis-aligned bounding box (AABB) or minimum bounding box (OBB), for example, using these extreme values ​​as boundaries to generate an axis-aligned cuboid target box (AABB) with its center coordinates at ((X...)). min +X max ) / 2, (Y min +Y max ) / 2, (Z min +Z max ) / 2), with dimensions (X max -X min Y max -Y min Z max -Z min Generate target boxes, repeat the above process, and obtain the initial set of target boxes generated by point clustering.

[0047] Step S200: Calculate the percentage of overlapping area between any two target boxes in the initial target box set.

[0048] In this embodiment, the target boxes of the initial target box set are traversed, and the actual overlap area ratio of each pair of target boxes is calculated.

[0049] Figure 2 This is a flowchart illustrating the calculation of the overlap area ratio of two target boxes in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application. Figure 2 As shown, in one implementation of this application, calculating the percentage of overlapping area between two target boxes in the initial target box set includes the following steps S210 to S240.

[0050] Step S210: Calculate the intersection area of ​​the two target boxes;

[0051] Step S220: Calculate the minimum bounding rectangle area of ​​the two target boxes;

[0052] Step S240: Obtain the percentage of overlapping area between the two target boxes based on the ratio of the intersection area to the area of ​​the minimum bounding rectangle.

[0053] In this embodiment, the separating axis theorem is used to calculate the overlapping area of ​​two rectangles with facing angles. Figure 3 The flowchart shown is a process for calculating the intersection area of ​​two target boxes in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application. Figure 3 As shown, in one implementation of this application, calculating the intersection area of ​​the two target boxes includes the following steps S211 to S213.

[0054] Step S211: Determine the normal directions corresponding to the four sides of the rectangle of each target box as the separation axis;

[0055] Step S212: Obtain the coordinates of the four vertices of the rectangle of each target box, and project all vertices of the rectangle of each target box onto the corresponding separation axis to obtain their respective projection ranges;

[0056] Step S213: Determine whether the two target boxes overlap based on the projection interval on the separation axis, and calculate the intersection area of ​​the two target boxes in the overlapping state.

[0057] In this embodiment, the overlapping area between two rotated rectangles (i.e., non-axis-aligned rectangles) is calculated based on the Separating Axis Theorem (SAT). For two convex polygons (such as rectangles), if there exists an axis (i.e., the separating axis) such that the projections of the two polygons onto this axis do not overlap, then they do not intersect. For rectangles, the separating axes are the normal directions of the two sides of each rectangle (a total of 4 axes).

[0058] This embodiment uses the normals of the four sides of the target bounding box as separation axes and derives the overlapping area by utilizing the overlap of vertex projection intervals. For a rectangular target bounding box, only the normal directions corresponding to its four sides (a total of 4 separation axes) need to be checked, eliminating the need to check all possible axes and greatly simplifying the calculation.

[0059] Taking two bounding boxes A and B as examples, the normal directions of their four sides are extracted as separation axes, resulting in a total of 8 candidate separation axes. If the two boxes are axis-aligned, only 4 are needed. For rectangles, the normal direction of each side is perpendicular to that side. For example, if one side of a rectangle is along the positive X-axis, its normal direction is along the positive Y-axis. If the bounding boxes are axis-aligned (AABB), the separation axes are directly the X-axis and Y-axis (because the normals of the four sides only correspond to these two directions).

[0060] Figure 4 This diagram illustrates the principle of calculating the overlap area of ​​two target boxes in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application. Figure 4 As shown, each target box is a rectangle. The normal directions corresponding to the four sides of the two rectangles are determined as separation axes, for a total of 4 axes. Each rectangle has 2 unique normal axes. Then, the coordinates of the four vertices of each rectangle are calculated, which can be derived based on the center point, half width, half height, and orientation angle. Then, all vertices of the two rectangles are projected onto each separation axis to obtain their respective projection intervals. Finally, based on the projection intervals on the separation axes, it is determined whether the two target boxes overlap, and the intersection area of ​​the two target boxes is calculated according to the overlap status.

[0061] Specifically, the following is an example of calculating the overlap area of ​​two target boxes:

[0062] For a rectangular target box, calculate the two-dimensional coordinates of its four vertices based on its center coordinates, dimensions, and rotation angle (θ, θ=0 when aligned with axes). Let rectangle R1 have center (cx1, cy1), half-width w1 / 2, half-height h1 / 2, and facing angle θ1; the normal axes of its sides are (cosθ1, sinθ1) and (-sinθ1, cosθ1).

[0063] Let rectangle R2 have center (cx2, cy2), half width w2 / 2, half height h2 / 2, and facing angle θ2; the normal axes of its sides are (cosθ2, sinθ2) and (-sinθ2, cosθ2);

[0064] Calculate the coordinates of vertex R1: (cx1 ± w1 / 2cosθ1 ∓ h1 / 2sinθ1, cy1 ± w1 / 2sinθ1 ± h1 / 2cosθ1);

[0065] Calculate the coordinates of vertex R2: (cx2±w2 / 2cosθ2 ∓ h2 / 2sinθ2, cy2 ± w2 / 2sinθ2 ± h2 / 2cosθ2);

[0066] For each separation axis, project all vertices of the two bounding boxes onto that axis. The projection calculation method is: the projection value of a vertex (x, y) onto axis u(ux, uy) = x × ux + y × uy (i.e., vector dot product). Take the maximum (max_p) and minimum (min_p) projection values ​​for each bounding box to obtain the projection interval [min_p, max_p] of that bounding box on the current separation axis. For each separation axis, check if the projection intervals of the two bounding boxes overlap. If the projection intervals on any axis do not overlap, the two boxes do not overlap, the overlap area is 0, and no further calculation is needed. If the projection intervals on all separation axes overlap, calculate the overlap length for each axis: overlap length = min (box A_max_p, box B_max_p) - max (box A_min_p, box B_min_p) (the result must be ≥ 0, otherwise 0).

[0067] If it is an axis-aligned bounding box (AABB): the overlap area is calculated directly using the overlap length of the X and Y axes. Overlap area = X-axis overlap length × Y-axis overlap length.

[0068] If it is a rotated bounding box (OBB): the intersection area of ​​the polygons needs to be derived using the Shoelace formula by taking the overlap length of all separation axes, and finally the overlap area is obtained.

[0069] When two rectangles (A and B) overlap and the overlapping area is a convex polygon, its vertices include:

[0070] 1) Intersection of the sides of two rectangles: The points where the sides of rectangle A and rectangle B intersect (a maximum of 4 intersection points, since each rectangle has 4 sides, there are a maximum of 4×4=16 combinations, but the actual effective intersection points are usually ≤4).

[0071] 2) Vertices completely contained by the other: Vertices of rectangle A are completely inside rectangle B, or vertices of rectangle B are completely inside rectangle A (this needs to be determined by the containment relationship between the point and the polygon).

[0072] Check the four vertices of rectangle A. If a vertex is inside rectangle B, add it to the overlapping region vertex set. Check the four vertices of rectangle B. If a vertex is inside rectangle A, add it to the overlapping region vertex set. Remove duplicate vertices and arrange the vertices in clockwise or counterclockwise order to ensure a convex polygon is formed.

[0073] Figure 5 This diagram illustrates the principle of calculating the overlap area of ​​two target boxes in the projection interval on the separation axis in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application. Figure 5 As shown, in one implementation of this application, determining whether two target boxes overlap based on the projection interval on the separation axis, and calculating the intersection area of ​​the two target boxes in the overlap state, includes:

[0074] 1) If the projection intervals on any of the separation axes do not overlap, then the intersection area is determined to be 0;

[0075] 2) If the projection intervals on all the separation axes overlap, then the two target boxes are determined to overlap; if the intersection area is a convex polygon, the vertices of the convex polygon are composed of the intersection points of the two rectangular sides and the vertices contained by the other, and the area of ​​the convex polygon is obtained as the intersection area.

[0076] Wherein, if the overlapping area is a convex polygon, the vertices of the convex polygon are composed of the intersection of the sides of the two rectangles and the vertices contained in the other rectangle, and the overlapping area of ​​the two target boxes is calculated using the Gaussian area formula. That is, if the overlapping area is a convex polygon, its vertices are composed of the intersection of the sides of the two rectangles and the vertices contained in the other rectangle, and the overlapping area is calculated using the Gaussian area formula.

[0077] In this embodiment, the projection intervals of the two rectangles are calculated for each of the four separating axes. After confirming that the projections of all axes overlap, the intersection points of the edges are solved, such as the intersection points of the edges of R1 and R2, and the interior points, such as the points where the vertex of R1 is inside R2.

[0078] Sort the vertices of the overlapping region clockwise and calculate the overlapping area using the Shoelace formula:

[0079]

[0080] Where (x1,y1),...,(x n ,y n ) represents the vertices of the overlapping region, xi+1=x1, yi+1=y1.

[0081] Substitute the vertex coordinates into the formula above in order to calculate the value of each pair of adjacent vertices. Then, sum the results, take the absolute value of the sum, multiply it by 1 / 2, and obtain the area of ​​the overlapping region.

[0082] In this embodiment, the method for calculating the minimum bounding rectangle area of ​​the two target boxes is as follows:

[0083] Determine the minimum x-coordinate of all vertices of the two rectangles. min Maximum value x_max The minimum value of the y-coordinate y_ min Maximum value y _max :

[0084] x_ min = min (x1,...,x8), x_ max = max (x1,...,x8);

[0085] y_ min = min(y1,...,y8), y _max = max(y1,...,y8);

[0086] Then the area S_ of the minimum circumscribed rectangle min = (x_ max - x_ min ) * (y_ max - y_ min ).

[0087] Specifically, the percentage of overlapping area is calculated as the ratio of the intersection area to the area of ​​the smallest bounding rectangle, that is:

[0088] Overlapping area ratio η=S_ overlap / S_ min When S_ min When η = 0, η = 0. Where S_ overlap Let S_ be the area of ​​the intersection. min This represents the area of ​​the minimum bounding rectangle.

[0089] This embodiment employs a rotating rectangle overlap calculation method, which calculates the overlap area of ​​two rectangles with orientation angles based on the Separating Axis Theorem (SAT). This effectively solves the problem of boundary size estimation deviation caused by point cloud sparsity in traditional methods, thereby improving the accuracy and reliability of target detection.

[0090] Step S300: Determine the output target box based on the overlap area ratio and update the initial target box set.

[0091] In this embodiment, a dynamic merging strategy is executed based on the percentage of overlapping area. Figure 7 The flowchart shown is a process for determining the output target box based on the overlap area ratio in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application. Figure 7 As shown, in one implementation of this application, the step of determining the output target box based on the overlap area ratio and updating the initial target box set includes the following steps S310 to S340.

[0092] Step S310: Detect whether the percentage of overlapping area is within a preset threshold range: if yes, continue to step S320; if no, continue to step S330 and step S340.

[0093] Step S320: Then retain the target boxes with higher confidence as the output target boxes.

[0094] Step S330: When the percentage of overlapping area is greater than the maximum threshold of the preset threshold range, the two target boxes are merged to form a single target box, and this target box is used as the output target box.

[0095] Merging two bounding boxes (A and B) generates a new bounding box that contains both of them:

[0096] 1) Merging Axis-Aligned Bounding Boxes (AABB): Directly take the extreme values ​​of the two boxes on the X and Y axes to form a new bounding box:

[0097] The coordinates of the top-left corner of the new bounding box are: (min(A.xmin, B.xmin), min(A.ymin, B.ymin))

[0098] The coordinates of the bottom right corner of the new bounding box are: (max(A.xmax,B.xmax),max(A.ymax,B.ymax))

[0099] The center coordinates of the new frame are calculated based on the extreme values ​​mentioned above, and the size is the maximum difference between the width and height.

[0100] 2) Merging rotated bounding boxes (OBBs) requires generating a minimum bounding box that contains all vertices of the two rotated boxes:

[0101] Collect the coordinates of all 8 vertices of the two boxes; calculate the convex hull of these vertices to obtain the smallest convex polygon containing all vertices; fit the smallest bounding rectangle (with rotation angle) to the convex hull as the merged target box.

[0102] In addition to spatial coordinates, other attributes of the two target bounding boxes can also be fused, such as the velocity and reflection intensity of millimeter-wave radar:

[0103] a) Velocity fusion: Take the average velocity of the two boxes (if they are for the same target, the velocities should be close);

[0104] b) Confidence fusion: If the box has a confidence score, the maximum value or weighted average value can be taken;

[0105] c) Point cloud count: The number of point clouds within the merged box is the sum of the point cloud counts of the two boxes (duplicates need to be removed).

[0106] Step S340: When the percentage of overlapping area is less than the minimum threshold of the preset threshold range, the two target boxes are respectively used as output target boxes.

[0107] That is, when the overlap area ratio is greater than the preset threshold, target boxes are merged; when the overlap area ratio is within the preset threshold range, the target box with the highest confidence is retained; when the overlap area ratio is less than the preset threshold, the original state is maintained.

[0108] For example, when the overlap area ratio η is greater than 0.6, target boxes are merged; when the overlap area ratio η is in the range of 0.3 to 0.6, the target box with the highest confidence is retained; when the overlap area ratio η is less than 0.3, the original state is maintained.

[0109] In this embodiment, a multi-frame verification mechanism can be used to confirm the target box. Specifically, the multi-frame verification mechanism can be used to confirm the target box when the overlap area ratio is less than a preset threshold, or when the overlap area ratio is greater than the maximum threshold of the preset threshold range.

[0110] Specifically, in one implementation of this application, the method further includes: comparing the overlapping data of the target box detected in the current frame with the overlapping data of the target box detected in at least one previous frame; when a preset triggering condition is met, merging the target box of the current frame with the target box of at least one previous frame, and using the merged target box as the output target box.

[0111] Specifically, in this embodiment, the target box information detected in the current frame is recorded and compared with the previous two frames. If the same overlap is detected in multiple consecutive frames (e.g., three frames), the target box merging process is triggered. The same overlap means that the preset triggering conditions are met simultaneously.

[0112] In one implementation of this application, the preset triggering conditions include: consistent overlap state, consistent overlap region shape, and consistent overlap area.

[0113] 1) Consistent overlap: The overlap states of the two target boxes must be the same, either both overlapping or both non-overlapping;

[0114] 2) Consistent shape of overlapping regions: The vertex set of overlapping regions (after deduplication) is exactly the same or the error is within the threshold range;

[0115] 3) Consistent overlapping area: The difference in overlapping area is within a set threshold range, such as 0.1m² or less.

[0116] Specifically, in this embodiment, a specific implementation method for confirming the target box using a multi-frame verification mechanism is as follows:

[0117] 1) Initialize the frame buffer queue to store the target bounding box overlap information of the most recent three frames. The queue capacity is 3, and only the latest three frames are retained.

[0118] 2) When the overlap area ratio is less than a preset threshold, for example, when the current frame detects an overlap area ratio η1 < 0.3, record the following information and store it in the queue:

[0119] The IDs of the two overlapping bounding boxes (ID1, ID2);

[0120] The position coordinates of the two bounding boxes are (cx1, cy1) and (cx2, cy2).

[0121] The dimensions of the two bounding boxes are (w1, h1) and (w2, h2).

[0122] The orientation angles θ1 and θ2 of the two target boxes;

[0123] The overlap area ratio η1 of the current frame.

[0124] 3) Determine if the queue already contains the data of the first two frames, i.e., if the queue length is ≥ 3:

[0125] If the cache is not full, such as if there are only 1 or 2 frames of data, the merging process will not be performed, and the new frame information will continue to be cached.

[0126] If the queue is full, extract the overlapping information of the first three frames: η1 is the current frame, η2 is the previous frame, and η3 is the previous two frames.

[0127] 4) Verify the overlap consistency of three consecutive frames:

[0128] 4-1) Consistent overlapping states:

[0129] Consistency condition 1: The IDs of the overlapping target boxes in the three frames are all ID1 and ID2, and there are no newly added or disappeared target boxes;

[0130] 4-2) The overlapping areas have the same shape:

[0131] Consistency condition 2: The positional deviation between two target boxes in three frames is ≤ a preset threshold (e.g., x / y direction deviation ≤ 0.5m);

[0132] Consistency condition 3: The size deviation of two target boxes in three frames is ≤ a preset threshold (e.g., width / height deviation ≤ 10%).

[0133] Consistency condition 4: The orientation angle deviation between two target boxes in three frames is ≤ a preset threshold (e.g., angle deviation ≤ 5°).

[0134] 4-2) Consistent overlapping area:

[0135] Consistency condition 5: The overlap area ratio of the three frames all satisfy η1, η2, η3 < 0.3, and the numerical fluctuation is ≤ 0.1, such as η1 = 0.25, η2 = 0.23, η3 = 0.27, and the fluctuation is within the allowable range.

[0136] If all five consistency conditions are met, it is determined that the same overlap is detected in three consecutive frames, triggering a merging process. Even if η < 0.3 for a single frame, frame merging is still performed. If any condition is not met, the earliest frame data in the queue is cleared, the latest two frames are retained, and the next frame information is cached.

[0137] After the merge process is executed, the frame buffer queue is cleared, and the monitoring of the overlap of the new target boxes is restarted.

[0138] This embodiment achieves adaptive processing of complex road scenes based on the overlap area ratio and multi-frame verification mechanism, effectively reducing the false detection rate and improving the accuracy and stability of target box detection results.

[0139] In this embodiment, after determining the target bounding boxes, the target bounding box list can be updated, and then the optimized target bounding boxes, i.e., the target detection results, can be output.

[0140] After determining the output target bounding boxes, updating the target bounding box list is a crucial step in maintaining the continuity and accuracy of target tracking. In this embodiment, the filtered and merged valid target bounding boxes are combined with historical tracking information to form the latest target status list.

[0141] Figure 7 The flowchart shown is a process for updating the initial target box set in a target box detection optimization method based on millimeter-wave radar according to an embodiment of this application. Figure 7 As shown, in this embodiment, updating the initial target box set includes:

[0142] Step S350: Match any target box in the previous frame with any target box in the target box list of the previous frame in the initial target box set, and determine whether the two target boxes are the same target: if yes, continue to step S360: update the target state corresponding to the target box list, wherein the target state includes the target position, target confidence and target velocity; if no, continue to step S370: confirm that the target box is a new target, and add the target box to the target box list.

[0143] In this embodiment, the specific process of updating the initial target box set includes:

[0144] 1) Relationship between new and old goals

[0145] By matching the bounding boxes output in the current frame with the list of bounding boxes in the previous frame, we can determine which targets are consecutive occurrences of the same object.

[0146] Matching metrics include, but are not limited to: IoU (Intersection over Union), centroid distance, and velocity consistency.

[0147] Matching rule: When the IoU of the target bounding boxes in two frames is ≥0.3 (adjustable), they are determined to be the same target.

[0148] 2) Target status update

[0149] For targets that have been successfully associated, merge information from the current frame and historical frames to update their status:

[0150] Location: Take the weighted average of the current bounding box and the historical predicted locations, such as using Kalman filtering;

[0151] Confidence level: dynamically adjusted based on the number of consecutively occurring frames; for example, the confidence level increases by 0.1 for each consecutively occurring frame.

[0152] Velocity: Calculated based on positional changes between consecutive frames, applicable to the velocity measurement characteristics of millimeter-wave radar.

[0153] 3) New target processing

[0154] New target: A bounding box in the current frame that does not match any historical target is marked as a new target and added to the list;

[0155] In addition, this embodiment can also handle the disappearance of targets: historical targets that have not been matched for N consecutive frames (e.g., 3 frames) are removed from the target box list to avoid redundancy in the target box list.

[0156] The update of the target box list in this embodiment can form a complete closed loop of "association of old and new targets - status update - addition / disappearance judgment".

[0157] This embodiment optimizes the update process of the target box list by introducing a multi-frame verification mechanism and a dynamic threshold strategy, which can improve the system's target ID switching capability in complex interchange scenarios and reduce the occurrence of target recognition errors.

[0158] The method in this embodiment, while ensuring accuracy, employs an efficient algorithm design, significantly reducing the complexity of the detection process and meeting the real-time target classification and detection requirements in vehicle-side scenarios. Compared to traditional methods that use fixed thresholds to determine overlap, the dynamic merging strategy in this embodiment can more flexibly adapt to different degrees of overlap, effectively improving the flexibility and adaptability of target detection and overcoming the limitations of complex road environments.

[0159] The scope of protection of the target box detection optimization method based on millimeter-wave radar described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0160] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the target box detection optimization method based on millimeter-wave radar provided in any embodiment of this application.

[0161] In the embodiments of this application, any combination of one or more storage media may be used. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0162] This application also provides an electronic device. Figure 8 The diagram shown is a structural schematic of the electronic device 100 provided in an embodiment of this application. In some embodiments, the electronic device may be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), or other terminal device. Furthermore, the target bounding box detection optimization method based on millimeter-wave radar provided in this application can also be applied to databases, servers, and service response systems based on terminal artificial intelligence. This application does not impose any limitations on the specific application scenarios of the target bounding box detection optimization method based on millimeter-wave radar.

[0163] like Figure 8 As shown, the electronic device 100 provided in this application embodiment includes a memory 101 and a processor 102.

[0164] The memory 101 is used to store computer programs; preferably, the memory 101 includes various media that can store program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card or optical disk.

[0165] Specifically, memory 101 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic device 100 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 101 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0166] The processor 102 is connected to the memory 101 and is used to execute the computer program stored in the memory 101 so that the electronic device 100 executes the target box detection optimization method based on millimeter-wave radar provided in any embodiment of this application.

[0167] Optionally, the processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0168] Optionally, in this embodiment, the electronic device 100 may further include a display 103. The display 103 is communicatively connected to the memory 101 and the processor 102, and is used to display the relevant GUI interactive interface of the target box detection optimization method based on millimeter-wave radar.

[0169] In summary, this application employs a rotating rectangle overlap calculation method, using the Separating Axis Theorem (SAT) to calculate the overlap area of ​​two rectangles with facing angles. This effectively solves the problem of bounding box size estimation bias caused by point cloud sparsity in traditional methods, improving the accuracy and reliability of target detection. This application designs a dynamic merging strategy, which, based on the overlap area ratio and multi-frame verification mechanism, achieves adaptive processing for complex road scenes, effectively reducing the false detection rate and improving the accuracy and stability of detection results. By introducing a multi-frame verification mechanism and a dynamic threshold strategy, this application optimizes the target box list update process, improving the system's target ID switching capability in complex interchange scenarios and reducing target recognition errors. While ensuring accuracy, this application uses an efficient algorithm design, significantly reducing the complexity of the detection process and meeting the needs of real-time target classification and detection in vehicle-side scenarios. Compared to the fixed threshold method used in traditional methods to determine overlap, this application's dynamic merging strategy can more flexibly adapt to different degrees of overlap, effectively improving the flexibility and adaptability of target detection and overcoming the limitations of complex road environments. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0170] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A target bounding box detection optimization method based on millimeter-wave radar, characterized in that, include: Obtain the initial set of target bounding boxes generated based on millimeter-wave radar; Calculate the percentage of overlap between any two target boxes in the initial target box set; The output target bounding box is determined based on the overlap area ratio, and the initial target bounding box set is updated. It also includes: comparing the overlapping data of the target box detected in the current frame with the overlapping data of the target box detected in at least one previous frame; when a preset triggering condition is met, merging the target box of the current frame with the target box of at least one previous frame, and using the merged target box as the output target box. The preset triggering conditions include: consistent overlap state, consistent overlap region shape, and consistent overlap area.

2. The target bounding box detection optimization method based on millimeter-wave radar according to claim 1, characterized in that, The calculation of the overlap area ratio of two target boxes in the initial target box set includes: Calculate the intersection area of ​​the two target boxes; Calculate the area of ​​the minimum bounding rectangle of the two target boxes; The percentage of overlap between the two target boxes is obtained based on the ratio of the intersection area to the area of ​​the minimum bounding rectangle.

3. The target bounding box detection optimization method based on millimeter-wave radar according to claim 2, characterized in that, The calculation of the intersection area of ​​the two target boxes includes: The normal directions corresponding to the four sides of the rectangle of each target box are determined as the separation axes; The coordinates of the four vertices of the rectangle of each target box are obtained respectively, and all vertices of the rectangle of each target box are projected onto the corresponding separation axis to obtain their respective projection ranges; Based on the projection interval on the separation axis, determine whether the two target boxes overlap, and calculate the intersection area of ​​the two target boxes in the overlapping state.

4. The target bounding box detection optimization method based on millimeter-wave radar according to claim 3, characterized in that, The step of determining whether two target boxes overlap based on the projection interval on the separation axis, and calculating the intersection area of ​​the two target boxes in the overlapping state, includes: If the projection intervals on any of the separation axes do not overlap, then the intersection area is determined to be 0; If all the projection intervals on the separation axes overlap, then the two target boxes are determined to overlap; if the intersection area is a convex polygon, the vertices of the convex polygon are composed of the intersection points of the two rectangular sides and the vertices contained by the other, and the area of ​​the convex polygon is obtained as the intersection area.

5. The target bounding box detection optimization method based on millimeter-wave radar according to claim 1, characterized in that, The step of determining the output target box based on the overlap area ratio and updating the initial target box set includes: Detect whether the percentage of overlapping area is within a preset threshold range: If so, retain the target box with higher confidence as the output target box; If not, when the percentage of overlapping area is greater than the maximum threshold of the preset threshold range, the two target boxes are merged to form a single target box, and this target box is used as the output target box; when the percentage of overlapping area is less than the minimum threshold of the preset threshold range, the two target boxes are used as the output target boxes respectively.

6. The target bounding box detection optimization method based on millimeter-wave radar according to claim 1, characterized in that, Updating the initial set of target boxes includes: Match any target bounding box in the previous frame with any target bounding box in the target bounding box list of the previous frame in the initial target bounding box set, and determine whether the two target bounding boxes are the same target: If so, update the corresponding target status in the target box list, wherein the target status includes the target position, target confidence level, and target velocity; If not, then confirm that the target box is a new target and add the target box to the target box list.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the target box detection optimization method based on millimeter-wave radar as described in any one of claims 1 to 6.

8. An electronic device, characterized in that, The electronic device includes: Processor and memory; The memory stores program instructions; The processor is configured to run the program instructions to execute the target box detection optimization method based on millimeter-wave radar as described in any one of claims 1 to 6.

Images