Cart 3D frame correction method and system based on multi-sensor fusion

Abstract

The invention provides a cart 3D frame correction method and system based on multi-sensor fusion, and relates to the technical field of automatic driving environment awareness, and the method comprises the steps: obtaining a detection result outputted by a BEV model and a 3D detection frame generated by a laser radar point cloud model; performing time synchronization processing on the detection result and the 3D detection frame to obtain multi-source detection data after time alignment; screening out a to-be-evaluated target from the multi-source detection data according to a preset triggering condition; projecting a to-be-evaluated target to the laser radar coordinate system, and performing time compensation on the spatial position by using the speed of the to-be-evaluated target; calculating the space overlapping proportion of the BEV detection frame of the target to be evaluated after time compensation and the laser radar 3D detection frame; according to the space overlapping proportion, whether frame splitting exists in the to-be-evaluated target is judged; if yes, replacing the plurality of split laser radar 3D detection frames with a BEV detection frame to generate a corrected 3D detection frame; and otherwise, outputting the laser radar 3D detection frame.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving environmental perception technology, and in particular to a method and system for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion. Background Technology

[0002] In autonomous driving systems, accurately and stably perceiving the three-dimensional position and shape of surrounding vehicles is fundamental for safety decision-making and planning control. For large vehicles (such as trucks and buses), due to their length and complex structure, the accuracy of their 3D detection bounding boxes is particularly crucial for collision avoidance and ensuring driving safety. In practical applications, due to sensor characteristics, environmental interference, and algorithm limitations, 3D detection of large vehicles still faces stability issues, often resulting in a "box splitting" phenomenon where the same vehicle is identified as multiple incomplete bounding boxes, severely impacting the reliability of subsequent target tracking and behavior prediction.

[0003] Currently, LiDAR point clouds are commonly used for 3D target detection. This type of method directly generates 3D detection boxes by clustering or deep learning regression of the point cloud, and has advantages in ranging accuracy.

[0004] However, in existing technologies, LiDAR is prone to "box splitting" when detecting large vehicles due to point cloud fragmentation. This causes the same target to be identified as multiple incomplete 3D boxes, leading to tracking failures such as ID jumps and speed reversals, which seriously affect the decision-making safety of autonomous driving. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion. This method can solve the technical problem that in the prior art, LiDAR detection of large vehicles is prone to "box splitting" due to point cloud fragmentation, which causes the same target to be identified as multiple incomplete 3D bounding boxes, resulting in tracking failures such as ID jumps and speed reversals, which seriously affect the decision-making safety of autonomous driving.

[0006] A first aspect of this invention proposes a method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion, comprising: S1: Obtain the detection results output by the BEV model and the 3D detection box generated by the LiDAR point cloud model; S2: Perform time synchronization processing on the detection results and 3D detection boxes to obtain time-aligned multi-source detection data; S3: Select the target to be evaluated from the multi-source detection data according to the preset trigger conditions; S4: Project the target to be evaluated onto the lidar coordinate system and use the velocity of the target to be evaluated to perform time compensation on the spatial position; S5: Calculate the spatial overlap ratio between the BEV detection box and the 3D detection box of the target to be evaluated after time compensation; S6: Based on the spatial overlap ratio, determine whether the target to be evaluated has a box split; if so, replace the multiple split LiDAR 3D detection boxes with BEV detection boxes to generate corrected 3D detection boxes; otherwise, output the LiDAR 3D detection boxes.

[0007] A second aspect of the present invention provides a 3D frame correction system for a large vehicle based on multi-sensor fusion, comprising: a processor and a memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the large vehicle 3D frame correction method based on multi-sensor fusion as described in the first aspect.

[0008] A third aspect of the present invention provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the large vehicle 3D frame correction method based on multi-sensor fusion as described in the first aspect.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, by fusing BEV detection and LiDAR 3D bounding boxes, and combining preset trigger conditions and an overlap ratio judgment mechanism, selective correction is applied to large vehicle targets. This effectively solves the "bounding box splitting" problem caused by LiDAR point cloud fragmentation, thereby avoiding the incorrect association of the same physical target with multiple IDs, leading to repeated tracking and contradictory motion states. While suppressing bounding box splitting and ensuring the stability of target IDs and the consistency of motion states, the selective correction mechanism preserves the original high-precision advantage of LiDAR, significantly improving the system's perception robustness and the safety of autonomous driving decisions in complex scenarios. Attached Figure Description

[0010] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0011] Figure 1 This is a flowchart illustrating a method for correcting the 3D frame of a large vehicle based on multi-sensor fusion, provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of frame overlap calculation provided in an embodiment of the present invention.

[0013] Figure 3 This is a schematic diagram of a 3D frame correction system for a large vehicle based on multi-sensor fusion, provided in an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] The following description, in conjunction with the accompanying drawings, details the method for correcting the 3D frame of a large vehicle based on multi-sensor fusion, through specific embodiments and application scenarios.

[0016] Reference manual attached Figure 1 The diagram shows a flowchart of a method for correcting the 3D frame of a large vehicle based on multi-sensor fusion, provided by an embodiment of the present invention.

[0017] This invention provides a method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion, which may include the following steps: S1: Obtain the detection results output by the BEV model and the 3D detection bounding box generated by the LiDAR point cloud model.

[0018] The BEV model refers to a deep learning perception model based on bird's-eye view. It typically uses forward-looking or multi-view camera images as input, transforms and infers them through a neural network to a two-dimensional space from a top-down perspective, and outputs the detection boxes, categories, and attribute information of various obstacles on this top-down plane.

[0019] The detection results specifically refer to the perception results output by the BEV model.

[0020] Among them, the LiDAR point cloud model refers to an algorithm or model for three-dimensional target detection based on LiDAR point cloud data.

[0021] Among them, the 3D detection box refers to the three-dimensional bounding box output by the LiDAR point cloud model, which is used to represent the position, size and orientation of an obstacle in three-dimensional space.

[0022] In one possible implementation, the detection results output by the BEV model are cached in a history buffer for a preset duration.

[0023] It should be noted that those skilled in the art can set the preset duration according to actual needs, and this invention does not limit this.

[0024] The history buffer refers to a data structure or memory area used to temporarily store historical detection results output by the BEV model. Its core function is to retain BEV perception data from a past period to support time synchronization with asynchronous LiDAR detection data.

[0025] Among them, the cache duration refers to the time span during which the BEV detection results are stored in the historical buffer.

[0026] For example, using HistoryBufferT to cache BEV detection results, the cache time is 0.3 seconds, and the LiDAR point cloud model generates a 3D detection bounding box.

[0027] In one possible implementation, the 3D detection box generated by the lidar point cloud model is replaced by a 3D detection box generated by millimeter-wave radar clustering.

[0028] Among them, millimeter-wave radar clustering refers to a technique used in alternative implementations to generate initial 3D detection boxes. Specifically, it involves using target reflection point clouds acquired by millimeter-wave radar (Radar), grouping radar points belonging to the same physical target into groups using clustering algorithms, and estimating the target's position, size, and heading in three-dimensional space based on these point sets, thereby forming a 3D detection box.

[0029] Specifically, millimeter-wave radar replaces lidar for initial clustering (suitable for rainy or foggy days).

[0030] S2: Perform time synchronization processing on the detection results and 3D detection boxes to obtain time-aligned multi-source detection data.

[0031] Time synchronization refers to the process of aligning data from different sensors or sensing models, which have different acquisition or processing timestamps, in a consistent time dimension using a specific alignment method.

[0032] In one possible implementation, S2 specifically involves: completing time synchronization by finding the detection result that is closest to the detection time of the 3D detection box.

[0033] For example, use GetIndexClosestToTime to get the most recent BEV detection.

[0034] In this embodiment of the invention, time synchronization is achieved by finding the BEV detection result closest to the LiDAR detection time, which effectively eliminates the time asynchrony error between multi-sensor data and provides a reliable time reference for subsequent accurate spatial overlap calculation and frame split determination, thereby ensuring the accuracy of the correction algorithm and the real-time performance of the system.

[0035] S3: Select the target to be evaluated from the multi-source detection data according to the preset trigger conditions.

[0036] Among them, the target to be evaluated refers to a single BEV detection target that is identified as needing to undergo frame split risk assessment after being screened by preset trigger conditions.

[0037] Optionally, the preset trigger conditions include: The target type is large vehicle.

[0038] The deviation between the target heading angle and the vehicle's heading angle is within the preset angle range.

[0039] The target's position in the vehicle coordinate system is located within a preset rectangular area.

[0040] It should be noted that those skilled in the art can set the preset angle range and the size of the preset rectangular area according to actual needs, and the present invention does not limit these settings.

[0041] Among them, the autonomous vehicle coordinate system refers to a right-handed three-dimensional Cartesian coordinate system established with the autonomous vehicle itself as the reference origin.

[0042] Specifically, the trigger condition range is set as follows: the BEV is a large vehicle type. If there is a large vehicle directly in front (heading ±15°). Within a certain rectangular area in front of and to the left and right of the vehicle: 0 ≤ x And -3.5≤ y ≤3.5. If both conditions are met, proceed to step S4.

[0043] In this embodiment of the invention, based on a screening mechanism with preset triggering conditions, the invention achieves precise triggering and selective processing of target correction for large vehicles, effectively avoiding unnecessary computational overhead and the risk of erroneous correction.

[0044] S4: Project the target to be evaluated onto the lidar coordinate system and use the target's velocity to perform time compensation on its spatial position. The lidar coordinate system refers to a right-handed three-dimensional Cartesian coordinate system with the lidar sensor's mounting center as the origin. The axes of this coordinate system are typically aligned with the physical orientation of the sensor itself.

[0045] Time compensation refers to the calculation process used in sensor fusion to correct target position deviations caused by delays in sensor data acquisition, processing, or transmission.

[0046] Spatial position refers to the three-dimensional coordinates of the target being evaluated at a specific moment on its trajectory.

[0047] In this embodiment of the invention, by projecting the target to be evaluated onto the lidar coordinate system and performing velocity-time compensation, precise spatiotemporal alignment of cross-sensor data is achieved. This not only eliminates spatial misalignment and time delay errors caused by differences in coordinate systems and acquisition timing, but also ensures the accuracy of subsequent box overlap calculation and split determination. As a result, the robustness of large vehicle target correction and the reliability of fusion results in dynamic scenes are significantly improved.

[0048] Reference manual attached Figure 2 The diagram illustrates a frame overlap calculation method provided by an embodiment of the present invention.

[0049] S5: Calculate the spatial overlap ratio between the BEV detection frame and the LiDAR 3D detection frame of the target to be evaluated after time compensation.

[0050] Among them, the BEV detection box refers to the two-dimensional rectangular box output by the BEV perception model that represents the outer contour of the target object on the bird's-eye view plane.

[0051] Among them, the lidar 3D detection box refers to the cuboid box that represents the outer contour of the target object in three-dimensional space, directly output by the lidar point cloud perception model.

[0052] Among them, the spatial overlap ratio refers to the ratio of the area of ​​the overlapping region between a BEV detection box and a LiDAR 3D detection box in the same coordinate system to the area of ​​the smaller of the two.

[0053] Specifically, the overlapping area of ​​the two predicted contours is calculated, and the overlap ratio is also calculated. The formula for calculating the overlap ratio is: .

[0054] in, This indicates the BEV detection box. Indicates the first i A 3D detection frame for LiDAR. Indicates the area of ​​the overlapping region. This indicates the area of ​​the BEV detection frame. This represents the area of ​​the LiDAR 3D detection frame. This indicates taking the minimum value. express and The overlap ratio between them.

[0055] In this embodiment of the invention, by calculating the normalized spatial overlap ratio between the BEV detection box and the LiDAR 3D detection box, an objective and quantitative metric is provided for box splitting determination. This ratio effectively characterizes the geometric consistency of detection boxes from different sources, enabling the system to accurately identify multiple LiDAR boxes that highly overlap with the same BEV box. This accurately triggers subsequent correction logic, avoiding misjudgments and omissions that may be caused by subjective thresholds or simple intersection judgments, and significantly improving the accuracy of box splitting detection and the overall reliability of the algorithm.

[0056] S6: Based on the spatial overlap ratio, determine whether the target to be evaluated has a split bounding box. If so, replace the multiple split LiDAR 3D bounding boxes with BEV bounding boxes to generate corrected 3D bounding boxes. Otherwise, output the LiDAR 3D bounding boxes.

[0057] Among them, box splitting refers to the phenomenon in LiDAR 3D target detection where a single physical target is incorrectly detected as two or more incomplete and discontinuous 3D bounding boxes due to uneven point cloud distribution, occlusion, or algorithm limitations.

[0058] The corrected 3D detection box refers to the unified and complete three-dimensional bounding box generated after processing by the method of the present invention, which is used to replace the original split box.

[0059] In this embodiment of the invention, by intelligently replacing the split frames of the LiDAR with BEV detection frames, the frame splitting phenomenon in the perception of large vehicle targets is fundamentally eliminated, ensuring the continuous stability of target ID and motion state, and significantly improving the reliability of the perception system output and the accuracy of downstream fusion tracking.

[0060] Optionally, the specific rule for determining box splitting is as follows: one BEV detection box corresponds to at least two LiDAR 3D detection boxes, and the spatial overlap ratio between the BEV detection box and the corresponding LiDAR 3D detection box is greater than a set threshold.

[0061] It should be noted that those skilled in the art can set the threshold value according to actual needs, and this invention does not limit it.

[0062] Specifically, one BEV box corresponds to multiple (>=2) LiDAR boxes. If there is box splitting, the split boxes in LiDAR Detect are combined into one BEV box.

[0063] In one possible implementation, the step of replacing the split multiple LiDAR 3D detection boxes with BEV detection boxes to generate corrected 3D detection boxes specifically includes sub-steps S601 to S603: S601: Project the BEV detection box of the target to be evaluated onto the lidar coordinate system.

[0064] S602: Time compensation is performed on the projected detection frame using the velocity information of the target to be evaluated.

[0065] S603: Replace the split multiple LiDAR 3D detection boxes with the projected and compensated BEV detection boxes to generate corrected 3D detection boxes.

[0066] In this embodiment of the invention, by using a frame splitting determination and selective replacement mechanism based on spatial overlap ratio, multiple split LiDAR frames belonging to the same physical entity can be accurately identified, and a more complete BEV detection frame can be used to replace and fuse them, thereby effectively eliminating target ID jumps, trajectory breaks and speed estimation contradictions caused by frame splitting, generating a unified and stable corrected 3D detection frame, providing high-quality input for the downstream perception fusion module, and significantly improving the stability and safety of the entire autonomous driving system in multi-target tracking, especially in large vehicle following scenarios.

[0067] In one possible implementation, the process after S6 includes: S7: Perform post-fusion processing on the corrected 3D detection frame or LiDAR 3D detection frame.

[0068] Post-fusion processing refers to the entire process in an autonomous driving perception system of fusing data from multiple independent perception sources, based on temporal and spatial alignment, and through algorithms such as data association and state estimation, to generate a unified, stable, and accurate list of dynamic obstacle information.

[0069] Specifically, the observation noise from the modified 3D detection box is adaptively adjusted. In the context of state estimation, observation noise refers to the statistical representation of the uncertainty or error of sensor observations (i.e., measurements), which is usually mathematically modeled in the form of a covariance matrix.

[0070] The selective correction of the lidar model's bounding box can be performed within the lidar sensing model or in the lidar sensing post-processing step. Finally, the corrected lidar sensing results are sent to the post-sensing fusion process as one of the inputs for post-fusion.

[0071] Furthermore, since the corrected 3D detection box originates from the BEV perception model, its measurement noise characteristics differ from those of the original LiDAR detection box. To optimize the post-fusion effect, this embodiment adaptively adjusts state estimation algorithms such as Kalman filtering: the observation noise covariance matrix corresponding to the observations from the corrected 3D detection box is adaptively amplified. For example, this noise covariance can be adjusted to N times the noise covariance of the original LiDAR detection box, where N is a real number greater than 1. Through this adaptive adjustment, the post-fusion algorithm can more accurately characterize the uncertainty of measurements from different sources, thereby generating a more stable and smoother target trajectory, further improving the system's tracking robustness in scenarios such as following other vehicles and changing lanes.

[0072] For example, in post-sensory fusion processing, asynchronous detection data from multiple sensors are first time-aligned and interpolated to the same timestamp. Then, the Hungarian algorithm is used to correlate and match targets detected by different sensors to identify observations belonging to the same physical entity. Next, Kalman filtering is used to fuse and update the states of the correlated targets, adaptively adjusting the observation noise from the corrected 3D detection boxes, for example, adjusting its noise covariance to twice that of the uncorrected LiDAR detection box noise covariance. Furthermore, the target trajectory lifecycle is managed, handling target emergence and disappearance. Finally, stable and accurate obstacle dynamic information is output, including position, velocity, heading, and category.

[0073] In this embodiment of the invention, by introducing post-fusion processing and adaptively adjusting the observation noise of the corrected 3D detection box, the perception correction result of the present invention is deeply fused with the downstream multi-target tracking system and the performance is optimized.

[0074] It should be noted that the following alternative solutions can be adopted to further improve performance: Replacing the laser coordinate system projection with BEV coordinate system mapping reduces the cumulative error introduced by coordinate transformation. Optimizing the rectangular trigger area into a fan-shaped area centered on the vehicle with a heading angle of ±20° can increase the coverage of large vehicle targets in curve conditions by approximately 18%. Adjusting the area overlap determination criterion in the overlap ratio calculation to an Intersection over Union (IoU) greater than 0.15 can reduce the false positive rate of frame splitting by approximately 25%. Furthermore, using infrared sensors to assist or replace millimeter-wave radar for compensation fusion enhances the system's perception penetration and robustness in low-visibility environments such as rain and fog.

[0075] In practical applications, by fusing highly complete BEV perception results with high-precision LiDAR point cloud detection, and based on temporal and spatial synchronization, a closed-loop correction process was designed, including target selection, overlap calculation, split determination, and bounding box replacement. This process accurately locates and intelligently replaces only high-risk large vehicle bounding boxes, fundamentally solving tracking failures such as ID jumps and trajectory breaks, while maximizing the preservation of the inherent accuracy advantages of LiDAR. Through backend optimizations such as noise adaptation, seamless collaboration between the correction results and the existing fusion system is ensured, ultimately achieving a significant improvement in the stability of large vehicle target perception and the accuracy of perception across all scenarios.

[0076] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, by fusing BEV detection and LiDAR 3D bounding boxes, and combining preset trigger conditions and an overlap ratio judgment mechanism, selective correction is applied to large vehicle targets. This effectively solves the "bounding box splitting" problem caused by LiDAR point cloud fragmentation, thereby avoiding the incorrect association of the same physical target with multiple IDs, leading to repeated tracking and contradictory motion states. While suppressing bounding box splitting and ensuring the stability of target IDs and the consistency of motion states, the selective correction mechanism preserves the original high-precision advantage of LiDAR, significantly improving the system's perception robustness and the safety of autonomous driving decisions in complex scenarios.

[0077] Reference manual attached Figure 3 The diagram shows a structural schematic of a large vehicle 3D frame correction system based on multi-sensor fusion provided by an embodiment of the present invention.

[0078] This invention provides a 3D frame correction system 20 for large vehicles based on multi-sensor fusion, comprising: a processor 201 and a memory 202; The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-mentioned method for correcting the 3D frame of a large vehicle based on multi-sensor fusion and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0079] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0080] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0081] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0082] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0083] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0085] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0086] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0087] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0088] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] This invention provides a readable storage medium that stores a program or instructions on the storage medium. When the program or instructions are executed by a processor, they implement the steps of the above-described method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion, and achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion, characterized in that, include: S1: Obtain the detection results output by the BEV model and the 3D detection box generated by the LiDAR point cloud model; S2: Perform time synchronization processing on the detection results and the 3D detection box to obtain time-aligned multi-source detection data; S3: Select the target to be evaluated from the multi-source detection data according to the preset triggering conditions; S4: Project the target to be evaluated onto the lidar coordinate system, and use the velocity of the target to be evaluated to perform time compensation on the spatial position; S5: Calculate the spatial overlap ratio between the BEV detection box and the 3D detection box of the target to be evaluated after time compensation; S6: Based on the spatial overlap ratio, determine whether the target to be evaluated has a box split; if so, replace the split multiple lidar 3D detection boxes with the BEV detection box to generate a corrected 3D detection box; Otherwise, output the 3D detection frame of the LiDAR.

2. The method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion according to claim 1, characterized in that, The detection results output by the BEV model are cached in the history buffer for a preset duration.

3. The method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion according to claim 1, characterized in that, The 3D detection box generated by the lidar point cloud model is replaced by the 3D detection box generated by millimeter-wave radar clustering.

4. The method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion according to claim 1, characterized in that, S2 specifically involves: completing time synchronization by finding the detection result that is closest to the detection time of the 3D detection box.

5. The method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion according to claim 1, characterized in that, The preset triggering conditions include: The target type is large vehicle type; The deviation between the target heading angle and the vehicle's heading angle is within the preset angle range; The target's position in the vehicle coordinate system is located within a preset rectangular area.

6. The method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion according to claim 1, characterized in that, The specific rule for determining the box splitting is as follows: one BEV detection box corresponds to at least two LiDAR 3D detection boxes, and the spatial overlap ratio between the BEV detection box and the corresponding LiDAR 3D detection box is greater than a set threshold.

7. The method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion according to claim 1, characterized in that, The step of replacing the split plurality of LiDAR 3D detection boxes with the BEV detection box to generate a corrected 3D detection box specifically includes: S601: Project the BEV detection frame of the target to be evaluated onto the lidar coordinate system; S602: Time compensation is performed on the projected detection frame using the velocity information of the target to be evaluated; S603: Replace the split plurality of said LiDAR 3D detection boxes with the projected and compensated BEV detection boxes to generate said corrected 3D detection boxes.

8. The method for correcting the 3D bounding box of a large vehicle based on multi-sensor fusion according to claim 1, characterized in that, Following S6, the following is also included: S7: Perform post-fusion processing on the corrected 3D detection frame or the lidar 3D detection frame.

9. A 3D frame correction system for large vehicles based on multi-sensor fusion, characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the multi-sensor fusion-based 3D frame correction method for large vehicles as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the multi-sensor fusion-based 3D frame correction method for large vehicles as described in any one of claims 1 to 8.

Images