A dynamic environment adaptive AGV multi-sensor perception fusion method

Abstract

The present application belongs to the field of AGV outdoor perception, and specifically relates to a dynamic environment self-adaptive AGV multi-sensor perception fusion method, which comprises the following steps: target obstacles are detected by using a camera, a laser radar and a millimeter wave radar respectively, and multiple sensors are tracked and associated for multiple frames to generate single-sensor tracking results with unique identification ID, and the tracking results of each sensor are converted into a self-vehicle coordinate system through external parameters, and the single-sensor detection tracking results are sent to a fusion module in real time; the fusion module performs asynchronous frame-by-frame fusion on the tracking results of each sensor, associates and fuses target and observation target through two-stage ID matching and data matching, and dynamically updates the state information of the fusion target; the current weather type is inferred in real time based on an environment weather classification model, and the fusion parameter configuration is dynamically switched according to the weather type. The method can perceive the outdoor environment condition in advance, make the AGV adaptively switch the sensor fusion strategy, provide accurate input for AGV obstacle avoidance planning, and improve the driving safety of the AGV.

Description

Technical Field

[0001] This invention belongs to the field of AGV outdoor sensing, specifically a dynamic environment adaptive AGV multi-sensor sensing fusion method. Background Technology

[0002] Conventional AGV outdoor sensing relies on single or multiple sensors such as cameras, LiDAR, and millimeter-wave radar to detect and fuse obstacle positions in real time, providing input for subsequent obstacle avoidance / around planning. However, outdoor weather conditions for AGVs are complex and variable, and the performance of sensor target detection varies depending on the characteristics of each sensor. For example, cameras experience blurred lenses and low target recall in rain and snow. LiDAR experiences increased noise in rain and snow, and severe scattering in foggy weather, reducing target accuracy. Millimeter-wave radar performs worse than the former two in general weather, but it has strong penetration and high detection rate in rain, snow, and fog. Therefore, existing multi-sensor fusion solutions designed only for general scenarios are not suitable for complex and variable outdoor weather environments, reducing the adaptability of AGV sensing functions and thus lowering AGV operational safety. Summary of the Invention

[0003] The purpose of this invention is to provide a dynamic environment adaptive multi-sensor perception fusion method for AGVs, ensuring that AGVs can accurately locate surrounding obstacles during low-speed autonomous driving in various weather conditions. Utilizing the target information provided by this method, the effectiveness of obstacle avoidance / bypass planning and decision-making is improved, ensuring safe driving and overcoming the shortcomings of prior art's multi-sensor perception fusion methods in terms of accuracy loss during dynamic weather changes.

[0004] The technical solution adopted by this invention to achieve the above objectives is: a dynamic environment adaptive AGV multi-sensor perception fusion method, comprising the following steps:

[0005] S1: Detect target obstacles using cameras, lidar, and millimeter-wave radar respectively, perform multi-frame tracking and association of multiple sensors, generate single-sensor tracking results with unique identifiers, convert the tracking results of each sensor into the vehicle coordinate system through external parameters, and send the single-sensor detection and tracking results to the fusion module in real time.

[0006] S2: The fusion module asynchronously fuses the tracking results of each sensor generated in step S1 frame by frame. It associates the fusion target with the observed target through two stages of ID matching and data matching, and dynamically updates the status information of the fusion target.

[0007] S3: During the execution of step S2, the current weather type is inferred in real time based on the environmental weather classification model, and the fusion parameter configuration is dynamically switched according to the weather type.

[0008] The camera and lidar detect target obstacles, specifically as follows:

[0009] Label specific target categories in the AGV factory area and fine-tune the pre-trained target detection model;

[0010] The camera uses the YOLOv10 model for visual detection; the LiDAR uses the PointPillars model for point cloud detection.

[0011] The camera detection bounding box is converted into a 3D detection bounding box using monocular depth ranging.

[0012] Given the pixel difference y between the bottom edge of the camera detection frame and the optical center, the depth is:

[0013] d = f * H / y

[0014] Where f is the focal length and H is the camera height setting;

[0015] The length, width, and height (w, h, l) of the 3D detection box are given using category prior values, and the turning angle yaw is 0 by default, thus completing the conversion of the visual 2D detection box to the 3D detection box.

[0016] The step of performing multi-frame tracking and correlation on multiple sensors to generate a single-sensor tracking result with a unique identifier ID is as follows:

[0017] S1-1: An improved ByteTrack algorithm is used for LiDAR target tracking, with a value range of (-1, 1). The matching index is replaced with the 3D detection bounding box GIoU, i.e.:

[0018]

[0019] Where A and B are the volumes of the two 3D bounding boxes, IoU is the intersection-union ratio of the volumes of the two bounding boxes, and C is the volume of the largest bounding cuboid of the two bounding boxes;

[0020] S1-2: In the improved ByteTrack algorithm, the motion model adopts a uniformly accelerated linear motion Kalman filter;

[0021] S1-3: Confidence filtering of millimeter-wave detection and tracking results.

[0022] The single sensor detection and tracking result is: single sensor detection and tracking result (t, sor, objs), where t is the data source timestamp, sor is the sensor category, and objs is the target set;

[0023] Each target includes information (id,x,y,z,w,h,l,yaw,cls,conf,(u,v,a,h)), where cls is the category, conf is the confidence score, and (u,v,a,h) is the visual 2D detection box information, which is used for matching calculation in the fusion stage.

[0024] Step S2 includes the following steps:

[0025] S2-1: The fusion module collects the detection and tracking results of each sensor. After accumulating N frames, it sorts the frames according to the data source timestamp and discards data that is earlier than the previous batch fusion time.

[0026] S2-2: Set the sensor target as the observation target and the target of the fusion module as the fusion target. The two are associated and matched, and the fusion target is updated with the observation target. When the initial sensor of the fusion target is the same as the sensor of the current frame, and the tracking ID of the fusion target sensor is the same as the tracking ID of the observation target, the two are matched.

[0027] S2-3: Exclude the targets already matched in step S2-2, determine the consistency of category and size between the remaining observed targets and the fused targets, and exclude inconsistent target pairs from association; calculate the association index for each sensor type, apply the association algorithm, and if the association is successful, they are considered to be matched;

[0028] S2-4: Update the data of the fusion targets that have been matched in steps S2-2 to S2-3;

[0029] S2-5: For unmatched observation targets in step S2-3, if they are visual or laser targets, and the confidence level exceeds the set threshold and the continuous observation exceeds the set number of frames, they are initialized as new fusion targets. If they are millimeter-wave targets, it is additionally determined whether the target location is within the drivable area of ​​the factory map.

[0030] S2-6: For fusion targets that were not matched in step S2-3, if the time since the last update exceeds the set time limit threshold, then remove the fusion target.

[0031] S2-7: Send the set of fusion targets (t, objs) with confidence greater than the threshold to the planning and control module in real time; where t is the data source timestamp, objs is the target set, and each target includes information (id, x, y, z, w, h, l, yaw, vx, vy, cls, conf).

[0032] In steps S2-3, the calculation of correlation indicators and application of correlation algorithms according to sensor type specifically includes:

[0033] For the camera, the correlation metric is the pixel 2D detection bounding box IoU, which is associated with the fusion target using the Hungarian algorithm.

[0034] For LiDAR: the 2D detection box in pixel space is obtained by projection, and the association index is calculated as the weighted sum of GIoU2D and GIoUBev; where GIoU2D is the GIoU value of the pixel 2D detection box, and GIoUBev is the GIoU value of the BEV space detection box. The Hungarian algorithm is used to associate it with the fusion target.

[0035] For millimeter-wave radar: millimeter-wave targets with GIoUBev greater than the threshold are all associated with the fused target, allowing one-to-many matching relationships.

[0036] Step S2-4 specifically includes:

[0037] For position (x, y, z) and velocity (vx, vy, vz): the values ​​are updated using an error state Kalman filter; if the current frame is a millimeter-wave radar, the velocity is updated to the average velocity of all matched millimeter-wave targets;

[0038] For the size (w,h,l): if the current frame is a laser frame, the laser observation size is assigned; if it is not a laser frame, the validity period of the last assignment is determined; if the validity period has expired, the prior size is restored; if the current frame is a millimeter-wave radar frame, the sizes (w,h,l) of the matched objects are overlaid as patches onto the fused object size.

[0039] For the yaw angle: If the current frame is a lidar frame, the yaw angle is assigned to the lidar observation yaw angle; if it is not a lidar frame, the validity period of the last assignment is checked. If the validity period has expired, the yaw angle is restored to the default value of 0.

[0040] For target confidence: If the current frame is a visual frame or a laser frame, set the weighting parameters for the old and new confidence and weight them according to the parameters. If it is a millimeter wave frame, multiply the confidence by an upward factor.

[0041] Step S3 includes the following steps:

[0042] S3-1: Classifies weather into four types: normal, rainy, snowy, and foggy. Collects a large amount of data on visual, laser, and millimeter-wave target detection and tracking results under different weather conditions;

[0043] S3-2: Summarize the target detection and tracking results data collected in step S3-1 according to the sensor type;

[0044] S3-3: Using weather type as the label and the statistical indicators from step S3-2 as the features, train an XGBoost model for environmental classification offline and validate it using the K-fold cross-validation method.

[0045] S3-4: Deploy the environmental classification model generated in step S3-3 onto the AGV vehicle body to infer the weather status in real time during driving; and configure different fusion parameters for different weather conditions, and adaptively switch parameters according to the predicted weather status.

[0046] The statistical indicators include: mean / median / variance of intra-frame target confidence, mean / median of variance of historical consecutive frames of tracked target confidence, and mean / median of category information entropy of historical consecutive frames of tracked target.

[0047] The information entropy is: i represents the target category.

[0048] In steps S3-4, the step of inferring the weather conditions in real time during driving specifically involves:

[0049] A time synchronization module is used to match data from each sensor, and the statistical indicators in step S3-2 are calculated in real time. The statistical inference results are calculated in a 10-minute window, and the predicted mode is taken as the final predicted state.

[0050] The fusion parameters include: visual target initial confidence threshold, laser target initial confidence threshold, millimeter wave target initial confidence threshold, maximum GIoU distance threshold for data association, maximum size difference threshold for data association, old confidence weighting parameters, millimeter wave matching confidence upscaling coefficient, and timeliness for cleaning up unupdated targets.

[0051] The present invention has the following beneficial effects and advantages:

[0052] 1. This invention can sense outdoor environmental conditions in advance, enabling AGV to adaptively switch sensor fusion strategies, providing accurate input for AGV obstacle avoidance / bypass planning, and improving AGV driving safety.

[0053] 2. This invention significantly improves the perception robustness under severe weather conditions such as rain, snow, and fog by dynamically switching fusion parameters (such as confidence threshold and association algorithm threshold) in a real-time weather classification model.

[0054] 3. This invention provides localized weather inference based on sensor data statistical indicators (confidence variance, information entropy), avoiding reliance on network weather interfaces, reducing communication overhead, and adapting to local microclimates.

[0055] 4. The fusion target state update mechanism of the present invention (such as laser frame priority update size and millimeter wave frame priority update speed) ensures the reliability of key state quantities and provides high-precision input for obstacle avoidance planning. Attached Figure Description

[0056] Figure 1 Flowchart of the AGV multi-sensor perception fusion method of the present invention. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0058] like Figure 1 The diagram shown is a flowchart of the AGV multi-sensor perception fusion method of the present invention. This embodiment, in conjunction with an example, illustrates how a forward-looking monocular camera, a surround-view lidar, and a target-level millimeter-wave radar fuse perception results in dynamic environmental weather conditions. It is divided into three stages.

[0059] Phase 1: Sensor Detection and Tracking

[0060] 1) Collect outdoor factory area data for AGVs, including forward-looking camera images and LiDAR point clouds, and manually label them. In addition to common categories such as pedestrians and cars in the KITTI dataset, the categories also include specific categories that affect the movement of AGVs, such as AGV vehicles, flowers and trees, and materials.

[0061] 2) We mixed publicly available datasets with self-collected datasets to fine-tune the pre-trained object detection model. For visual processing, we used the YOLOv10 model, and for point cloud processing, we used the PointPillars model.

[0062] 3) During the AGV's operation, each sensor periodically detects target obstacles.

[0063] 4) Each sensor performs multi-frame tracking and association to obtain a unique target identifier (ID). The ByteTrack algorithm is used to track visual targets. An improved ByteTrack algorithm is used to track lidar targets; specifically, the matching index of the original algorithm is changed to... The value range is (-1, 1), where A and B are the volumes of the two 3D detection boxes, IoU is the intersection-union ratio of the two detection box volumes, and C is the volume of the largest circumscribed cuboid of the two detection boxes. The Kalman filter in the original algorithm is replaced with a uniformly accelerated linear motion Kalman filter. Confidence filtering is applied to the millimeter-wave detection and tracking results.

[0064] 5) The 2D visual bounding box is converted into a 3D bounding box. The target depth is determined using monocular depth ranging. Specifically, the pixel difference between the bottom edge of the bounding box and the optical center is y, f is the focal length, and H is the camera height setting; therefore, the depth d = f * H / y. The length, width, and height (w, h, l) are given using class prior values. The steering angle yaw is set to 0 by default.

[0065] 6) The tracking results of each sensor are converted into the vehicle coordinate system via external parameters.

[0066] 7) Send single sensor detection and tracking results (t, sor, objs) in real time, where t is the data source timestamp, sor is the sensor category, and objs is the target set;

[0067] Each target includes information (id,x,y,z,w,h,l,yaw,cls,conf,(u,v,a,h)), where cls is the category, conf is the confidence score, and (u,v,a,h) is the visual 2D detection box information, which is used for matching calculation in the fusion stage.

[0068] Phase Two: Asynchronous Fusion of Multiple Sensors

[0069] (1) The fusion module collects the detection results of each sensor. For every N frames accumulated, the frames are sorted according to the data source timestamp and the data earlier than the previous batch fusion time is discarded.

[0070] (2) Asynchronous frame-by-frame fusion is performed on the frame queue in step (1). The fusion process is as follows: the sensor target is the observation target, the target of the fusion module is the fusion target, the two are associated and matched, and the fusion target is updated with the observation target.

[0071] (3) The first stage of fusion is ID matching. When the initial sensor of the fusion target is the same as the sensor of the current frame, and the tracking ID of the fusion target sensor is the same as the tracking ID of the observed target, the two are matched.

[0072] (4) The second stage of fusion is data matching. First, target pairs with successfully matched IDs are excluded. Then, the consistency of category and size between the observed target and the fused target is judged, and inconsistent target pairs are excluded from association. Then, association indices are calculated according to sensor type, and association algorithms are applied. For cameras, the association index is the pixel 2D detection box IoU, and the Hungarian algorithm is used to associate with the fused target. For lidar, the 2D detection box in pixel space is first obtained through projection, and then the association index is calculated as the weighted sum of GIoU2D and GIoUBev, where GIoU2D is the pixel 2D detection box GIoU value, and GIoUBev is the BEV (Bird-Eye-View) spatial detection box GIoU value. The Hungarian algorithm is used to associate with the fused target. For millimeter-wave radar, due to its limited detection accuracy, it often connects a group of targets into one target or splits one target into multiple targets. Therefore, one-to-many brute-force matching is used, and millimeter-wave targets with GIoUBev greater than the threshold are associated with the fused target.

[0073] (5) Update the fusion targets matched in step (4). For position (x, y, z) and velocity (vx, vy, vz), the values ​​are updated using an error state Kalman filter. Due to the high accuracy of millimeter-wave velocity measurement, if the current frame is a millimeter-wave radar frame, the velocity is updated to the average velocity of all matched millimeter-wave targets. For size (w, h, l), if the current frame is a laser frame, the laser observation size is assigned; if it is not a laser frame, the validity period of the last assignment is checked, and if it exceeds the validity period, the size is restored to the prior size. If the current frame is a millimeter-wave radar frame, the sizes (w, h, l) of the matched objects are overlaid as patches onto the fusion object size. For yaw angle, if the current frame is a laser radar frame, the yaw angle is assigned to the laser observation yaw angle; if it is not a laser frame, the validity period of the last assignment is checked, and if it exceeds the validity period, the yaw angle is restored to the default value of 0. For the target confidence level, if the current frame is a visual frame or a laser frame, set the weighting parameters for the old and new confidence levels and weight them according to the parameters. If it is a millimeter wave frame, multiply the confidence level by an upward coefficient.

[0074] (6) For the unmatched observation targets in step (4), if they are visual targets or laser targets, and the confidence level exceeds a certain threshold and the continuous observation exceeds a certain number of frames, they are initialized as new fusion targets. If they are millimeter wave targets, it is also necessary to determine whether the target location is within the drivable area of ​​the factory map.

[0075] (7) For the fusion targets that were not matched in step (4), if the time since the last update exceeds a certain threshold, they are removed.

[0076] (8) Send the set of fusion targets (t,objs) with confidence greater than the threshold to the planning and control module in real time, where t is the data source timestamp, objs is the target set, and each target includes information (id,x,y,z,w,h,l,yaw,vx,vy,cls,conf).

[0077] Phase 3: Adaptive Switching of Fusion Strategies

[0078] a. Weather conditions are categorized into four types: normal, rainy, snowy, and foggy. Extensive data on visual, laser, and millimeter-wave target detection and tracking results under different weather conditions are collected.

[0079] b. For the detection and tracking results collected in step a, the following indicators are statistically analyzed by sensor type: mean / median / variance of intra-frame target confidence, mean / median of variance of historical consecutive frames of tracked target confidence, and mean / median of category information entropy of historical consecutive frames of tracked target. The formula for calculating information entropy is... i represents the target category.

[0080] c. Using weather type as the label and the statistical index from step b as the feature, an XGBoost model for environmental classification is trained offline and validated using the K-fold cross-validation method. Compared to weather data obtained through network interfaces, which is applicable to large geographical areas but not localized ones, this method directly infers outdoor conditions from sensor detection results, reducing unnecessary network resource consumption and exhibiting a degree of robustness.

[0081] d. Deploy the environmental classification model generated in step c onto the AGV vehicle body to infer weather conditions in real time during operation. Specifically, a time synchronization module is used to match data from each sensor, calculate the indicators from step b in real time, statistically analyze the inference results in 10-minute windows, and take the predicted mode as the final predicted state.

[0082] e. Configure different fusion parameters under different weather conditions, and adaptively switch parameters based on the weather conditions predicted in step d. Specifically, the parameter configuration is shown in the table below:

[0083] Table 1

[0084]

[0085] f. If the detection model in the first stage is changed, the environment classification model shall be retrained and deployed in steps a to c.

[0086] In conjunction with the embodiments of the present invention, the present invention significantly improves the obstacle perception accuracy and continuity of AGVs in adverse weather conditions such as rain, snow, and fog, providing reliable assurance for autonomous driving safety decisions and promoting the all-weather application of outdoor AGVs.

[0087] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0088] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A dynamic environment adaptive AGV multi-sensor perception fusion method, characterized in that, Includes the following steps: S1: Detect target obstacles using cameras, lidar, and millimeter-wave radar respectively, perform multi-frame tracking and association of multiple sensors, generate single-sensor tracking results with unique identifiers, convert the tracking results of each sensor into the vehicle coordinate system through external parameters, and send the single-sensor detection and tracking results to the fusion module in real time. S2: The fusion module asynchronously fuses the tracking results of each sensor generated in step S1 frame by frame. It associates the fusion target with the observed target through two stages of ID matching and data matching, and dynamically updates the status information of the fusion target. S3: During the execution of step S2, the current weather type is inferred in real time based on the environmental weather classification model, and the fusion parameter configuration is dynamically switched according to the weather type.

2. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 1, characterized in that, The camera and lidar detect target obstacles, specifically as follows: Label specific target categories in the AGV factory area and fine-tune the pre-trained target detection model; The camera uses the YOLOv10 model for visual detection; the LiDAR uses the PointPillars model for point cloud detection. The camera detection bounding box is converted into a 3D detection bounding box using monocular depth ranging. Given the pixel difference y between the bottom edge of the camera detection frame and the optical center, the depth is: d = f * H / y Where f is the focal length and H is the camera height setting; The length, width, and height (w, h, l) of the 3D detection box are given using category prior values, and the turning angle yaw is 0 by default, thus completing the conversion of the visual 2D detection box to the 3D detection box.

3. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 1, characterized in that, The step of performing multi-frame tracking and correlation on multiple sensors to generate a single-sensor tracking result with a unique identifier ID is as follows: S1-1: An improved ByteTrack algorithm is used for LiDAR target tracking, with a value range of (-1, 1). The matching index is replaced with the 3D detection bounding box GIoU, i.e.: Where A and B are the volumes of the two 3D bounding boxes, IoU is the intersection-union ratio of the volumes of the two bounding boxes, and C is the volume of the largest circumscribed cuboid of the two bounding boxes; S1-2: In the improved ByteTrack algorithm, the motion model adopts a uniformly accelerated linear motion Kalman filter; S1-3: Confidence filtering of millimeter-wave detection and tracking results.

4. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 1, characterized in that, The single sensor detection and tracking result is: single sensor detection and tracking result (t, sor, objs), where t is the data source timestamp, sor is the sensor category, and objs is the target set; Each target includes information (id,x,y,z,w,h,l,yaw,cls,conf,(u,v,a,h)), where cls is the category, conf is the confidence score, and (u,v,a,h) is the visual 2D detection box information, which is used for matching calculation in the fusion stage.

5. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 1, characterized in that, Step S2 includes the following steps: S2-1: The fusion module collects the detection and tracking results of each sensor. After accumulating N frames, it sorts the frames according to the data source timestamp and discards data that is earlier than the previous batch fusion time. S2-2: Set the sensor target as the observation target and the target of the fusion module as the fusion target. The two are associated and matched, and the fusion target is updated with the observation target. When the initial sensor of the fusion target is the same as the sensor of the current frame, and the tracking ID of the fusion target sensor is the same as the tracking ID of the observation target, the two are matched. S2-3: Exclude the targets already matched in step S2-2, determine the consistency of category and size between the remaining observed targets and the fused targets, and exclude inconsistent target pairs from association; calculate the association index for each sensor type, apply the association algorithm, and if the association is successful, they are considered to be matched; S2-4: Update the data of the fusion targets that have been matched in steps S2-2 to S2-3; S2-5: For unmatched observation targets in step S2-3, if they are visual or laser targets, and the confidence level exceeds the set threshold and the continuous observation exceeds the set number of frames, they are initialized as new fusion targets. If they are millimeter-wave targets, it is additionally determined whether the target location is within the drivable area of ​​the factory map. S2-6: For fusion targets that were not matched in step S2-3, if the time since the last update exceeds the set time limit threshold, then remove the fusion target; S2-7: Send the set of fusion targets (t, objs) with confidence greater than the threshold to the planning and control module in real time; where t is the data source timestamp, objs is the target set, and each target includes information (id, x, y, z, w, h, l, yaw, vx, vy, cls, conf).

6. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 5, characterized in that, In steps S2-3, the calculation of correlation indicators and application of correlation algorithms according to sensor type specifically includes: For the camera, the correlation metric is the pixel 2D detection bounding box IoU, which is associated with the fusion target using the Hungarian algorithm. For LiDAR: the 2D detection box in pixel space is obtained by projection, and the association index is calculated as the weighted sum of GIoU2D and GIoUBev; where GIoU2D is the GIoU value of the pixel 2D detection box, and GIoUBev is the GIoU value of the BEV space detection box. The Hungarian algorithm is used to associate it with the fusion target. For millimeter-wave radar: millimeter-wave targets with GIoUBev greater than the threshold are all associated with the fused target, allowing one-to-many matching relationships.

7. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 5, characterized in that, Step S2-4 specifically includes: For position (x, y, z) and velocity (vx, vy, vz): the values ​​are updated using an error state Kalman filter; if the current frame is a millimeter-wave radar, the velocity is updated to the average velocity of all matched millimeter-wave targets; For the size (w,h,l): if the current frame is a laser frame, the laser observation size is assigned; if it is not a laser frame, the validity period of the last assignment is checked; if the validity period has expired, the prior size is restored; if the current frame is a millimeter-wave radar frame, the sizes (w,h,l) of the matched objects are overlaid as patches onto the fused object size. For the yaw angle: If the current frame is a lidar frame, the yaw angle is assigned to the lidar observation yaw angle; if it is not a lidar frame, the validity period of the last assignment is checked. If the validity period has expired, the yaw angle is restored to the default value of 0. For target confidence: If the current frame is a visual frame or a laser frame, set the weighting parameters for the old and new confidence and weight them according to the parameters. If it is a millimeter wave frame, multiply the confidence by an upward factor.

8. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 1, characterized in that, Step S3 includes the following steps: S3-1: Classify weather into four types: normal, rainy, snowy, and foggy; collect a large amount of visual, laser, and millimeter-wave target detection and tracking data under different weather conditions; S3-2: Summarize the target detection and tracking results data collected in step S3-1 according to the sensor type; S3-3: Using weather type as the label and the statistical indicators from step S3-2 as the features, train an XGBoost model for environmental classification offline and validate it using the K-fold cross-validation method. S3-4: Deploy the environmental classification model generated in step S3-3 onto the AGV vehicle body to infer the weather status in real time during driving; and configure different fusion parameters for different weather conditions, and adaptively switch parameters according to the predicted weather status.

9. The dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 8, characterized in that, The statistical indicators include: mean / median / variance of intra-frame target confidence, mean / median of variance of historical consecutive frames of tracked target confidence, and mean / median of category information entropy of historical consecutive frames of tracked target. The information entropy is: i represents the target category.

10. A dynamic environment adaptive AGV multi-sensor perception fusion method according to claim 8, characterized in that, In steps S3-4, the step of inferring the weather conditions in real time during driving specifically involves: A time synchronization module is used to match data from each sensor, and the statistical indicators in step S3-2 are calculated in real time. The statistical inference results are calculated in a 10-minute window, and the predicted mode is taken as the final predicted state. The fusion parameters include: visual target initial confidence threshold, laser target initial confidence threshold, millimeter wave target initial confidence threshold, maximum GIoU distance threshold for data association, maximum size difference threshold for data association, old confidence weighting parameters, millimeter wave matching confidence upscaling coefficient, and timeliness for cleaning up unupdated targets.

Images