AGV dynamic obstacle avoidance method based on machine vision-laser radar fusion
By integrating machine vision and LiDAR technology, the obstacle avoidance problem of AGVs in complex dynamic environments has been solved, achieving high-precision dynamic obstacle detection and tracking, and improving the obstacle avoidance accuracy, response speed, adaptability, and efficiency of AGVs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-11
- Publication Date
- 2026-04-03
AI Technical Summary
Existing AGV environmental perception and obstacle avoidance technologies suffer from problems such as large sensor ranging errors, data mismatch, and path planning delays. These issues lead to untimely obstacle avoidance and unreasonable path planning in complex dynamic environments, affecting operational efficiency and safety.
By employing a fusion approach combining machine vision and LiDAR, and through data preprocessing, spatiotemporal synchronization, feature fusion, and path planning algorithms, high-precision dynamic obstacle detection and tracking are achieved. The path is optimized by combining dynamic windowing and ant colony algorithms, thereby improving obstacle avoidance accuracy and response speed.
It significantly improves the obstacle avoidance accuracy and response speed of AGVs in complex and dynamic environments, enhances environmental adaptability and operational efficiency, and reduces deployment and maintenance costs.
Smart Images
Figure CN121785301A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of AGV dynamic obstacle avoidance technology, and in particular to an AGV dynamic obstacle avoidance method based on machine vision-LiDAR fusion. Background Technology
[0002] With the rapid development of industrial automation and intelligent manufacturing, Automated Guided Vehicles (AGVs), as core equipment for material handling and production process automation, have been widely used in warehousing and logistics, intelligent manufacturing, and port terminals. Dynamic obstacle avoidance capability is one of the core technologies for AGVs to achieve autonomous navigation, directly affecting their operating efficiency, operational safety, and environmental adaptability.
[0003] Current AGV environmental perception and obstacle avoidance technologies mainly rely on single sensors or simple multi-sensor combinations, which have significant technical limitations:
[0004] At the sensor level, while a single LiDAR solution can provide accurate distance information, it is prone to ranging errors when facing low-reflectivity objects such as reflective metal surfaces and glass. Furthermore, in adverse weather conditions such as rain and fog, the effective detection range drops drastically to less than 30 meters, and near-field noise increases more than fivefold. Simultaneously, the 10Hz refresh rate of mainstream LiDAR is only one-fifth that of visual sensors, leading to lag in tracking moving targets in dynamic scenes. A moving target 200 meters away can shift more than 3 meters between two scans, severely impacting dynamic obstacle avoidance accuracy. On the other hand, a single machine vision solution is significantly affected by lighting conditions. In strong light, shadow, or low-light environments, the reduced image signal-to-noise ratio increases the obstacle detection miss rate by more than 40%, and the target detection accuracy of deep learning models decreases by an average of 35%.
[0005] In multi-sensor fusion technology, existing solutions face several insurmountable technical bottlenecks: First, insufficient time synchronization accuracy. The difference in sampling frequency between lidar (typically 10 frames / second) and vision sensors (typically 25-30 frames / second) leads to data mismatch, and traditional soft synchronization methods are prone to nanosecond-level time deviations. Second, limited spatial registration accuracy. Traditional calibration methods are susceptible to vibration and temperature changes in complex industrial environments, resulting in different sensor data not accurately corresponding to the same physical space, with dynamic obstacle recognition errors reaching over 0.5 meters. Third, low feature fusion efficiency. The failure to effectively combine the semantic information (color, texture) of vision sensors with the geometric information (distance, contour) of lidar leads to a high obstacle classification error rate.
[0006] At the path planning level, traditional algorithms such as the A* algorithm are slow to respond in dynamic environments and need to rebuild the global map when faced with sudden obstacles, resulting in obstacle avoidance delays. On the other hand, local obstacle avoidance algorithms such as the artificial potential field method are prone to getting trapped in local optima, which can lead to path oscillations or stagnation in multi-obstacle scenarios. These technical shortcomings cause existing AGVs to frequently experience untimely obstacle avoidance, unreasonable path planning, or even collisions in complex dynamic environments, severely restricting the application of AGVs in highly dynamic and complex scenarios.
[0007] Although some solutions attempt to employ multi-sensor fusion technology, shortcomings remain in key areas such as spatiotemporal synchronization, feature fusion, and dynamic decision-making. Time synchronization often relies on simple timestamp alignment, failing to address the inherent latency issues of sensors. Spatial registration depends on manual calibration or low-precision automatic calibration methods, which are insufficient for dynamic environments. Feature fusion often uses simple stitching methods, failing to fully leverage the complementary advantages of different sensors. Therefore, developing a high-precision, robust dynamic obstacle avoidance method based on multi-sensor fusion has become an urgent need for the development of AGV technology. Summary of the Invention
[0008] In order to overcome the shortcomings of the prior art, one of the objectives of this invention is to provide a dynamic obstacle avoidance method for AGVs based on machine vision-LiDAR fusion.
[0009] One of the objectives of this invention is achieved through the following technical solution:
[0010] A dynamic obstacle avoidance method for AGVs based on machine vision-LiDAR fusion is characterized by the following steps:
[0011] (1) Data acquisition steps: The machine vision module and the lidar module installed on the AGV simultaneously acquire raw data of the surrounding environment, wherein the machine vision module acquires environmental image data and the lidar module acquires environmental point cloud data.
[0012] (2) Data preprocessing steps: The acquired image data is denoised, enhanced, and distortion corrected; the acquired point cloud data is filtered, downsampled, and subjected to extrinsic parameter correction. The image denoising uses Gaussian filtering, and its two-dimensional Gaussian kernel function is:
[0013]
[0014] in, These are the pixel position coordinates. The standard deviation is used; the point cloud downsampling employs voxel filtering, which is achieved by calculating the index of the point cloud in the voxel grid:
[0015]
[0016] in, This indicates rounding down; voxelsize is the voxel size. For the three-dimensional coordinates of the point cloud, A three-dimensional index for voxels;
[0017] (3) Spatiotemporal synchronization step: The preprocessed image data and point cloud data are time-aligned using a time synchronization protocol, and the spatial transformation relationship between sensors is established through a calibration algorithm to achieve spatial registration of image data and point cloud data; the spatial registration adopts the Iterative Closest Point (ICP) algorithm, and its objective function is:
[0018]
[0019] in, For points in the source point cloud, For the corresponding point in the target point cloud, Let be a rotation matrix. It is a translation vector;
[0020] (4) Feature fusion step: Visual features are extracted from the synchronized image data, and lidar features are extracted from the synchronized point cloud data. The visual features and lidar features are fused using a feature fusion algorithm to obtain fused features. The feature fusion algorithm adopts weighted fusion.
[0021]
[0022] in, As a visual feature, Features of lidar and These are the weighting coefficients. For adjustment items, ;
[0023] (5) Obstacle detection and tracking steps: Based on the fusion feature recognition of obstacles in the environment, determine whether the obstacle is a dynamic obstacle, and use a tracking algorithm to update the position and motion state of dynamic obstacles in real time; the method for determining dynamic obstacles is: when the position change of an obstacle exceeds a set threshold within a preset time interval, it is determined to be a dynamic obstacle, and the position change is calculated as follows:
[0024]
[0025] in, and The positions of the obstacles at different times. To set a threshold; the tracking algorithm uses Kalman filtering, and its state prediction equation is:
[0026]
[0027] The state update equation is:
[0028]
[0029] in, To predict the state, Here is the state transition matrix. For the control matrix, To control the input, To predict the error covariance, For process noise covariance, For Kalman gain, For the observation matrix, For the observed values, To measure the noise covariance, It is the identity matrix;
[0030] (6) Path planning and adjustment steps: Based on the position and movement state of obstacles, combined with the current position and target position of the AGV, the AGV's driving path is generated or adjusted in real time through a dynamic path planning algorithm to achieve dynamic obstacle avoidance; the dynamic path planning algorithm adopts the dynamic window method (DWA), and its evaluation function is:
[0031]
[0032] in, For linear velocity, Angular velocity, Rate the direction. Rate the speed. Scoring the distance to obstacles. These are the weighting coefficients.
[0033] Furthermore, the machine vision module in step (1) includes at least one camera, and the lidar module includes a 360° scanning lidar.
[0034] Furthermore, the preprocessing of image data in step (2) includes denoising using the Gaussian filter or median filter, enhancement using histogram equalization, and distortion correction using camera intrinsic parameters; the preprocessing of lidar data includes downsampling using the voxel filter and removing outliers using statistical filtering, wherein the representative point of each voxel in the voxel filter is the centroid of all points within that voxel:
[0035]
[0036] in, voxels The center of mass, voxels The point set within, Let be the number of points in the point set. Let be the coordinates of the points in the set of points.
[0037] Furthermore, in step (3), the time synchronization protocol combines the gPTP universal precise time protocol with the PPS second pulse signal to achieve nanosecond-level time alignment; spatial registration uses the Iterative Closest Point (ICP) algorithm to calculate the transformation matrix, and iteratively optimizes the objective function. To achieve the conversion of point cloud data to an image coordinate system.
[0038] Furthermore, the visual features in step (4) include the color features, shape features and texture features of the obstacle, which are extracted by a convolutional neural network (CNN); the lidar features include the distance features, contour features and position features of the obstacle; the feature fusion algorithm adopts the weighted fusion, serial fusion or deep learning fusion model based on attention mechanism.
[0039] Furthermore, in step (5), the tracking algorithm employs the Kalman filter, extended Kalman filter, or particle filter algorithm to predict the trajectory of dynamic obstacles in real time, wherein the state vector of the Kalman filter... It contains information about the location and speed of obstacles.
[0040] Furthermore, in step (6), the Dynamic Window Method (DWA) searches for the optimal control quantity in the velocity space based on the position and motion state of the obstacle, the motion constraints of the AGV, and the position of the target point. The path is adjusted in real time by maximizing the evaluation function.
[0041] Furthermore, step (6) also includes: when multiple dynamic obstacles are detected, the obstacle avoidance path of the AGV is globally optimized using ant colony algorithm or auction algorithm to avoid local optimal solutions in multi-obstacle scenarios.
[0042] An AGV dynamic obstacle avoidance system implementing the method of any one of claims 1-8, characterized in that it comprises:
[0043] Machine vision module: used to acquire environmental image data;
[0044] LiDAR module: used to collect environmental point cloud data;
[0045] Data preprocessing unit: used to preprocess image data and point cloud data, including performing Gaussian filtering and voxel filtering as described in claim 1;
[0046] Spatiotemporal synchronization unit: used to realize temporal alignment and spatial registration of image data and point cloud data, including executing the ICP algorithm as described in claim 1;
[0047] Feature fusion unit: used to extract visual features and LiDAR features and fuse them, including executing the weighted fusion algorithm as described in claim 1;
[0048] Obstacle detection and tracking unit: used to identify obstacles and track the motion state of dynamic obstacles, including executing the dynamic obstacle judgment and Kalman filtering algorithm as described in claim 1;
[0049] Path planning unit: used to generate or adjust the AGV's driving path in real time based on obstacle information, including executing the DWA algorithm as described in claim 1;
[0050] Control unit: Used to control the movement of the AGV according to the planned path.
[0051] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the AGV dynamic obstacle avoidance method based on machine vision-LiDAR fusion as described in any one of claims 1-8.
[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0053] 1. Significantly Improved Environmental Perception Robustness: By leveraging the complementary strengths of visual and LiDAR data, the environmental adaptability limitations of single sensors are overcome. LiDAR provides precise 3D spatial distance information, compensating for the blind spots in low-light environments; the visual sensor provides rich semantic features (color, texture, shape), solving the problem of LiDAR detecting low-reflectivity objects. The Gaussian filtering (two-dimensional Gaussian kernel function) and voxel filtering algorithms used in the preprocessing stage reduce visual image noise and point cloud data volume, respectively, improving the image signal-to-noise ratio by over 30% and reducing point cloud data volume by 60% while retaining key feature information.
[0054] 2. Significantly Improved Spatiotemporal Synchronization Accuracy: A hard synchronization method combining the gPTP universal precise time protocol and PPS second pulse signal is adopted to achieve nanosecond-level time alignment, resolving the data mismatch problem caused by the difference in sampling frequencies between the LiDAR and the vision sensor; spatial registration is optimized through the Iterative Closest Point (ICP) algorithm, whose objective function... It can effectively calculate the optimal transformation matrix, control the sensor spatial calibration error within 0.02 meters, and ensure that data from different sensors accurately correspond to the same physical space.
[0055] 3. Performance optimization of dynamic obstacle detection and tracking: The proposed feature weighted fusion algorithm By combining visual semantic features with LiDAR geometric features, the accuracy of dynamic obstacle recognition reaches over 95%, an improvement of 25-40% compared to single-sensor solutions. Real-time tracking of dynamic obstacles is achieved using a Kalman filter algorithm, and state prediction equations are used to... The updated equation accurately predicts the motion trajectory, controlling the prediction error of dynamic target position to within 5cm, thus solving the problem of tracking lag for fast-moving targets in traditional schemes.
[0056] 4. Enhanced Real-Time Performance and Security of Path Planning: The path planning algorithm based on the Dynamic Window Method (DWA) optimizes the evaluation function through multiple objectives. The system rapidly searches for the optimal control value in the velocity space, ensuring that the AGV achieves smooth obstacle avoidance while satisfying motion constraints. The ant colony algorithm global optimization strategy, designed for multi-obstacle scenarios, effectively avoids local optima, reducing the AGV's path adjustment response time in complex dynamic environments to less than 100ms and improving operational efficiency by over 20%.
[0057] 5. Excellent System Compatibility and Engineering Practicality: The fusion framework and algorithm modules proposed in this invention are adaptable to different types of visual sensors and LiDAR devices, and the standardized preprocessing and calibration process lowers the hardware selection threshold. Simultaneously, the system possesses good scalability, which can be improved by adjusting the weighted fusion coefficients. By adjusting parameters such as path evaluation function weights, the system can quickly adapt to the needs of different application scenarios such as warehousing, manufacturing, and ports, significantly reducing the deployment and maintenance costs of AGV systems.
[0058] In summary, this invention significantly improves the obstacle avoidance accuracy, response speed, and environmental adaptability of AGVs in complex dynamic environments by organically combining deep fusion of machine vision and LiDAR, high-precision spatiotemporal synchronization, efficient feature fusion, and dynamic path planning algorithms, providing key technical support for achieving highly reliable and efficient autonomous navigation.
[0059] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description
[0060] Figure 1 This is a flowchart of the dynamic obstacle avoidance method in this embodiment. Detailed Implementation
[0061] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.
[0062] It should be noted that when a component is described as "fixed to" another component, it can be directly on the other component or may have a component in between. When a component is considered "connected to" another component, it can be directly connected to the other component or may have a component in between. When a component is considered "set on" another component, it can be directly set on the other component or may have a component in between. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.
[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0064] I. System Hardware Configuration
[0065] The AGV dynamic obstacle avoidance system hardware platform used in this embodiment includes:
[0066] Machine vision module: 2-megapixel global shutter industrial camera (model: BasleracA2040-120uc), resolution 1920×1080, frame rate 30fps, lens focal length 8mm, horizontal field of view 62°, mounted on the front of the top of the AGV, 1.2m above the ground.
[0067] LiDAR module: 16-line LiDAR (model: VelodyneVLP-16), horizontal field of view 360°, vertical field of view ±15°, ranging range 0.5-100m, ranging accuracy ±3cm, frame rate 10Hz, installed on the same side below the camera, 0.8m above the ground.
[0068] Synchronization module: It adopts the gPTP universal precision time protocol (IEEE802.1AS) and PPS second pulse signal to realize the time synchronization between the camera and the lidar, and the synchronization error is controlled within 50ns.
[0069] Computing units: Intel Xeon E3-1230v5 processor (4 cores, 8 threads), Nvidia GTX 1080 Ti GPU (11GB VRAM), 16GB DDR4 memory, ensuring real-time algorithm operation.
[0070] AGV body: Two-wheel differential drive AGV, maximum speed 1.5m / s, minimum turning radius 0m, equipped with wheel odometer and IMU inertial measurement unit.
[0071] II. Specific Implementation Steps of the Method
[0072] 1. Data Acquisition Steps
[0073] In an industrial warehousing environment, the machine vision module and the LiDAR module are activated simultaneously upon startup of the AGV:
[0074] The camera captures RGB image data at 30fps, with each frame being 1920×1080×3 bytes in size, and transmits it to the computing unit via the MIPI-CSI interface.
[0075] The lidar outputs point cloud data at 10Hz, with each frame containing approximately 30,000 points. Each point contains three-dimensional coordinates (x, y, z) and reflection intensity information, which is transmitted via an Ethernet interface.
[0076] The system adds timestamps to the raw data. The camera timestamp is triggered by the PPS signal, and the lidar timestamp is calibrated via the gPTP protocol.
[0077] 2. Data Preprocessing Steps
[0078] Image preprocessing:
[0079] Noise reduction: Two-dimensional Gaussian filtering is used, and the Gaussian kernel function is:
[0080]
[0081] in With a kernel size of 3×3, the signal-to-noise ratio of the processed image is improved by 32%.
[0082] Enhancement: Use CLAHE to limit contrast adaptive histogram equalization, with the enhancement factor set to 2.0.
[0083] Distortion correction: Correction is performed based on the camera intrinsic parameter matrix and distortion coefficients. The intrinsic parameter matrix is obtained using Zhang's calibration method.
[0084]
[0085] Point cloud preprocessing:
[0086] Filtering: Statistical filtering is used to remove outliers, with the number of neighborhood points set to 15 and the standard deviation factor set to 1.5.
[0087] Downsampling: Voxel filtering is used, with voxel sizes set to 5cm×5cm×5cm. The voxel index is calculated as follows:
[0088]
[0089] The point cloud data volume is reduced by 65% while retaining key contour features. The representative point of each voxel is the voxel center:
[0090]
[0091] External parameter correction: Calculate the rotation matrix R and translation vector t of the radar to the camera based on the calibration board data.
[0092] 3. Spatiotemporal synchronization steps
[0093] Time alignment: Data synchronization is achieved through timestamp matching. For the k-th frame image from the camera (timestamp T_cam) and the m-th frame point cloud from the LiDAR (timestamp T_lidar), matching is performed when |T_cam-T_lidar| < 5ms. If the threshold is exceeded, linear interpolation is used to generate intermediate frames.
[0094] Spatial registration: The Iterative Closest Point (ICP) algorithm is used, with the objective function being:
[0095] Wherein, the source point cloud p_i represents LiDAR points, and the target point cloud q_i represents visual feature points. The maximum number of iterations is set to 20, the convergence threshold is 0.01m, and the spatial error after registration is controlled within 0.02m.
[0096] 4. Feature Fusion Steps
[0097] Visual feature extraction: The YOLOv5s network is used to extract the color, shape and texture features of obstacles, and the output feature vector has a dimension of 512.
[0098] LiDAR feature extraction: Extracting distance features (Euclidean distance from the AGV), contour features (point cloud bounding box), and position features (3D coordinates) of obstacles, with a feature vector dimension of 64.
[0099] Feature fusion: A weighted fusion algorithm is used.
[0100] Among them, visual feature weights The optimal weight allocation is determined through cross-validation.
[0101] 5. Obstacle Detection and Tracking Steps
[0102] Dynamic obstacle detection: Calculate the change in position of the obstacle within a 100ms time interval.
[0103] when Objects larger than 0.1m are considered dynamic obstacles (such as moving forklifts or pedestrians).
[0104] Kalman filter tracking:
[0105] State vector: (Position and velocity)
[0106] State transition matrix:
[0107] Process noise covariance Q (based on uniform velocity model):
[0108]
[0109] in
[0110] Measurement noise covariance R: (Based on lidar ranging accuracy)
[0111] After tracking, the error in predicting the position of dynamic obstacles was controlled within 5cm.
[0112] 6. Route Planning and Adjustment Steps
[0113] Dynamic Window Method (DWA): The evaluation function is: Among them, the directional score weight Speed score weighting Distance rating weight .
[0114] Velocity space constraint: linear velocity angular velocity .
[0115] Multi-obstacle optimization: When more than 3 dynamic obstacles are detected, the ant colony algorithm is enabled for global optimization, with the pheromone evaporation coefficient set to 0.1 and the heuristic factor set to 2.0.
[0116] Practical process record:
[0117] 1. Experimental Environment Setup
[0118] Test scenario: 20m×15m industrial warehouse environment, including shelves (static obstacles), mobile forklifts (dynamic obstacles, speed 0.8m / s), and pedestrians (dynamic obstacles, speed 1.2m / s).
[0119] Comparison of options:
[0120] Option A: Single LiDAR obstacle avoidance solution
[0121] Option B: Single machine vision obstacle avoidance solution
[0122] Option C: This invention integrates an obstacle avoidance scheme.
[0123] Evaluation metrics: obstacle detection accuracy, dynamic tracking error, obstacle avoidance success rate, path length optimization rate, and response time.
[0124] 2. Experimental Data Recording
[0125]
[0126]
[0127]
[0128] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.
Claims
1. A dynamic obstacle avoidance method for AGVs based on machine vision-LiDAR fusion, characterized in that, Includes the following steps: (1) Data acquisition steps: The machine vision module and the lidar module installed on the AGV simultaneously acquire raw data of the surrounding environment, wherein the machine vision module acquires environmental image data and the lidar module acquires environmental point cloud data. (2) Data preprocessing steps: The acquired image data is denoised, enhanced, and distortion corrected; the acquired point cloud data is filtered, downsampled, and subjected to extrinsic parameter correction. The image denoising uses Gaussian filtering, and its two-dimensional Gaussian kernel function is: in, These are the pixel position coordinates. The standard deviation is used; the point cloud downsampling employs voxel filtering, which is achieved by calculating the index of the point cloud in the voxel grid: in, This indicates rounding down; voxelsize is the voxel size. For the three-dimensional coordinates of the point cloud, A three-dimensional index for voxels; (3) Spatiotemporal synchronization step: The preprocessed image data and point cloud data are time-aligned using a time synchronization protocol, and the spatial transformation relationship between sensors is established through a calibration algorithm to achieve spatial registration of image data and point cloud data; the spatial registration adopts the Iterative Closest Point (ICP) algorithm, and its objective function is: in, For points in the source point cloud, For the corresponding point in the target point cloud, For rotation matrix, It is a translation vector; (4) Feature fusion step: Visual features are extracted from the synchronized image data, and lidar features are extracted from the synchronized point cloud data. The visual features and lidar features are fused using a feature fusion algorithm to obtain fused features. The feature fusion algorithm adopts weighted fusion. in, As a visual feature, Features of lidar and These are the weighting coefficients. For adjustment items, ; (5) Obstacle detection and tracking steps: Based on the fusion feature recognition of obstacles in the environment, determine whether the obstacle is a dynamic obstacle, and use a tracking algorithm to update the position and motion state of dynamic obstacles in real time; the method for determining dynamic obstacles is: when the position change of an obstacle exceeds a set threshold within a preset time interval, it is determined to be a dynamic obstacle, and the position change is calculated as follows: in, and The positions of the obstacles at different times. To set a threshold; the tracking algorithm uses Kalman filtering, and its state prediction equation is: The state update equation is: in, To predict the state, Here is the state transition matrix. For the control matrix, To control the input, To predict the error covariance, For process noise covariance, For Kalman gain, For the observation matrix, For the observed values, To measure the noise covariance, It is the identity matrix; (6) Path planning and adjustment steps: Based on the position and movement state of obstacles, combined with the current position and target position of the AGV, the AGV's driving path is generated or adjusted in real time through a dynamic path planning algorithm to achieve dynamic obstacle avoidance; the dynamic path planning algorithm adopts the dynamic window method (DWA), and its evaluation function is: in, Linear velocity, Angular velocity, Rate the direction. Rate the speed. Scoring the distance to obstacles. These are the weighting coefficients.
2. The method according to claim 1, characterized in that, The machine vision module in step (1) includes at least one camera, and the lidar module includes a 360° scanning lidar.
3. The method according to claim 1, characterized in that, The preprocessing of image data in step (2) includes denoising using Gaussian filtering or median filtering, enhancement using histogram equalization, and distortion correction using camera intrinsic parameters; the preprocessing of lidar data includes downsampling using the voxel filtering and removing outliers using statistical filtering, wherein the representative point of each voxel in the voxel filtering is the centroid of all points within that voxel: in, voxels The center of mass, voxels The point set within, Let be the number of points in the point set. Let be the coordinates of the points in the set of points.
4. The method according to claim 1, characterized in that, The time synchronization protocol in step (3) adopts a combination of gPTP universal precise time protocol and PPS second pulse signal to achieve nanosecond-level time alignment; Spatial registration employs the Iterative Closest Point (ICP) algorithm to calculate the transformation matrix, and the objective function is iteratively optimized. To achieve the conversion of point cloud data to an image coordinate system.
5. The method according to claim 1, characterized in that, The visual features in step (4) include the color features, shape features, and texture features of the obstacle, which are extracted by a convolutional neural network (CNN); the lidar features include the distance features, contour features, and position features of the obstacle. The feature fusion algorithm employs the weighted fusion, serial fusion, or deep learning fusion model based on the attention mechanism.
6. The method according to claim 1, characterized in that, In step (5), the tracking algorithm employs the Kalman filter, extended Kalman filter, or particle filter algorithm to predict the trajectory of dynamic obstacles in real time. The state vector of the Kalman filter... It contains information about the location and speed of obstacles.
7. The method according to claim 1, characterized in that, The Dynamic Window Method (DWA) in step (6) searches for the optimal control quantity in the velocity space based on the position and motion state of the obstacle, the motion constraints of the AGV, and the position of the target point. The path is adjusted in real time by maximizing the evaluation function.
8. The method according to claim 1, characterized in that, Step (6) further includes: when multiple dynamic obstacles are detected, the obstacle avoidance path of the AGV is globally optimized using ant colony algorithm or auction algorithm to avoid local optimal solutions in multi-obstacle scenarios.
9. An AGV dynamic obstacle avoidance system implementing the method of any one of claims 1-8, characterized in that, include: Machine vision module: used to acquire environmental image data; LiDAR module: used to collect environmental point cloud data; Data preprocessing unit: used to preprocess image data and point cloud data, including performing Gaussian filtering and voxel filtering as described in claim 1; Spatiotemporal synchronization unit: used to realize temporal alignment and spatial registration of image data and point cloud data, including executing the ICP algorithm as described in claim 1; Feature fusion unit: used to extract visual features and LiDAR features and fuse them, including executing the weighted fusion algorithm as described in claim 1; Obstacle detection and tracking unit: used to identify obstacles and track the motion state of dynamic obstacles, including executing the dynamic obstacle judgment and Kalman filtering algorithm as described in claim 1; Path planning unit: used to generate or adjust the AGV's driving path in real time based on obstacle information, including executing the DWA algorithm as described in claim 1; Control unit: Used to control the movement of the AGV according to the planned path.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the AGV dynamic obstacle avoidance method based on machine vision-LiDAR fusion as described in any one of claims 1-8.