Multi-sensor fusion intelligent AGV path planning method and system

Abstract

The invention discloses a multi-sensor fused intelligent AGV path planning method and system, and the method comprises the steps: enabling an AGV to carry a 360-degree laser radar sensor, a binocular vision sensor, an infrared thermal imaging sensor, an ultrasonic array sensor and other sensors, processing data through a low-rank multi-source feature fusion module LMF, and constructing a high-precision two-dimensional grid map through a VF-Cart algorithm; the global path planning uses an improved MRRT * algorithm, and the algorithm combines the dynamic data processing capability of a Markov decision process MDP and the efficient search capability of RRT * to ensure the path optimality; local path optimization adopts an FPA * algorithm fusing an artificial potential field method APF and a dynamic heuristic search algorithm DHPA *, local optimum is avoided, and intelligent obstacle avoidance is realized in combination with deep reinforcement learning DRL; according to the method, centimeter-level obstacle detection and dynamic pedestrian recognition are realized in a complex dynamic environment, the path planning efficiency and the motion stability of the AGV are remarkably improved, and the scheduling time is shortened.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to an intelligent AGV path planning method and system that integrates multiple sensors. Background Technology

[0002] With the advancement of technology, mobile robot technology is playing an increasingly important role in the field of artificial intelligence. It is widely used in service robots, autonomous vehicles, and drones, significantly reducing human labor and improving production efficiency. Path planning, a key technology for the autonomous movement of mobile robots, helps them find optimal or near-optimal paths in complex environments to reach their target location from the starting point. Path planning must consider robot motion constraints and environmental obstacles, while simultaneously satisfying objectives such as shortest time and shortest path, which is of great significance for improving robot performance, adaptability, and reliability.

[0003] In the field of AGV path planning, traditional methods mainly rely on non-learning algorithms, such as the sampling-based Rapid Expanding Random Tree (RRT) algorithm and the heuristic search-based A* algorithm. Taking the A* algorithm as an example, it searches for the optimal path by combining a cost function and a heuristic function, and can efficiently find the shortest feasible path from the starting point to the target point in a static environment. This algorithm uses a grid map or topology map to represent the environment and gradually expands the path by evaluating the cost value of each node until the target location is reached.

[0004] However, traditional path planning methods have significant limitations, especially in their poor adaptability in dynamic environments. Because these algorithms rely on pre-built environmental models, they need to replan the path once the scene changes (such as the addition of obstacles or adjustment of task objectives), leading to a decrease in real-time performance. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a multi-sensor fusion intelligent AGV path planning method that can improve the adaptability of AGVs to complex environments; on the other hand, it provides a multi-sensor fusion intelligent AGV path planning system.

[0006] Technical solution: The intelligent AGV path planning method of the present invention includes the following steps:

[0007] (1) The AGV vehicle uses a 360° lidar, binocular vision, infrared thermal imager and ultrasonic array module to collect multi-dimensional data, perceive the environment and dynamically avoid pedestrian obstacles in the operating environment.

[0008] (2) Use the low-rank multi-source feature fusion module (LMF) to fuse the multi-sensor, multi-dimensional data collected in step (1);

[0009] (3) The VF-Cart algorithm is used to process the fused multi-dimensional data to construct a global two-dimensional grid map and obtain an indoor and outdoor AGV operating environment map;

[0010] (4) Based on the indoor and outdoor AGV operating environment map, the improved RRT* algorithm MRRT* is used to perform global path planning for AGV vehicles; the MRRT* algorithm combines Markov Decision Process (MDP) and RRT*, that is, it utilizes the ability of MDP to process dynamic data and the efficient path search capability of RRT* in complex environments.

[0011] (5) During the process of the AGV moving forward according to the global optimal path obtained in step (4), it detects pedestrians or obstacles on the path in real time through multiple sensors. If an obstacle is detected, step (6) is executed; otherwise, step (7) is executed.

[0012] (6) An FPA* fusion algorithm is proposed. The algorithm uses the gravitational field of the artificial potential field method (APF) to guide the early movement direction of the AGV, and combines the dynamic heuristic search algorithm (DHPA*) to dynamically adjust the heuristic function. Combined with real-time environmental information, it finds a feasible path from the current position to the target point, optimizes the local path of the AGV during its operation, and introduces the deep reinforcement learning (DRL) method to assist in obstacle avoidance decision-making and complete the obstacle avoidance function.

[0013] (7) The AGV continues to travel along the planned path until it reaches the target point corresponding to the planned path.

[0014] This invention significantly enhances the AGV's ability to recognize complex scenes such as changing lighting and dynamic obstacles through multi-sensor collaborative perception; it effectively eliminates multi-source data noise based on low-rank feature fusion, improving the robustness of environmental modeling; it generates a raster map containing semantic information through the VF-Cart algorithm, supporting the AGV's accurate positioning in unstructured environments; the improved MRRT* algorithm, which incorporates Markov decision processes, enables global path planning to adaptively handle dynamic environmental changes; a real-time multimodal detection mechanism ensures rapid response to sudden obstacles; local path adjustment maintains path optimality in narrow spaces or dense obstacle scenarios through the fusion of potential field guidance and dynamic heuristic search; and the introduction of deep reinforcement learning further enhances the AGV's autonomous decision-making ability in complex pedestrian environments; ultimately ensuring the AGV's stable navigation to the target point in complex environments. This scheme, through a multi-level adaptive mechanism, balances global planning efficiency and real-time local obstacle avoidance, significantly improving the AGV's autonomous adaptability and path planning reliability.

[0015] Preferably, step 1 includes:

[0016] (1.1) The 360° lidar measures the distance to surrounding objects by emitting a laser beam and receiving reflected light. It also acquires three-dimensional environmental point cloud data around the AGV vehicle through rotational scanning, performing 360° horizontal scanning and multi-angle vertical scanning to generate a high-precision three-dimensional point cloud map. The environmental coordinate vector is:

[0017] z k =(d k ,θ k k = 1, 2, 3, ..., N

[0018] In the formula, z k For 3D point cloud data, d k For lidar at θ k Distance value obtained from the angle;

[0019] (1.2) The binocular vision module acquires environmental images through a binocular camera and uploads them to the onboard computer. It uses the ORB feature extraction method to extract feature points in the image that have drastic changes in gray value or large edge curvature, which are used for the identification and tracking of dynamic pedestrians and obstacles, and assist the AGV in completing path planning.

[0020] (1.3) The infrared thermal imaging module is connected to the data processing module and is used to detect the environment around the AGV and transmit it to the data processing module to realize pedestrian detection in environments where binocular vision sensors and lidar are difficult to identify.

[0021] (1.4) The ultrasonic array module is used for near-range obstacle detection in AGVs, identifying targets within a few centimeters to a few meters and detecting small obstacles or near-range collision risks in the chassis blind spot. The distance calculation formula is:

[0022] H = 0.5s = 0.5vt

[0023] In the formula, H is the distance between the obstacle and the ultrasonic rangefinder, s is the round-trip distance of the ultrasonic wave, v is the speed of the ultrasonic wave, and t is the round-trip time.

[0024] High-precision 3D environment modeling is achieved through 360° LiDAR, providing centimeter-level ranging capability and omnidirectional spatial perception. Combined with ORB feature extraction from binocular vision sensors, the ability to recognize textures and track motion of dynamic obstacles is enhanced. Infrared thermal imagers compensate for the perception deficiencies of traditional sensors in low light, smoke, and other adverse conditions, achieving reliable pedestrian detection through thermal radiation characteristics. Ultrasonic array modules detect nearby obstacles in chassis blind spots, effectively preventing collisions with low or transparent objects. The collaborative work of multiple sensors forms a three-dimensional perception network, enabling the AGV to possess all-weather, omnidirectional environmental perception capabilities in complex lighting, dynamic pedestrian flow, and unstructured environments, significantly improving its adaptability and safety in complex industrial scenarios.

[0025] Preferably, step 2 includes:

[0026] (2.1) The low-rank multi-source feature fusion (LMF) method is adopted to calculate the outer product of different modal features, explore the correlation and complementarity between modalities, and at the same time ensure the integrity of each modal feature, making full use of multi-sensor data;

[0027] (2.2) The weight tensor x is decomposed into the outer product of four mode-specific low-rank factors, calculated as follows:

[0028]

[0029] In the formula, the smallest r makes the above decomposition valid, which is called the effective rank of the tensor; β is the weight matrix of the nth mode in the i-th low-rank factor; i Indicates additional weight; The outer product of tensors is used to generate low-rank factors; α is a coefficient used to balance the relationship between the low-rank approximation and the regularization term.

[0030] (2.3) The final multi-sensor feature fusion data h is obtained by multiplying the corresponding elements of the input modal feature X by the low-rank factor and summing the results. The calculation formula is as follows:

[0031]

[0032] In the formula, The Hadamard product of four modal tensors, x n This represents the input feature of the nth modality.

[0033] The Low-Rank Multi-Source Feature Fusion (LMF) module achieves efficient fusion of multi-sensor data. Specifically, it involves: establishing deep correlations between different modal features using outer product calculations; transforming high-dimensional weight tensors into four low-rank factors through low-rank decomposition, significantly reducing computational complexity while preserving key feature information; and finally, achieving adaptive weighted fusion of LiDAR, visual, infrared, and ultrasonic data through Hadamard product operations. This technology effectively solves the problems of high dimensionality and redundancy in multi-source sensor data, improving feature extraction efficiency while ensuring information integrity. This enables AGVs to quickly generate robust environmental feature representations, providing a more accurate and stable multi-modal perception data foundation for subsequent path planning.

[0034] Preferably, step 3 includes:

[0035] (3.1) The point cloud data acquired by the 360° lidar scan is processed by voxel filtering, and all points in the grid are replaced by the centroid of each grid. The centroid is then projected onto a two-dimensional plane to construct a voxel grid index:

[0036]

[0037] Where: Φ voxel (x1, y1, z1) represents the projection of the three-dimensional voxel mesh onto the two-dimensional plane. The original coordinates of the centroid of the voxel mesh. The point cloud is represented by its three-dimensional coordinates, and f() is the projection function.

[0038] (3.2) Perform state estimation on the point cloud projected onto the two-dimensional grid to predict its next state information. The calculation formula is as follows:

[0039] h t =H(Φ voxel (x1,y1,z1)+βK t )+δ+∈

[0040] In the formula, h t K represents the state and pose of a point cloud. t H() represents the amount of control input point cloud, δ is the state transition function, β and ∈ are new variables;

[0041] (3.3) By comparing the predicted point cloud with the data before prediction, overlapping point clouds are assigned low weights and non-overlapping point clouds are assigned high weights. First, the observation model function V() is used to calculate the observation value of each point cloud, and then the weights are assigned according to the probability comparison results:

[0042] v t =V1(h t )+V2(δ)

[0043]

[0044] In the formula, v t For each observation of the point cloud, w t For the weighted result, P(v t |h t ) represents the pose h in a given state. t v was observed below t The probability of;

[0045] (3.4) Perform filtering and fusion screening. Project the weighted point cloud data onto the world coordinate system and fuse it with the voxelized data of the corresponding grid. Perform a weighted operation on the point cloud weights and then screen. Update the projected state information and point cloud weights first:

[0046]

[0047] In the formula, h t+1 The updated state pose; To accumulate the state pose at each time step; εt For new noise disturbances; w t+1 The updated weighting result; P(v t |h t+1 ) represents the state pose h after the update. t+1 v was observed below t The probability of; For new variables;

[0048] (3.5) After filtering, update the state and density of the filtered point cloud information:

[0049]

[0050] In the formula, θ n (x1, y1, z1) represents the updated point cloud density values, w t+1 As weight, For the updated point cloud state information, h t+1 For the point cloud state information of the new stage, Values ​​distributed per unit space;

[0051] (3.6) In the Cartographer algorithm, the position and orientation of the AGV are represented by ξ = (ξ x ,ξ y ,ξ θ ) indicates that ξ x and ξ y ξ is the distance the AGV moves in the x and y directions. θ It is its direction of motion angle, and the environmental data frame collected by the 360° lidar is denoted as L={l k} k＝1....K ,l k ∈R 2 , where each l k It is a two-dimensional vector, transformed by pose F ξ Mapping each point in the data frame to a subgraph is calculated using the following formula:

[0052]

[0053] In the formula, s is the position of the original point, and ρ x ρ y The coordinates of the reference point;

[0054] (3.7) Before the scan frame is added to the subgraph, the pose ξ is processed by the Ceres solver to obtain the pose information of the scan frame. The calculation formula is as follows:

[0055]

[0056] In the formula, F ξ lξ For point cloud data after pose transformation, M smooth It is a smoothing function;

[0057] (3.8) When a map is constructed from multiple subgraphs, matching the scan frame only with the current subgraph can easily lead to accumulated errors. The Cartographer algorithm optimizes the LiDAR data frame and AGV pose through sparse pose adjustment. The calculation formula is as follows:

[0058]

[0059] In the formula, These represent the sub-image pose and the scan frame pose, respectively. ij Let A be the relative pose of the scanned frames in the subgraph, A be the optimized pose, ρ be the loss function used to measure the error, and E be the relative pose of the scanned frames in the subgraph. 2 To calculate the error between the lidar data frame and the AGV pose;

[0060] (3.9) The Cartographer algorithm performs loop closure detection using a branch-and-bound scan matching algorithm. The calculation formula is as follows:

[0061]

[0062] In the formula, W is the search window, and M is the search window. nearest For the extension of the M function, R(δ) is the regularization term. It is the regularization parameter.

[0063] The VF-Cart algorithm, based on the Cartographer algorithm, preprocesses point cloud data by introducing weighted voxel filtering. This method combines particle filtering with weighting and secondary filtering of point cloud data acquired by 360° LiDAR. The weighted and filtered point cloud data, along with pose data acquired by IMU and wheeled odometer, is input into the backend optimization section. The VF-Cart algorithm achieves high-precision environmental modeling and real-time positioning: weighted voxel filtering effectively reduces noise while preserving key point cloud features; state prediction and dynamic weighting mechanisms significantly improve the sensitivity of dynamic obstacle recognition; pose transformation and Ceres solver achieve sub-meter positioning accuracy; and sparse pose adjustment and branch-bound loop closure detection eliminate accumulated errors, ensuring global map consistency. This technology enables AGVs to construct centimeter-level precision 2D grid maps in complex environments while maintaining positioning stability, providing a highly reliable spatial basis for path planning, and is particularly suitable for dynamically changing and structurally complex industrial scenarios.

[0064] Preferably, step 4 includes:

[0065] (4.1) In a two-dimensional environment, the RRT* algorithm starts from the starting point and generates the connection starting point M. iand target point M g A random tree with a compensation range of r is constructed. The initial position of the AGV is taken as the starting point and set as the root node of the random tree T. The target point is the end point of the AGV.

[0066] (4.2) Random sampling is performed in a two-dimensional environment to obtain sampling points q. rand ;

[0067] (4.3) The nearest point search is to find a node q in the search tree. nearest , making q nearest With random sample point q rand Find the minimum distance between them, traverse the search tree, and take the search tree and random sample points q as input. rand The output is the nearest node q. nearest The expression is:

[0068]

[0069] In the formula, q nearest Represents the nearest node, q rand Let v represent a random sampling point, v represent a certain search tree, and T represent the set of search trees;

[0070] (4.4) Introducing Markov Decision Process (MDP), defining states, actions, transition probabilities, and reward functions to optimize path planning. Actions are represented by the symbol 'a', and all possible actions constitute the action set A. State transition probabilities and immediate rewards are closely related to the current state and action. The AGV is in state S. t The set of optional actions is A. t In different states, the action set A t They may be different;

[0071] (4.5) In a certain state, the possible actions of the AGV follow a certain probability distribution, which is called the policy and is represented by the symbol π(a|s). The policy π(a|s) represents the probability distribution of the AGV choosing action a in state s. It is only related to the current state and does not depend on historical information. It determines the behavior of the AGV in the current state. Although the policy is fixed at a certain moment, the AGV can dynamically adjust and optimize the policy over time to achieve the optimal decision in each state. The formula is as follows:

[0072] π(a|s)=P[A t =a|S t =s]

[0073] (4.6) According to step (4.3), the random sampling point and the nearest node are input into the growth function to generate a new node q. new The expression is as follows:

[0074]

[0075] In the formula, step represents the movement step length of the AGV, which can be adjusted according to the environment and requirements;

[0076] (4.7) In state S t The AGV performs action a1 (a1∈A) t When ), the state transition is not fixed at S. t →S t+1 Instead, it is stored in the state transition probability matrix P in the form of a probability distribution. StSt+1|a In this context, it exhibits the characteristics of stochastic dynamic programming, and its expression is as follows:

[0077]

[0078] (4.8) Collision detection determines whether a new node conflicts with an obstacle: For a circular obstacle, calculate the shortest distance from its center to the line connecting the new node and the nearest node. If the distance is less than or equal to the radius of the obstacle, the path is not feasible; otherwise, the path is considered to have not crossed the obstacle and is feasible.

[0079] For a rectangular obstacle, first determine whether the new node is inside or on its boundary. If it is inside, the path is not feasible. If it is outside, then check whether the line connecting the new node and the nearest node intersects the obstacle. If they intersect, the path is not feasible. Here, the lines connecting points A and D to the sampling points are used as boundary lines, and their expressions are as follows:

[0080]

[0081] In the formula, the variable k represents the slope of the straight line;

[0082] (4.9) If through point q nearest If a straight line satisfies a certain condition, it means that the line does not intersect any obstacle. For collision detection, it is assumed that the boundary of each rectangular obstacle has a boolean value. i When bool i When the value is 1, it indicates that the line intersects with the obstacle; when boolean, it indicates that the line intersects with the obstacle. i When = 0, it indicates that there is no intersection. Therefore, the collision detection function needs to include an auxiliary subroutine to manipulate these Boolean values, as shown in the following formula:

[0083] j(I,N,P)=((y P -y I (xx-x) I ))>((y N -y I (x) P -xI ))

[0084] bool i =(j(q) nearest ,v1,v2)≠j(q new ,v1,v2))

[0085] (j(q nearest ,q new v1)≠j(q) nearest ,q new ,v2))

[0086] In the formula: j(I,N,P) is a newly added sub-function whose input parameters represent the boundary of the rectangular obstacle in a specific way, and q nearest The closest point; q new v1 and v2 are the new sampling points; v1 and v2 are the two vertices of the rectangular obstacle.

[0087] (4.10) The strategy π(a|s) determines the action a of the AGV in state s, while the state transition probability The state transition after the action is determined, and both the strategy π(a|s) and the state transition probability jointly affect the entire transfer process of the AGV. Substituting the values ​​into the calculation, we obtain state S. t The state-value function V under policy π π (S t The calculation formula is as follows:

[0088]

[0089] In the formula, using To represent the expectation E(R) t+1 |A t =a), which simplifies the above equation to:

[0090]

[0091] Wherein, strategy π(a|S) t and state transition probability matrix As weights, the probabilities of all possible actions and the values ​​of subsequent states are weighted and summed to calculate state S. t The value of this, and thus all subsequent transitions to state S. t+1 The state value is taken into account;

[0092] By designing an algorithm to solve the model, the optimal policy π for each state can be obtained. * That is, the globally optimal path;

[0093] (4.11) The quality of a node is evaluated using a cost function Di. Di represents the cost of a node by calculating the distance between the node and its parent node. This cost includes both spatial distance and implicit time cost. The expression for Di is:

[0094] D i =||q new -M n ||2+ΣD j j = 1, 2, ..., i-1

[0095] In the formula, i is the index of the random tree node, Di is the cost function value of node i; M n It is the nth node in the random tree;

[0096] (4.12) Based on the RRT algorithm, the RRT* algorithm introduces a parent node reconnection strategy. If there are obstacles between a child node and its parent node, the reconnection operation is performed again. When a new neighboring node is added to the random tree, the algorithm checks among nodes whose cost function is less than one compensation range r to see if there is a parent node with a smaller cost function, and updates the parent node to ensure the relative optimality of the path. The calculation formula is as follows:

[0097] M n =T i (x1,y1,0,minC,p i ),||T i -q new || <r

[0098] In the formula: T i Let be the i-th node in the random tree; x1 is the x-coordinate of the new sampling point; y1 is the y-coordinate of the new sampling point; minC is the current minimum cost function; p i This is the index of the parent node of this node;

[0099] (4.13) Introduce the function of finding neighboring nodes to identify q new The nodes around the point will be stored in q. neighbor Next, the algorithm incorporates a mechanism for selecting the parent node. After a new node passes collision detection, its parent node is determined by estimating the cost function. In addition, the search tree is continuously optimized through pruning until the optimal path is found.

[0100] (4.14) The parent node selection process uses a function to find the neighboring nodes of the new node, then iterates through all nodes in the array and calculates the path length from the starting point to the latest node through the neighboring points. The pruning operation determines whether the parent node relationship needs to be adjusted by comparing the path cost of the nodes. It compares the path length formed by two specific nodes through their respective parent nodes with the path length formed through the latest node. If a point is the parent node of the latest node and the path it forms is relatively long, then pruning is not necessary. If the path formed by the new node as the parent node is shorter, then the relevant nodes are pruned. All new connections need to pass the collision detection. If the detection fails, the original connection is retained.

[0101] The improved MRRT* algorithm significantly enhances the path planning capabilities of AGVs in dynamic environments: a dynamic state transition model based on Markov decision processes enables real-time response to environmental changes; rapid spatial exploration is achieved through intelligent sampling and growth functions; path safety is ensured by a precise collision detection mechanism; multi-objective optimization is achieved using state value functions and dynamic cost evaluation; and path quality is continuously optimized through parent node reconnection and pruning strategies. This technology enables AGVs to quickly generate globally optimal paths in environments with complex obstacle distributions, while maintaining the ability to quickly replan for sudden obstacles, balancing path smoothness, safety, and efficiency, making it particularly suitable for industrial logistics scenarios requiring real-time dynamic adjustments.

[0102] Preferably, step 5 includes:

[0103] The AGV car travels along a certain path indoors or outdoors using the MRRT* algorithm to find the global optimal path in step (4). It scans nearby obstacles using the multi-sensor it is equipped with and determines whether there is a step that needs to avoid obstacles. If there is, it jumps to step (6); otherwise, it continues to travel.

[0104] The AGV is guided to travel efficiently and safely by a globally optimal path planned using the MRRT* algorithm, while multiple sensors are used to perceive the dynamics of the surrounding environment in real time. When an obstacle is detected, the system immediately triggers an obstacle avoidance mechanism to ensure the continuity of travel; if there is no obstacle, it runs stably along the original path, taking into account both global path optimization and local dynamic obstacle avoidance capabilities, significantly improving the autonomy, reliability, and adaptability of the AGV in complex indoor and outdoor scenarios.

[0105] Preferably, step 6, which involves using the FPA* algorithm, which integrates the Artificial Potential Field (APF) method and the Dynamic Heuristic Search Algorithm (DHPA*), for local path adjustment, includes:

[0106] (6.1) Create open and close lists, initialize the open and closed lists, and add the starting point and the target point to the open list respectively; at the same time, set the APF algorithm to give the starting point and the target point an initial gravitational potential field so that the AGV is guided by the target gravity from the beginning.

[0107] (6.2) Check if the open list is empty. If it is empty, it means there is no path to follow and the path search has failed. If it is not empty, continue to the next step.

[0108] (6.3) The DHPA* algorithm searches for surrounding child nodes of the current node and calculates the cost, selecting the child node with the lowest cost as the new parent node. Simultaneously, it uses a Close list to record searched nodes and an Open list to store nodes to be expanded. The evaluation function f(n) assesses the merits of each node. The evaluation function f(n) is calculated as follows:

[0109]

[0110] In the formula, f(n) is the evaluation function of node n, which is used to select the node with the minimum cost for expansion; g(n) is the actual cost from the starting point to node n; Z is the weight factor; h(n) is the heuristic estimated cost from node n to the target point; h(ρ) is the heuristic estimated cost from the parent node p of node n to the target point.

[0111] (6.4) Select the node with the minimum cost function from the open table as the current node, obtain its position, and check whether the current node meets the preset "preserve region" condition. If it does not meet the condition, adjust the current path direction and turn; if it does meet the condition, continue to the next path planning step.

[0112] (6.5) The APF algorithm introduces the concept of potential field in path planning to add various virtual force fields to the environment of the AGV. The gravitational potential field is related to the distance and direction from the current position of the AGV to the target point: the greater the distance, the greater the gravitational force on the robot. The direction of the gravitational force is from the current position to the target point, thereby guiding the AGV to gradually approach the target point. The expression of the gravitational potential field function is as follows:

[0113]

[0114] In the formula, U att (q) is the gravitational potential field function, representing the gravitational potential energy at position q; η is the proportionality coefficient used to adjust the strength of the gravitational potential field, ρ 2 (q,q g () represents the distance from the current position q to the target point q. g The Euclidean distance;

[0115] (6.6) The gravitational force generated by the gravitational potential field is calculated using the negative gradient of its potential function, expressed as:

[0116]

[0117] In the formula, ρ(q,q) g () is a vector whose magnitude is the distance between the current position and the target point, and whose direction is from the current position to the target point. Represents the gravitational potential field function U att The gradient of (q) represents the direction of gravity;

[0118] (6.7) The repulsive potential field only works within a certain range around the obstacle. Outside this range, the AGV is no longer affected by the repulsive force of the obstacle. Its expression is:

[0119]

[0120] In the formula, k is the repulsive force coefficient, ρ(q,q0) is the vector pointing from the obstacle to the current point, and its magnitude is the distance from the current point to the obstacle; ρ0 is the radius of action of the repulsive potential field; when 0≤|ρ(q,q0)|≤ρ0, the repulsive force on the AGV increases as the distance between the vehicle and the obstacle decreases; when |ρ(q,q0)|≥ρ0, the AGV is no longer subject to the repulsive force.

[0121] (6.8) The gravitational force generated by the repulsive potential field is its negative gradient, and its functional expression is as follows:

[0122]

[0123] The net force on the AGV is: F(q) = F at (q)+F mq (q);

[0124] (6.9) When the starting point and the ending point are close, a smaller weight coefficient is used for a detailed search to find the shortest path. However, this method is only suitable for small-scale environments. In large-scale environments, although the search time will be shortened, the path may not be the shortest. Therefore, the weight coefficient Z is dynamically adjusted to avoid the algorithm getting trapped in local optima. The expression is:

[0125]

[0126] In the formula, h(n) is the heuristic function, and the threshold of Z is set to 1.5;

[0127] (6.10) In the path search process, the DHPA* algorithm improves the A* algorithm's method of selecting nodes for search based on the actual cost g(n) from the starting point to the current node and the heuristically estimated cost h(n) from the current node to the destination. The DHPA* algorithm uses the sum of the diagonal distance and the distance from the current node's parent node to the destination as the heuristic distance. The formulas for calculating the heuristic distance h(n) and the heuristic distance h(p) of the current node's parent node are as follows:

[0128]

[0129] h(p) = |x1 - x2| + |y1 - y2|

[0130] In the formula, L is the grid size; x and y are the minimum and second minimum absolute values ​​of the coordinate difference between the current node n and the target node; the diagonal distance is... Multiples of the horizontal or vertical distance; yL is the straight-line distance from the parent node to the endpoint; x and y are calculated as follows:

[0131] x = min(|x1-x2|,|y1-y2|)

[0132] y = ||x1-x2|-|y1-y2|||

[0133] In the formula, (x1,y1) and (x2,y2) are the coordinates of the current node n and the target node, respectively;

[0134] (6.11) In step (6.3), the evaluation function f(n) introduces a turning penalty term P to reduce unnecessary turns of the AGV. It is determined by evaluating the two angles formed by the vector from the current node to the endpoint and the parent node and the starting point. The degree of turning when expanding to the next node is evaluated. The specific calculation formula is as follows:

[0135] P t =C1θ1+C2θ2

[0136]

[0137] In the formula, P t θ2 is the turning penalty term; C1 and C2 are weight coefficients, C1 is the angle between PC and CG; θ2 is the angle between CS and CG, C is the current node, P is the parent node of C, S is the starting point, and G is the ending point.

[0138] (6.12) The DHPA* algorithm is the main body, and the APF algorithm is integrated to form the FPA* algorithm. When generating random points, a gravitational potential field is applied to them, so that the current position of the AGV is affected by three force fields: the gravitational potential field of the target point, the gravitational potential field of the random point, and the repulsive potential field of the obstacle.

[0139] By integrating the Artificial Potential Field (APF) method with Dynamic Heuristic Search (DHPA*), the FPA* hybrid algorithm achieves intelligent obstacle avoidance and path optimization for AGVs in dynamic obstacle environments. The APF-based gravity-repulsion model provides real-time motion direction guidance for the AGV, ensuring rapid response to sudden obstacles. The dynamically adjusted heuristic function weights and optimized diagonal distance heuristic significantly improve search efficiency. The introduction of a turning penalty term effectively enhances path smoothness and reduces unnecessary turning energy consumption. The dual-algorithm collaboration achieves a balance between global guidance and local optimization, enabling the AGV to intelligently avoid dynamic obstacles while maintaining path continuity. This is particularly suitable for real-time path adjustment requirements in human-machine hybrid operation scenarios, and overall improves the obstacle avoidance agility and motion smoothness of the AGV.

[0140] Preferably, the DRL agent mentioned in step 6 is a decision-making module integrated into the AGV vehicle control system. It learns the optimal path planning and obstacle avoidance strategy through deep reinforcement learning algorithms, receives state information from the environment, outputs control actions, and continuously optimizes the AGV vehicle's decision-making strategy based on feedback from the reward function. The introduction of deep reinforcement learning (DRL) to assist obstacle avoidance decision-making includes:

[0141] (6.13) Collect historical path planning data and real-time interaction data of AGV vehicles, and generate data samples. These samples cover the operating status, control actions and corresponding reward values ​​of AGV vehicles in different environments.

[0142] (6.14) Extract control actions from data samples to form an action space, including the speed adjustment and steering angle adjustment of the AGV vehicle. Extract the working state formed by executing the control actions to form a state space, which includes the position, speed, position and distance of the AGV vehicle and surrounding obstacles.

[0143] (6.15) Set a reward function to match a reward value for each control action. The design goal of the reward function is to encourage the AGV to reach the target point quickly and safely, while avoiding collisions with obstacles. Successfully avoiding obstacles and approaching the target point will result in a positive reward, while colliding with obstacles or deviating from the target direction will result in a negative reward. The calculation formula is as follows:

[0144]

[0145] r = -ar * +b+∈

[0146] In the formula, r* represents the total cost after compensation, and A i Represents control cost, α i s represents the control cost coefficient. i P represents the percentage of the control variable. imax Indicates the upper limit of the control variable; Bj c represents the cost compensation value for exceeding the decision limit. j x is the over-limit compensation coefficient for the motion. j To output the decision value, x s γ is the action limit; r is the reward value; a and b are the linear coefficients, respectively. i and δ j It is a compensation term related to control; ∈ is a random disturbance term;

[0147] (6.16) Establish an experience base to store a quadruple sample (s, a, r, s') formed by the current state s, control action a, reward value r for performing the control action, and the next state s' obtained after performing the control action, in order to train the DRL agent.

[0148] (6.17) Set the state action value function Q(s,a) as the evaluation index to judge the quality of the control action performed in the current state. The calculation formula is as follows:

[0149]

[0150] Where, r t+n Let γ be the reward value for performing control action a at time t+n, and γ be the discount factor.

[0151] (6.18) Set up the Actor network and Critic network, receive the batch input quadruplets (s, a, r, s'), and use the Adam optimizer to calculate the gradient according to the set state action value function Q, and update the network parameters of the Actor network and Critic network accordingly.

[0152] This technology endows AGVs with autonomous decision-making and optimization capabilities through a deep reinforcement learning (DRL) module: it accumulates experience by building a training set based on historical data; it accurately describes complex environmental interactions through multi-dimensional state / action space modeling; an innovatively designed composite reward function balances path efficiency and motion stability; an experience playback mechanism improves learning efficiency; and a dual-network architecture enables continuous iterative optimization of strategies. This technology gives AGVs online learning capabilities, allowing them to adaptively adjust obstacle avoidance strategies and exhibit human-like decision-making intelligence in unknown dynamic environments, significantly improving their autonomous adaptability in unstructured scenarios and long-term task reliability.

[0153] Preferably, the objective functions optimized by the MRRT* algorithm and the FPA* algorithm in steps (4) and (6) include:

[0154] In AGV path planning, whether it's global planning or local planning, three objective functions are involved: the total path length f1(X). i ), the proximity of the path to the infeasible region f2(X) i), path smoothness f3(X) i The mean squared error (RMSE) is used to perform a weighted evaluation of the multi-objective function.

[0155] Shortest path length L min ;

[0156] Completely avoid infeasible areas;

[0157] Completely smooth.

[0158] By employing a multi-objective optimization framework, this method unifies and quantifies the total path length, obstacle avoidance safety, and smoothness, and uses mean squared error (RMSE) for weighted evaluation, achieving a synergistic balance between global and local optimization in AGV path planning. Mathematically, this method ensures that under idealized constraints of shortest path, absolute obstacle avoidance, and perfect smoothness, it approximates the optimal solution through a weighted compromise, thereby simultaneously improving the AGV's driving efficiency, safety, and motion stability in complex environments. This provides a highly robust decision-making basis for real-time path adjustment in dynamic obstacle scenarios.

[0159] Secondly, the intelligent AGV path planning system of the present invention includes:

[0160] Multi-source environmental perception module: Composed of 360° lidar, binocular vision, infrared thermal imaging and ultrasonic array module, used for multi-dimensional data acquisition, environmental perception and dynamic pedestrian obstacle avoidance of the operating environment;

[0161] Low-rank multi-source feature fusion module: used to fuse the multi-sensor, multi-dimensional data collected in step (1) using the low-rank multi-source feature fusion module LMF;

[0162] Map building module: Used to process the fused multi-dimensional data using the VF-Cart algorithm to build a global two-dimensional grid map, thus obtaining an indoor and outdoor AGV operating environment map;

[0163] Global Path Planning Module: Based on the indoor and outdoor AGV operating environment map, it uses the improved RRT* algorithm MRRT* to perform global path planning for AGV vehicles; the MRRT* algorithm combines Markov Decision Process (MDP) and RRT*, that is, it utilizes the ability of MDP to process dynamic data and the efficient path search capability of RRT* in complex environments.

[0164] Dynamic obstacle detection module: used to detect pedestrians or obstacles on the path in real time using multiple sensors while moving according to the globally optimal path;

[0165] Local path optimization module: An FPA* fusion algorithm is proposed. This algorithm uses the gravitational field of the Artificial Potential Field (APF) method to guide the AGV's initial movement direction, and combines the Dynamic Heuristic Search (DHPA*) algorithm to dynamically adjust the heuristic function. Combined with real-time environmental information, it finds a feasible path from the current position to the target point, optimizes the local path of the AGV during its operation, and introduces the Deep Reinforcement Learning (DRL) method for obstacle avoidance auxiliary decision-making to complete the obstacle avoidance function.

[0166] Motion control module: Executes the final planned path instructions to drive the AGV to the target point.

[0167] Thirdly, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program capable of being loaded by the processor and executing the multi-sensor fusion intelligent AGV path planning method.

[0168] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned intelligent AGV path planning method based on multi-sensor fusion.

[0169] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. Through multi-sensor collaborative work, it achieves comprehensive and multi-dimensional perception of the environment, significantly improving the obstacle detection capability and environmental adaptability of AGV in dynamic environments; 2. The LMF module effectively removes noise and redundant information, fully preserving the correlation and complementarity characteristics of multi-source data, providing high-quality data support for path planning; 3. The improved VF-Cart algorithm, through particle filtering weighting and secondary screening, can effectively handle the uncertainty of sensor data and generate a passable map that accurately reflects the distribution of environmental obstacles; 4. The MRRT* algorithm, combining the dynamic decision-making capability of MDP and the efficient search characteristics of RRT, enables AGV to effectively cope with the path planning requirements in dynamic environments; 5. The fusion algorithm of APF and DHPA* (FPA*) achieves fast path guidance and real-time obstacle avoidance, and combined with the auxiliary decision-making of DRL, further improves the accuracy and reliability of obstacle avoidance in dynamic environments. Attached Figure Description

[0170] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0171] Figure 2 This is a schematic diagram of the VF-Cart algorithm flow of the present invention;

[0172] Figure 3 This is a schematic diagram of the MRRT* algorithm flow of the present invention;

[0173] Figure 4 This is a schematic diagram of the FPA* algorithm flow of the present invention;

[0174] Figure 5 This is a schematic diagram of the overall process of AGV (global + local) path planning in this invention. Detailed Implementation

[0175] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0176] like Figure 1 and Figure 5 As shown, this invention discloses a multi-sensor fusion intelligent AGV path planning method and system. First, the AGV is equipped with multiple sensors, including a 360° LiDAR, binocular vision, infrared thermal imaging, and ultrasonic array, to collect multi-dimensional map environment data, achieving centimeter-level obstacle detection and dynamic pedestrian recognition. The collected multi-sensor data is fused using a low-rank multi-source feature fusion module (LMF). A two-dimensional grid map is constructed using the VF-Cart algorithm. Based on the optimized environment map, the AGV employs MRRT (Mean Modulated Path Recognition). * The algorithm performs global path planning, and the improved MRRT * The algorithm combines Markov Decision Process (MDP) and Regression Theory (RRT). * Combining the capabilities of MDP in processing dynamic data with RRT * The AGV achieves efficient pathfinding in complex environments, thus obtaining the globally optimal path. Simultaneously, to prevent obstacles from preventing the AGV from reaching the target point during the globally optimal pathfinding process, the AGV employs an Artificial Potential Field (APF) gravitational field to guide its initial movement direction, in conjunction with the DHPA (Dual Path Approach). * Combined algorithm FPA * Optimize local paths, DHPA * By dynamically adjusting the weight coefficients, the AGV (Automated Guided Vehicle) effectively avoids getting stuck in local optima. Furthermore, it incorporates Deep Reinforcement Learning (DRL) for obstacle avoidance decision-making, thus completing the obstacle avoidance function. This invention enables the AGV to accurately and quickly plan the optimal path in complex dynamic environments, reducing AGV scheduling time.

[0177] Specifically, the intelligent AGV path planning method includes the following steps:

[0178] Step 1: The AGV vehicle is equipped with a 360° lidar, binocular vision, infrared thermal imaging and ultrasonic array module to collect multi-dimensional data of the operating environment, perceive the environment and dynamically avoid pedestrian obstacles.

[0179] (1.1) The 360° lidar measures the distance to surrounding objects by emitting a laser beam and receiving reflected light. It also acquires three-dimensional environmental point cloud data around the AGV vehicle through rotational scanning, performing 360° horizontal scanning and multi-angle vertical scanning to generate a high-precision three-dimensional point cloud map. The environmental coordinate vector is:

[0180] z k =(d k ,θ k k = 1, 2, 3, ..., N

[0181] Among them, z k For 3D point cloud data, d k For lidar at θ k The distance value obtained from the angle.

[0182] (1.2) The binocular vision module acquires environmental images through a binocular camera and uploads them to the onboard computer. The ORB feature extraction method is used to extract feature points in the image with drastic changes in gray value or large edge curvature, which are used for the identification and tracking of dynamic pedestrians and obstacles, and to assist the AGV in completing path planning.

[0183] (1.3) The infrared thermal imaging module is connected to the data processing module and is used to detect the environment around the AGV and transmit it to the data processing module. It can effectively detect human body heat and accurately detect pedestrians even when binocular vision sensors and lidar are difficult to identify.

[0184] (1.4) The ultrasonic array module is used for near-range obstacle detection in AGVs. It can quickly identify targets within a few centimeters to a few meters, and is especially effective in detecting small obstacles or near-range collision risks in the chassis blind spot. The distance formula is as follows:

[0185] H = 0.5s = 0.5vt

[0186] Where H is the distance between the obstacle and the ultrasonic rangefinder, s is the round-trip distance of the ultrasonic wave, v is the speed of the ultrasonic wave, and t is the round-trip time.

[0187] Step 2: The collected multi-sensor, multi-dimensional data is fused using the Low-Rank Multi-Source Feature Fusion (LMF) module.

[0188] (2.1) This scheme adopts the low-rank multi-source feature fusion (LMF) method. By calculating the outer product of different modal features, the correlation and complementarity between modalities are explored, while ensuring the integrity of each modal feature, so as to make full use of multi-sensor data.

[0189] (2.2) The LMF algorithm decomposes the weight tensor x into the outer product of four mode-specific low-rank factors. The specific calculation formula is as follows:

[0190]

[0191] The smallest r that makes the above decomposition valid is called the effective rank of the tensor. Let β be the weight matrix of the j-th mode in the i-th low-rank factor. i Indicates additional weight. α represents the outer product of tensors, used to generate low-rank factors, and α is a coefficient used to balance the relationship between the low-rank approximation and the regularization term.

[0192] (2.3) The output h is obtained by multiplying the corresponding elements of the input modal features X by a low-rank factor and summing the results, thus obtaining the final multi-sensor feature fusion data. The specific calculation formula is as follows:

[0193]

[0194] in, The Hadamard product of four modal tensors, x n The input features represent the nth modality. The LMF algorithm reduces the computational burden and adapts to different numbers of modality fusions by not constructing a high-dimensional tensor W.

[0195] Step 3: Use the VF-Cart algorithm to construct a two-dimensional grid map to obtain an indoor and outdoor AGV operating environment map. The VF-Cart algorithm process is as follows: Figure 2 As shown.

[0196] (3.1) The VF-Cart algorithm is based on the Cartographer algorithm. It introduces weighted voxel filtering to preprocess point cloud data. This method combines particle filtering, weights the point cloud collected by 360° LiDAR and performs secondary filtering. The weighted and filtered point cloud data, along with the pose data collected by IMU and wheel odometry, are input into the back-end optimization part.

[0197] (3.2) First, the point cloud data obtained by the 360° lidar scan is processed by voxel filtering, and all points in the grid are replaced by the centroid of each grid. Then, the centroid is projected onto a two-dimensional plane to construct a voxel grid index.

[0198]

[0199] Where: φ voxel (x1, y1, z1) represents the projection of the three-dimensional voxel mesh onto the two-dimensional plane. The original coordinates of the centroid of the voxel mesh. Let f() represent the three-dimensional coordinates of the point cloud.

[0200] (3.3) Next, the state of the point cloud projected onto the two-dimensional grid is estimated to predict its next state information. The specific calculation formula is as follows:

[0201] h t =H(Φ voxel (x1,y1,z1)+βK t )+δ+∈

[0202] Where: h t K represents the state and pose of a point cloud. t H() represents the quantity controlling the input point cloud, δ is the state transition function, β and ∈ are the noise disturbance during state transition, and β and ∈ are new variables.

[0203] (3.4) By comparing the predicted point cloud with the data before prediction, overlapping point clouds are assigned low weights, and non-overlapping point clouds are assigned high weights. First, the observation model function V() is used to calculate the observed value of each point cloud, and then weights are assigned based on the probability comparison results:

[0204] v t =V1(h t )+V2(δ)

[0205]

[0206] In the formula, v t For each observation of the point cloud, w t For the weighted result, P(v t |h t ) represents the pose h in a given state. t v was observed below t The probability of.

[0207] (3.5) Finally, filtering and fusion are performed. The weighted point cloud data is projected onto the world coordinate system and fused with the voxelized data of the corresponding grid. The point cloud weights are then weighted before filtering, and the projected state information and point cloud weights are updated first:

[0208]

[0209] In the formula, h t+1 The updated state pose; To accumulate the state pose at each time step; ε t For new noise disturbances; w t+1 The updated weighting result; P(v t |h t+1 ) represents the state pose h after the update. t+1 v was observed below t The probability of; This is a new variable.

[0210] (3.6) After filtering is complete, the state and density of the filtered point cloud information can be updated:

[0211]

[0212] The updated point cloud density value is θ n (x1, y1, z1), with weight w t+1 The updated point cloud status information is as follows: h t+1 For the point cloud state information of the new stage, This represents the unit spatial distribution value.

[0213] (3.7) In the Cartographer algorithm, the position and orientation of the AGV are represented by ξ = (ξ x ,ξ y ,ξ θ ) indicates that ξ x and ξ y ξ is the distance the AGV moves in the x and y directions. θ It is its direction of motion angle. The environmental data frame acquired by the 360° lidar is denoted as L={l k} k＝1....K ,l k ∈R 2 , where each l k It is a two-dimensional vector, transformed by pose F ξ Mapping each point in the data frame to a subgraph is calculated using the following formula:

[0214]

[0215] In the formula, s is the position of the original point, and ρ x ρ y The coordinates are the reference point.

[0216] (3.8) Before the scan frame is added to the submap, the pose ξ is processed by the Ceres solver to handle the nonlinear least squares problem, thereby obtaining the pose information of the scan frame to improve the mapping accuracy. The specific calculation formula is as follows:

[0217]

[0218] In the formula, F ξ l ξ For point cloud data after pose transformation, M smooth It is a smoothing function.

[0219] (3.9) When a map is constructed from multiple subgraphs, matching the scan frame only with the current subgraph can easily lead to accumulated errors. The Cartographer algorithm optimizes the LiDAR data frame and AGV pose through sparse pose adjustment. The specific calculation formula is as follows:

[0220]

[0221] In the formula: These represent the sub-image pose and the scan frame pose, respectively. ij Let A be the relative pose of the scanned frames in the subgraph, A be the optimized pose, ρ be the loss function used to measure the error, and E be the relative pose of the scanned frames in the subgraph. 2 To calculate the error between the lidar data frame and the AGV pose.

[0222] (3.10) The Cartographer algorithm improves the speed of loop closure detection and increases computational efficiency by using a branch-and-bound scan matching algorithm. The calculation formula is as follows:

[0223]

[0224] In the formula: W is the search window, M nearest For the extension of the M function, R(δ) is the regularization term. It is the regularization parameter.

[0225] Step 4: Using the environmental map obtained in Step 3, apply an improved RRT. * Algorithm MRRT * Perform global path planning for AGV vehicles; the MRRT * The algorithm includes Markov Decision Process (MDP) and RRT. * Combining the capabilities of MDP in processing dynamic data with RRT * MRRT provides efficient path search capabilities in complex environments, improving global path planning efficiency. * The algorithm flow is as follows Figure 3 As shown.

[0226] (4.1) In a two-dimensional environment, the RRT* algorithm starts from the starting point and generates the connection starting point M. i and target point M g A random tree with a compensation range of r is constructed. The initial position of the AGV is taken as the starting point and set as the root node of the random tree T. The target point is the end point of the AGV.

[0227] (4.2) In a two-dimensional environment, a point q is randomly sampled. rand .

[0228] (4.3) The nearest point search is to find a node q in the search tree. nearest , making qnearest With random sample point q rand The minimum distance between them is required, which requires traversing the search tree. The input is the search tree and random sample points q. rand The output is the nearest node q. nearest The expression is:

[0229]

[0230] In the formula: q nearest Represents the nearest node, q rand Let v represent a random sampling point, v represent a certain search tree, and T represent the set of search trees.

[0231] (4.4) Markov Decision Process (MDP) is introduced to enhance the handling of dynamic obstacles. Path planning is optimized by defining states, actions, transition probabilities, and reward functions. Actions are typically represented by the symbol 'a', and all possible actions constitute the action set A. State transition probabilities and immediate rewards are closely related to the current state and action. The AGV is in state S... t The set of optional actions is A. t In different states, the action set A t They may be different.

[0232] (4.5) Under a certain state, the possible actions of the AGV follow a certain probability distribution, which is called the policy, denoted by the symbol π(a|s), and the specific formula is as follows:

[0233] π(a|s)=P[A t =a|S t =s]

[0234] (4.6) The strategy π(a|s) represents the probability distribution of the AGV choosing action a in state s. It depends only on the current state and does not rely on historical information. It determines the behavior of the AGV in the current state. Although the strategy is fixed at a certain moment, the AGV can dynamically adjust and optimize the strategy over time to achieve the optimal decision in each state.

[0235] (4.7) According to step (4.3), the random sampling point and the nearest node are input into the growth function to generate a new node q. new The specific expression is as follows:

[0236]

[0237] In the formula: step represents the movement step length of the AGV, which can be adjusted according to the environment and requirements.

[0238] (4.8) In state S tThe AGV performs action a1 (a1∈A) t When ), the state transition is not fixed at S. t →S t+1 Instead, it is stored in the state transition probability matrix P in the form of a probability distribution. StSt+1|a In this context, it exhibits the characteristics of stochastic dynamic programming, and its specific expression is as follows:

[0239]

[0240] Since the indoor and outdoor movements of AGVs are highly random and dynamic, using MDP to process dynamic data for AGVs can significantly improve path planning performance.

[0241] (4.9) Collision detection determines whether a new node conflicts with an obstacle: For a circular obstacle, calculate the shortest distance from its center to the line connecting the new node and the nearest node. If this distance is less than or equal to the obstacle's radius, the path is not feasible; otherwise, the path is considered feasible as it does not cross the obstacle.

[0242] For a rectangular obstacle, first determine whether the new node is inside or on its boundary. If it is inside, the path is not feasible; if it is outside, then check whether the line connecting the new node and the nearest node intersects the obstacle. If they intersect, the path is not feasible. The lines connecting points A and D to the sampling points are considered boundary lines, and their specific expressions are as follows:

[0243]

[0244] In the above formula, the variable k represents the slope of the line.

[0245] (4.10) If through point q nearest A straight line that satisfies a certain condition indicates that it does not intersect any obstacle. For collision detection, it is assumed that the boundary of each rectangular obstacle has a boolean value (bool). i When bool i When the value is 1, it indicates that the line intersects with the obstacle; when boolean, it indicates that the line intersects with the obstacle. i When the value is 0, it indicates that there is no intersection. Therefore, the collision detection function needs to include an auxiliary subroutine to manipulate these Boolean values, as shown in the following formula:

[0246] judge(I,N,P)=((y P -y I (x) N -x I ))>((y N -y I (x) P -x I ))

[0247] bool i =(j(q) nearest ,v1,v2)≠j(q new ,v1,v2))

[0248] (j(q nearest ,q new v1)≠j(q) nearest ,q new ,v2))

[0249] In the formula: j(I,N,P) is a newly added sub-function whose input parameters represent the boundary of the rectangular obstacle in a specific way, and q nearest The closest point; q new v1 and v2 are the new sampling points; v1 and v2 are the two vertices of the rectangular obstacle.

[0250] (4.11) The strategy π(a|s) determines the action a of the AGV in state s, while the state transition probability The state transition after the action is determined. Both the policy π(a|s) and the state transition probability jointly influence the entire transition process of the AGV. Substituting the values ​​into the calculation, we can obtain state S. t The state-value function V under policy π π (S t The specific calculation formula is as follows:

[0251]

[0252] In the formula, we can use To represent the expectation E(R) t+1 |A t =a), which simplifies the above expression to:

[0253]

[0254] Wherein, strategy π(a|S) t ) and state transition probability As weights, the probabilities of all possible actions and the values ​​of subsequent states are weighted and summed to calculate state S. t The value of this, and thus all subsequent transitions to state S. t+1 The state value is taken into account.

[0255] By designing an algorithm to solve the model, the optimal policy π for each state can be obtained. * That is, the globally optimal path.

[0256] (4.12) The cost function Di is used to evaluate the quality of a node. It reflects the cost of a node by calculating the distance between the node and its parent node. This cost includes both spatial distance and implicit time cost. The expression for Di is:

[0257] D i =||q new -M n ||2+∑D j j = 1, 2, ..., i-1

[0258] In the formula, i is the index of the random tree node, Di is the cost function value of node i; M n It is the nth node in the random tree.

[0259] (4.13) Based on the RRT algorithm, the RRT* algorithm introduces a parent node reconnection strategy. If there are obstacles between a child node and its parent node, the reconnection operation is performed again. When a new neighboring node is added to the random tree, the algorithm will check if there is a parent node with a smaller cost function among nodes whose cost function is less than one compensation range r, and update the parent node to ensure the relative optimality of the path. The RRT* algorithm can gradually optimize the path in this way. The specific calculation formula is as follows:

[0260] M n =T i (x1,y1,0,minC,p i ),||T i -q new || <r

[0261] T i Let be the i-th node in the random tree; x1 is the x-coordinate of the new sampling point; y1 is the y-coordinate of the new sampling point; minC is the current minimum cost function; p i This is the index of the parent node of this node.

[0262] The RRT* algorithm introduces an optimization process of reselecting parent nodes and rewiring based on RRT. It reduces computational cost through incremental optimization, quickly finds the initial path, and continuously iterates and optimizes until the target point is reached.

[0263] (4.14) Introduce the function of finding neighboring nodes and identify q. new The nodes around the point will be stored in q. neighbor Next, a mechanism for selecting a parent node is incorporated into the algorithm. After a new node passes collision detection, its parent node is determined by estimating the cost function. Furthermore, the search tree is continuously optimized through pruning until the optimal path is found.

[0264] (4.15) The parent node selection process involves using a function to find the neighboring nodes of the new node, then traversing all nodes in the array and calculating the path length from the starting point through neighboring nodes to the latest node. The pruning operation determines whether to adjust parent node relationships by comparing the path costs of nodes. If the path length formed by two specific nodes through their respective parent nodes is longer than the path length formed through the latest node, pruning is unnecessary. If the path formed by the new node as the parent node is shorter, then the relevant nodes are pruned. All new connections must undergo collision detection; if a collision fails, the original connection is retained.

[0265] Step 6: An FPA* fusion algorithm is proposed. This algorithm uses the Artificial Potential Field (APF) method to guide the AGV's initial movement direction with a gravitational field, and combines it with the Dynamic Heuristic Search Algorithm DHPA* to dynamically adjust the heuristic function. By incorporating real-time environmental information, it quickly finds a feasible path from the current position to the target point, avoiding the problem of unreachable targets. It optimizes the local path during the AGV's operation and introduces Deep Reinforcement Learning (DRL) for obstacle avoidance assistance, effectively completing the obstacle avoidance function. The FPA* algorithm is as follows: Figure 4 As shown.

[0266] (6.1) Create open and close lists, initialize the open and closed lists, and add the starting point and the target point to the open list respectively; at the same time, set the APF algorithm to give the starting point and the target point an initial gravitational potential field so that the AGV is guided by the target gravity from the beginning.

[0267] (6.2) Determine if the open list is empty. If it is empty, it means there is no path to follow and the path search has failed. If it is not empty, continue to the next step.

[0268] (6.3) The DHPA* algorithm searches for surrounding child nodes of the current node and calculates the cost, selecting the child node with the lowest cost as the new parent node. Simultaneously, a Close list records the searched nodes, an Open list stores nodes to be expanded, and an evaluation function f(n) assesses the merits of each node. The evaluation function f(n) is calculated as follows:

[0269]

[0270] In the formula, f(n) is the evaluation function of node n, which is used to select the node with the minimum cost for expansion; g(n) is the actual cost from the starting point to node n; Z is the weight factor; h(n) is the heuristic estimated cost from node n to the target point; and h(p) is the heuristic estimated cost from the parent node p of node n to the target point.

[0271] (6.4) Select the node with the minimum cost function from the open table as the current node and obtain its position. Check whether the current node meets the "preserve region" condition. If not, the current path direction needs to be adjusted and a turn is required. If it meets the condition, continue to the next step of path planning.

[0272] (6.5) The APF algorithm introduces the concept of a potential field in path planning to add various virtual force fields to the environment of the AGV. The gravitational potential field is related to the distance and direction from the current position of the AGV to the target point: the greater the distance, the greater the gravitational force on the robot, and the direction of the gravitational force is from the current position to the target point, thus guiding the AGV to gradually approach the target point. The expression of the gravitational potential field function is as follows:

[0273]

[0274] In the formula, U att (q) is the gravitational potential field function, representing the gravitational potential energy at position q; η is the proportionality coefficient used to adjust the strength of the gravitational potential field, ρ 2 (q,q g () represents the distance from the current position q to the target point q. g The Euclidean distance.

[0275] (6.6) The gravitational force generated by the gravitational potential field is calculated using the negative gradient of its potential function, expressed as:

[0276]

[0277] In the formula, ρ(q,q) g () is a vector whose magnitude is the distance between the current position and the target point, and whose direction is from the current position to the target point. Represents the gravitational potential field function U att The gradient of (q) represents the direction of gravity.

[0278] (6.7) The repulsive potential field only works within a certain range around the obstacle. Outside this range, the AGV is no longer affected by the repulsive force of the obstacle, thus avoiding the inability to reach the target point due to obstacles when approaching it. Its expression is:

[0279]

[0280] (6.8) The gravitational force generated by the repulsive potential field is its negative gradient, and its functional expression is as follows:

[0281]

[0282] In the formula: k is the repulsive force coefficient; ρ(q,q0) is the vector pointing from the obstacle to the current point, and its magnitude is the distance from the current point to the obstacle; ρ0 is the radius of the repulsive potential field. When 0≤|ρ(q,q0)|≤ρ0, the repulsive force on the AGV increases as the distance between the vehicle and the obstacle decreases; when |ρ(q,q0)|≥ρ0, the AGV is no longer subject to repulsive force.

[0283] The net force on the AGV is F(q) = F at (q)+F mq (q).

[0284] (6.9) When the starting and ending points are close, the algorithm uses a smaller weight coefficient for a detailed search to find the shortest path. However, this method is only suitable for small-scale environments. In large-scale environments, although the search time will be shortened, the path may not be the shortest. Therefore, the weight coefficient Z is dynamically adjusted to avoid the algorithm getting trapped in local optima. The expression is:

[0285]

[0286] In the formula: h(n) is the heuristic function; the threshold is set to 1.5, and α is an adjustment parameter used to adjust the weight coefficient Z.

[0287] (6.10) The DHPA* algorithm improves upon the A* algorithm's method of selecting nodes for search by using the actual cost g(n) from the starting point to the current node and the heuristically estimated cost h(n) from the current node to the destination. The DHPA* algorithm uses the sum of the diagonal distance and the distance from the current node's parent node to the destination as the heuristic distance. This reduces the number of searches and nodes, significantly improving search efficiency. The formulas for calculating the heuristic distance h(n) and the heuristic distance h(p) from the current node's parent node are as follows:

[0288]

[0289] h(p) = |x1 - x2| + |y1 - y2|

[0290] In the formula, L is the grid size; x and y are the minimum and second minimum absolute values ​​of the coordinate difference between the current node n and the target node; the diagonal distance is... Multiples of the horizontal or vertical distance; yL is the straight-line distance from the parent node to the endpoint; x and y are calculated as follows:

[0291] x=min(|x1-x2|,|y1-y2|); y=||x1-x2|-|y1-y2|||

[0292] (6.11) In step (6.3), a turning penalty term P is introduced into the evaluation function f(n) to reduce unnecessary turns by the AGV. This is determined by evaluating the two angles formed by the vector from the current node to the endpoint and the parent node and the starting point. The degree of turning when expanding to the next node is evaluated, minimizing the number of turns during the AGV's journey and thus optimizing path selection. The specific calculation formula is as follows:

[0293] P t =C1θ1+C2θ2

[0294] in:

[0295]

[0296] In the formula: P t θ1 is the turning penalty term; C1 and C2 are weight coefficients; θ1 is the angle between PC and CG; θ2 is the angle between CS and CG; C is the current node; P is the parent node of C; S is the starting point; and G is the ending point.

[0297] (6.12) The DHPA* algorithm is the main body, which is combined with the APF algorithm to form the FPA* algorithm. When generating random points, a gravitational potential field is applied to them, so that the current position of the AGV is affected by three force fields: the gravitational potential field of the target point, the gravitational potential field of the random point, and the repulsive potential field of the obstacle.

[0298] Based on a similar inventive concept, this invention also provides an intelligent AGV path planning system corresponding to the aforementioned intelligent AGV path planning method, comprising:

[0299] Multi-source environmental perception module: Composed of 360° lidar, binocular vision, infrared thermal imaging and ultrasonic array module, used for multi-dimensional data acquisition, environmental perception and dynamic pedestrian obstacle avoidance of the operating environment;

[0300] Low-rank multi-source feature fusion module: used to fuse the multi-sensor, multi-dimensional data collected in step (1) using the low-rank multi-source feature fusion module LMF;

[0301] Map building module: Used to process the fused multi-dimensional data using the VF-Cart algorithm to build a global two-dimensional grid map, thus obtaining an indoor and outdoor AGV operating environment map;

[0302] Global Path Planning Module: Based on the indoor and outdoor AGV operating environment map, it uses the improved RRT* algorithm MRRT* to perform global path planning for AGV vehicles; the MRRT* algorithm combines Markov Decision Process (MDP) and RRT*, that is, it utilizes the ability of MDP to process dynamic data and the efficient path search capability of RRT* in complex environments.

[0303] Dynamic obstacle detection module: used to detect pedestrians or obstacles on the path in real time using multiple sensors while moving according to the globally optimal path;

[0304] Local Path Optimization Module: An FPA* fusion algorithm is proposed. This algorithm uses the gravitational field of the Artificial Potential Field (APF) method to guide the initial movement direction of the AGV, and combines the Dynamic Heuristic Search (DHPA*) algorithm to dynamically adjust the heuristic function. Combined with real-time environmental information, it finds a feasible path from the current position to the target point. The FPA* fusion algorithm is used to optimize the local path of the AGV during its operation, and the Deep Reinforcement Learning (DRL) method is introduced to assist in obstacle avoidance decision-making, thus completing the obstacle avoidance function.

[0305] Motion control module: Executes the final planned path instructions to drive the AGV to the target point.

[0306] The present invention also discloses an electronic device.

[0307] Specifically, the electronic device can be a desktop computer, laptop computer, handheld computer, or cloud server, etc. This computer device may include, but is not limited to, a processor and memory. The processor and memory can be connected via a bus or other means. The processor can be a Central Processing Unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, graphics processing units (GPUs), embedded neural network processing units (NPUs) or other dedicated deep learning coprocessors, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0308] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor executes various functional applications and data processing by running non-transitory software programs, instructions, and modules stored in memory. Memory may include a program storage area and a data storage area. The program storage area may store the control unit and the application program required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, memory may include high-speed random access memory and non-transitory memory. In some embodiments, memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0309] The present invention also discloses a computer-readable storage medium.

[0310] Specifically, the computer-readable storage medium is used to store a computer program, which, when executed by a processor, implements the methods described in the above method implementation.

[0311] Those skilled in the art will understand that all or part of the processes in the methods described above can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

Claims

1. A multi-sensor fusion intelligent AGV path planning method, characterized in that, Comprise the following steps: (1) The AGV car passes through the 360°laser radar, binocular vision, infrared thermal imaging and ultrasonic array module, multi-dimensional data collection, environment perception and dynamic pedestrian obstacle avoidance of running environment are carried out; (2) The low rank multi-source feature fusion module LMF is used to fuse the multi-dimensional data collected in step (1); (3) The VF-Cart algorithm is used to process the fused multi-dimensional data, a global two-dimensional grid map is constructed, and an indoor and outdoor AGV running environment map is obtained; (4) Based on the indoor and outdoor AGV running environment map, the improved RRT*algorithm MRRT*is used for global path planning of AGV vehicle;The MRRT*algorithm combines Markov decision process MDP and RRT*, that is, the ability of MDP to process dynamic data and the efficient path search ability of RRT*in complex environment; (5) In the process of advancing according to the global optimal path obtained in step (4), the AGV car detects pedestrians or obstacles on the path in real time through multiple sensors, and if an obstacle is detected, step (6) is executed, otherwise step (7) is executed; (6) A kind of FPA*fusion algorithm is proposed, which uses artificial potential field APF to guide the direction of AGV early movement, and combines dynamic heuristic search algorithm DHPA*to dynamically adjust heuristic function, combines real-time environmental information, finds a feasible path from the current position to the target point, optimizes the local path of AGV car in the process of running, and introduces deep reinforcement learning DRL method for obstacle avoidance auxiliary decision-making to complete the obstacle avoidance function; (7) The AGV car continues to run according to the planned path until it runs to the corresponding target point in the planned path.

2. The intelligent AGV path planning method of claim 1, wherein, Step 1 includes: (1.1) The 360°laser radar measures the distance from the surrounding objects by emitting laser beams and receiving reflected light, and obtains the three-dimensional environmental point cloud data around the AGV car by rotating scanning, which generates a high-precision three-dimensional point cloud map by 360°scanning in the horizontal direction and multi-angle scanning in the vertical direction, and the environmental coordinate vector is: z k = (d k , θ k )k = 1, 2, 3, …, N In the formula, z k is three-dimensional point cloud data, d k is a distance value obtained by the laser radar at the θ k angle; (1.2) The binocular vision module collects environmental images through binocular cameras and uploads them to the onboard computer, and uses ORB feature extraction method to extract feature points with large gray value changes or large edge curvature in the image, which is used for dynamic pedestrian and obstacle recognition and tracking to assist AGV to complete path planning; (1.3) The infrared thermal imaging module is connected with the data processing module, which is used for detecting the environment around AGV and transmitting to the data processing module, and realizing pedestrian detection in the environment that cannot be recognized by binocular vision sensor and laser radar; (1.4) The ultrasonic array module is used for AGV close-range obstacle detection, which can identify the target distance within a few centimeters to a few meters and find small obstacles or close-range collision risks in the blind area of chassis, and the distance calculation formula is: H=0.5s=0.5vt In the formula, H is the distance between the obstacle and the ultrasonic range finder, s is the round trip distance of ultrasonic wave, v is the speed of ultrasonic wave, and t is the round trip time.

3. The intelligent AGV path planning method of claim 1, wherein, Step 2 includes: (2.1) The low-rank multi-source feature fusion LMF method is used to calculate the exterior product of different modal features, to mine the correlation and complementarity between modalities, and to ensure the integrity of each modal feature, and to make full use of multi-sensor data; (2.2) The weight tensor x is decomposed into the exterior product of four modal specific low rank factors, and the calculation formula is: where the smallest r for which the above decomposition is valid is called the effective rank of the tensor; is the weight matrix for the n-th modality in the i-th low-rank factor; β i denotes an additional weight; denotes the outer product of tensors, used to generate the low-rank factors; α is a coefficient used to balance the relationship between the low-rank approximation and the regularization term; (2.3) The final multi-sensor feature fusion data h is obtained by multiplying the low rank factor with the corresponding element of the input modal feature X and summing up, and the calculation formula is: wherein, denotes the Hadamard product of the four modal tensors, x n represents the input features of the n-th modality.

4. The intelligent AGV path planning method of claim 1, wherein, Step 3 includes: (3.1) First, the point cloud data obtained by 360° laser radar scanning is filtered by voxel, and the gravity point of each grid is used to replace all points in the grid, and then the gravity point is projected onto the two-dimensional plane to construct the voxel grid index: wherein: Φ voxel (x1, y1, z1) represents the projection of the three-dimensional voxel grid on the two-dimensional plane, represents the original coordinates of the voxel grid center of gravity, represents the three-dimensional coordinates of the point cloud, and f() is a projection function; (3.2) State estimation is performed on the point cloud projected onto the two-dimensional grid to predict the next step state information, and the calculation formula is as follows: h t = H(Φ voxel (x1,y1,z1)+βK t )+δ+∈ where h t denotes the state pose of the point cloud, K t denotes the control input to the point cloud, H() is the state transition function, δ is the noise disturbance in state transition, and β and ∈ are new variables; (3.3) By comparing the predicted point cloud with the predicted data, the coincident point cloud is given a low weight, and the non-coincident point cloud is given a high weight, and then the observation value of each point cloud is calculated using the observation model function V(), and then the weight is assigned according to the probability comparison result: v t = V1(h t ) + V2(δ) where v t is the observation value for each point cloud, w t is the weighting result, P(v t |h t ) represents the probability of observing v t given the state pose h t . (3.4) Filtering and fusion screening is performed, the point cloud data with weight is projected into the world coordinate system, and the voxel data of the corresponding grid is fused, the point cloud weight is weighted and screened, and the projected state information and point cloud weight are updated: where h t+1 is the updated state pose; is the accumulated state pose for each time step; ε t is the new noise disturbance; w t+1 is the updated weighted result; P(v t |h t+1 represents the probability of observing v t+1 given the updated state pose h t ; is the new variable; (3.5) After filtering, the state and density of the filtered point cloud information are updated: In the formula, θ n (x1,y1,z1) is the updated point cloud density value, w t+1 is the weight, θ n (x1,y1,z1) is the updated point cloud state information, h t+1 is the point cloud state information of the new stage, θ E is the unit space distribution value; (3.6) In the Cartographer algorithm, the position and pose of the AGV are represented by ξ = (ξ x , ξ y , ξ θ ), where ξ x and ξ y are the moving distances of the AGV in the x and y directions, and ξ θ is the moving direction angle. The environmental data frame collected by the 360° laser radar is denoted as L = {l k} k＝1....K , l k ∈ R 2 , where each l k is a two-dimensional vector, and each point in the data frame is mapped to the subgraph by the pose transformation F ξ , and the calculation formula is as follows: where s is the position of the original point, p x , p y are the coordinates of the reference point; (3.7) Before the pose ξ is added to the subgraph, the Ceres solver is used to solve the nonlinear least squares problem to obtain the pose information of the scan frame, and the calculation formula is as follows: In the formula, F ξ l ξ is the point cloud data after the pose transformation, M smooth is a smoothing function; (3.8) When the map is composed of multiple subgraphs, the scan frame is only matched with the current subgraph, which is easy to produce cumulative error, and the Cartographer algorithm optimizes the laser radar data frame and AGV pose through sparse pose adjustment method, and the calculation formula is as follows: In the formula, respectively, subgraph pose and scan frame pose, ξ ij is the relative pose of the scan frame in the subgraph, A is the optimized pose, ρ is the loss function, which is used to measure the error, E 2 is the error between the laser radar data frame and the AGV pose; (3.9) The Cartographer algorithm performs loop detection through branch and bound scan matching algorithm, and the calculation formula is as follows: where W is the search window, M nearest As an extension of the M function, R(δ) is a regularization term, is a regularization parameter.

5. The intelligent AGV path planning method of claim 1, wherein, Step 4 includes: (4.1) In the two-dimensional environment, the RRT* algorithm generates a random tree connecting the starting point M i and the target point M g , with a compensation range r, taking the initial position of the AGV as the starting point and setting it as the root node of the random tree T, and the target point as the end point of the AGV. (4.2) Randomly sampling within the two-dimensional environment to obtain a sample point q rand ; (4.3) Nearest point search is finding a node q in a search tree nearest such that the distance between q nearest and a random sample point q rand is the smallest, traversing the search tree, input is the search tree and a random sample point q rand , output is the nearest node q nearest , expression is: where q nearest represents the nearest node, q rand represents a random sampling point, v represents a search tree, and T represents a set of search trees (4.4) Introduce Markov Decision Process MDP, define state, action, transition probability and reward function optimization path planning, action symbol a represents, all possible actions constitute the action set A, the state transition probability and the immediate reward are closely related to the current state and action, the optional action set of AGV car in state S t is A t , the action set A t may be different under different states; (4.5) In a certain state, the action of AGV car follows a certain probability distribution, which is called strategy Policy, and is denoted by symbol π(a|s). The strategy π(a|s) represents the probability distribution of AGV car selecting action a in state s, which is only related to the current state and does not depend on historical information. It determines the behavior of AGV car in the current state. Although the strategy is fixed at a certain time, AGV car can dynamically adjust and optimize the strategy over time to make optimal decisions in each state. The formula is as follows: π(a|s) = P[A t = a|S t = s] (4.6) According to step (4.3), the random sampling point is input into the nearest node growing function to generate a new node q new The expression is as follows: In the formula, step represents the motion step of AGV car, which can be adjusted according to the environment and demand; (4.7) In state S t The AGV performs action a1 (a1∈A) t When ), the state transition is not fixed at S. t →S t+1 Instead, it is stored in the state transition probability matrix P in the form of a probability distribution. StSt+1|a In this context, it exhibits the characteristics of stochastic dynamic programming, and its expression is as follows: (4.8) Collision detection judges whether the new node conflicts with the obstacle: for circular obstacles, calculate the shortest distance from the center to the line connecting the new node and the nearest node, if the distance is less than or equal to the radius of the obstacle, the path is not feasible; otherwise, it is considered that the path does not cross the obstacle and is feasible. For rectangular obstacles, firstly, it is judged whether the new node is inside or on the boundary of the obstacle. If it is inside, the path is not feasible. If it is outside, it is further checked whether the line connecting the new node and the nearest node intersects with the obstacle. If it intersects, the path is not feasible. The lines connecting point A and point D with the sampling points are taken as the boundary lines, and the expressions are as follows: In the formula, the variable k represents the inclination of the straight line; (4.9) If a straight line through point q nearest satisfies a certain condition, it means that the straight line does not intersect any obstacle, in order to perform collision detection, it is assumed that the boundary of each rectangular obstacle is provided with a Boolean value bool i : when bool i is 1, it means that the straight line intersects the obstacle; when bool i = 0, it means that there is no intersection, therefore, the collision detection function needs to include a helper subroutine to operate these Boolean values, the formula is as follows: j(I, N, P) = ((y P - y I )(x N - x I )) ((y N - y I )(x P - x I )) bool i = (j(q nearest ,v1,v2) ≠ j(q new ,v1,v2)) (j(q nearest ,q new ,v1)≠j(q nearest ,q new ,v2)) where: j(I, N, P) is a new sub-function whose input parameters I, N, P represent the boundary of the rectangular obstacle in a specific way, q nearest is the closest point; q new is the new sample point; v1, v2 are two vertices of the rectangular obstacle; (4.10) policy π(a|s) determines the action a of AGV in state s, and state transition probability determines the state transition after action, both of which jointly affect the entire transition process of AGV, by substituting policy π(a|s) and state transition probability into the calculation, the state S t is obtained π Under the policy π, the state value function V t (S t ), the calculation formula is as follows: where E(R t+1 |A t = a), the above equation can be simplified as:​ where the policy π(a|S t ) and state transition probability matrix The value of state S t is calculated by weighting and summing the probabilities of all possible actions and the values of the subsequent states as weights, and thus the state values of all the states S t+1 transited to subsequently are taken into account. By designing algorithm to solve the model, the optimal strategy π of each state can be obtained * That is, the global optimal path; (4.11) The node is evaluated by the cost function Di, which reflects the cost of the node by calculating the distance between the node and its parent node. The cost includes both spatial distance and implicit time cost. The expression of Di is as follows: D i = ||q new -M n ||2+∑D j ,j = 1,2,..., i - 1 In the formula, i is the serial number of a random tree node, Di is the cost function value of node i; M n is the nth node in the random tree; (4.12) On the basis of the RRT algorithm, the RRT* algorithm introduces a parent node reconnection strategy. If there is an obstacle between the child node and the parent node, the reconnection operation is performed again. After a new nearby point is added to the random tree, the algorithm judges whether there is a parent node with a smaller cost function in the nodes within a range of 1 compensation r of the cost function. The parent node is updated to ensure the relative optimality of the path, and the calculation formula is as follows: M n = T i (x1, y1, 0, minC, p i ), ||T i -q new || <r wherein: T i is the i-th node on the random tree; x1 is the horizontal coordinate of the newly sampled point; y1 is the vertical coordinate of the newly sampled point; minC is the current minimum cost function; p i is the serial number of the parent node of the node; (4.13) Introducing the function to find the neighboring nodes, identify q new nodes around the point and store in q neighbor Then, the mechanism to select the parent node is incorporated into the algorithm, after the new node passes the collision detection, the cost function is evaluated to determine its parent node, in addition, the search tree is continuously optimized by the pruning function until the best path is found; (4.14) The parent node selection process finds the neighboring nodes of the new node through the function, and then traverses all nodes in the array. The path length of the starting point reaching the latest node through the neighboring nodes is calculated. The pruning operation decides whether the parent node relationship needs to be adjusted by comparing the path cost of the nodes. By comparing the path length formed by the two specific nodes through their respective parent nodes with the path length formed through the latest node, if one of the nodes as the parent node of the latest node forms a longer path, pruning is unnecessary. If the path formed by the new node as the parent node is shorter, the related nodes are pruned. All new connections need to pass through the collision detection. If the detection fails, the original connection is retained.

6. The intelligent AGV path planning method of claim 1, wherein, Step 5 comprises: Through the global optimal path found by the MRRT* algorithm in step (4), the AGV car travels along a certain path indoors or outdoors. Through the multi-sensor carried, the nearby obstacles are scanned, and it is judged whether there is a step that needs to be avoided. If there is, it jumps to step (6). Otherwise, it will continue to travel.

7. The intelligent AGV path planning method of claim 1, wherein, The FPA* algorithm for local path adjustment using the fusion artificial potential field method APF and the dynamic heuristic search algorithm DHPA* in step 6 comprises: (6.1) Create open and close lists, initialize the open list and the closed list, and add the starting point and the target point to the open list. At the same time, set the APF algorithm to give the starting point and the target point an initial gravitational potential field, so that the AGV car is guided by the target force at the beginning; (6.2) It is judged whether the open list is empty. If it is empty, it means that there is no path to walk, and the path search fails. If it is not empty, the next step is continued; (6.3) The DHPA* algorithm finds the surrounding child nodes through the current node and calculates the cost. The child node with the smallest cost is selected as the new parent node. At the same time, the Close table records the searched nodes, the Open table saves the nodes to be expanded, and the evaluation function f(n) evaluates the advantages and disadvantages of each node. The calculation method of the evaluation function f(n) is as follows: In the formula, f(n) is the evaluation function of node n, which is used to select the node with the minimum cost for expansion; g(n) is the actual cost from the starting point to node n; Z is a weight factor; h(n) is the heuristic estimated cost from node n to the target point; h(p) is the heuristic estimated cost from the parent node p of node n to the target point; (6.4) Select the node with the minimum cost function from the open table as the current node, and obtain its position, check whether the current node meets the conditions of the preset "maintenance area", if not, adjust the current path direction and turn; if yes, continue to the next step of path planning; (6.5) The APF algorithm introduces the concept of potential field in path planning, which is used to add various virtual force fields to the environment where the AGV is located. The attractive potential field is related to the distance and direction from the current position of the AGV to the target point: the greater the distance, the greater the attraction force the robot receives, and the direction of the attractive force is from the current position to the target point, thereby guiding the AGV to gradually approach the target point. The expression of the attractive potential field function is as follows: where U att (q) is the gravitational potential field function representing the gravitational potential energy at position q; η is a scaling factor used to adjust the strength of the gravitational potential field, ρ 2 (q, q g ) is the Euclidean distance from the current position q to the target point q g . (6.6) The attractive force generated by the attractive potential field is calculated by the negative gradient of its potential field function, and the expression is: where p(q, q g ) is a vector whose length is the distance between the current position and the target point and whose direction is from the current position to the target point, denotes the gradient of the attractive potential function U att (q) and indicates the direction of the attractive force; (6.7) The repulsive potential field only works within a certain range around the obstacle, and beyond this range, the AGV is no longer affected by the repulsive force of the obstacle. Its expression is: In the formula, k is the repulsive force coefficient, ρ(q, q0) is the vector from the obstacle to the current point, and its length is the distance from the current point to the obstacle; ρ0 is the action radius of the repulsive potential field; when 0≤|ρ(q, q0)|≤ρ0, the repulsive force of the AGV increases as the distance between the vehicle and the obstacle decreases; when |ρ(q, q0)|≥ρ0, the AGV is no longer affected by the repulsive force; (6.8) The attractive force generated by the attractive potential field is calculated by the negative gradient of its potential field function, and the expression is: The resultant force on the AGV is: F(q) = F at (q) + F mq (q); (6.9) When the distance between the starting point and the end point is close, a smaller weight coefficient is used for detailed search to find the shortest path, but this method is only suitable for small-scale environments. In large-scale environments, although the search time is shortened, the path may not be the shortest. Therefore, the weight coefficient Z is dynamically adjusted to avoid the algorithm falling into a local optimal solution, and the expression is: In the formula, h(n) is the heuristic function, and the threshold value of Z is set to 1.5; (6.10) In the path search process, the DHPA* algorithm improves the method of A* algorithm for selecting nodes for search by using the actual cost g(n) from the starting point to the current node and the heuristic estimated cost h(n) from the current node to the end point. The DHPA* algorithm uses the sum of the diagonal distance and the distance from the parent node of the current node to the end point as the heuristic distance. The calculation formulas of the heuristic distance h(n) and the heuristic distance h(p) of the parent node of the current node are as follows: h(p) = |x1-x2| + |y1-y2| In the formula, L is the grid size; x and y are the minimum and second minimum of the absolute values of the coordinate differences of the current node n to the target node; the diagonal distance is twice the horizontal or vertical distance; yL is the straight-line distance from the parent node to the end point; and x and y are calculated as follows: x = min(|x1-x2|, |y1-y2|) y = ||x1-x2|-|y1-y2||| In the formula, (x1, y1) and (x2, y2) are the coordinates of the current node n and the target node respectively. (6.11) In step (6.3), the evaluation function f(n) introduces a turning penalty term P to reduce unnecessary turns of the AGV, which is determined by evaluating the two angles formed by the vector from the current node to the end point and the parent node and the start point, to determine the degree of turning when expanding the next node, the specific calculation formula is as follows: P t = C1θ1+ C2θ2 In the formula, P t is a turning penalty term; C1, C2 are weight coefficients, C1 is the included angle of PC and CG; θ2 is the included angle of CS and CG, C is the current node, P is the parent node of C, S is the starting point, and G is the terminal point; (6.12) DHPA* algorithm is the main body, combined with APF algorithm, to form FPA* algorithm, when generating random points, the gravitational potential field is applied to the random points, so that the current position of the AGV is affected by three force fields: the gravitational potential field of the target point, the gravitational potential field of the random point and the repulsive potential field of the obstacle.

8. The intelligent AGV path planning method of claim 1, wherein, The DRL agent of step 6 is a decision-making module integrated into the AGV control system, which learns the optimal path planning and obstacle avoidance strategy through deep reinforcement learning algorithm, receives state information from the environment, outputs control actions, and continuously optimizes the decision-making strategy of the AGV according to the feedback of the reward function, the introduction of deep reinforcement learning DRL auxiliary obstacle avoidance decision-making includes: (6.13) Collect historical path planning data and real-time interaction data of AGV, generate data samples, these samples cover the running state, control action and corresponding reward value of AGV in different environments; (6.14) Extract control actions from data samples to form action space, including AGV speed adjustment and steering angle adjustment, extract working state formed according to the execution of control actions to form state space, state space includes AGV position, speed, position and distance of surrounding obstacles; (6.15) Set up reward function, match reward value for each execution of control action, the design goal of reward function is to encourage AGV to quickly and safely reach the target point while avoiding collision with obstacles, successful obstacle avoidance and approach to target point will get positive reward, while collision with obstacles or deviation from target direction will get negative reward, the calculation formula is as follows: where r* represents the total cost after compensation, A i represents the control cost, a i represents the control cost coefficient, s i represents the control variable percentage, P imax represents the upper limit of the control variable; B j represents the cost compensation value of decision out-of-limit, c j is the action out-of-limit compensation coefficient, x j is the output decision value, x s is the action limit value; r is the reward value; a and b are linear coefficients, γ i and δ j are compensation items related to control; ∈ is a random disturbance item; (6.16) Establish an experience library to store four-tuple samples (s, a, r, s') formed by current state s, control action a, reward value r of executing control action and next state s' obtained after executing control action, to train DRL agent; (6.17) Set state action value function Q(s, a) as evaluation index to judge the pros and cons R value of executing control action in current state, the calculation formula is as follows: where r t+n is the reward value of performing control action a at time t + n, and γ is the discount factor. (6.18) Set Actor network and Critic network, receive batch input four-tuple (s, a, r, s'), and calculate gradient using Adam optimizer according to the set state action value function Q, and feedback to update network parameters of Actor network and Critic network.

9. The intelligent AGV path planning method of claim 1, wherein, The objective function optimized by the MRRT* algorithm and the FPA* algorithm in steps (4) and (6) includes: In AGV path planning, whether global planning or local planning, three objective functions are involved, i.e., the total length of the path f1(X i ), the proximity of the path to the infeasible region f2(X i ), and the smoothness of the path f3(X i ). The root mean square error (RMSE) is used to evaluate the multi-objective functions by weighting:

10. A multi-sensor fusion intelligent AGV path planning system, characterized in that, It includes: Multi-source environmental perception module: composed of 360° laser radar, binocular vision, infrared thermal imaging and ultrasonic array module, used for multi-dimensional data acquisition, environmental perception and dynamic pedestrian obstacle avoidance of running environment; Low-rank multi-source feature fusion module: for using a low-rank multi-source feature fusion module LMF to fuse the multi-sensor multi-dimensional data collected in step (1); Map construction module: for processing the fused multi-dimensional data by using a VF-Cart algorithm to construct a global two-dimensional grid map, and obtaining an indoor and outdoor AGV running environment map; Global path planning module: for performing global path planning of the AGV vehicle based on the indoor and outdoor AGV running environment map by using an improved RRT* algorithm MRRT*; the MRRT* algorithm combines Markov decision process MDP and RRT*, that is, the ability of MDP to process dynamic data and the efficient path search ability of RRT* in a complex environment; Dynamic obstacle detection module: for detecting pedestrians or obstacles on the path in real time by using multiple sensors during the process of advancing along the globally optimal path; Local path optimization module: a FPA* fusion algorithm is proposed, which uses the artificial potential field APF gravitational field to guide the AGV motion direction in the early stage, combines the dynamic heuristic search algorithm DHPA* to dynamically adjust the heuristic function, combines the real-time environmental information to find a feasible path from the current position to the target point, optimizes the local path of the AGV vehicle during operation, and introduces a deep reinforcement learning DRL method for obstacle avoidance auxiliary decision-making to complete the obstacle avoidance function; Motion control module: executes the final planned path instruction to drive the AGV to the target point.

Images