A dynamic path planning method for AGV based on neural network

Abstract

The present invention relates to the technical field of automatic guided vehicles, and in particular to a method for dynamic path planning of AGVs based on neural networks. The content includes: real-time collection of environmental data, and pre-processing of environmental data; environmental perception and path prediction through an adaptive deep neural network; the environmental perception generates an environmental model through an improved convolutional neural network; the path prediction generates an initial path through an adaptive deep neural network; path optimization and dynamic obstacle avoidance are performed based on the initial path to obtain path optimization results at all levels; and the path optimization results at all levels are integrated to form a final path. The technical problems of being unable to fully and accurately perceive the environment, being difficult to cope with complex dynamic environmental changes, being unable to update the path in real time, being difficult to achieve path optimization in complex environments, and being difficult to dynamically adjust according to environmental changes are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic guided vehicles, and in particular to an AGV dynamic path planning method based on a neural network. Background Art

[0002] With the continuous advancement of Industry 4.0 and smart manufacturing, Automated Guided Vehicles (AGVs) are increasingly being used in warehousing, logistics, manufacturing and other fields. In addition, with the development of smart cities and smart transportation, AGVs are also beginning to emerge in emerging fields such as smart parking lots. AGVs can automatically handle and transport materials, as well as park and retrieve vehicles in smart parking lots, greatly improving production efficiency, operational safety and parking management efficiency. However, AGVs still face many challenges in autonomous navigation in complex and dynamic environments, especially in path planning and dynamic obstacle avoidance.

[0003] Traditional AGV path planning methods mainly rely on preset paths and rules. This method can work well in static and known environments, but in actual applications, AGVs often need to operate in dynamic and unknown environments. This dynamic environment requires AGVs to have higher path planning and environmental perception capabilities, be able to update paths in real time, avoid dynamic obstacles, and ensure safe and efficient completion of work tasks.

[0004] In recent years, with the development of artificial intelligence and deep learning technology, intelligent path planning methods have gradually become a research hotspot. The application of deep neural networks (DNN) and reinforcement learning (RL) in path planning shows great potential. Deep neural networks can generate accurate environmental models by processing a large amount of sensor data, providing important support for path planning. Reinforcement learning algorithms can continuously optimize path planning strategies through interaction with the environment, and achieve path optimization and dynamic obstacle avoidance in complex environments.

[0005] The existing AGV path planning methods have at least the following technical problems: they are unable to fully and accurately perceive the environment, resulting in inaccurate environmental modeling; they are difficult to cope with complex dynamic environmental changes and are unable to update the path in real time, resulting in low efficiency and poor flexibility in path planning; they are difficult to achieve path optimization in complex environments, and the quality and efficiency of path planning need to be improved; they are difficult to make dynamic adjustments based on environmental changes, resulting in insufficient robustness and flexibility in path planning. Summary of the invention

[0006] The present invention provides an AGV dynamic path planning method based on a neural network to solve the technical problems that the environment cannot be perceived comprehensively and accurately, resulting in inaccurate environmental modeling; it is difficult to cope with complex dynamic environmental changes and the path cannot be updated in real time, resulting in low efficiency and poor flexibility in path planning; it is difficult to achieve path optimization in complex environments, and the quality and efficiency of path planning need to be improved; it is difficult to make dynamic adjustments according to environmental changes, resulting in insufficient robustness and flexibility in path planning.

[0007] The present invention provides a neural network-based AGV dynamic path planning method, which specifically includes the following technical solutions:

[0008] A neural network-based AGV dynamic path planning method comprises the following steps:

[0009] S1. Real-time collection of environmental data and preprocessing of the environmental data; environmental perception and path prediction through an adaptive deep neural network; the environmental perception generates an environmental model through an improved convolutional neural network; the path prediction generates an initial path through an adaptive deep neural network;

[0010] S2. Perform path optimization and dynamic obstacle avoidance based on the initial path to obtain path optimization results at each level; integrate the path optimization results at each level to form the final path.

[0011] Preferably, the S1 specifically includes:

[0012] The environmental data collected in real time include lidar data, camera data and ultrasonic data; the environmental data is preprocessed to generate an input data set; the input layer of the improved convolutional neural network receives the input data set, extracts features through multiple convolutional layers, and generates an environmental feature map; the environmental feature map is reduced in dimension through an adaptive pooling layer; and an environmental model is generated through an adaptive fully connected layer.

[0013] Preferably, the S1 specifically includes:

[0014] The output of the environmental model is input into an adaptive deep neural network, which is combined with a recurrent neural network and a graph neural network to perform path prediction. The recurrent neural network is used to capture timing information, and the graph neural network is used to perform graph structure processing on the timing information to generate an initial path.

[0015] Preferably, the S2 specifically includes:

[0016] Based on the generated initial path, a multi-level adaptive reinforcement learning algorithm is used for path optimization and dynamic obstacle avoidance.

[0017] Preferably, the S2 specifically includes:

[0018] In the implementation process of the multi-level adaptive reinforcement learning algorithm, the initial path is scored based on the scoring function to obtain the scoring result of the initial path; and the path optimization decision is made based on the scoring result of the initial path.

[0019] Preferably, the S2 specifically includes:

[0020] Each level of the multi-level adaptive reinforcement learning algorithm has an independent state space, action space, reward function, and state transfer function.

[0021] Preferably, the S2 specifically includes:

[0022] The strategy function for path optimization is constructed based on the scoring results of the initial path and the state-action value function.

[0023] Preferably, the S2 specifically includes:

[0024] For any level in the multi-level adaptive reinforcement learning algorithm, the path is adjusted through the path optimization strategy function to obtain the path optimization result.

[0025] The beneficial effects of the technical solution of the present invention are:

[0026] 1. Collect environmental data including lidar data, camera data and ultrasonic data, pre-process the environmental data to form an input data set, use the improved convolutional neural network to extract features of the input data set, and generate a high-precision environmental model; through the three-dimensional point cloud data provided by the lidar sensor and the visual image data captured by the camera, as well as the distance and obstacle position information obtained by the ultrasonic sensor, the AGV can fully perceive the environment, improving the accuracy and reliability of environmental perception.

[0027] 2. The adaptive deep neural network combined with the recurrent neural network and graph neural network can capture time series information and graph structure information to perform path prediction in complex environments. Through the recurrent neural network and graph neural network, path prediction is more accurate and robust, and can cope with a variety of complex environmental changes.

[0028] 3. A multi-level adaptive reinforcement learning algorithm is used to achieve path optimization in complex environments through multi-level learning and adjustment. Each level makes path adjustments based on environmental data to ensure the diversity and flexibility of path planning. The path optimization decision of the multi-level adaptive reinforcement learning algorithm is guided by the scoring function, which improves the efficiency and quality of path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of a neural network-based AGV dynamic path planning method;

[0030] Figure 2 It is the dynamic path planning diagram of AGV. DETAILED DESCRIPTION

[0031] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0033] The following is a detailed description of a specific scheme of a neural network-based AGV dynamic path planning method provided by the present invention in conjunction with the accompanying drawings.

[0034] See attached Figure 1 , which shows a flow chart of a neural network-based AGV dynamic path planning method provided by an embodiment of the present invention, the method comprising the following steps:

[0035] S1. Collect environmental data in real time and pre-process the environmental data; perform environmental perception and path prediction through an adaptive deep neural network; the environmental perception generates an environmental model through an improved convolutional neural network; the path prediction generates an initial path through an adaptive deep neural network.

[0036] See attached Figure 2 , environmental perception and path prediction are performed through adaptive deep neural networks. For environmental perception, the pre-processed environmental data is processed through an improved convolutional neural network to generate an environmental model. The specific steps include: sensor data acquisition, real-time environmental data is collected through lidar sensors, cameras and ultrasonic sensors; data preprocessing, filtering, denoising and other pre-processing operations on the collected environmental data; environmental modeling, feature extraction of pre-processed environmental data through an improved convolutional neural network to generate an environmental model. The adaptive deep neural network is also used for path prediction, generating an initial path based on the output of the environmental model. After the output of the environmental model is input into the adaptive deep neural network, the adaptive deep neural network outputs the initial path.

[0037] The implementation process of environmental perception and path prediction through adaptive deep neural network is as follows:

[0038] Sensor data collection is performed in real time by lidar sensors, cameras, and ultrasonic sensors. The specific environmental data collected includes:

[0039] LiDAR data: LiDAR sensors emit lasers in a high-speed rotating manner, measure the reflected laser pulses, and obtain three-dimensional point cloud data of the environment, which includes information such as the distance, shape, and position of each object in the environment;

[0040] Camera data: The camera captures visual image data of the environment, obtaining information such as color, texture, and shape. Multiple cameras can be used to obtain visual information from multiple angles to improve perception accuracy.

[0041] Ultrasonic data: Ultrasonic sensors emit ultrasonic waves, measure the reflected ultrasonic signals, and obtain information such as distance and obstacle location. Ultrasonic sensors are usually used for close-range detection to supplement the data of lidar and cameras.

[0042] The collected environmental data is preprocessed by filtering and denoising to form a unified input data set; the above filtering and denoising are both based on existing technologies and will not be described in detail here. Environmental modeling uses an improved convolutional neural network, and the specific implementation steps include:

[0043] The input layer of the improved convolutional neural network receives the input data set; feature extraction is performed through multiple convolutional layers, each convolutional layer is composed of multiple dynamically adjusted filters to extract environmental features at different levels and generate an environmental feature map; the dimension of the environmental feature map is reduced through the adaptive pooling layer to reduce the computational complexity; the multiple convolutional layers and the adaptive pooling layers are stacked and finally connected to the adaptive fully connected layer to output the environmental model.

[0044] Path prediction is performed by an adaptive deep neural network. The specific implementation steps include: inputting the output of the environment model into the input layer of the adaptive deep neural network; performing feature conversion through a hidden layer based on an adaptive nonlinear activation function (such as adaptive ReLU) to learn complex path planning strategies; and outputting the initial path through the output layer.

[0045] Specifically, in the process of environmental perception and path prediction, the mathematical formula used is as follows: Let the input data set be X, which contains lidar data, camera data, and ultrasonic data. After convolution and adaptive pooling operations, the environmental feature map F is generated, and then the environmental model is generated through the adaptive fully connected layer. The formula for the adaptive convolution operation is as follows:

[0046]

[0047] in, Represents the value of the kth channel of the environmental feature map F at position (i, j); each filter corresponds to one channel; and b k (t) is the weight and bias of the kth filter at position (m,n) dynamically adjusted over time t; X (i+m-1，j+n-1) Represents the value of position (i+m-1, j+n-1) in the input data set X; σ is an adaptive nonlinear activation function (such as adaptive ReLU); M×N is the filter size.

[0048] The adaptive pooling operation formula is as follows:

[0049]

[0050] Among them, P ij (t) represents the value of the feature map after pooling at position (i, j) dynamically adjusted over time t; s(t) is the size of the pooling window dynamically adjusted over time; is the value of the kth channel of the environmental feature map F at position (i+m-1, j+n-1). ij (t) Merge to obtain the complete pooling feature map P = [P ij (t)], the merging process is to put all P ij (t) form a matrix or tensor.

[0051] The formula of the adaptive fully connected layer is as follows:

[0052] E=σ(W f (t)P+b f (t))

[0053] Where E is the output of the environment model; σ is an adaptive nonlinear activation function, such as adaptive ReLU; W f (t) and b f (t) is the weight and bias of the fully connected layer dynamically adjusted over time t; P is the feature map after pooling.

[0054] The output E of the environment model is input into the adaptive deep neural network, which combines the recurrent neural network and the graph neural network for path prediction. First, the recurrent neural network is used to capture the timing information:

[0055] h t =φ(W ih (t)E t +W hh (t)h t-1 +b h (t))

[0056] Among them, h t represents the hidden state of the recurrent neural network at time t; W ih (t), W hh(t) are the weight matrices from input layer to hidden layer and from hidden layer to hidden layer respectively; b h (t) is the bias vector of the hidden layer; φ is the adaptive nonlinear activation function. The recurrent neural network captures the environmental characteristics and path information of the current time step through the hidden state of each time step. The hidden state can be regarded as the node feature of the graph neural network, that is, each hidden state represents a node.

[0057] The captured time series information is processed into a graph structure through a graph neural network to generate an initial path:

[0058]

[0059] Among them, Y represents the predicted output of the initial path; is the set of neighbor nodes of node v; α uv is the adaptive attention weight, that is, the weight of node v to neighbor node u at time t; h u and h v are the hidden states of node v and its neighbor node u; W o (t) and b o (t) is the weight and bias of the output layer; ψ is the adaptive nonlinear activation function.

[0060] S2. Perform path optimization and dynamic obstacle avoidance based on the initial path to obtain path optimization results at each level; integrate the path optimization results at each level to form the final path.

[0061] See attached Figure 2 After obtaining the initial path, a multi-level adaptive reinforcement learning algorithm is used to optimize the path and avoid dynamic obstacles. The multi-level adaptive reinforcement learning algorithm achieves path optimization in complex environments through multi-level learning and adjustment. The specific steps include:

[0062] First, the initial path Y is scored, and the scoring function takes into account the length of the path segment, the collision risk, and the complexity of the environment:

[0063]

[0064] Where S(Y) represents the scoring result of the initial path Y; n is the number of path segments of the initial path Y; d κ represents the length of the κth path segment; r κ represents the collision risk of the κth path segment; c κ represents the environmental complexity of the κth path segment; w1, w2, w3 are the weight coefficients of the path segment length, collision risk and environmental complexity, respectively.

[0065] The scoring result S(Y) of the initial path is used to guide the path optimization decision of the multi-level adaptive reinforcement learning algorithm. In the multi-level adaptive reinforcement learning algorithm, the scoring result of the initial path is used as part of the reward function to optimize the path planning.

[0066] Specifically, the multi-level adaptive reinforcement learning algorithm includes multiple levels, each of which has an independent state space S, action space A, reward function R and state transfer function T, and performs path optimization in a hierarchical manner.

[0067] For any level in the multi-level adaptive reinforcement learning algorithm, the path is adjusted based on the collected environmental data. The formula is as follows:

[0068]

[0069] Among them, g l (a|s) represents the strategy function of the path optimization at the lth level, that is, the probability distribution of taking action a in state s at the lth level; A is the action space, which represents the set of all possible actions; ∑ a∈A exp(Q l (s,a)+S(Y)) is a normalization factor, which ensures that the strategy function gl(a|s) of the path optimization at the first level is a valid probability distribution with a value between 0 and 1, and the sum of the probabilities of all possible actions is 1; Q l (s,a) represents the state-action value function of the lth level, which represents the value of taking action a in state s at the lth level. The specific calculation formula is:

[0070]

[0071] in, is the expected value, which refers to the weighted average of a random variable under a given probability distribution; R l (s,a) is the immediate reward of the lth level, which means the immediate reward obtained by taking action a in state s at the lth level; T l (s'|s,a) is the state transition function of the lth level, which represents the probability of transitioning from state s to state s' after taking action a at the lth level; V l (s') is the state value function of the lth level, which represents the value of the state s' in the lth level. Its calculation formula is: a' is all possible actions in state s'; Q l (s',a') represents the value of taking action a' in state s'. The calculation formula of the state transition function at the lth level is:

[0072]

[0073] Among them, Tl (s'|s,a) is the state transition function of the lth level, which represents the probability of transitioning from state s to state s' after taking action a at the lth level; exp(-||s'-sa·Δt|| 2 ) represents the state transfer error; Δt represents the time required to transfer from state s to state s'; S is the set of state spaces, which represents the set of all states that the AGV may reach; f(o) is the obstacle influence function.

[0074] The path optimization results g at each level l (a|s) is integrated to form the final path. The comprehensive path optimization formula is as follows:

[0075]

[0076] Among them, Y * is the final path; L is the number of levels.

[0077] In summary, a neural network-based AGV dynamic path planning method was completed.

[0078] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0079] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0080] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A neural network-based AGV dynamic path planning method, characterized in that: The following steps are involved: S1. The environmental data collected in real time includes lidar data, camera data and ultrasonic data. The environmental data is preprocessed to generate an input data set. The input layer of the improved convolutional neural network receives the input data set, extracts features through multiple convolutional layers, and generates an environmental feature map. The environmental feature map is reduced in dimension through an adaptive pooling layer. The environmental model is then generated through an adaptive fully connected layer. The adaptive convolution operation formula is as follows: in, Represents the value of the kth channel of the environmental feature map F at position (i, j); and b k (t) is the weight and bias of the k-th filter at position (m, n) dynamically adjusted over time t; X (i+m-1，j+n-1) represents the value of position (i+m-1, j+n-1) in the input data set X; σ is the adaptive nonlinear activation function; M×N is the filter size; The adaptive pooling operation formula is as follows: Among them, P ij (t) represents the value of the feature map after pooling at position (i, j) dynamically adjusted over time t; s(t) is the size of the pooling window dynamically adjusted over time; is the value of the kth channel of the environmental feature map F at position (i+m-1, j+n-1); The formula of the adaptive fully connected layer is as follows: E=σ(W f (t)P+b f (t)) Where E is the output of the environment model; σ is the adaptive nonlinear activation function; W f (t) and b f (t) is the weight and bias of the fully connected layer dynamically adjusted over time t; P is the feature map after pooling; Environmental perception and path prediction are performed through an adaptive deep neural network; the environmental perception generates an environmental model through an improved convolutional neural network; the path prediction generates an initial path through an adaptive deep neural network; S2. Based on the initial path, a multi-level adaptive reinforcement learning algorithm is used to perform path optimization and dynamic obstacle avoidance. In the implementation process of the multi-level adaptive reinforcement learning algorithm, the initial path is scored based on the scoring function to obtain the scoring result of the initial path. The specific implementation formula is: Where S(Y) represents the scoring result of the initial path Y; n is the number of path segments of the initial path Y; d κ represents the length of the κth path segment; r κ represents the collision risk of the κth path segment; c κ represents the environmental complexity of the κth path segment; w1, w2, w3 are the weight coefficients of the path segment length, collision risk and environmental complexity respectively; Make path optimization decisions based on the scoring results of the initial path; integrate the path optimization results at all levels to form the final path.

2. The AGV dynamic path planning method based on neural network according to claim 1 is characterized in that: The S1 specifically includes: The output of the environmental model is input into an adaptive deep neural network, which is combined with a recurrent neural network and a graph neural network to perform path prediction. The recurrent neural network is used to capture timing information, and the graph neural network is used to perform graph structure processing on the timing information to generate an initial path.

3. The AGV dynamic path planning method based on neural network according to claim 1 is characterized in that: The S2 specifically includes: Each level of the multi-level adaptive reinforcement learning algorithm has an independent state space, action space, reward function, and state transfer function.

4. The AGV dynamic path planning method based on neural network according to claim 1 is characterized in that: The S2 specifically includes: The strategy function for path optimization is constructed based on the scoring results of the initial path and the state-action value function.

5. The AGV dynamic path planning method based on neural network according to claim 4 is characterized in that: The S2 specifically includes: For any level in the multi-level adaptive reinforcement learning algorithm, the path is adjusted through the path optimization strategy function to obtain the path optimization result.

Images