Intelligent warehouse agv group path planning method based on communication cooperation

Abstract

The application discloses a kind of intelligent warehousing AGV group path planning method based on communication cooperation, specifically relates to the technical field of intelligent warehousing, the two-dimensional space of the intelligent warehousing to be solved is divided by grid method;Reward function, action space and neural network are set, and sensor is additionally installed to trolley itself, obtains positioning information and observation space information;Initially, the strategy of first step is obtained by observation space information combined with reinforcement learning algorithm;Then, the strategy and message of this time step are obtained by the communication of the partial message generated by last step, combined with the observation space information of current step trolley, and the process is repeated, to generate complete path planning answer for trolley;Finally, under the constraint condition set, the path planning of all trolleys is completed, that is, the path planning of vehicle group is completed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent warehousing, and in particular to a path planning method for an intelligent warehousing AGV group based on communication collaboration. Background Art

[0002] Warehousing plays a vital role in logistics and is crucial to overall operational performance. Traditional warehouse operations are primarily manpower-intensive, characterized by low efficiency, prone to errors, and high labor costs. Considering the entire warehousing area, the systemic risks associated with human error lead to low productivity and a perceived waste of human resources. The intelligent logistics era has ushered in significant changes in warehousing. Through the application of real-time sensing, information processing, and artificial intelligence technologies, warehousing has become more intelligent, freeing up significant human and material resources and significantly improving operational efficiency and real-time responsiveness in order processing.

[0003] Warehousing has undergone a long evolution, from simple storage facilities to advanced logistics centers. With rapid technological advancements, more and more industrial plants are adopting highly automated smart warehouse systems to enhance their competitiveness. Artificial intelligence and machine learning algorithms are widely used to predict demand patterns, optimize inventory management, and improve cargo transportation route planning. Leveraging big data, warehouses are becoming more intelligent, enabling real-time decision-making and better resource allocation. The combination of automation and robotics is revolutionizing the warehousing industry, including the deployment of automated guided vehicles (AGVs), drones, and robotic pickers. These technologies are optimizing warehouse operations, improving efficiency, and reducing the risk of workplace injuries. The future of warehousing holds an exciting prospect, where technology, sustainability, and adaptability converge. Future warehousing will evolve from automation and AI to sustainability and customization, a dynamic and transformative journey. Summary of the Invention

[0004] To this end, the present invention provides a path planning method for an intelligent warehouse AGV group based on communication collaboration to solve the problems raised in the background technology.

[0005] In order to achieve the above objectives, the present invention provides the following technical solution: a method for intelligent warehouse AGV group path planning based on communication collaboration, comprising the following steps:

[0006] S1: Use the grid map method to divide the two-dimensional space of the intelligent warehouse to be solved, and convert the real space where the AGV group moves into a data structure that is easy for computer storage and processing;

[0007] S2: The AGV uses its own sensor information for positioning. With itself as the center, it observes the surrounding environment and perceives spatial information, including the positions of other AGVs and obstacles. It also designs reward functions, action spaces, and neural networks.

[0008] S3: When the time step is equal to 1, the observation space information obtained in S2 is input into the deep neural network for preprocessing. The output head generates the message used in the next time step. The strategy of the automatic guided vehicle (AGV) at the current time step is calculated by the reinforcement learning method NoisyNet-Dueling.

[0009] S4: When the time step is greater than or equal to 2, for a certain AGV, the message output by other AGVs in its communication range in the previous time step is used as the input of the AGV's communication module in the current time step. After passing through multiple calculation blocks, the AGV obtains a message fused by communication. The message obtained by the AGV after communication is connected with the AGV's own observation space information obtained through preprocessing to obtain the final message. Finally, the output head outputs the AGV's strategy and message in the current time step.

[0010] S5: until the planning time is less than or equal to the maximum time step, each automated guided vehicle AGV obtains the strategy of the current time step from S4, so that the automated guided vehicle AGV completes the movement from the starting position to the target position, thereby completing the path planning of the automated guided vehicle AGV group.

[0011] Preferably, the specific steps of dividing the intelligent warehouse two-dimensional space in step S1 are:

[0012] S11: Use the grid map method to model the intelligent warehouse environment. According to the complex environment of the intelligent warehouse system, the objects in it are classified into automatic guided vehicles (AGVs), obstacles, and free grids.

[0013] S12: The grid map method marks the free grid as 1 and the grid occupied by obstacles as 0.

[0014] Preferably, the observation space, reward function and action space in step S2 are specifically:

[0015] In the context of smart warehousing, design the observation space, reward function, and action space of the entire path planning method;

[0016] The observation space includes the position of the AGV within its limited field of view, the position of obstacles, the target position of the AGV, the heuristic channel, and the direction vector of the AGV pointing to the target position. The specific information is recorded in the grid of the rasterized map.

[0017] The reward function is a composite reward, which includes the sum of intrinsic and extrinsic rewards;

[0018] The extrinsic reward is: if the AGV moves one step, it will receive a reward of -0.5; if the AGV remains stationary and the position is the target position, it will receive a reward of 0; if the AGV remains stationary and the position is not the target position, it will receive a reward of -0.5; if the AGV collides, it will receive a reward of -3; if the AGV causes a blockage, it will receive a reward of -2; if the AGV reaches the target position, it will receive a reward of +20;

[0019] The intrinsic reward is as follows: each AGV has an independent buffer with a capacity of K, which is empty at the beginning of each round; the storage locations in the buffer represent the locations that the AGV is already familiar with; first, a rectangular coordinate system is established for the grid map, and the current position is represented by the coordinates (x, y). At each step, each AGV that deviates from the target first calculates the Euclidean distance between its current position and all the storage locations in the buffer; if the maximum distance is greater than a threshold τ, the AGV generates an intrinsic reward;

[0020]

[0021] Among them, r i represents the intrinsic reward; τ checks whether the automated guided vehicle AGV has walked out of the qualitative area familiar to it at its current position in the current event; the value of a depends on the scale of the task reward, while the value of b determines the sign of the intrinsic reward, and c is a predefined hyperparameter, the value of which determines the sign of the intrinsic reward; the present invention sets a = 0.7, b = 1; the final reward is calculated as the sum of the intrinsic reward and the extrinsic reward; after calculating any intrinsic reward, each automated guided vehicle AGV determines that its current maximum distance is greater than or equal to the threshold τ, then adds the current position coordinates (x, y) to the buffer, and if the buffer is at full capacity K and the current element is about to enter the buffer, the random element in the buffer will be replaced by the current element;

[0022] The action space consists of five directions, including the four basic directions of front, back, left, and right, as well as remaining still.

[0023] Preferably, the specific steps of pre-processing in step S3 are:

[0024] S31: The position of the AGV within the limited field of view, the position of the obstacle, the target position of the AGV, and the heuristic channel. This 7×7×7 tensor observation channel matrix passes through 3 convolutional layers and 1 maximum pooling layer. Then, the process is repeated once and finally passes through 1 convolutional layer.

[0025] S32: At the same time, the direction vector of the automatic guided vehicle AGV pointing to the target position is transmitted through a fully connected layer;

[0026] S33: Then, the results of the encoding of the two parts of the observation value S31 and S32 are concatenated and then passed through two fully connected layers;

[0027] S34: Finally, the observed value encoding result of S33 and the hidden state of the previous time step The input is fed into the LSTM, giving each automated guided vehicle (AGV) the ability to integrate past information and complete the observation encoding, which is the preprocessing process.

[0028] Preferably, the specific method for generating a strategy for the automatic guided vehicle AGV at the current time step in step S3 is:

[0029] NoisyNet-Dueling adds noise to the linear fully connected layer of the original Dueling DQN to generate a strategy for each automatic guided vehicle (AGV). The method of adding noise to the parameters is as follows:

[0030] The original linear fully connected layer, the forward calculation formula of the fully connected layer is y = wx + b, where w is the weight and b is the bias; and the linear layer with noise added is expressed as Among them, q w +k w ⊙v w Instead of w, q b +k b ⊙v b Replace b;

[0031] This method adds noise to the parameters. Noise needs to be reset after each training, that is, the random value v needs to be reset. w and v b , the noise is calculated as follows:

[0032]

[0033] The function represented by f is q w , k w ,q b , k b It is learnable, v w and v b is a noise random variable, v i represents the input noise, v j represents the output noise;

[0034] During training, the loss function of the output strategy is:

[0035]

[0036] in, is the total return of the automated guided vehicle (AGV), is the reward received by the AGV at time t, s t ,a t is the state and action of the automatic guided vehicle AGV at time t, is the parameter of the target network, μ is the online parameter, γ is the discount factor, Evaluate the value of the action at time t, MSE is the mean square error.

[0037] Preferably, the specific steps of the automatic guided vehicle AGV obtaining the communication fusion message in step S4 are:

[0038] S41: Output the message of the last time step of all AGVs within the communication range Enter Embedding to enhance the information specific to each AGV;

[0039] S42: The message after embedding is input into the calculation block. First, the message goes through the layer normalization method. Then, the dynamic relationship between the messages is encoded using the multi-head attention mechanism in the improved Transformer encoder. Then, the message goes through the gated recurrent unit, layer normalization method, feedforward neural network, gated recurrent unit in sequence. Finally, the intermediate message of the automatic guided vehicle AGV is obtained.

[0040] S43: The AGV encodes the message after communication into node features and edge features in the graph neural network to build communication connections, and then samples the communication connections. Remove each message in the dataset and determine whether the message has any impact on the decision of the AGV. Use KL-divergence to estimate the impact of the message on the AGV.

[0041] D=D KL (P(a i |a -i ,o)||P(a i |a -i ,o,e))

[0042] Among them, o is the joint observation value, a i For the movement of the automatic guided vehicle AGV itself, a -i For joint actions other than the AGV, the smaller D is, the smaller the impact of the message is. If the message does not affect the final decision, the corresponding communication connection on the AGV is marked as redundant and the message is removed. Otherwise, the message is retained.

[0043] S44: Repeat the calculation of the communication mechanism S42 and S43 twice. The final message obtained by the automatic guided vehicle AGV under the communication is

[0044] Preferably, the specific steps for completing the path planning of the automatic guided vehicle AGV group in step S5 are:

[0045] S51: S5 determines the strategy of each AGV at each time step, and each AGV moves one step or remains stationary;

[0046] S52: When all AGVs complete their respective path planning tasks, the AGV group path planning ends. If an AGV is damaged due to a collision and cannot move, the AGV path planning task is considered to have ended. If the maximum time step of the plan is exceeded, the AGV group path planning task is terminated.

[0047] The present invention has the following advantages:

[0048] 1. This invention uses automated guided vehicles (AGVs), which can reduce labor costs and floor space. Furthermore, not every task requires continuous human intervention. Combining robots, drones, and AGVs can enable partial or full autonomous execution of certain tasks.

[0049] 2. This invention uses reinforcement learning algorithms to plan AGV routes, making inbound and outbound operations in smart warehouses faster and more accurate, effectively shortening operation time while improving accuracy. This AI-driven approach can optimize AGV routes, reduce process bottlenecks, and enable dynamic task allocation and on-site handshakes.

[0050] 3. The present invention introduces communication into the path planning algorithm, which enhances the cooperation between automatic guided vehicles (AGVs) and effectively reduces the collision rate.

[0051] 4. The present invention enhances the communication content during communication, effectively extracts the more useful information in the communication, prevents the automatic guided vehicle AGV from being overwhelmed by a large amount of information during the communication process, and improves the communication quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Fig. 1 A schematic diagram of the process provided by the present invention;

[0053] Fig. 2 This is a diagram of the communication module of the automatic guided vehicle AGV provided by the present invention. DETAILED DESCRIPTION

[0054] The following embodiments of the present application are illustrated by way of specific examples, and other advantages and effects of the present application will be apparent to those skilled in the art from this disclosure. It is readily apparent to one skilled in the art that the described embodiments are only a part of embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0055] The embodiment provides a communication cooperation-based intelligent warehouse AGV group path planning method, as shown in the figure, which comprises the following steps: Figs. 1-2

[0056] S1: using a grid map method to divide the two-dimensional space of the intelligent warehouse to be solved, and converting the real space of the AGV group movement into a data structure easy to store and process by a computer;

[0057] S2: the AGV uses its own sensor information for positioning, perceives the surrounding environment around the AGV, obtains observation space information such as the positions of other AGVs and obstacles, and designs a reward function, an action space and a neural network;

[0058] S3: when the time step is equal to 1, the observation space information obtained in S2 is input into a deep neural network for preprocessing, a message used in the next time step is generated by an output head, and the strategy of the AGV in the current time step is calculated by a reinforcement learning method NoisyNet-Dueling;

[0059] S4: when the time step is greater than or equal to 2, for an AGV, taking the car i as an example, the messages output by other AGVs in the previous time step within the communication range of the car i are input into the communication module of the car i in the current time step, and after a plurality of calculation blocks, the car i obtains the messages fused by communication after communication. The messages obtained by the AGV after communication are connected with the observation space information of the AGV obtained by preprocessing, to obtain the final messages. Finally, the strategy and the messages of the AGV in the current time step are output by the output head;

[0060] S5: until the planning time is less than or equal to the maximum time step, each AGV obtains the strategy in the current time step from S4, so that the AGV completes the movement from the starting position to the target position, thereby completing the path planning of the AGV group.

[0061] The specific steps of dividing the two-dimensional space of the intelligent warehouse in S1 of the example are as follows:

[0062] S11: using a grid map method to model the intelligent warehouse environment, and classifying and defining the objects in the intelligent warehouse system as AGVs, obstacles and free grids according to the complex environment of the intelligent warehouse system;​

[0063] S12: The grid map method marks the free grid as 1 and the grid occupied by obstacles as 0.

[0064] In this example, the observation space, reward function, and action space in step S2 are specifically:

[0065] In the scenario of smart warehousing, the observation space, reward function and action space of the entire path planning method are designed.

[0066] The observation space includes the position of the AGV (car i) within its limited field of view, the position of obstacles, the target position of the car, the heuristic channel, and the direction vector of the AGV (car i) pointing to the target position. The specific information is recorded in the grid of the rasterized map.

[0067] The reward function is a composite reward, which includes the sum of intrinsic and extrinsic rewards. The extrinsic reward is: for each AGV step, a reward of -0.5 is given; if the AGV remains stationary and the position is the target position, a reward of 0 is given; if the AGV remains stationary and the position is not the target position, a reward of -0.5 is given; if the AGV collides, a reward of -3 is given; if the AGV causes an obstruction, a reward of -2 is given; if the AGV reaches the target position, a reward of +20 is given.

[0068] The intrinsic reward is: Each AGV has an independent buffer with a capacity of K, which is empty at the beginning of each round. The storage location of the buffer represents the location that the AGV is already familiar with. In each step, each AGV that deviates from the target first calculates the Euclidean distance between its current location (establishing a rectangular coordinate system for the grid map, using coordinates (x, y) to represent the current location) and all the storage locations in the buffer. If the maximum distance is greater than the threshold τ, the AGV generates an intrinsic reward

[0069]

[0070] Among them, r i Represents the intrinsic reward; τ checks whether the AGV has walked out of the qualitative area it is familiar with at its current position in the current event. The value of a depends on the scale of the task reward, while the value of b determines the sign of the intrinsic reward. c is a predefined hyperparameter, and the value of c determines the sign of the intrinsic reward. The present invention sets a=0.7 and b=1. The final reward is calculated as the sum of the intrinsic reward and the extrinsic reward. After calculating any intrinsic reward, each AGV determines that its current maximum distance is greater than or equal to the threshold τ, then adds the current position (x, y) to the buffer. If the buffer is at full capacity K and the current element is about to enter the buffer, the random element in the buffer will be replaced by the current element.

[0071] The action space consists of five directions, including the four basic directions of front, back, left, and right, as well as remaining still.

[0072] The specific steps of pre-processing in step S3 of this example are:

[0073] S31: The position of the AGV (car i) within its limited field of view, the position of the obstacle, the target position of the car, and the heuristic channel. This 7×7×7 tensor observation channel matrix passes through three convolutional layers and one maximum pooling layer. Then, the process is repeated once and finally passes through one convolutional layer.

[0074] S32: At the same time, the direction vector of the AGV (car i) pointing to the target position is passed through a fully connected layer;

[0075] S33: Then, the results of the encoding of the two parts of the observation value S31 and S32 are concatenated and then passed through two fully connected layers;

[0076] S34: Finally, the observed value encoding result of S33 and the hidden state of the previous time step The input is fed into LSTM, giving each AGV the ability to integrate its own past information and complete the observation encoding, which is the preprocessing process.

[0077] The specific method for AGV to generate a strategy at the current time step in step S3 of this example is:

[0078] We use a method called NoisyNet-Dueling to generate policies for each AGV by adding noise to the linear fully connected layers of the original Dueling DQN. The method of adding noise to the parameters is as follows.

[0079] The original linear fully connected layer, the forward calculation formula of the fully connected layer is y = wx + b, where w is the weight and b is the bias. The linear layer with noise added is expressed as Among them, q w +k w ⊙v w Instead of w, q b +k b ⊙v b Replace b.

[0080] This method adds noise to the parameters. Noise needs to be reset after each training, that is, the random value v needs to be reset. w and v b , the noise is calculated as follows:

[0081]

[0082] The function represented by f is qw , k w ,q b , k b It is learnable, v w and v b is a noise random variable, v i represents the input noise, v j represents the output noise.

[0083] During training, the loss function of the output strategy is

[0084]

[0085] in, is the total reward of car i, is the reward received by car i at time t, s t ,a t is the state and action of car i at time t, is the parameter of the target network, μ is the online parameter, γ is the discount factor, Evaluate the value of the action at time t, MSE is the mean square error.

[0086] The specific steps for small car i in S4 to obtain the communication fusion message are as follows:

[0087] S41: Output the message of the last time step of all AGVs within the communication range Enter Embedding to enhance the information specific to each AGV;

[0088] S42: The message after embedding is input into the calculation block. First, the message goes through the layer normalization method. Then, the dynamic relationship between the messages is encoded using the multi-head attention mechanism in the improved Transformer encoder. Then, the message goes through the gated recurrent unit, layer normalization method, feedforward neural network, gated recurrent unit in sequence. Finally, the intermediate message of car i is obtained.

[0089] S43: Car i encodes the message after communication into node features and edge features in the graph neural network to build a communication connection, and then samples the communication connection. Remove each message in the dataset and determine whether the message has any impact on the decision of car i. Use KL-divergence to estimate the impact of the message on car i.

[0090] D=D KL (P(a i |a -i ,o)||P(a i |a -i ,o,e));

[0091] Among them, o is the joint observation value, a i For the movement of the automatic guided vehicle AGV itself, a -i = is the joint action other than car i. The smaller D is, the smaller the impact of the message is. If the message does not affect the final decision, the corresponding communication connection on car i is marked as redundant and the message is removed. Otherwise, the message is retained.

[0092] S44: Repeat the calculation of communication mechanisms S42 and S43 twice. The final message obtained by car i under the communication is

[0093] The specific steps for completing the path planning of the AGV group in step S5 of this example are:

[0094] S51: S5 determines the strategy of each AGV at each time step, and each AGV moves one step or remains stationary;

[0095] S52: When all AGVs complete their respective path planning tasks, the AGV group path planning ends. If an AGV is damaged due to a collision and cannot move, it is considered that the AGV path planning task has ended. If the maximum time step of the plan is exceeded, the AGV group path planning task ends.

[0096] According to the above examples, it can be seen that in order to address the problems of traditional warehousing systems such as requiring a large amount of manpower, low efficiency, prone to errors, and high labor costs, the method of this embodiment establishes a two-dimensional plan map of the warehouse map, models the warehouse using the grid map method, and uses the reinforcement learning algorithm NoisyNet-Duel ing to plan the path for the AGV, thereby improving the operating efficiency of the AGV. At the same time, by establishing efficient communication between AGVs and enhancing cooperation between AGVs, safe and efficient AGV group collaborative operation is achieved; at the same time, the method is highly reusable and extensible, and only a small part of the content needs to be modified to apply it in other smart warehousing scenarios.

[0097] Although this embodiment has been described in detail above using general explanations and specific examples, it will be apparent to those skilled in the art that modifications or improvements may be made to this embodiment. Therefore, such modifications or improvements, without departing from the spirit of this embodiment, are within the scope of protection claimed in this embodiment.

Claims

1. A communication-based collaborative intelligent warehouse AGV group path planning method, characterized by: The steps include: S1: Use the grid map method to divide the two-dimensional space of the intelligent warehouse to be solved, and convert the real space where the AGV group moves into a data structure; S2: The AGV uses its own sensor information for positioning. With itself as the center, it perceives the surrounding environment's observation space information, including the positions of other AGVs and obstacles, and designs a reward function, action space, and neural network. S3: When the time step is equal to 1, the observation space information obtained in S2 is input into the deep neural network for preprocessing. The output head generates the message used in the next time step. The strategy of the automatic guided vehicle (AGV) at the current time step is calculated by the reinforcement learning method NoisyNet-Dueling. S4: When the time step is greater than or equal to 2, for a certain AGV, the message output by other AGVs in its communication range in the previous time step is used as the input of the AGV's communication module in the current time step. After passing through multiple calculation blocks, the AGV obtains a message fused by communication. The message obtained by the AGV after communication is connected with the AGV's own observation space information obtained through preprocessing to obtain the final message. Finally, the output head outputs the AGV's strategy and message in the current time step. S5: until the planning time is less than or equal to the maximum time step, each automated guided vehicle AGV obtains the strategy of the current time step from S4, so that the automated guided vehicle AGV completes the movement from the starting position to the target position, thereby completing the path planning of the automated guided vehicle AGV group.

2. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The specific steps of dividing the intelligent storage two-dimensional space in step S1 are: S11: Use the grid map method to model the intelligent warehouse environment. According to the complex environment of the intelligent warehouse system, the objects in it are classified into automatic guided vehicles (AGVs), obstacles, and free grids. S12: The grid map method marks the free grid as 1 and the grid occupied by obstacles as 0.

3. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The observation space, reward function and action space in step S2 are specifically: In the context of smart warehousing, design the observation space, reward function, and action space of the entire path planning method; The observation space includes the position of the AGV within its limited field of view, the position of obstacles, the target position of the AGV, the heuristic channel, and the direction vector of the AGV pointing to the target position. The specific information is recorded in the grid of the rasterized map. The reward function is a composite reward, which includes the sum of intrinsic and extrinsic rewards; The extrinsic rewards are designed as follows: If the AGV moves one step, it will get a reward value of -0.5; If the AGV remains stationary and the position is the target position, the reward value is 0; If the AGV remains stationary and the position is not the target position, it will receive a reward of -0.

5. If the AGV collides, it will receive a reward of -3. If the AGV causes a blockage, it will receive a reward of -2. If the AGV reaches the target location, it will receive a reward of +20. The intrinsic reward is: each AGV has an independent buffer with a capacity of K, which is empty at the beginning of each round; the storage location of the buffer represents the location that the AGV is already familiar with; First, a rectangular coordinate system is established for the grid map, with the coordinates (x, y) representing the current position. At each step, each AGV that deviates from the target first calculates the Euclidean distance between its current position and all stored positions in the buffer. If the maximum distance is greater than a threshold τ, the AGV generates an intrinsic reward. Among them, r i represents the intrinsic reward; τ checks whether the automated guided vehicle AGV has walked out of the qualitative area it is familiar with at its current position in the current event; the value of a depends on the scale of the task reward, while the value of b determines the sign of the intrinsic reward, and c is a predefined hyperparameter, the value of c determines the sign of the intrinsic reward; the final reward is calculated as the sum of the intrinsic reward and the extrinsic reward; after calculating any intrinsic reward, each automated guided vehicle AGV determines that its current maximum distance is greater than or equal to the threshold τ, then adds the current position coordinates (x, y) to the buffer, and if the buffer is at full capacity K and the current element is about to enter the buffer, the random element in the buffer will be replaced by the current element; The action space consists of five directions, including the four basic directions of front, back, left, and right, as well as remaining still.

4. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The specific steps of the pre-processing in step S3 are: S31: The position of the AGV within the limited field of view, the position of the obstacle, the target position of the AGV, and the heuristic channel. This 7×7×7 tensor observation channel matrix passes through 3 convolutional layers and 1 maximum pooling layer. Then, the process is repeated once and finally passes through 1 convolutional layer. S32: At the same time, the direction vector of the automatic guided vehicle AGV pointing to the target position is transmitted through a fully connected layer; S33: Then, the results of the encoding of the two parts of the observation value S31 and S32 are concatenated and then passed through two fully connected layers; S34: Finally, the observed value encoding result of S33 and the hidden state of the previous time step The input is fed into the LSTM, giving each automated guided vehicle (AGV) the ability to integrate past information and complete the observation encoding, which is the preprocessing process.

5. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The specific method for the automatic guided vehicle AGV to generate a strategy at the current time step in step S3 is: NoisyNet-Dueling adds noise to the linear fully connected layer of the original Dueling DQN to generate a strategy for each automatic guided vehicle (AGV). The method of adding noise to the parameters is as follows: The original linear fully connected layer, the forward calculation formula of the fully connected layer is y = wx + b, where w is the weight and b is the bias; and the linear layer with noise added is expressed as Among them, q w +k w ⊙v w Instead of w, q b +k b ⊙v b Replace b; This method adds noise to the parameters. Noise needs to be reset after each training, that is, the random value v needs to be reset. w and v b , the noise is calculated as follows: The function represented by f is q w , k w ,q b , k b It is learnable, v w and v b is a noise random variable, v i represents the input noise, v j represents the output noise; During training, the loss function of the output strategy is: in, is the total return of the automated guided vehicle (AGV), is the reward received by the AGV at time t, s t ,a t is the state and action of the automatic guided vehicle AGV at time t, is the parameter of the target network, μ is the online parameter, γ is the discount factor, Evaluate the value of the action at time t, MSE is the mean square error.

6. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The specific steps for the automatic guided vehicle AGV to obtain the communication fusion message in step S4 are: S41: Output the message of the last time step of all AGVs within the communication range Enter Embedding to enhance the information specific to each AGV; S42: The message after embedding is input into the calculation block. First, the message goes through the layer normalization method. Then, the dynamic relationship between the messages is encoded using the multi-head attention mechanism in the improved Transformer encoder. Then, the message goes through the gated recurrent unit, layer normalization method, feedforward neural network, gated recurrent unit in sequence. Finally, the intermediate message of the automatic guided vehicle AGV is obtained. S43: The AGV encodes the message after communication into node features and edge features in the graph neural network to build communication connections, and then samples the communication connections. Remove each message in the dataset and determine whether the message has any impact on the decision of the AGV. Use KL-divergence to estimate the impact of the message on the AGV. D=D KL (P(a i |a -i ,o)||P(a i |a -i ,o,e)); Among them, o is the joint observation value, a i For the movement of the automatic guided vehicle AGV itself, a -i For joint actions other than the AGV, the smaller D is, the smaller the impact of the message is. If the message does not affect the final decision, the corresponding communication connection on the AGV is marked as redundant and the message is removed. Otherwise, the message is retained. S44: Repeat the calculation of the communication mechanism S42 and S43 twice. The final message obtained by the automatic guided vehicle AGV under the communication is 7. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The specific steps for completing the path planning of the automatic guided vehicle AGV group in step S5 are: S51: S5 determines the strategy of each AGV at each time step, and each AGV moves one step or remains stationary; S52: When all AGVs complete their respective path planning tasks, the AGV group path planning ends. If an AGV is damaged due to a collision and cannot move, the AGV path planning task is considered to have ended. If the maximum time step of the plan is exceeded, the AGV group path planning task is terminated.

Images