Intelligent storage AGV group path planning method based on communication cooperation

Abstract

The invention discloses an intelligent storage AGV group path planning method based on communication cooperation, and particularly relates to the technical field of intelligent storage, and the method comprises the steps: dividing an intelligent storage two-dimensional space needing to be solved through a grid method; a reward function, an action space and a neural network are set, a sensor is additionally arranged on the trolley, and positioning information and observation space information are obtained; firstly, a first-step strategy is obtained by observing spatial information and combining a reinforcement learning algorithm; then, communication is carried out through part of messages generated in the previous step length, the part of messages are combined with observation space information of the trolley in the current step length, a strategy and messages of the time step length are obtained, the process is repeated, and a complete path planning answer is generated for the trolley; and finally, under a set constraint condition, completing the path planning of all the trolleys, namely completing the path planning of the trolley group.

Description

Technical Field

[0001] The present invention relates to the field of intelligent warehousing, and in particular to a path planning method for an intelligent warehousing AGV group based on communication collaboration. Background Art

[0002] Warehousing plays a vital role in logistics and plays a key role in overall operational performance. Traditional warehouse operations are mainly manpower-based, with problems such as low efficiency, prone to errors and high labor costs. Taking the warehousing area into consideration, the systemic risks brought by human errors lead to low productivity, and human resource investment is also regarded as a waste. Warehousing in the era of intelligent logistics has brought about major changes. Through the application of real-time sensing technology, information processing technology and artificial intelligence technology, warehousing has become more intelligent, liberating a large amount of human and material resources, and significantly improving operational efficiency and real-time response capabilities of order processing.

[0003] Warehousing has come a long way from being a simple storage facility to an advanced logistics center. With the rapid development of technology, more and more industrial plants are adopting highly automated smart warehouse systems to improve their competitiveness. Artificial intelligence and machine learning algorithms are widely used to predict demand patterns, optimize inventory management, and improve cargo transportation route planning. With the help of big data, warehouses have become more intelligent, enabling real-time decision-making and better resource allocation. The combination of automation and robotics is revolutionizing the warehousing industry, including the deployment of automated guided vehicles (AGVs), drones, and robotic pickers, which optimize warehouse operations, improve efficiency, and reduce the risk of workplace injuries. The future of warehousing is full of exciting prospects, where technology, sustainability, and adaptability merge with each other. The future of warehousing will move from automation and artificial intelligence to sustainability and customization, becoming a dynamic and transformative journey in the field. Summary of the invention

[0004] To this end, the present invention provides a path planning method for an intelligent warehouse AGV group based on communication collaboration to solve the problems raised in the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solution: a path planning method for an intelligent warehouse AGV group based on communication collaboration, comprising the following steps:

[0006] S1: The two-dimensional space of intelligent warehouse to be solved is divided using the grid map method, and the real space where the AGV group moves is converted into a data structure that is easy for computer storage and processing;

[0007] S2: The AGV uses its own sensor information for positioning. With itself as the center, it observes the spatial information of the surrounding environment, including the positions of other AGVs and obstacles, and designs reward functions, action spaces, and neural networks.

[0008] S3: When the time step is equal to 1, the observation space information obtained in S2 is input into the deep neural network for preprocessing, and the output head generates the message used in the next time step. The strategy of the automatic guided vehicle AGV at the current time step is calculated by the reinforcement learning method NoisyNet-Dueling;

[0009] S4: When the time step is greater than or equal to 2, for a certain AGV, the message output by other AGVs in the previous time step within its communication range is used as the input of the communication module of the current time step of the AGV. After multiple calculation blocks, the AGV obtains a message fused by the communication after communication. The message obtained by the AGV after communication is connected with the observation space information of the AGV itself obtained by preprocessing to obtain the final message. Finally, the output head outputs the strategy and message of the current time step of the AGV.

[0010] S5: until the planning time is less than or equal to the maximum time step, each automated guided vehicle AGV obtains the strategy of the current time step from S4, so that the automated guided vehicle AGV completes the movement from the starting position to the target position, thereby completing the path planning of the automated guided vehicle AGV group.

[0011] Preferably, the specific steps of dividing the intelligent storage two-dimensional space in step S1 are:

[0012] S11: Use the grid map method to model the intelligent warehousing environment. According to the complex environment of the intelligent warehousing system, the objects in it are classified into AGV, obstacles and free grids;

[0013] S12: The grid map method marks the free grid as 1 and the grid occupied by obstacles as 0.

[0014] Preferably, the observation space, reward function and action space in step S2 are specifically:

[0015] In the scenario of smart warehousing, design the observation space, reward function and action space of the entire path planning method;

[0016] The observation space includes the position of the AGV within the limited field of view, the obstacle position, the target position of the AGV, the heuristic channel, and the direction vector of the AGV pointing to the target position. The specific information is recorded in the grid of the rasterized map.

[0017] The reward function is a combined reward, which includes the sum of intrinsic rewards and extrinsic rewards;

[0018] The external rewards are: if the AGV moves one step, a reward value of -0.5 is obtained; if the AGV remains stationary and the position is the target position, a reward value of 0 is obtained; if the AGV remains stationary and the position is not the target position, a reward value of -0.5 is obtained; if the AGV collides, a reward value of -3 is obtained; if the AGV causes a blockage, a reward value of -2 is obtained; if the AGV reaches the target position, a reward value of +20 is obtained;

[0019] The intrinsic reward is: each AGV has an independent buffer with a capacity of K, which is empty at the beginning of each round; the storage location of the buffer represents the location that the AGV is already familiar with; first, a rectangular coordinate system is established for the grid map, and the current position is represented by the coordinates (x, y). In each step, each AGV that deviates from the target first calculates the Euclidean distance between its current position and all the storage locations in the buffer; if the maximum distance is greater than the threshold τ, the AGV generates an intrinsic reward;

[0020]

[0021] Among them, r i represents the intrinsic reward; τ checks whether the automated guided vehicle AGV has walked out of the qualitative area familiar to it at the current position in the current event; the value of a depends on the scale of the task reward, while the value of b determines the sign of the intrinsic reward, and c is a predefined hyperparameter, and the value of c determines the sign of the intrinsic reward; the present invention sets a=0.7, b=1; the final reward is calculated as the sum of the intrinsic reward and the extrinsic reward; after calculating any intrinsic reward, each automated guided vehicle AGV determines that its current maximum distance is greater than or equal to the threshold τ, then adds the current position coordinates (x, y) to the buffer, and if the buffer is at full capacity K and the current element is about to enter the buffer, the random element in the buffer will be replaced by the current element;

[0022] The action space consists of five directions, including four basic directions: front, back, left, right, and staying still.

[0023] Preferably, the specific steps of preprocessing in step S3 are:

[0024] S31: The position of the AGV within the limited field of view, the position of the obstacle, the target position of the AGV, and the heuristic channel. The observation channel matrix of this 7×7×7 tensor passes through 3 convolutional layers and 1 maximum pooling layer. Then, the process is repeated once and finally passes through 1 convolutional layer.

[0025] S32: At the same time, the direction vector of the automatic guided vehicle AGV pointing to the target position is transmitted through a fully connected layer;

[0026] S33: Then, the results of the encoding of the two parts of observation values ​​in S31 and S32 are concatenated and then passed through two fully connected layers;

[0027] S34: Finally, the observed value encoding result of S33 and the hidden state of the previous time step The input is fed into LSTM, giving each automated guided vehicle (AGV) the ability to integrate past information and complete the observation encoding, which is the preprocessing process.

[0028] Preferably, the specific method for generating a strategy for the automatic guided vehicle AGV at the current time step in step S3 is:

[0029] NoisyNet-Dueling adds noise to the linear fully connected layer of the original Dueling DQN to generate a strategy for each automatic guided vehicle AGV. The method of adding noise to the parameters is as follows:

[0030] The original linear fully connected layer, the forward calculation formula of the fully connected layer is y = wx + b, where w is the weight and b is the bias; and the linear layer with noise added is expressed as Among them, q w +k w ⊙v w Instead of w, q b +k b ⊙v b Replace b;

[0031] This method adds noise to the parameters. Noise needs to be reset after each training, that is, the random value v needs to be reset. w and v b , the noise is calculated as follows:

[0032]

[0033] The function represented by f is q w , k w ,q b , k b is learnable, v w and v b is a noise random variable, v i represents the input noise, v j represents the output noise;

[0034] During training, the loss function of the output strategy is:

[0035]

[0036] in, is the total return of the automated guided vehicle, is the reward received by the AGV at time t, s t , a t is the state and action of the automatic guided vehicle AGV at time t, is the parameter of the target network, μ is the online parameter, γ is the discount factor, Evaluate the value of the action at time t, MSE is the mean square error.

[0037] Preferably, the specific steps of the automatic guided vehicle AGV obtaining the communication fusion message in step S4 are:

[0038] S41: Output the message of the last time step of all AGVs within the communication range Enter Embedding to enhance the information specific to each AGV;

[0039] S42: The message after embedding is input into the calculation block. First, the message is normalized by the layer method. Then, the dynamic relationship between the messages is encoded by the multi-head attention mechanism in the improved Transformer encoder. Then, the message is passed through the gated recurrent unit, layer normalization method, feedforward neural network, gated recurrent unit in turn. Finally, the intermediate message of the automatic guided vehicle AGV is obtained.

[0040] S43: The AGV encodes the communicated message into node features and edge features in the graph neural network to build a communication connection, and then samples the communication connection. Remove each message in the AGV to determine whether the message has an impact on the decision of the AGV, and use KL-divergence to estimate the impact of the message on the AGV

[0041] D=D KL (P(a i |a -i ,o)||P(a i |a -i , o, e))

[0042] Among them, o is the joint observation value, a i is the movement of the automatic guided vehicle AGV itself, a -i For joint actions other than the AGV, the smaller D is, the smaller the impact of the message is. If the message does not affect the final decision, the corresponding communication connection on the AGV is marked as redundant and the message is removed. Otherwise, the message is retained.

[0043] S44: Repeat the calculation of communication mechanisms S42 and S43 twice. The final message obtained by the automatic guided vehicle AGV under the action of communication is:

[0044] Preferably, the specific steps for completing the path planning for the AGV group in step S6 are:

[0045] S61: S5 determines the strategy of each automatic guided vehicle AGV at each time step, and each automatic guided vehicle AGV moves one step or remains stationary;

[0046] S62: When all the automatic guided vehicles (AGVs) complete their respective path planning tasks, the path planning of the AGV group ends. If the automatic guided vehicle (AGV) is damaged due to a collision and cannot move, it is considered that the path planning task of the automatic guided vehicle (AGV) has ended. If the maximum time step of the planning is exceeded, the path planning task of the AGV group ends.

[0047] The present invention has the following advantages:

[0048] 1. The present invention uses an automated guided vehicle, which can reduce labor costs and floor space. At the same time, not every task requires continuous human intervention. Combining robots, drones and automated guided vehicles can achieve partial or full autonomous execution of certain tasks.

[0049] 2. The present invention uses reinforcement learning algorithms to plan the AGV path, making the warehousing and outbound operations of the smart warehouse faster and more accurate, effectively shortening the operation time and improving the accuracy of the operation. The artificial intelligence-driven method can optimize the AGV driving route, reduce process blockages, and realize dynamic task allocation and on-site handshakes.

[0050] 3. The present invention introduces communication into the path planning algorithm, enhances the cooperation between automatic guided vehicles (AGVs), and effectively reduces the collision rate.

[0051] 4. The present invention enhances the communication content during communication, effectively extracts the more useful information in the communication, prevents the automatic guided vehicle AGV from being overwhelmed by a large amount of information during the communication process, and improves the communication quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of the process provided by the present invention;

[0053] Figure 2 This is a diagram of the automatic guided vehicle AGV communication module provided by the present invention. DETAILED DESCRIPTION

[0054] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] This embodiment provides a method for intelligent warehouse AGV group path planning based on communication collaboration, such as Figure 1-Figure 2 As shown, it includes the following steps:

[0056] S1: The two-dimensional space of intelligent warehouse to be solved is divided using the grid map method, and the real space where the automatic guided vehicle (AGV) group moves is converted into a data structure that is easy for computer storage and processing;

[0057] S2: The AGV uses its own sensor information for positioning, takes itself as the center, perceives the surrounding environment, obtains observation space information such as the position of other AGVs and obstacles, and designs reward functions, action spaces, and neural networks;

[0058] S3: When the time step is equal to 1, the observation space information obtained in S2 is input into the deep neural network for preprocessing, and the output head generates the message used in the next time step. The strategy of the AGV in the current time step is calculated by the reinforcement learning method NoisyNet-Dueling;

[0059] S4: When the time step is greater than or equal to 2, for a certain AGV, take car i as an example, the message output by other AGVs in its communication range in the previous time step is used as the input of the communication module of car i in the current time step. After multiple calculation blocks, car i obtains the message fused by communication after communication. The message obtained by AGV after communication is connected with the observation space information obtained by AGV after preprocessing to obtain the final message. Finally, the output head outputs the strategy and message of AGV in the current time step;

[0060] S5: until the planning time is less than or equal to the maximum time step, each AGV obtains the strategy of the current time step from S4, so that the AGV completes the movement from the starting position to the target position, thereby completing the path planning of the AGV group.

[0061] The specific steps of dividing the intelligent warehouse two-dimensional space in step S1 of this example are:

[0062] S11: Use the grid map method to model the intelligent warehousing environment. According to the complex environment of the intelligent warehousing system, the objects in it are classified into AGV, obstacles and free grids;

[0063] S12: The grid map method marks the free grid as 1 and the grid occupied by obstacles as 0.

[0064] In this example, the observation space, reward function and action space in step S2 are specifically:

[0065] In the scenario of smart warehousing, the observation space, reward function and action space of the entire path planning method are designed.

[0066] The observation space includes the position of the AGV (car i) within its limited field of view, the position of obstacles, the target position of the car, the heuristic channel, and the direction vector of the AGV (car i) pointing to the target position. The specific information is recorded in the grid of the rasterized map.

[0067] The reward function is a combined reward, which includes the addition of intrinsic rewards and extrinsic rewards. The extrinsic reward is: if the AGV moves one step, it will get a reward value of -0.5; if the AGV remains stationary and the position is the target position, it will get a reward value of 0; if the AGV remains stationary and the position is not the target position, it will get a reward value of -0.5; if the AGV collides, it will get a reward value of -3; if the AGV causes a blockage, it will get a reward value of -2; if the AGV reaches the target position, it will get a reward value of +20.

[0068] The intrinsic reward is: Each AGV has an independent buffer with a capacity of K, which is empty at the beginning of each round. The storage locations in the buffer represent the locations that the AGV is already familiar with. In each step, each AGV that deviates from the target first calculates the Euclidean distance between its current position (establishing a rectangular coordinate system for the grid map, using coordinates (x, y) to represent the current position) and all the storage locations in the buffer. If the maximum distance is greater than the threshold τ, the AGV generates an intrinsic reward

[0069]

[0070] Among them, r i Represents the intrinsic reward; τ checks whether the AGV has walked out of the qualitative area familiar to it at its current position in the current event. The value of a depends on the scale of the task reward, while the value of b determines the sign of the intrinsic reward. c is a predefined hyperparameter, and the value of c determines the sign of the intrinsic reward. The present invention sets a=0.7 and b=1. The final reward is calculated as the sum of the intrinsic reward and the extrinsic reward. After calculating any intrinsic reward, each AGV determines that its current maximum distance is greater than or equal to the threshold τ, then adds the current position (x, y) to the buffer. If the buffer is at full capacity K and the current element is about to enter the buffer, the random element in the buffer will be replaced by the current element.

[0071] The action space consists of five directions, including four basic directions: front, back, left, right, and staying still.

[0072] The specific steps of preprocessing in step S3 of this example are:

[0073] S31: The position of the AGV (car i) within its limited field of view, the position of the obstacle, the target position of the car, and the heuristic channel. This 7×7×7 tensor observation channel matrix passes through 3 convolutional layers and 1 maximum pooling layer. Then, the process is repeated once and finally passes through 1 convolutional layer.

[0074] S32: At the same time, the direction vector of AGV (car i) pointing to the target position is transmitted through a fully connected layer;

[0075] S33: Then, the results of the encoding of the two parts of observation values ​​in S31 and S32 are concatenated and then passed through two fully connected layers;

[0076] S34: Finally, the observed value encoding result of S33 and the hidden state of the previous time step The input is fed into LSTM, giving each AGV the ability to integrate its past information and complete the observation encoding, which is the preprocessing process.

[0077] The specific method for AGV to generate a strategy at the current time step in step S3 of this example is:

[0078] A method of adding noise to the linear fully connected layer of the original Dueling DQN is used to generate a policy for each AGV. The method is called NoisyNet-Dueling. The method of adding noise to the parameters is as follows.

[0079] The original linear fully connected layer, the forward calculation formula of the fully connected layer is y = wx + b, where w is the weight and b is the bias. The linear layer with added noise is expressed as Among them, q w +k w ⊙v w Instead of w, q b +k b ⊙v b Replace b.

[0080] This method adds noise to the parameters. Noise needs to be reset after each training, that is, the random value v needs to be reset. w and v b , the noise is calculated as follows:

[0081]

[0082] The function represented by f is qw , k w ,q b , k b is learnable, v w and v b is a noise random variable, v i represents the input noise, v j represents the output noise.

[0083] During training, the loss function of the output strategy is

[0084]

[0085] in, is the total return of car i, is the reward received by car i at time t, s t , a t is the state and action of car i at time t, is the parameter of the target network, μ is the online parameter, γ is the discount factor, Evaluate the value of the action at time t, MSE is the mean square error.

[0086] The specific steps for small car i in S4 to obtain the communication fusion message are:

[0087] S41: Output the message of the last time step of all AGVs within the communication range Enter Embedding to enhance the information specific to each AGV;

[0088] S42: The message after embedding is input into the calculation block. First, the message is normalized by the layer method. Then, the dynamic relationship between messages is encoded by the multi-head attention mechanism in the improved Transformer encoder. Then, the message is passed through the gated recurrent unit, layer normalization method, feedforward neural network, gated recurrent unit in turn. Finally, the intermediate message of car i is obtained.

[0089] S43: Car i encodes the message after communication into node features and edge features in the graph neural network to build a communication connection, and then samples the communication connection. Remove each message in the equation to determine whether the message has any impact on the decision of car i, and use KL-divergence to estimate the impact of the message on car i

[0090] D=D KL (P(a i |a -i ,o)||P(a i |a -i , o, e));

[0091] Among them, o is the joint observation value, a i is the movement of the automatic guided vehicle AGV itself, a -i is the joint action other than car i. The smaller D is, the smaller the impact of the message is. If the message does not affect the final decision, the corresponding communication connection on car i is marked as redundant and the message is removed. Otherwise, the message is retained.

[0092] S44: Repeat the calculation of communication mechanisms S42 and S43 twice. The final message obtained by car i under the communication is:

[0093] The specific steps for completing the path planning of the AGV group in step S6 of this example are:

[0094] S61: S5 determines the strategy of each AGV at each time step, and each AGV moves one step or remains stationary;

[0095] S62: When all AGVs complete their respective path planning tasks, the AGV group path planning ends. If the AGV is damaged due to collision and cannot move, it is considered that the AGV path planning task ends. If the maximum time step of the planning is exceeded, the AGV group path planning task ends.

[0096] According to the above examples, in order to address the problems of traditional warehousing systems, such as requiring a large amount of manpower, low efficiency, prone to errors, and high labor costs, the method of this embodiment establishes a two-dimensional plan view of the warehouse map, models the warehouse using the grid map method, and uses the reinforcement learning algorithm NoisyNet-Dueling to plan the path for the AGV, thereby improving the operating efficiency of the AGV. At the same time, by establishing efficient communication between AGVs and enhancing cooperation between AGVs, safe and efficient AGV group collaborative operation is achieved; at the same time, the method is highly reusable and extensible, and only a small part of the content needs to be modified to be applied in other smart warehousing scenarios.

[0097] Although the present embodiment has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present embodiment. Therefore, these modifications or improvements made without departing from the spirit of the present embodiment all fall within the scope of protection claimed in the present embodiment.

Claims

1. A method for intelligent warehouse AGV group path planning based on communication collaboration, characterized by: The steps include: S1: The two-dimensional space of intelligent warehouse to be solved is divided using the grid map method, and the real space where the AGV group moves is converted into a data structure; S2: The AGV uses its own sensor information for positioning, and uses itself as the center to perceive the surrounding environment's observation space information, including the positions of other AGVs and obstacles, and designs reward functions, action spaces, and neural networks; S3: When the time step is equal to 1, the observation space information obtained in S2 is input into the deep neural network for preprocessing, and the output head generates the message used in the next time step. The strategy of the automatic guided vehicle AGV at the current time step is calculated by the reinforcement learning method NoisyNet-Duel ing; S4: When the time step is greater than or equal to 2, for a certain AGV, the message output by other AGVs in the previous time step within its communication range is used as the input of the communication module of the current time step of the AGV. After multiple calculation blocks, the AGV obtains a message fused by the communication after communication. The message obtained by the AGV after communication is connected with the observation space information of the AGV itself obtained by preprocessing to obtain the final message. Finally, the output head outputs the strategy and message of the current time step of the AGV. S5: until the planning time is less than or equal to the maximum time step, each automated guided vehicle AGV obtains the strategy of the current time step from S4, so that the automated guided vehicle AGV completes the movement from the starting position to the target position, thereby completing the path planning of the automated guided vehicle AGV group.

2. According to claim 1, a communication-coordinated intelligent warehouse AGV group path planning method is characterized by: The specific steps of dividing the intelligent storage two-dimensional space in step S1 are: S11: Use the grid map method to model the intelligent warehousing environment. According to the complex environment of the intelligent warehousing system, the objects in it are classified into AGV, obstacles and free grids; S12: The grid map method marks the free grid as 1 and the grid occupied by obstacles as 0.

3. According to claim 1, a communication-coordinated intelligent warehousing AGV group path planning method is characterized by: The observation space, reward function and action space in step S2 are specifically: In the scenario of smart warehousing, design the observation space, reward function and action space of the entire path planning method; The observation space includes the position of the AGV within the limited field of view, the obstacle position, the target position of the AGV, the heuristic channel, and the direction vector of the AGV pointing to the target position. The specific information is recorded in the grid of the rasterized map. The reward function is a combined reward, which includes the sum of intrinsic rewards and extrinsic rewards; The external rewards are designed as follows: If the AGV moves one step at a time, it will get a reward value of -0.5; If the AGV remains stationary and the position is the target position, the reward value is 0; If the AGV remains stationary and the position is not the target position, a reward value of -0.5 is obtained; If the AGV collides, the reward value is -3; If the AGV causes a blockage, the reward value is -2; If the AGV reaches the target location, it will receive a reward of +20; The intrinsic reward is: each AGV has an independent buffer with a capacity of K, which is empty at the beginning of each round; the storage location of the buffer represents the location that the AGV is already familiar with; First, a rectangular coordinate system is established for the grid map, and the current position is represented by the coordinates (x, y). In each step, each AGV that deviates from the target first calculates the Euclidean distance between its current position and all storage positions in the buffer respectively; if the maximum distance is greater than the threshold τ, the AGV generates an intrinsic reward; Among them, r i represents the intrinsic reward; τ checks whether the AGV has walked out of the qualitative area familiar to it at its current position in the current event; the value of a depends on the scale of the task reward, while the value of b determines the sign of the intrinsic reward, c is a predefined hyperparameter, and the value of c determines the sign of the intrinsic reward; the final reward is calculated as the sum of the intrinsic reward and the extrinsic reward; after calculating any intrinsic reward, each AGV determines that its current maximum distance is greater than or equal to the threshold τ, then adds the current position coordinates (x, y) to the buffer, and if the buffer is at full capacity K and the current element is about to enter the buffer, the random element in the buffer will be replaced by the current element; The action space consists of five directions, including four basic directions: front, back, left, right, and staying still.

4. According to claim 1, a communication-coordinated intelligent warehouse AGV group path planning method is characterized by: The specific steps of pre-processing in step S3 are: S31: The position of the AGV within the limited field of view, the position of the obstacle, the target position of the AGV, and the heuristic channel. The observation channel matrix of this 7×7×7 tensor passes through 3 convolutional layers and 1 maximum pooling layer. Then, the process is repeated once and finally passes through 1 convolutional layer. S32: At the same time, the direction vector of the automatic guided vehicle AGV pointing to the target position is transmitted through a fully connected layer; S33: Then, the results of the encoding of the two parts of observation values ​​in S31 and S32 are concatenated and then passed through two fully connected layers; S34: Finally, the observed value encoding result of S33 and the hidden state of the previous time step The input is fed into LSTM, giving each automated guided vehicle (AGV) the ability to integrate past information and complete the observation encoding, which is the preprocessing process.

5. According to claim 1, a communication-coordinated intelligent warehouse AGV group path planning method is characterized by: The specific method for generating the strategy of the automatic guided vehicle AGV at the current time step in step S3 is: NoisyNet-Dueling adds noise to the linear fully connected layer of the original Dueling DQN to generate a strategy for each automatic guided vehicle AGV. The method of adding noise to the parameters is as follows: The original linear fully connected layer, the forward calculation formula of the fully connected layer is y = wx + b, where w is the weight and b is the bias; and the linear layer with noise added is expressed as Among them, q w +k w ⊙v w Instead of w, q b +k b ⊙v b Replace b; This method adds noise to the parameters. Noise needs to be reset after each training, that is, the random value v needs to be reset. w and v b , the noise is calculated as follows: The function represented by f is q w , k w ,q b , k b is learnable, v w and v b is a noise random variable, v i represents the input noise, v j represents the output noise; During training, the loss function of the output strategy is: in, is the total return of the automated guided vehicle, is the reward received by the AGV at time t, s t ,a t is the state and action of the automatic guided vehicle AGV at time t, is the parameter of the target network, μ is the online parameter, γ is the discount factor, Evaluate the value of the action at time t, MSE is the mean square error.

6. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The specific steps for the automatic guided vehicle AGV to obtain the communication fusion message in step S4 are: S41: Output the message of the last time step of all AGVs within the communication range Enter Embedding to enhance the information specific to each AGV; S42: The message after embedding is input into the calculation block. First, the message is normalized by the layer method. Then, the dynamic relationship between the messages is encoded by the multi-head attention mechanism in the improved Transformer encoder. Then, the message is passed through the gated recurrent unit, layer normalization method, feedforward neural network, gated recurrent unit in turn. Finally, the intermediate message of the automatic guided vehicle AGV is obtained. S43: The AGV encodes the communicated message into node features and edge features in the graph neural network to build a communication connection, and then samples the communication connection. Remove each message in the AGV to determine whether the message has an impact on the decision of the AGV, and use KL-divergence to estimate the impact of the message on the AGV D=D KL (P(a i |a -i ,o)||P(a i |a -i ,o,e)); Among them, o is the joint observation value, a i is the movement of the automatic guided vehicle AGV itself, a -i For joint actions other than the AGV, the smaller D is, the smaller the impact of the message is. If the message does not affect the final decision, the corresponding communication connection on the AGV is marked as redundant and the message is removed. Otherwise, the message is retained. S44: Repeat the calculation of communication mechanisms S42 and S43 twice. The final message obtained by the automatic guided vehicle AGV under the action of communication is:

7. The method for intelligent warehouse AGV group path planning based on communication collaboration according to claim 1 is characterized by: The specific steps for completing the path planning of the AGV group in step S6 are: S61: S5 determines the strategy of each automatic guided vehicle AGV at each time step, and each automatic guided vehicle AGV moves one step or remains stationary; S62: When all the automatic guided vehicles (AGVs) complete their respective path planning tasks, the path planning of the AGV group ends. If the automatic guided vehicle (AGV) is damaged due to a collision and cannot move, it is considered that the path planning task of the automatic guided vehicle (AGV) has ended. If the maximum time step of the planning is exceeded, the path planning task of the AGV group ends.

Images