Multi-AGV task scheduling method and device based on multi-strategy whale optimization algorithm combined with reinforcement learning

Abstract

The invention discloses a multi-AGV task scheduling method and device based on a multi-strategy whale optimization algorithm combined with reinforcement learning. The method comprises the following steps: S1, establishing a mathematical model for multi-AGV task scheduling; s2, taking the balance between the path cost and the AGV vacancy rate as an optimization index, and constructing an AGV scheduling optimization model; s3, combining a multi-strategy whale optimization algorithm with a reinforcement learning method framework, and determining a Markov decision process including state, action, reward and value definition; and S4, solving the AGV scheduling optimization model by adopting a multi-strategy whale optimization algorithm combined with reinforcement learning. According to the method, the multi-strategy improved WOA is realized by initializing the population through chaotic mapping, adjusting the non-convergence factor and introducing reinforcement learning to update the whale individual position updating strategy, the calculation precision of the WOA is improved, the algorithm convergence is accelerated, the capacity of the WOA for jumping out of local optimum and searching for global optimum is enhanced, a feasible scheme is provided for multi-AGV task scheduling, and the method is suitable for large-scale popularization and application. And the method has certain advancement and feasibility.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-AGV task scheduling in intelligent warehousing, and relates to a method and device for multi-AGV task scheduling based on a multi-strategy whale optimization algorithm combined with reinforcement learning. Background Art

[0002] Automated Guided Vehicle (AGV) is a flexible and intelligent logistics processing robot widely used in intelligent warehousing systems. AGV task allocation involves assigning different tasks to multiple AGVs to achieve efficient logistics transportation and automated operations. The goal of task allocation is to maximize the utilization of AGVs, reduce task completion time, and improve overall system efficiency. The task allocation problem is a relatively complex permutation and combination problem. Single AGV task allocation can no longer meet current actual production needs. Therefore, how to reasonably allocate multiple AGV tasks to achieve the shortest transportation time and the highest transportation efficiency is a current research hotspot.

[0003] The patent "A method and system for AGV task scheduling based on an improved genetic algorithm" aims to solve the problem that the early task allocation method cannot meet the current AGV task allocation needs, and provides a method based on an improved genetic algorithm to optimize task allocation. The patent "Robot task scheduling method and related device with improved population crossover strategy" comprehensively considers the total running time of the robot and the balance between the robot's tasks, and improves the crossover strategy of the NSGA-Ⅱ algorithm. The improved NSGA-Ⅱ algorithm adopts a combined partial matching crossover strategy (CPMX) to improve the crossover efficiency of the algorithm and balance the robot's task allocation. The patent "AGV task scheduling method based on simulated annealing algorithm" discloses an AGV task scheduling method based on a simulated annealing algorithm, which aims to solve the optimization problem of scientifically and reasonably allocating shelf handling tasks among multiple AGVs in an intelligent warehousing system, with the goal of minimizing the completion time of all tasks. The patent "A method for allocating AGV tasks based on multi-agent deep reinforcement learning" aims at the problem of poor adaptability and scalability of centralized decision-making methods, and provides an AGV task allocation method based on multi-agent deep reinforcement learning that can improve the information potential field reward function to solve the reward sparsity problem, provide continuous rewards for AGVs, and implicitly guide AGVs to move to different cargo targets. The patent "A dynamic crossover genetic algorithm for multi-AGV task scheduling" aims at the problem that AGV scheduling has poor effect in practical applications, and uses a multi-objective optimization model with the least power consumption of single and group AGVs. Constraints such as AGV power loss and load capacity are introduced into the scheduling model to make the model more suitable for practical applications. Secondly, appropriate improvements have been made to the genetic algorithm to obtain better results with faster computing efficiency.

[0004] From the research in recent years, it can be seen that most of the studies take a single goal as the optimization target, ignoring the balance of multi-AGV task execution. In addition, most of the studies rely on classic optimization algorithms, such as genetic algorithms, which will limit the optimization potential of the scheduling system. Summary of the invention

[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a method and device for multi-AGV task scheduling based on a multi-strategy whale optimization algorithm combined with reinforcement learning, with the aim of achieving a balance between path cost and AGV vacancy rate during task scheduling and maximizing the overall efficiency of the AGV scheduling system.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions: Figure 1 As shown:

[0007] The multi-AGV task scheduling method based on the multi-strategy whale optimization algorithm combined with reinforcement learning includes the following steps:

[0008] S1. Establish a mathematical model for multi-AGV task scheduling, as follows:

[0009] The specific parameters are defined as follows:

[0010] m: number of tasks;

[0011] n: number of AGVs;

[0012] k: the number of tasks in the task queue of each AGV;

[0013] ti: task number, i∈{1,2,…,m};

[0014] r j : AGV number, j∈{1,2,…,n};

[0015] x i,j,k :Task t i Assigned to car r j The kth task in the task queue;

[0016] D j : Total cost of AGV j’s journey;

[0017] T j : Total time cost of AGV j;

[0018] I: AGV vacancy rate.

[0019] The constraints are as follows:

[0020]

[0021] Among them, constraint (1) means that each task can only be assigned to one item in the task queue of an AGV, constraint (2) means that the number of tasks in the task queue of each AGV cannot exceed k, and constraint (3) means that the tasks in the task queue of each AGV must be executed in sequence.

[0022] S2. Taking the balance between path cost and AGV vacancy rate as the optimization index, an AGV scheduling optimization model is constructed, as follows:

[0023] The vacancy rate of AGV refers to the proportion of the time that AGV is idle to the total time within a certain period of time. For urgent orders, if there is an idle AGV, it will be assigned to the idle AGV. Under normal circumstances, it will choose to wait for all AGVs to complete the task before assigning a new task batch. In order to reduce the waiting time of the task, the vacancy rate of AGV is used as one of the optimization goals of task allocation. Because AGVs work at the same time, the total task time is the maximum time cost of AGV, and the waiting time of the AGV that completes the task first is the longest, so the difference between the maximum time cost and the minimum time cost divided by the maximum time cost is taken as the vacancy rate.

[0024] The scheduling optimization target is a comprehensive consideration between the total path cost D and the AGV vacancy rate I. The weighted sum method is used for calculation. The fitness function calculation formula is as follows:

[0025] Fitness=w1×D+w2×I (4)

[0026] Among them, w1 and w2 are the weights of path cost and vacancy rate respectively. The smaller the total path cost and vacancy rate, the smaller the fitness and the better the result.

[0027] S3, combine the multi-strategy whale optimization algorithm with the reinforcement learning method framework to determine the Markov decision process (MDP), including the definition of state, action, reward, and value. The specific steps of step S3 include:

[0028] S301, state selection:

[0029] The population characteristics are measured by considering the average population fitness, population standard deviation, and best individual fitness as single indicators.

[0030]

[0031]

[0032] Among them, i represents the i-th individual in the population, T is the maximum number of iterations, and t is the current number of iterations.

[0033] In formula (5), the average population fitness of the current iteration is divided by the average population fitness of the first generation for normalization; formula (6) is the population standard deviation, which is also normalized by dividing it by the standard deviation of the first generation population; formula (7) is the best individual fitness, which is normalized by dividing it by the best fitness of the first generation population.

[0034] The state space is divided by the changes in the average population fitness, the population standard deviation, and the best individual fitness.

[0035] S302, action selection:

[0036] There are four ways to update individual positions in the multi-strategy whale optimization algorithm: search or surround, bubble net attack, and Brownian motion. Among them, search and surround are mentioned together because the choice of these two actions is mainly determined by The modulus value determines the action selection, so it is regarded as a method. The action set is set to three actions, [0, 1, 2], which correspond to the actions of search or encirclement, bubble net attack, and Brownian motion in the multi-strategy whale optimization algorithm.

[0037] S303, Reward Selection:

[0038] The reward is determined based on the changes in the average population fitness, population standard deviation, and best individual fitness. The goal is to learn the position update action that optimizes the scheduling target. The calculation formula is as follows:

[0039] R=R AF +R DF +R F (8)

[0040]

[0041] Among them, the smaller the algorithm fitness, the better the result. Therefore, the best individual fitness of the population will not be greater than the initial best individual fitness, so there is no possibility of >1; the average population fitness decreases, indicating that most individuals tend to the best individual fitness; the increase in the population standard deviation indicates that the fitness values ​​of individuals in the population vary greatly and the population diversity is higher.

[0042] S304, Exploration and Utilization:

[0043] The purpose of exploration and exploitation is mainly to obtain the global optimal solution, where exploitation is to harvest the current maximum benefit, and exploration is to obtain the maximum benefit of the entire Q learning process. The action selection strategy is the ε-greedy strategy, which pre-sets the value of ε before Q learning. When the agent makes a decision, there is a probability of ε to randomly select an action, and there is a probability of 1-ε to select the action that the agent thinks is the best, so that the agent has the possibility of generating better solutions in addition to using historical experience. The calculation formula for exploration and exploitation is as follows:

[0044]

[0045] S4, using a multi-strategy whale optimization algorithm combined with reinforcement learning to solve the AGV scheduling optimization model and obtain the optimal scheduling solution. The specific steps of step S4 include:

[0046] S401, Whale Optimization Algorithm Coding:

[0047] Assume there are n tasks, m vehicles, and the position vector is expressed as The vector dimension is the number of tasks, the vector position is the task number, x i The value range is [-m,m].

[0048] S402, individual position vector conversion:

[0049] In the WOA algorithm, the individual position vector elements are continuous values, while the task scheduling solution is discrete values. Linear transformation is used to map the continuous value to a finite number of discrete values. A simple linear transformation method is to use equidistance discretization. The transformation formula is as follows:

[0050]

[0051] Among them, round means rounding down, the continuous value is x, the number of discrete values ​​is n, and the range of discrete values ​​is [a, b].

[0052] S403, initializing reinforcement learning parameters;

[0053] S404, chaos map initialization population:

[0054] Kent mapping initializes the whale population. Through chaotic mapping, the solutions can be distributed as evenly as possible in the solution space. The Kent mapping formula is as follows:

[0055]

[0056] Among them, the mapping parameter α∈(0,1).

[0057] When α is within the acceptable range, the system is in a chaotic state. The Lyapunov index is an important indicator to measure the sensitivity of a chaotic system to initial conditions. For a chaotic system, a positive Lyapunov index usually indicates that the system has chaotic properties. The larger the Lyapunov index, the stronger the chaos of the system. The Lyapunov formula is as follows:

[0058]

[0059] Where λ is the Lyapunov exponent, which represents the exponential growth rate of the system state. x(t) is the state vector of the system at time t. n is the length of the time series, t i is the i-th time point in the time series.

[0060] According to the Lyapunov exponents obtained under different mapping parameters, the mapping parameters with the strongest chaos in the system are determined.

[0061] S405, calculating the fitness values ​​of the individuals in the population, selecting the best individual as the initial optimal solution, and calculating the initial population state.

[0062] S406: Select an action according to the greedy strategy.

[0063] S407, update parameter a, l, p:

[0064] The present invention provides a nonlinear convergence factor a, the formula is as follows:

[0065]

[0066] Among them, t is the current iteration number, T max is the maximum number of iterations.

[0067] S408. Update individual position according to the selected action:

[0068] The individuals in the population select a position update strategy according to the obtained actions. In order to increase the diversity of the population, the present invention provides a Brownian motion position update strategy.

[0069] The Brownian motion position update formula is as follows:

[0070]

[0071] in, represents the position of the current whale after t iterations, μ is a random number that follows a uniform distribution. It represents the multiplication of element-wise average, rand defines a random number in the range [0,1], the value of the sign function sign[rand-12] is 1, 0, -1, and Brownian represents a random step size that follows the normal distribution.

[0072] The random step length Δx of Brownian motion i Follows the standard normal distribution:

[0073]

[0074] Considering the time interval and step size, the step generation formula is:

[0075]

[0076] Where time_Interval is the time interval. The square root is used to adjust the standard deviation of the step length so that it is proportional to the time interval, which conforms to the nature of Brownian motion. step_size is a parameter for adjusting the step size, which is used to control the amount of movement in each step.

[0077] S409, calculate individual fitness values, update the optimal solution, and calculate the new population state.

[0078] S410, calculate reward R and update Q table;

[0079] S411. Repeat steps S406-S410. If the iteration termination condition is met, output the final result of multi-AGV task allocation.

[0080] The second aspect of the present invention relates to a multi-AGV task scheduling device based on a multi-strategy whale optimization algorithm combined with reinforcement learning, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the multi-AGV task scheduling method based on a multi-strategy whale optimization algorithm combined with reinforcement learning of the present invention.

[0081] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the multi-AGV task scheduling method of the present invention based on a multi-strategy whale optimization algorithm combined with reinforcement learning.

[0082] The present invention realizes multi-strategy improvement of WOA by initializing the population through chaotic mapping, adjusting the non-convergence factor and introducing reinforcement learning to update the individual position update strategy of whales, improves the calculation accuracy of WOA, accelerates the convergence of the algorithm and enhances the ability of WOA to jump out of the local optimum and search for the global optimum, and provides a feasible solution for multi-AGV task scheduling, which has certain advancement and feasibility.

[0083] The advantages of the present invention are: initializing the population through chaotic mapping, adjusting the non-convergence factor and introducing reinforcement learning to adjust the individual whale position update strategy to achieve multi-strategy improved WOA, improving the calculation accuracy of WOA, accelerating the convergence of the algorithm and enhancing the ability of WOA to jump out of the local optimum and search for the global optimum. For multi-AGV task scheduling, minimizing the path cost and AGV vacancy rate is used as the objective function, and tasks are allocated based on the multi-strategy improved WOA, and the solution with the minimum fitness is obtained as the optimal solution, thereby achieving a balance between the path cost and the vacancy rate during task scheduling, and maximizing the overall efficiency of the AGV scheduling system. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 is a flow chart of the method of the present invention;

[0085] Figure 2 A flowchart of the multi-strategy whale optimization method combined with reinforcement learning adopted by the present invention;

[0086] Figure 3 This is the parameter change trend of the multi-AGV task scheduling optimization model in the embodiment of the present invention. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the novel technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments:

[0088] like Figure 2 As shown, this is a flowchart of the multi-strategy whale optimization algorithm combined with reinforcement learning used in the present invention. Based on the standard WOA, the algorithm adopts a chaotic mapping initialization population strategy, improves the nonlinear convergence factor α strategy and Brownian motion, and performs multi-strategy improvements on WOA, thereby improving the calculation accuracy of WOA, accelerating the convergence of the algorithm and enhancing the ability of WOA to jump out of the local optimum. On the basis of the multi-strategy whale optimization algorithm, reinforcement learning is introduced to learn the position update strategy, so that the algorithm can more effectively find the global optimal solution in a more complex search space.

[0089] The present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

[0090] Example 1

[0091] This embodiment relates to a multi-AGV task scheduling method based on a multi-strategy whale optimization algorithm combined with reinforcement learning, comprising the following steps:

[0092] S1. Establish a mathematical model for multi-AGV task scheduling, as follows:

[0093] Given 10 tasks and 5 AGVs, each AGV can be assigned an unlimited number of tasks.

[0094] S2. Taking the balance between path cost and AGV vacancy rate as the optimization index, an AGV scheduling optimization model is constructed, as follows:

[0095] The vacancy rate of AGV refers to the proportion of the time that AGV is idle to the total time within a certain period of time. For urgent orders, if there is an idle AGV, it will be assigned to the idle AGV. Under normal circumstances, it will choose to wait for all AGVs to complete the task before assigning a new task batch. In order to reduce the waiting time of the task, the vacancy rate of AGV is used as one of the optimization goals of task allocation. Because AGVs work at the same time, the total task time is the maximum time cost of AGV, and the waiting time of the AGV that completes the task first is the longest, so the difference between the maximum time cost and the minimum time cost divided by the maximum time cost is taken as the vacancy rate.

[0096] The magnitudes of D and I are usually quite different, which will have a significant impact on the calculation when performing weighted sum, so the data needs to be normalized. The path cost and vacancy rate of the first individual in the initial population are used as the benchmark, denoted as D0 and I0 respectively, and its fitness is denoted as 1.

[0097] The fitness function formula is adjusted to:

[0098] Fitness=w1×D+w2×I (4)

[0099] Among them, the path cost and vacancy rate of the current individual are divided by the benchmark path cost and vacancy rate respectively, and normalized to obtain a fitness value that fluctuates around 1, w1 takes 0.6, and w2 takes 0.4.

[0100] S3, combine the multi-strategy whale optimization algorithm with the reinforcement learning method framework to determine the Markov decision process (MDP), including the definition of state, action, reward, and value. The specific steps of step S3 include:

[0101] S301, state selection:

[0102] The population characteristics are measured by considering the average population fitness, population standard deviation, and best individual fitness as single indicators.

[0103]

[0104] Among them, i represents the i-th individual in the population, T is the maximum number of iterations, and t is the current number of iterations.

[0105] It is divided into 8 levels according to the changes in average population fitness, population standard deviation, and best individual fitness.

[0106] S302, action selection:

[0107] There are four ways to update individual positions in the multi-strategy whale optimization algorithm: search or surround, bubble net attack, and Brownian motion. Among them, search and surround are mentioned together because the choice of these two actions is mainly determined by The modulus value determines the action selection, so it is regarded as a way. Set the action set to three actions, [0, 1, 2], which correspond to the actions of search or encirclement, bubble net attack, and Brownian motion in the multi-strategy whale optimization algorithm.

[0108] S303, Reward Selection:

[0109] The reward is determined based on the changes in the average population fitness, population standard deviation, and best individual fitness. The goal is to learn the position update action that optimizes the scheduling target. The calculation formula is as follows:

[0110] R=R AF +R DF +R F (8)

[0111]

[0112] S304, Exploration and Utilization

[0113] The action selection strategy is the ε-greedy strategy, which sets the value of ε in advance before Q learning. When the agent makes a decision, there is a probability of ε to randomly select an action, and there is a probability of 1-ε to select the action that the agent thinks is the best, so that the agent can not only use historical experience, but also have the possibility of generating better solutions. The calculation formula for exploration and utilization is as follows:

[0114]

[0115] S4, using a multi-strategy whale optimization algorithm combined with reinforcement learning to solve the AGV scheduling optimization model and obtain the optimal scheduling solution. The specific steps of step S4 include:

[0116] S401, Whale Optimization Algorithm Coding:

[0117] Given 10 tasks, 5 vehicles, the position vector is represented as The vector dimension is the number of tasks, the vector position is the task number, x i The value range is [-5,5]. The multi-strategy whale optimization algorithm sets the population size to 30, the dimension to 30, and the maximum number of iterations to 500.

[0118] S402, individual position vector conversion:

[0119] In the WOA algorithm, the individual position vector elements are continuous values, while the task scheduling solution is discrete values. Linear transformation is used to map the continuous value to a finite number of discrete values. A simple linear transformation method is to use equidistance discretization. The transformation formula is as follows:

[0120]

[0121] Among them, round means rounding down, the continuous value is x, the number of discrete values ​​is n, and the discrete values ​​are {1,2,3,4,5}.

[0122] S403, initialize reinforcement learning parameters, alpha is 0.1, discount rate gamma is 0.9, and exploration rate epsilon is 0.1.

[0123] S404, chaos mapping initialization population:

[0124] Kent mapping initializes the whale population. Through chaotic mapping, the solutions can be distributed as evenly as possible in the solution space. The Kent mapping formula is as follows:

[0125]

[0126] Among them, the mapping parameter α∈(0,1).

[0127] When α is within the acceptable range, the system is in a chaotic state. The Lyapunov index is an important indicator to measure the sensitivity of a chaotic system to initial conditions. For a chaotic system, a positive Lyapunov index usually indicates that the system has chaotic properties. The larger the Lyapunov index, the stronger the chaos of the system. The Lyapunov formula is as follows:

[0128]

[0129] Where λ is the Lyapunov exponent, which represents the exponential growth rate of the system state. x(t) is the state vector of the system at time t. n is the length of the time series, t i is the i-th time point in the time series.

[0130] In the range of α, with a step size of 0.1, different mapping parameters are selected to obtain the Lyapunov index and determine the mapping parameter with the strongest chaos in the system. As shown in the table below, when α is 0.5, the Lyapunov index is the largest, so the mapping parameter α is 0.5.

[0131]

[0132] S405, calculating individual fitness values, selecting the best individual as the initial optimal solution, and calculating the initial population state;

[0133] S406: Select an action according to the greedy strategy.

[0134] S407, update parameter a, l, p:

[0135] The present invention provides a nonlinear convergence factor a, the formula is as follows:

[0136]

[0137] Among them, t is the current iteration number, T max is the maximum number of iterations.

[0138] S408. Update individual position according to the selected action:

[0139] The individuals in the population select a position update strategy according to the obtained actions. In order to increase the diversity of the population, the present invention provides a Brownian motion position update strategy.

[0140] In the task scheduling application, there is a continuous-discrete conversion step. If the random step of Brownian motion is too small, the result of discrete conversion after the position update may not change. In addition, the random step of Brownian motion is positive or negative. Therefore, the formula is simplified to increase the probability of position update. The Brownian motion position update formula is adjusted to:

[0141]

[0142] in, It represents the current position of the whale at iteration t, and Brownian represents the random step size that follows the normal distribution.

[0143] The step size generation formula is adjusted to:

[0144]

[0145] Among them, the time interval time_Interval is 1.

[0146] S409, calculating individual fitness values, updating the optimal solution, and calculating the new population state;

[0147] S410, calculate reward R and update Q table.

[0148] S411, repeat steps S406-S410, if the iteration termination condition is reached, output the final result of multi-AGV task allocation, and the parameter change trend of multi-AGV task scheduling optimization model is as follows: Figure 3 shown.

[0149] Compared with the prior art, this embodiment is based on a multi-strategy whale optimization algorithm combined with reinforcement learning, and designs a scheduling function with the balance of total path and AGV vacancy rate as the scheduling objective. It can optimize the standard whale optimization algorithm, achieve the balance of path cost and vacancy rate during task scheduling, and quickly solve the multi-AGV task allocation problem in intelligent warehousing, thereby maximizing the overall efficiency of the AGV scheduling system, and has certain advancement and feasibility.

[0150] Example 2

[0151] The present embodiment relates to a multi-AGV task scheduling device based on a multi-strategy whale optimization algorithm combined with reinforcement learning, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the multi-AGV task scheduling method based on a multi-strategy whale optimization algorithm combined with reinforcement learning of Example 1.

[0152] Example 3

[0153] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the multi-AGV task scheduling method based on the multi-strategy whale optimization algorithm combined with reinforcement learning of Example 1 is implemented.

[0154] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Multi-AGV task scheduling method based on multi-strategy whale optimization algorithm combined with reinforcement learning, It is characterized in that The following steps are involved: S1. Establish a mathematical model for multi-AGV task scheduling; S2, taking the balance between path cost and AGV vacancy rate as the optimization index, constructing the AGV scheduling optimization model; S3. Combine the multi-strategy whale optimization algorithm with the reinforcement learning method framework to determine the Markov decision process (MDP), including the definition of state, action, reward, and value; S4. Use the multi-strategy whale optimization algorithm combined with reinforcement learning to solve the AGV scheduling optimization model and obtain the optimal scheduling solution.

2. The multi-AGV task scheduling method based on the multi-strategy whale optimization algorithm combined with reinforcement learning according to claim 1, It is characterized in that In step S1 The constraints are as follows: Among them, constraint (1) means that each task can only be assigned to one item in the task queue of an AGV, constraint (2) means that the number of tasks in the task queue of each AGV cannot exceed k, and constraint (3) means that the tasks in the task queue of each AGV must be executed in sequence; The specific parameters are defined as follows: m: number of tasks; n: number of AGVs; k: the number of tasks in the task queue of each AGV; t i : task number, i∈{1,2,…,m}; r j : AGV number, j∈{1,2,…,n}; x i,j,k :Task t i Assigned to car r j The kth task in the task queue; D j : Total cost of AGV j’s journey; T j : Total time cost of AGV j; I: AGV vacancy rate.

3. The multi-AGV task scheduling method based on the multi-strategy whale optimization algorithm combined with reinforcement learning according to claim 1, It is characterized in that In step S2, the scheduling optimization target is a comprehensive consideration between the total path cost D and the AGV vacancy rate I, and the fitness value is calculated using a weighted sum method; The vacancy rate of AGV refers to the proportion of the time that AGV is idle to the total time within a certain period of time. Because AGVs work at the same time, the total task time is the maximum time cost of AGV, and the waiting time of the AGV that completes the task first is the longest, so the difference between the maximum time cost and the minimum time cost is divided by the maximum time cost as the vacancy rate. The scheduling optimization target is a comprehensive consideration between the total path cost D and the AGV vacancy rate I. The weighted sum method is used for calculation. The fitness function calculation formula is as follows: Fitness=in 1 ×D+w 2 ×I (4) Among them, w 1 and w 2 are the weights of path cost and vacancy rate respectively; the smaller the total path cost and vacancy rate, the smaller the fitness and the better the result.

4. The multi-AGV task scheduling method based on the multi-strategy whale optimization algorithm combined with reinforcement learning according to claim 1, It is characterized in that Step S3 specifically includes: S301, state selection: The population characteristics are measured by considering the average population fitness, population standard deviation, and best individual fitness as single indicators; Among them, i represents the i-th individual in the population, T is the maximum iteration number, and t is the current iteration number; In formula (5), the average population fitness of the current iteration is divided by the average population fitness of the first generation for normalization; formula (6) is the population standard deviation, which is also divided by the standard deviation of the first generation population for normalization; formula (7) is the best individual fitness, which is divided by the best fitness of the first generation population for normalization; The state space is divided by the changes in the average population fitness, the population standard deviation, and the best individual fitness; S302, action selection: There are four ways to update individual positions in the multi-strategy whale optimization algorithm: search or surround, bubble net attack, and Brownian motion. Among them, search and surround are mentioned together because the choice of these two actions is mainly determined by The modulus value determines the action selection, so it is regarded as a method; the action set is set to three actions, [0, 1, 2], which correspond to the actions of search or encirclement, bubble net attack, and Brownian motion in the multi-strategy whale optimization algorithm; S303, Reward Selection: The reward is determined based on the changes in the average population fitness, population standard deviation, and best individual fitness. The goal is to learn the position update action that optimizes the scheduling target. The calculation formula is as follows: R=R AF +R DF +R F (8) Among them, the smaller the algorithm fitness, the better the result, so the best individual fitness of the population will not be greater than the initial best individual fitness, so there is no possibility of > 1; the average population fitness decreases, indicating that most individuals tend to the best individual fitness; the increase in the population standard deviation indicates that the fitness values ​​of individuals in the population are more different, and the diversity of the population is higher; S304, exploration and utilization. The purpose of exploration and utilization is mainly to obtain the global optimal solution, where utilization is to harvest the current maximum benefit, and exploration is to obtain the maximum benefit of the entire Q learning process; the action selection strategy is the ε-greedy strategy, which pre-sets the value of ε before Q learning. When the agent makes a decision, there is a probability of ε to randomly select an action, and there is a probability of 1-ε to select the action that the agent thinks is the best, so that the agent has the possibility of generating better solutions in addition to using historical experience; the calculation formula for exploration and utilization is as follows:

5. The multi-AGV task scheduling method based on the multi-strategy whale optimization algorithm combined with reinforcement learning according to claim 1, It is characterized in that Step S4 specifically includes: S401, Whale Optimization Algorithm Coding: Assume there are n tasks, m vehicles, and the position vector is expressed as The vector dimension is the number of tasks, the vector position is the task number, x i The value range is [-m,m]; S402, individual position vector conversion: In the WOA algorithm, the individual position vector elements are continuous values, while the task scheduling solution is discrete values. Linear transformation is used to map the continuous values ​​into a finite number of discrete value ranges. A simple linear transformation method is to use equidistance discretization. The transformation formula is as follows: Among them, round means round down, the continuous value is x, the number of discrete values ​​is n, and the range of discrete values ​​is [a, b]; S403, initializing reinforcement learning parameters; S404, chaos mapping initialization population: Kent mapping initializes the whale population. Through chaotic mapping, the solutions can be distributed as evenly as possible in the solution space. The Kent mapping formula is as follows: Among them, the mapping parameter α∈(0,1); When α is within the acceptable range, the system is in a chaotic state; the Lyapunov index is an important indicator to measure the sensitivity of the chaotic system to the initial conditions; for a chaotic system, a positive Lyapunov index usually indicates that the system has chaotic properties, and the larger the Lyapunov index, the stronger the chaos of the system; the Lyapunov formula is as follows: Where λ is the Lyapunov exponent, which represents the exponential growth rate of the system state; x(t) is the state vector of the system at time t; n is the length of the time series, t i is the i-th time point in the time series; According to the Lyapunov exponents obtained under different mapping parameters, the mapping parameters with the strongest chaos in the system are determined; S405, calculating the fitness values ​​of the individuals in the population, selecting the best individual as the initial optimal solution, and calculating the initial population state; S406, selecting an action according to a greedy strategy; S407, update parameter a, l, p: Provide a nonlinear convergence factor a, the formula is as follows: Among them, t is the current iteration number, T max is the maximum number of iterations; S408. Update individual position according to the selected action: Individuals in the population select a position update strategy based on the actions obtained; in order to increase the diversity of the population, a Brownian motion position update strategy is provided; The Brownian motion position update formula is as follows: in, represents the position of the current whale after t iterations, μ is a random number that follows a uniform distribution. Indicates the multiplication of element-wise average, rand defines a random number in the range [0,1], and the sign function The value of is 1, 0, -1, and Brownian represents a random step size that obeys the normal distribution; The random step length Δx of Brownian motion i Follows the standard normal distribution: Δx i ~N(0,1)(18) Considering the time interval and step size, the step generation formula is: Where time_Interval is the time interval. The square root is used to adjust the standard deviation of the step length so that it is proportional to the time interval, which conforms to the nature of Brownian motion. step_size is a parameter for adjusting the step size, which is used to control the amount of movement in each step. S409, calculating individual fitness values, updating the optimal solution, and calculating the new population state; S410, calculate reward R and update Q table; S411. Repeat steps S406-S410. If the iteration termination condition is met, output the final result of multi-AGV task allocation.

6. Multi-AGV task scheduling device based on multi-strategy whale optimization algorithm combined with reinforcement learning, It is characterized in that It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the multi-AGV task scheduling method based on a multi-strategy whale optimization algorithm combined with reinforcement learning as described in any one of claims 1-5.

7. A computer-readable storage medium, It is characterized in that A program is stored thereon, and when the program is executed by the processor, the multi-AGV task scheduling method based on the multi-strategy whale optimization algorithm combined with reinforcement learning as described in any one of claims 1-5 is implemented.

Images