Workshop AGV real-time scheduling method based on A-DDQN improved algorithm

Abstract

The invention discloses a workshop AGV real-time scheduling method based on an A-DDQN improved algorithm, and belongs to the field of warehouse logistics automatic distribution, and the method comprises the following steps: constructing a material constraint based on the information of a to-be-loaded material, and generating a time constraint according to the time needed by each stage and the operation information of an AGV; an artificial potential field method reward function is introduced to improve the Double-DQN algorithm to form an A-DDQN algorithm; solving the model of each stage according to an A-DDQN algorithm to obtain an optimal scheduling scheme; and operating the AGV according to the optimal scheduling scheme, comparing the actual operation time and the estimated completion time of a stage when the AGV completes the operation of the stage, and if the difference value is not smaller than a set threshold value, re-determining the optimal scheduling scheme of the next stage, and operating the AGV according to the new optimal scheduling scheme to the next stage. According to the method, rapid adaptation and decision optimization of the environment are realized, the scheduling safety is better, and the production efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of automatic distribution of warehousing logistics in intelligent workshops, and in particular relates to a workshop AGV real-time scheduling method based on an improved A-DDQN algorithm. Background Art

[0002] In the manufacturing industry, AGVs are widely used, which can achieve advantages such as material handling and flexible layout of production lines. Traditional AGV scheduling methods, such as an AGV scheduling method disclosed in the invention patent with application publication number CN111489007A, have the following workflow: the workstation sends demand information to the system, the system locates the workstation and the AGV that meets the demand, then plans the AGV route and determines whether the route is unobstructed, and finally selects the optimal route based on the judgment result, the system issues a transport instruction, and the AGV runs according to the instruction.

[0003] Traditional AGV scheduling methods fail to fully consider factors such as material dependencies and time constraints between vehicles, leading to problems such as low scheduling efficiency and high production costs. To this end, a real-time scheduling method for discrete workshop AGVs based on time and material constraints has emerged, which comprehensively optimizes AGV scheduling through material and time factors. This method analyzes the dependencies between materials, establishes the correspondence between materials and tasks, and sorts tasks according to material relationships. Taking into account factors such as vehicle availability and transportation capacity, tasks are adjusted. This method introduces time constraints, calculates the machine's remaining operating time, generates time constraints, and avoids competition and conflicts between vehicles through reasonable time planning, thereby improving machine operation efficiency.

[0004] In terms of path planning, traditional algorithms such as the A* algorithm, artificial potential field method, and sampling method perform well in simple mobile robot path planning problems such as single drones or single unmanned vehicles. However, these algorithms do not work well when faced with complex environments and multiple constraints. Therefore, more and more studies use heuristic intelligent algorithms with learning capabilities for path planning, such as deep reinforcement learning algorithms (such as A-DDQN), generative adversarial networks (GANs), graph neural networks (GNNs), and variational autoencoders (VAEs). These algorithms simulate complex behavioral patterns in nature and demonstrate stronger solving capabilities for high-dimensional, complex, and multi-constrained optimization problems. However, traditional reinforcement learning methods have limitations when dealing with large state spaces, and the algorithm has poor stability and performance. It cannot effectively avoid obstacles and maintain a safe distance, which can easily cause problems such as Figure 2 The AGV shown has conflicting paths and poor safety. Summary of the invention

[0005] The present invention provides a workshop AGV real-time scheduling method based on the A-DDQN improved algorithm to solve the problems of poor stability, performance and safety of the existing AGV real-time scheduling method.

[0006] In order to solve the above technical problems, the technical solution provided by the present invention is: The present invention relates to a workshop AGV real-time scheduling method based on an improved A-DDQN algorithm, which comprises the following steps: S1. Divide the transportation operation into loading stage, transportation stage and unloading stage, extract the information of materials to be loaded in each workshop and the operation information of AGV trolleys and establish a model, build material constraints based on the information of materials to be loaded, and generate time constraints based on the time required for each stage and the operation information of AGV trolleys; S2. The artificial potential field reward function is introduced to improve the Double-DQN algorithm to form the A-DDQN algorithm; S3. Solve the model of each stage according to the A-DDQN algorithm to obtain a suitable scheduling plan, and determine the best scheduling plan based on the AGV utilization rate, expected completion time and conflict risk; S4. Run the AGV car according to the optimal scheduling plan. Every time the AGV car completes a stage, compare the actual running time of the stage with the estimated completion time. If the difference is not less than the set threshold, return to S3 to re-determine the optimal scheduling plan for the next stage. The AGV car runs the next stage according to the new optimal scheduling plan.

[0007] Preferably, the step S1 constructs the material constraint based on the information of the material to be loaded, specifically: based on the material type, delivery address and delivery quantity of the material to be loaded, and taking into account the material consumption speed and priority, the material constraint is generated, and the expression of the material constraint is: (2), Among them, C represents the material constraint function, k represents the material type, d represents the delivery address, q represents the delivery quantity, v represents the material consumption speed, p represents the material priority, w1, w2 and w3 represent weights respectively, and w1+w2+w3=1, f(p) represents the function used to calculate the impact of the material priority on the material constraint based on the material priority p, g(d) represents the function used to calculate the impact based on the delivery address, and the material type k is an implicit factor that will affect the value of the material consumption speed v, the material priority p and the delivery quantity q.

[0008] Preferably, the expression of the S1 generation time constraint is: (1), Among them, T total represents the time constraint, T loadl represents the time required for the loading phase, T transrepresents the time required for the transportation phase, T unload Indicates the time required for the unloading phase.

[0009] Preferably, the Double-DQN algorithm in S2 is performed using two independent neural networks, one of which is used to determine the best action to be taken by the AGV in a given state, and the other is used to update the Q value of the best action. The Bellman equation for updating the Q value is defined as: (5), in, represents the Bellman equation of Double-DQN at time t, R t represents the immediate reward at time t, S t+1 represents the state of the car at time t+1, θ represents the weight parameter of the current online neural network, represents the weight parameter of the target neural network, γ represents the discount factor, α represents the learning rate, and its value range is 0 to 1. Q represents the Q value, which is used to measure the total future return expected by the car after taking a certain action. γ represents the discount factor, α represents the learning rate, and its value range is 0 to 1. Q represents the Q value. The Double-DQN algorithm gradually approaches the optimal scheduling plan by calculating the sum of the reward at the current moment and the maximum Q value of the next state.

[0010] Preferably, the expression of the artificial potential field method reward function introduced in S2 is: (6), Among them, R represents the reward function, s represents the state of the AGV, a represents the best action of the AGV, and U attr represents gravity, U rep Indicates repulsion; The calculation formulas of the attraction and repulsion are: (7), (8), Among them, r represents the Euclidean distance between the AGV and the target point, k attr represents the gravitational coefficient, k rep represents the repulsion coefficient, d represents the distance from the current point to the nearest obstacle of the AGV, and D represents the threshold value of the repulsion force generated by the obstacle.

[0011] Preferably, determining the best scheduling solution based on the utilization rate of the AGV, the estimated completion time and the conflict risk is specifically: calculating a comprehensive score based on the utilization rate of the AGV, the estimated completion time and the conflict risk, and determining the best scheduling solution based on the comprehensive score; The calculation formula of the utilization rate is: (9), Among them, ε represents the utilization rate, T work Indicates the actual working time of the AGV in the task, T total Indicates the total working time of the trolley, including waiting time or idle time; The calculation formula for the estimated completion time is: (10), Among them, T n estimated Indicates the estimated completion time of the corresponding stage, T n load represents the loading time of the corresponding stage, T n transport represents the transportation time of the corresponding stage, T n unload Indicates the unloading time of the corresponding stage; The calculation formula of the conflict risk is: (11), Among them, risk represents the risk of conflict, d i represents the minimum distance between the AGV and the i-th obstacle or other vehicles, P i represents the probability of conflict with the ith obstacle or vehicle, and n represents the total number of potential conflicts; The calculation formula of the comprehensive score is: (12), Among them, S tota represents the comprehensive score, T n max Indicates the maximum allowed value of the estimated completion time of the corresponding stage. ω1, ω2 and ω3 represent weights respectively, and ω1+ω2+ω3=1.

[0012] Preferably, the S1 also constructs a grid model, specifically, the factory is divided into multiple grid units, and discrete workshops are constructed. Each grid unit represents the driving state of the AGV in the area. The position of the grid unit is represented by row and column indexes. The formula for calculating the grid unit index of a point is: (3), (4), Among them, I represents the row index, J represents the column index, x and y represent the row coordinates and column coordinates of a point respectively, and x min Indicates the minimum row coordinate of the grid, y max Indicates the maximum column coordinate of the raster.

[0013] Preferably, the grid unit is a rectangular grid unit with a side length of 20 cm.

[0014] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects: 1. The workshop AGV real-time scheduling method based on the improved A-DDQN algorithm of the present invention introduces an artificial potential field method reward function to improve the Double-DQN algorithm to form an A-DDQN algorithm. The model of each stage of AGV scheduling is solved according to the A-DDQN algorithm. The A-DDQN algorithm uses two independent neural networks to respectively select and evaluate the actions of the AGV car, which effectively reduces the Q value over-estimation that may be caused by a single network, and the stability and performance of the algorithm are poor; at the same time, the artificial potential field method reward function enables it to effectively avoid obstacles and maintain a safe distance while pursuing the shortest path, thereby achieving rapid adaptation to the environment and decision optimization, and the safety of AGV car scheduling is better.

[0015] 2. The workshop AGV real-time scheduling method based on the improved A-DDQN algorithm of the present invention generates material constraints based on the material type, delivery address and delivery quantity of the materials to be loaded, and considers the material consumption speed and priority. Whenever the AGV car completes a stage, the actual running time of the stage is compared with the estimated completion time. If the difference is greater than the set threshold, the optimal scheduling plan for the next stage is re-determined. This time limit strategy based on material consumption speed, priority and type characteristics can effectively avoid the risk of machine failure due to excessive running time and can improve production efficiency.

[0016] 3. The workshop AGV real-time scheduling method based on the improved A-DDQN algorithm of the present invention compares the actual running time of a stage with the estimated completion time after the AGV car completes a stage. If the difference is not less than the set threshold, the optimal scheduling plan for the next stage is re-determined. The AGV car runs the next stage according to the new optimal scheduling plan to realize the real-time scheduling of the AGV car. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the real-time scheduling method of workshop AGV based on the improved A-DDQN algorithm; Figure 2 This is a schematic diagram of path conflict; Figure 3 This is an example diagram of the grid method environment model. DETAILED DESCRIPTION

[0018] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with examples. The following examples are used to illustrate the present invention but are not used to limit the scope of the present invention.

[0019] Embodiment: The present invention relates to a workshop AGV real-time scheduling method based on the A-DDQN improved algorithm, which comprises the following steps: S1. The transportation operation is divided into the loading stage, the transportation stage and the unloading stage. The information of the materials to be loaded in each workshop and the operation information of the AGV are extracted and a model is established. The information of the materials to be loaded in the workshop includes the location, quantity, average operation efficiency between workshops, etc. of the materials to be loaded. The operation information of the AGV obtains the real-time data of the materials through sensors and Internet of Things technology to ensure that the scheduling decision is based on the latest and most accurate information. The material constraints are constructed based on the information of the materials to be loaded, and the time constraints are generated according to the time required for each stage and the operation information of the AGV. The expression of the time constraint is: (1), Among them, T total represents the time constraint, T loadl represents the time required for the loading phase, T trans represents the time required for the transportation phase, T unload Indicates the time required for the unloading phase.

[0020] The specific steps of constructing the model are: First, define an integration function F, which converts the collected data (material location, quantity, workshop operation efficiency, etc.) into a unified format for subsequent processing. The expression of the integration function is: , in, represents one of the collected data, n represents the number or serial number of the collected data, and y represents the data converted into a unified format after being processed by the integration function F; Then, define a vector V to represent the collected data of each workshop, namely: , Among them, m is the number of the data after the unified format; Finally, a state equation S is used to describe the state of the AGV at any point in time. The state includes position, power, current load, etc. The expression of the state equation is: , Among them, pos(t) represents the position of the AGV at time t, bat(t) represents the power of the AGV at time t, and load(t) represents the position of the AGV at time t.

[0021] Constructing material constraints based on the information of materials to be loaded is specifically: based on the material type, delivery address and delivery quantity of the materials to be loaded, and taking into account the material consumption speed and priority, generate material constraints. The expression of material constraints is: (2), Where C represents the material constraint function, k represents the material type, which is used to distinguish different types of materials, d represents the delivery address, which is the specific location where the material needs to be delivered, q represents the delivery volume, that is, the number of materials that need to be delivered each time, v represents the material consumption speed, which represents the consumption rate of the material, p represents the priority of the material, w1, w2 and w3 represent the weights respectively, and w1+w2+w3=1, f(p) represents the function used to calculate the impact of the material priority p on the material constraint, g(d) represents the function used to calculate the impact of the delivery address d, Among them, the material type k is an implicit factor, which will affect the value of the material consumption speed v, the material priority p and the delivery quantity q.

[0022] Material constraints are built based on the information of the materials to be loaded, and time constraints are generated based on the time required for each stage and the AGV vehicle operation information. The so-called time constraints refer to: the planned time of the next stage is affected by the actual time spent in each stage, including: (1) The time spent in the loading phase of a single discrete workshop. According to the time spent in loading, the scheduling plan within the discrete feeding workshop is adjusted; (2) According to the actual loading time of the loading stage, the time constraint is modified, the constraint solution model is updated, and a scheduling scheme with high AGV utilization rate and meeting the loading time requirements is selected.

[0023] This step can also construct a grid model, specifically dividing the factory into a plurality of grid units, wherein the grid unit is a rectangular grid unit with a side length of 20 cm, such as Figure 3 As shown in the figure, each discrete workshop consists of multiple grid cells. Each grid cell represents the driving state of the AGV in the area. The position of the grid cell is represented by row and column indexes. The formula for calculating the grid cell index of a point is:

[0024] (3), (4), Among them, I represents the row index, J represents the column index, x and y represent the row coordinates and column coordinates of a point respectively, and x min Indicates the minimum row coordinate of the grid, y max Indicates the maximum column coordinate of the raster.

[0025] The model based on the material constraint and time constraint can use the material constraint to estimate the specific time that the machine can operate, thereby limiting the available time of the next AGV in the loading and transportation stage. Specifically, the material constraints are generated according to the material consumption rate, material priority, and material type and characteristics of the last unloading, and then the remaining time for the machine to operate is calculated based on these material constraints. In the next loading and transportation stage, this specific time is used to limit the available time of the AGV. This time limit strategy based on material consumption rate, priority, and type characteristics can effectively avoid the risk of machine failure due to excessive running time and can improve production efficiency.

[0026] S2. The artificial potential field reward function is introduced to improve the Double-DQN algorithm to form the A-DDQN algorithm; the Double-DQN algorithm uses two independent neural networks, one of which is an action selection network, which is used to determine the best action taken by the AGV in a given state, and the other is an evaluation network, which is used to update the Q value of the best action, so as to effectively reduce the overestimation of the Q value that may be caused by a single network. The Double-DQN algorithm gradually approaches the optimal scheduling solution by calculating the sum of the reward at the current moment and the maximum Q value of the next state. The Bellman equation for updating the Q value is defined as:

[0027] (5), in, represents the Bellman equation of Double-DQN at time t, R t represents the immediate reward at time t, S t+1 represents the state of the car at time t+1, θ represents the weight parameter of the current online neural network, represents the weight parameter of the target neural network, γ represents the discount factor, α represents the learning rate, and its value range is 0 to 1. Q represents the Q value, which is used to measure the total future return expected by the car after taking a certain action. γ represents the discount factor, α represents the learning rate, and its value range is 0 to 1. Q represents the Q value. The improved A-DDQN algorithm introduces artificial potential field reward on the basis of Double-DQN algorithm. The expression of artificial potential field reward function is: (6), Among them, R represents the reward function, s represents the state of the AGV, a represents the best action of the AGV, and U attr represents gravity, U rep Indicates repulsion; The calculation formulas of the attraction and repulsion are: 7), (8), Among them, r represents the Euclidean distance between the AGV and the target point, k attr represents the gravitational coefficient, k rep represents the repulsion coefficient, d represents the distance from the current point to the nearest obstacle of the AGV, and D represents the threshold value of the repulsion force generated by the obstacle.

[0028] In addition, by periodically copying the weights of the evaluation network to the action selection network, the parameters of the two networks are updated synchronously, thereby improving the long-term stability and consistency of the decision. The algorithm can also use the epsilon-greedy strategy to dynamically balance exploration and utilization, optimize the decision-making process based on actual operation data, and ensure the adaptability and efficiency of the model in a changing factory environment.

[0029] S3. Solve the model of each stage according to the A-DDQN algorithm to obtain a suitable scheduling plan, and determine the best scheduling plan based on the utilization rate of the AGV, the estimated completion time and the conflict risk. Specifically, calculate the comprehensive score based on the utilization rate of the AGV, the estimated completion time and the conflict risk, and determine the best scheduling plan based on the comprehensive score; The calculation formula of the utilization rate is: (9), Among them, ε represents the utilization rate, T work Indicates the actual working time of the AGV in the task, T total Indicates the total working time of the trolley, including waiting time or idle time; The calculation formula for the estimated completion time is: (10), Among them, T n estimated Indicates the estimated completion time of the corresponding stage, T n load represents the loading time of the corresponding stage, T n transport represents the transportation time of the corresponding stage, T n unload Indicates the unloading time of the corresponding stage; The calculation formula of the conflict risk is: (11), Among them, risk represents the risk of conflict, d i represents the minimum distance between the AGV and the i-th obstacle or other vehicles, P i represents the probability of conflict with the ith obstacle or vehicle, and n represents the total number of potential conflicts; The calculation formula of the comprehensive score is: (12), Among them, S tota represents the comprehensive score, T n max Indicates the maximum allowed value of the estimated completion time of the corresponding stage. ω1, ω2 and ω3 represent weights respectively, and ω1+ω2+ω3=1.

[0030] Specific steps are as follows Figure 1 As shown, including: S3.1. Input time constraints, material constraints, discrete workshops, AGV operation information and basic information of production tasks (materials to be loaded); S3.2. Based on the information input in S3.1, a suitable scheduling scheme is obtained according to the A-DDQN algorithm. Specifically, material constraints are generated according to the material consumption rate, material priority, and material type and characteristics of the last unloading. Then, the remaining time of the machine operation is calculated based on these material constraints. In the next loading and transportation stage, this specific time is used to limit the available time of the AGV, and then all suitable AGV scheduling schemes are obtained; S3.3. Determine the best AGV scheduling plan and material loading plan based on AGV utilization, estimated completion time and conflict risk, including the preset time T1, T2 and T3 of the three stages; S4. Run the AGV car according to the optimal scheduling plan. Every time the AGV car completes a stage, compare the actual running time of the stage with the estimated completion time. If the difference is not less than the set threshold, return to S3 to re-determine the optimal scheduling plan for the next stage. The AGV car runs the next stage according to the new optimal scheduling plan. The specific steps are as follows. Figure 1 As shown, including: S4.1. The AGV enters the first stage of execution according to the optimal scheduling plan, records the actual time t1 taken by the AGV to complete the first stage task, and calculates the error between the actual time t1 taken by the AGV to complete the first stage task and the preset time T1 taken by the first stage; S4.2. If the difference between the actual time t1 taken by the AGV to complete the first-stage task and the preset time T1 taken by the first stage is less than the set threshold, the AGV enters the second-stage task execution according to the original scheduling plan; if the difference between the actual time t1 taken by the AGV to complete the first-stage task and the preset time T1 taken by the first stage is not less than the set threshold, return to S3, recalculate the time constraint T2, obtain all new suitable scheduling plans according to the A-DDQN algorithm, and determine the best scheduling plan for the next stage. The AGV enters the second-stage task execution according to the new best scheduling plan; S4.3. The AGV enters the second stage of execution according to the optimal scheduling plan, records the actual time t2 taken by the AGV to complete the second stage task, and calculates the error between the actual time t2 taken by the AGV to complete the second stage task and the preset time T2 taken by the second stage; S4.4. If the difference between the actual time t1 taken by the AGV to complete the second-stage task and the preset time T2 taken by the second stage is less than the set threshold, the AGV enters the execution of the third-stage task according to the original scheduling plan; if the difference between the actual time t2 taken by the AGV to complete the second-stage task and the preset time T2 taken by the first stage is not less than the set threshold, return to S3, recalculate the time constraint T3, obtain all new suitable scheduling plans according to the A-DDQN algorithm, and determine the best scheduling plan for the next stage. The AGV enters the execution of the third-stage task according to the new best scheduling plan; S4.5.The AGV completes all tasks, checks all machines and derives the time constraint for the next task in the same way as S3 for use in the next task execution.

[0031] Effect example: According to the mathematical model and algorithm analysis in the embodiment, this effect example is set to test the feasibility and performance of the multi-task and multi-constraint cross-workshop AGV real-time scheduling method of the improved A-DDQN algorithm.

[0032] In the multi-workshop environment of a large manufacturing enterprise Q, the AGV scheduling system needs to handle multiple tasks while coordinating logistics between workshops, managing the priorities of different tasks, and dynamically changing environmental factors. Enterprise Q includes 8 workshops, involving the task of manufacturing 2 products. The endurance of each AGV is 10 hours. The goal of the scheduling system is to maximize the task efficiency and utilization of the AGV on the basis of meeting all operational constraints; the data in this effect example comes from a workshop in a large manufacturing enterprise. The material and delivery task data involved in the experiment are shown in Table 1, and the task scheduling requirements are shown in Table 2.

[0033] Table 1 Material and delivery task data

[0034] Table 2 Task scheduling requirements

[0035] The efficiency comparison before and after optimization using the embodiment method is shown in Table 3. Table 3 Comparison of workshop efficiency before and after optimization

[0036] As can be seen from Table 3, the multi-task multi-constraint cross-workshop AGV real-time scheduling method implemented using the improved A-DDQN algorithm and the artificial potential field method (APF) significantly improves the workshop operation efficiency. This efficiency improvement is reflected in several key aspects: First, by optimizing the task scheduling of AGV, the waiting time of machines and personnel is reduced, so that the production line can operate more smoothly, and the downtime caused by the AGV car not delivering materials in time is reduced. Secondly, the improvement in efficiency means that the workshop can handle more production tasks in the same time, which directly increases the output capacity of the entire workshop and enables the enterprise to meet higher market demand. Finally, the operation route and load of each AGV car are optimized to ensure that resources are fully utilized, which not only reduces energy consumption, but also reduces operating costs, because the efficiency of the AGV car is improved, and the invalid operation and empty load rate are significantly reduced. The implementation of this strategy has brought significant economic and social benefits to the enterprise, and also improved the ability of the enterprise to adapt to market changes.

[0037] The present invention is described in detail above in conjunction with the embodiments, but the contents described are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of application of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A workshop AGV real-time scheduling method based on the A-DDQN improved algorithm, characterized by: It includes the following steps: S1. Divide the transportation operation into loading stage, transportation stage and unloading stage, extract the information of materials to be loaded in each workshop and the operation information of AGV trolleys and establish a model, build material constraints based on the information of materials to be loaded, and generate time constraints based on the time required for each stage and the operation information of AGV trolleys; S2. The artificial potential field reward function is introduced to improve the Double-DQN algorithm to form the A-DDQN algorithm; S3. Solve the model of each stage according to the A-DDQN algorithm to obtain a suitable scheduling plan, and determine the best scheduling plan based on the AGV utilization rate, expected completion time and conflict risk; S4. Run the AGV car according to the optimal scheduling plan. Every time the AGV car completes a stage, compare the actual running time of the stage with the estimated completion time. If the difference is not less than the set threshold, return to S3 to re-determine the optimal scheduling plan for the next stage. The AGV car runs the next stage according to the new optimal scheduling plan.

2. The workshop AGV real-time scheduling method based on the A-DDQN improved algorithm according to claim 1 is characterized by: The S1 constructs the material constraint based on the information of the material to be loaded, specifically: based on the material type, delivery address and delivery quantity of the material to be loaded, and taking into account the material consumption speed and priority, the material constraint is generated. The expression of the material constraint is: （2）， Among them, C represents the material constraint function, k represents the material type, d represents the delivery address, q represents the delivery quantity, v represents the material consumption speed, p represents the material priority, w1, w2 and w3 represent weights respectively, and w1+w2+w3=1, f(p) represents the function used to calculate the impact of the material priority on the material constraint based on the material priority p, g(d) represents the function used to calculate the impact based on the delivery address, and the material type k is an implicit factor that will affect the value of the material consumption speed v, the material priority p and the delivery quantity q.

3. The workshop AGV real-time scheduling method based on the A-DDQN improved algorithm according to claim 1 is characterized in that: The expression of the S1 generation time constraint is: （1）， Among them, T total represents the time constraint, T loadl represents the time required for the loading phase, T trans represents the time required for the transportation phase, T unload Indicates the time required for the unloading phase.

4. The workshop AGV real-time scheduling method based on the A-DDQN improved algorithm according to claim 1 is characterized in that: The Double-DQN algorithm in S2 is performed using two independent neural networks, one of which is used to determine the best action to be taken by the AGV in a given state, and the other is used to update the Q value of the best action. The Bellman equation for updating the Q value is defined as: （5）， in, represents the Bellman equation of Double-DQN at time t, R t represents the immediate reward at time t, S t+1 represents the state of the car at time t+1, θ represents the weight parameter of the current online neural network, represents the weight parameter of the target neural network, γ represents the discount factor, α represents the learning rate, ranging from 0 to 1, and Q represents the Q value, which is used to measure the total future reward expected by the car after taking a certain action; The Double-DQN algorithm gradually approaches the optimal scheduling plan by calculating the sum of the reward at the current moment and the maximum Q value of the next state.

5. The workshop AGV real-time scheduling method based on the A-DDQN improved algorithm according to claim 4 is characterized in that: The expression of the artificial potential field reward function introduced by S2 is: （6）， Among them, R represents the reward function, s represents the state of the AGV, a represents the best action of the AGV, and U attr represents gravity, U rep Indicates repulsion; The calculation formulas of the attraction and repulsion are: （7）， （8）， Among them, r represents the Euclidean distance between the AGV and the target point, k attr represents the gravitational coefficient, k rep represents the repulsion coefficient, d represents the distance from the current point to the nearest obstacle of the AGV, and D represents the threshold value of the repulsion force generated by the obstacle.

6. The workshop AGV real-time scheduling method based on the A-DDQN improved algorithm according to claim 1 is characterized by: Determine the best scheduling plan based on the utilization rate of AGVs, the estimated completion time and the risk of conflicts. Specifically: calculate a comprehensive score based on the utilization rate of AGVs, the estimated completion time and the risk of conflicts, and determine the best scheduling plan based on the comprehensive score; The calculation formula of the utilization rate is: （9）， Among them, ε represents the utilization rate, T work Indicates the actual working time of the AGV in the task, T total Indicates the total working time of the trolley, including waiting time or idle time; The calculation formula for the estimated completion time is: （10）， Among them, T n estimated Indicates the estimated completion time of the corresponding stage, T n load represents the loading time of the corresponding stage, T n transport represents the transportation time of the corresponding stage, T n unload Indicates the unloading time of the corresponding stage; The calculation formula of the conflict risk is: （11）， Among them, risk represents the risk of conflict, d i represents the minimum distance between the AGV and the i-th obstacle or other vehicles, P i represents the probability of conflict with the ith obstacle or vehicle, and n represents the total number of potential conflicts; The calculation formula of the comprehensive score is: （12）， Among them, S tota represents the comprehensive score, T n max Indicates the maximum allowed value of the estimated completion time of the corresponding stage. ω1, ω2 and ω3 represent weights respectively, and ω1+ω2+ω3=1.

7. The workshop AGV real-time scheduling method based on the A-DDQN improved algorithm according to claim 1 is characterized by: The S1 also constructs a grid model, specifically dividing the factory into multiple grid units and constructing discrete workshops. Each grid unit represents the driving state of the AGV in the area. The position of the grid unit is represented by row and column indexes. The formula for calculating the grid unit index of a point is: （3）， （4）， Among them, I represents the row index, J represents the column index, x and y represent the row coordinates and column coordinates of a point respectively, and x min Indicates the minimum row coordinate of the grid, y max Indicates the maximum column coordinate of the raster.

8. The workshop AGV real-time scheduling method based on the A-DDQN improved algorithm according to claim 7 is characterized in that: The grid unit is a rectangular grid unit with a side length of 20 cm.

Images