Flexible job shop machine and AGV combined scheduling method, system and device

Abstract

The invention provides a flexible job shop machine and AGV combined scheduling method, system and device, and the method comprises the steps: carrying out the training of a Q-FOX through a certain number of iterations, obtaining an optimal hyper-parameter, and fixing the value of the hyper-parameter; solving the joint scheduling problem of the flexible job shop machine and the AGV through the value of the hyper-parameter; and constructing a decision process model capable of dynamically adjusting parameters, and searching an optimal solution with an optimal adaptive value from the whole population based on a population fitness function so as to automatically adjust the joint scheduling parameters of the flexible job shop machine and the AGV. According to the method disclosed by the invention, by utilizing the hyper-parameter automatic tuning function of the Q-FOX algorithm, the convergence speed of the algorithm can be improved, the solving time is shortened, and the robustness of the reinforcement learning algorithm is improved.

Description

Technical Field

[0001] The present invention relates to the field of flexible workshop scheduling, and in particular to a method, system and device for jointly scheduling machines and AGVs in a flexible workshop. Background Art

[0002] With the rapid development of logistics systems, the application of automated guided vehicles (AGVs) has gradually expanded to multiple fields, such as industry, military, transportation and electronics. AGVs have excellent environmental adaptability, strong anti-interference ability and excellent target recognition capabilities. Entering the 21st century, manufacturing companies are facing high-frequency and unpredictable market changes brought about by global competition. In order to ensure timely response to changes in market demand, companies must achieve efficient production. In this context, intelligent manufacturing systems have become the key to achieving these goals, and the need to improve the level of automation in industrial production has become increasingly prominent.

[0003] Intelligent manufacturing has become an important way to transform and upgrade the manufacturing industry. Intelligent decision-making represented by workshop scheduling is a key technology for enterprises to achieve intelligent manufacturing. The flexible job shop scheduling problem (FJSP) is a classic type of workshop scheduling, which is widely used in many discrete manufacturing industries, such as electronics and automobiles. With the development and application of a new generation of artificial intelligence technology, unmanned workshops have become an important trend in the future development of the manufacturing industry. Automated guided vehicles (AGVs) are widely used in many manufacturing industries for their high efficiency, stability and flexibility.

[0004] As an important part of the production system, AGV-based logistics scheduling is also one of the key ways to improve production efficiency. However, the existing research on production scheduling and logistics scheduling is independent of each other. Shop scheduling includes transportation time into processing time or assumes that transportation resources are unlimited. Logistics scheduling focuses more on the task allocation and route planning of AGV. Obviously, both ignore the interaction and interaction between production and logistics. In contrast, productivity can be improved by integrating the scheduling modeling of the two. Therefore, it is worthwhile to study the Flexible Job-Shop Scheduling Problem With Limited Automated Guided Vehicles (FJSPLA).

[0005] Reinforcement learning (RL) is a subset of artificial intelligence (AI) in which agents learn optimal actions by interacting with the environment, making it applicable to tasks that do not require labeled data or direct supervision. Hyperparameter (HP) tuning refers to selecting the best parameters that lead to the optimal solution in the RL algorithm. Therefore, we propose a flexible job shop machine-AGV joint scheduling method based on the Q-FOX algorithm. This method aims to accelerate the convergence of the reinforcement learning algorithm by automatically adjusting the hyperparameters of the reinforcement learning algorithm through the Q-FOX algorithm. It is used to solve the FJSPLA problem to achieve higher production efficiency, lower energy consumption and shorter production cycle. With this method, we can provide strong technical support for the future development of the manufacturing industry and promote the manufacturing industry to develop in a more intelligent and automated direction. For the flexible job shop scheduling problem (FJSPLA) with limited AGVs, commonly used heuristic algorithms include genetic algorithms (GA), ant colony optimization algorithms (ACO), particle swarm optimization algorithms (PSO), etc. These methods search the solution space by simulating the evolutionary process or group behavior in nature, and are suitable for scheduling problems with high dimensions and complex constraints. In recent years, reinforcement learning (RL) has also been gradually introduced into workshop scheduling, especially in adaptive scheduling tasks in dynamic environments. Reinforcement learning can learn scheduling strategies through interaction with the environment and has strong adaptability.

[0006] However, reinforcement learning still has an obvious disadvantage when solving flexible workshop scheduling problems, that is, a large number of parameters need to be manually adjusted. For example, the learning rate, discount factor, exploration and utilization balance parameters in reinforcement learning, etc. The selection of these hyperparameters has an important impact on the convergence speed and performance of the model. Manual parameter adjustment not only increases the complexity of the algorithm, but also may lead to instability in the optimization results, because different parameter combinations may lead to different learning effects. In practical applications, due to the uncertainty and complexity of the production scheduling environment, how to automatically adjust these parameters to improve the effect of reinforcement learning algorithms is still a problem to be solved. This makes the popularization of reinforcement learning face certain challenges, especially in production environments that need to adapt to changes efficiently and quickly.

[0007] Therefore, a flexible job shop machine and AGV joint scheduling method that can improve the algorithm convergence speed, shorten the solution time and improve the robustness of the reinforcement learning algorithm is particularly needed. Summary of the invention

[0008] The purpose of the present disclosure is to provide a method, system and device for joint scheduling of machines and AGVs in a flexible job shop, so as to at least solve one technical problem in the prior art.

[0009] The technical solution disclosed in the present invention is:

[0010] A method for jointly scheduling machines and AGVs in a flexible job shop, comprising:

[0011] After training Q-FOX for a certain number of iterations and obtaining the optimal hyperparameters, the values ​​of the hyperparameters are fixed;

[0012] Solving the problem of joint scheduling of machines and AGVs in a flexible job shop by using the value of the hyperparameter;

[0013] A decision-making process model with dynamically adjustable parameters is constructed. Based on the population fitness function, the optimal solution with the best fitness value is found from the entire population to automatically adjust the joint scheduling parameters of the flexible job shop machines and AGVs.

[0014] The training of Q-FOX through a certain number of iterations includes:

[0015] The distance between the red fox and its prey is obtained through the sound propagation time and the sound propagation speed;

[0016] The red fox updates its position by jumping, and in the exploration phase, it moves randomly with the current best position as the center;

[0017] The fitness function is used to give more weight to the average reward by doubling the average reward and subtracting the average training error, so as to maximize the reward in the learning process and obtain the optimal hyperparameters.

[0018] Initialize the FOX population, and store the position information of each FOX individual in the matrix X. In the process of approaching the target, the optimal solution is sought by iteratively updating the position;

[0019] The sound speed S is calculated based on the current optimal position of the population. The variable T represents the time from the detection sound wave being sent to the receiving sound wave. The variable it represents the number of iterations.

[0020] Sit=BestPos_it / Tit;_ represents the best position of the generation;

[0021] Indicates the time from the sending to the receiving of the best position detection sound wave of the first generation;

[0022] The distance between FOX and prey is obtained by the sound propagation time and sound propagation speed;

[0023] Dist_Pit = Sit × Tit × 0.5;

[0024] The red fox needs to jump to get close to the prey, which represents the time average of the sound propagation;

[0025] jump_hit = 0.5 × 9.81 × t 2 ;

[0026] t is calculated as follows:

[0027] According to FOX's treaty direction, there are:

[0028]

[0029] +1 indicates the position of the next generation; 1 and 2 are constants

[0030] In the exploration phase, the red fox moves randomly around the current best position Posbest, and:

[0031] Tmin=Min(Tit);

[0032] Finally, FOX starts to move randomly after the maximum number of iterations Maxit:

[0033] Pos it+1 =Posbest+rand(1,dim)×T min ×α.

[0034] After training Q-FOX for a certain number of iterations to obtain the optimal hyperparameters, the values ​​of the hyperparameters are fixed, including:

[0035] The importance of any objective is weighed through a multi-objective fitness function to ensure that Q-FOX can find the optimal solution and obtain the optimal hyperparameters;

[0036] The objectives of the multi-objective fitness function trade-off include: generating one or more of a process scheduling sequence, a machine selection sequence, and an AGV selection sequence.

[0037] The decision-making process model with dynamic parameter adjustment is constructed, based on the population fitness function, to find the best solution with the best fitness value from the entire population to automatically adjust the joint scheduling parameters of the flexible workshop machine and AGV, including:

[0038] Design a population evaluation function and perform normalization, and use the population state value as the state set of the Q learning model;

[0039] According to the different states of the population, set at least 2 groups of P c and P m , P c and P m Adjust and update actions taken as an agent;

[0040] The reward is designed by evaluating the best fitness of any agent and the average fitness of the group. t The best individual of the generation is better than the (t-1) generation, the agent will get the current P cis an efficient reward; if the average fitness of the tth generation is better than that of the (t-1)th generation, the agent will get the current P m Effective rewards.

[0041] The population evaluation function is designed and normalized, and the population state value is used as the state set of the Q learning model, including:

[0042] The action selection formula is as follows: Q(s t , a t )←(1-α)Q(s t , a t )++α(r t+1 +γmaxQ(s t+1 , a t+1 ));

[0043] Among them, Q(s t , a t ) is the current state s t When taking a t The Q value of the action; α represents the learning rate; r t+1 is in state s t The reward value after taking the action; γ is the discount factor; maxQ(s t+1 , a t+1 ) means s t+1 Execute a in the state t+1 The maximum value in the Q value table during action, and Q(s t+1 , a t+1 ) means s t+1 Execute a in the state t+1 The expected value of the Q value when taking an action.

[0044] The Q-learning model is:

[0045] S * =w1f * +w2d * +w3m * .

[0046] A flexible job shop machine and AGV joint scheduling system based on the flexible job shop machine and AGV joint scheduling method as described above comprises:

[0047] The first module is used to train Q-FOX through a certain number of iterations, and after obtaining the optimal hyperparameters, the values ​​of the hyperparameters are fixed;

[0048] A second module, which interacts with the first module (100) for solving the problem of joint scheduling of machines and AGVs in a flexible job shop through the value of the hyperparameter;

[0049] The third module interacts with the second module for data construction and is used to build a decision-making process model with dynamically adjustable parameters. Based on the population fitness function, it searches for the best solution with the best fitness value from the entire population to automatically adjust the joint scheduling parameters of the flexible job shop machines and AGVs.

[0050] An electronic device for joint dispatching of a machine and an AGV in a flexible workshop, comprising:

[0051] Storage medium for storing computer programs;

[0052] A processing unit exchanges data with the storage medium, and is used to execute the computer program through the processing unit when generating vehicle trajectory data, and perform the steps of the flexible job shop machine and AGV joint scheduling method as described in claims 1-7.

[0053] A computer readable storage medium:

[0054] The computer readable storage medium stores a computer program;

[0055] When the computer program is running, it executes the steps of the flexible job shop machine and AGV joint scheduling method as described above.

[0056] The beneficial effects of the present disclosure include at least:

[0057] The method disclosed in the present invention is based on the Q-FOX algorithm. After training Q-FOX for a certain number of iterations and obtaining the optimal hyperparameters, the values ​​of the hyperparameters are fixed to solve the joint scheduling problem of the machine and AGV in the flexible workshop. The Q-learning algorithm is used to adaptively adjust the crossover and mutation parameters to improve the search ability of the algorithm; at the same time, a Markov decision process (MDP) model with dynamically adjustable parameters is constructed. In the MDP model, in order to comprehensively evaluate the performance of the population, a state set based on the population fitness function is proposed, and a reward mechanism that effectively reduces the target value is designed; the method disclosed in the present invention utilizes the automatic hyperparameter tuning function of the Q-FOX algorithm to improve the convergence speed of the algorithm, while shortening the solution time and improving the robustness of the reinforcement learning algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of the Q-FOX algorithm described in the present disclosure.

[0059] Figure 2 is a flow chart of the present disclosure.

[0060] Figure 3 is a block diagram of the system described in the present disclosure. DETAILED DESCRIPTION

[0061] The present disclosure is further described below in conjunction with the accompanying drawings. Specific embodiment 1:

[0063] The present disclosure provides an embodiment:

[0064] In order to solve the problems of the prior art, such as Figure 1 and 2 A method for joint scheduling of machines and AGVs in a flexible job shop comprises: training Q-FOX through a certain number of iterations to obtain the optimal hyperparameters and then fixing the values ​​of the hyperparameters; solving the problem of joint scheduling of machines and AGVs in the flexible job shop through the values ​​of the hyperparameters; constructing a decision process model with dynamically adjustable parameters, and based on the population fitness function, searching for the best solution with the best fitness value from the entire population to automatically adjust the joint scheduling parameters of machines and AGVs in the flexible job shop.

[0065] Specifically, in this embodiment, the scheduling method is introduced in detail through the following contents:

[0066] Step 1: Encoding and decoding definition:

[0067] 1) Coding method: FJSP-T can be divided into four sub-problems: 1. Machine allocation; 2. Machine scheduling; 3. AGV allocation; 4. AGV scheduling. This problem is usually represented by three vectors. In order to achieve the crossover mutation of the three vectors and reduce the complexity of generating sub-populations, this study adopts a three-layer independent coding method to generate the operation scheduling sequence (OSS), machine selection sequence (MSS) and AGV selection sequence (ASS).

[0068] 2) Decoding method: Considering that the random selection method of sorting code will produce a certain number of invalid solutions, it is impossible to meet the coupling constraints between the movement of AGV equipment and the processing procedures, resulting in a decrease in population quality. Therefore, a heuristic decoding method is adopted to arrange the earliest transport AGV and the earliest completed machine according to each procedure of the initialized individual, thereby improving the population quality and reducing the generation of invalid solutions.

[0069] Step 2: Design action space, state space and reward function:

[0070] Status: This study intends to design a population evaluation function to judge the current status of the population. Taking into account the average fitness of the population, population diversity and the fitness of the best individual, a population evaluation function is designed and normalized, and the population status value is used as the state set of Q-Learning. The formula is as follows:

[0071] S * =w1f * +w2d * +w3m * :

[0072] Action: According to the different states of the population, this study intends to design multiple groups of P c and P m , the P c and P m Adjust and update the actions taken by the agent. The action selection formula is as follows:

[0073] Q(s t , a t )←(1-α)Q(s t , a t )+α(r t+1 +γmaxQ(s t+1 , a t+1 ));

[0074] In the action selection formula, Q(s t , a t ) is the current state s t When taking a t The Q value of the action, α represents the learning rate; r t+1 is in state s t The reward value after taking the action; γ is the discount factor. maxQ(s t+1 , a t+1 ) means s t+1 Execute a in the state t+1 The maximum value in the Q value table during action, and Q(s t+1 , a t+1 ) means s t+1 Execute a in the state t+1 The expected value of the Q value when taking an action.

[0075] Reward: This study intends to design rewards by evaluating the best fitness of individuals and the average fitness of the group. If the best individual in the tth generation is better than the (t-1)th generation, the agent will get the current P c is an efficient reward. If the average fitness of the tth generation is better than that of the (t-1)th generation, the agent will get the current P m Effective rewards.

[0076] Step 3: Introduction to FOX algorithm:

[0077] In the FOX algorithm, the red fox locates prey through ultrasound and uses the propagation time of sound to determine the distance from the prey, constantly adjusting the distance from the prey and finally capturing the prey. FOX algorithm process: First, the FOX population is initialized, and the position information of each individual is stored in the matrix X. In the process of approaching the target, the optimal solution is sought by iteratively updating the position, and the exploration and development strategies each account for 50%.

[0078] The sound speed S is calculated according to the current optimal position of the population. In addition, the variable T represents the time from the detection sound wave is sent to the reception, and the variable it represents the number of iterations.

[0079] Sit = BestPos_it / Tit;

[0080] The distance between the red fox and its prey is obtained by using the sound propagation time and sound propagation speed

[0081] Dist_Pit = Sit × Tit × 0.5;

[0082] The red fox needs to jump to get close to its prey, and at this time it needs to reach a suitable jumping height, where 9.81 represents the acceleration of gravity, and t represents the time average of sound propagation.

[0083] jump_hit = 0.5 × 9.81 × t;

[0084] t is calculated as follows:

[0085]

[0086] The red fox updates its position by jumping. When p>0.18, it jumps to the northeast, multiplied by the probability of successful hunting at this time c1. Otherwise, it jumps to the southwest, multiplied by the probability c2, where c1 and c2 are 0.82 and 0.18 respectively.

[0087]

[0088] In the exploration phase, the red fox moves randomly with the current best position Posbest as the center. To ensure that the random optimization in this phase can tend to a better solution, the algorithm first determines two parameters:

[0089] Tmain=Min(Tit);

[0090] Among them, Maxit is the maximum number of iterations, after which random movement begins:

[0091] Pos it+1 =Posbest+rand(1, dim)×T min ×a;

[0092] Step 3: Design the Q-FOX algorithm and obtain the optimal hyperparameters through the Q-FOX algorithm.

[0093] The Q-learning algorithm has several hyperparameters HP that affect the performance and behavior of the agent on the learned policy. The main hyperparameters are: step size (α), discount factor (γ), and epsilon (ε). All of these are variables and must be initialized before the learning phase begins. The α parameter is crucial for updating the action-value function at each time step. It determines the extent to which new information overwrites old information and its value is usually in the range [0, 1]. A lower α means that the algorithm is less sensitive to new information and takes longer to converge, while a higher α results in faster updates but may cause instability. Therefore, choosing the right α value is crucial for the best performance of any learning algorithm. The γ parameter affects the future rewards of the agent's decision and its value range is [0,1]. A larger value indicates that the agent emphasizes future rewards, while a smaller value means that the agent does not attach much importance to future rewards. This parameter controls the balance between current and future rewards, affecting the agent's decision-making process. The ε parameter balances exploration and exploitation when choosing an action. Its value range is [0,1], with higher values ​​resulting in more exploitation and lower values ​​resulting in more exploration. This parameter is crucial in determining the trade-off between exploration and exploitation and directly affects the agent's choice of action.

[0094] The development of a multi-objective fitness function is necessary to address the multiple goals of increasing reward and reducing error with minimal convergence time during training. This function needs to encapsulate the simultaneous optimization of these three objectives in a single fitness function. Therefore, the fitness function is as follows:

[0095]

[0096] This function is designed to give more weight to the average reward by doubling it and subtracting the average training error. This function is used on the last quarter of all episodes to ensure the stability of the learning process, so i = episodes-episodes / 4. For example, if learning episode n = 1000, start calculating the fitness function values ​​from episodes 750 to n. It should be noted that the value of T in this function represents the time it takes FOX to reach the best solution. The difficulty in using FOX with Q learning to obtain the best HP is that you need to think carefully about how to weigh the importance of each goal. This approach enables FOX to effectively seek solutions that best maximize the fitness function equation and maximize the rewards during the learning process.

[0097] The scheduling method described in this embodiment constructs a reinforcement learning environment based on the mathematical model of discrete manufacturing workshop scheduling, including the state space of workpieces, machines and carts, the scheduling decision action space and reward function, and uses the Q-FOX algorithm to find a high-quality solution to the flexible job shop scheduling problem (FJSPLA) with limited AGVs.

[0098] Specifically, the method for joint scheduling of flexible workshop machines and AGVs described in this embodiment provides a more automated and potentially more effective solution, which can significantly improve the algorithm convergence speed: the Q-FOX algorithm is used, and the FOX algorithm is combined to automatically tune the hyperparameters of the Q-Learning algorithm, which is more convenient than the traditional manual adjustment of parameters and can solve the challenges of manual adjustment; at the same time, the joint scheduling method can also significantly improve the quality of the solution: FOX is a new optimization algorithm inspired by the hunting behavior of red foxes, which aims to find the best solution with the best fitness value from the entire population. Through the efficient optimization capability of the FOX algorithm, the method described in this embodiment can quickly train effective hyperparameters and solve a solution set with high-quality characteristics. This means that in the problem of joint scheduling of flexible workshop machines and AGVs, a more balanced and excellent solution can be found using the joint scheduling method described in this embodiment.

[0099] In summary, this patent has achieved not only a breakthrough in solution efficiency through innovative technical means, but also a significant improvement in solution quality, providing an efficient and high-quality solution to the machine-AGV joint scheduling problem in flexible job shops. Specific embodiment 2:

[0101] The present disclosure also provides an embodiment:

[0102] like Figure 3 A flexible job shop machine and AGV joint scheduling system based on the flexible job shop machine and AGV joint scheduling method as described in specific embodiment 1 includes: a first module 100, a second module 200 and a third module 300; wherein the first module 100 is used to train Q-FOX through a certain number of iterations, and after obtaining the optimal hyperparameter, the value of the hyperparameter is fixed; the second module 200 interacts with the first module 100 for data, and is used to solve the flexible job shop machine and AGV joint scheduling problem through the value of the hyperparameter; the third module 300 interacts with the second module 200 for data, and is used to construct a decision process model with dynamically adjustable parameters, and based on the population fitness function, finds the best solution with the best fitness value from the entire population to automatically adjust the flexible job shop machine and AGV joint scheduling parameters. Specific embodiment 3:

[0104] The present disclosure also provides an embodiment:

[0105] An electronic device comprises: a storage medium and a processing unit; wherein the storage medium is used to store a computer program, and the processing unit exchanges data with the storage medium, and is used to execute the computer program through the processing unit when generating vehicle trajectory data, and perform the steps of the flexible job shop machine and AGV joint scheduling method as described in specific embodiment I. Specific embodiment 4:

[0107] A computer-readable storage medium stores a computer program; when the computer program is run, the computer program executes the steps of the flexible job shop machine and AGV joint scheduling method as described in the specific embodiment.

[0108] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, etc., or any suitable combination of the above.

[0109] The above disclosures are only several specific implementation scenarios of the present disclosure, but the present disclosure is not limited thereto, and any changes that can be thought of by a person skilled in the art should fall within the protection scope of the present disclosure. The above disclosure serial numbers are only for description and do not represent the advantages and disadvantages of the implementation scenarios.

Claims

1. A flexible job shop machine and AGV joint scheduling method based on the Q-FOX algorithm, characterized in that: include: After training Q-FOX for a certain number of iterations and obtaining the optimal hyperparameters, the values ​​of the hyperparameters are fixed; Solving the problem of joint scheduling of machines and AGVs in a flexible job shop by using the value of the hyperparameter; A decision-making process model with dynamically adjustable parameters is constructed. Based on the population fitness function, the optimal solution with the best fitness value is found from the entire population to automatically adjust the joint scheduling parameters of the flexible job shop machines and AGVs.

2. The method for jointly scheduling machines and AGVs in a flexible job shop according to claim 1, characterized in that: The training of Q-FOX through a certain number of iterations includes: The distance between the red fox and its prey is obtained through the sound propagation time and the sound propagation speed; The red fox updates its position by jumping, and in the exploration phase, it moves randomly around the current best position; The fitness function is used to give more weight to the average reward by doubling the average reward and subtracting the average training error, so as to maximize the reward in the learning process and obtain the optimal hyperparameters.

3. The method for jointly scheduling machines and AGVs in a flexible job shop according to claim 2, characterized in that: Initialize the FOX population, and store the position information of each FOX individual in the matrix X. In the process of approaching the target, the optimal solution is sought by iteratively updating the position; The sound speed S is calculated based on the current optimal position of the population. The variable T represents the time from the detection sound wave is sent to the reception. The variable i t Indicates the number of iterations; Yes t =BestPos_i t / Ti t ; BestPos_i t Indicates the i t The best position for the generation; Ti t Indicates the i t The time from the sending to the receiving of the sound wave for the best position detection; The distance between FOX and prey is obtained by the sound propagation time and sound propagation speed; Dist_Pi t =Yes t ×Ti t ×0.5; Red foxes need to jump to get close to their prey. t Represents the time average of sound propagation; jump_hi t =0.5×9.81×t 2 ; t is calculated as follows: According to FOX's contract direction, there are: Posi t +1 indicates the position of the next generation; The new position is obtained according to one of the following two equations, where c1 and c2 are 0.18 and 0.82 respectively; In the exploration phase, the red fox moves randomly around the current best position Posbest, and: Tmin=Min(Ti t ); Finally, FOX reaches the maximum number of iterations Maxi t Then start random movement: Mail it+1 =Posbest+rand(1,dim)×T min ×a.

4. The method for jointly scheduling machines and AGVs in a flexible job shop according to claim 1, characterized in that: After training Q-FOX for a certain number of iterations to obtain the optimal hyperparameters, the values ​​of the hyperparameters are fixed, including: The importance of any objective is weighed through a multi-objective fitness function to ensure that Q-FOX can find the optimal solution and obtain the optimal hyperparameters; The objectives of the multi-objective fitness function trade-off include: generating one or more of a process scheduling sequence, a machine selection sequence, and an AGV selection sequence.

5. The method for joint scheduling of machines and AGVs in a flexible job shop according to claim 1, characterized in that: The decision-making process model with dynamic parameter adjustment is constructed, based on the population fitness function, to find the best solution with the best fitness value from the entire population to automatically adjust the joint scheduling parameters of the flexible workshop machine and AGV, including: Design a population evaluation function and perform normalization, and use the population state value as the state set of the Q learning model; According to the different states of the population, set at least 2 groups of P c and P m , P c and P m Adjust and update the actions taken by the agent; Pc is the genetic algorithm crossover probability, and Pm is the genetic algorithm mutation probability; The reward is designed by evaluating the best fitness of any agent and the average fitness of the group. t The best individual of the generation is better than the (t-1) generation, the agent will get the current P c is an efficient reward; if the average fitness of the tth generation is better than that of the (t-1)th generation, the agent will get the current P m Effective rewards.

6. The method for jointly scheduling machines and AGVs in a flexible job shop according to claim 4, characterized in that: The population evaluation function is designed and normalized, and the population state value is used as the state set of the Q learning model, including: The action selection formula is as follows: Q(s t , α t )←(1-α)Q(s t , α t )+α(r t+1 +γmaxQ(s t+1 , α t+1 )); Among them, Q(s t , α t ) is the current state s t When taking α t The Q value of the action; α represents the learning rate; r t+1 is in state s t The reward value after taking the action; γ is the discount factor; maxQ(s t+1 , α t+1 ) means s t+1 Execute in state α t+1 The maximum value in the Q value table during action, and Q(s st+ , α t+1 ) means s t+1 Execute in state α t+1 The expected value of the Q value when taking an action.

7. The method for jointly scheduling machines and AGVs in a flexible job shop according to claim 4, characterized in that: The Q-learning model is: S * =w1f * +w2d * +w3m * : Among them, w1+w2+w3=1, w1=0.2, w2=0.4, w3=0.4; f * : represents the fitness of the best individual in the population; d * : represents the diversity of the population; m * : Represents the average fitness of the population.

8. A flexible job shop machine and AGV joint scheduling system based on the flexible job shop machine and AGV joint scheduling method according to any one of claims 1 to 7, characterized in that: The first module (100) is used to train Q-FOX through a certain number of iterations, and after obtaining the optimal hyperparameters, fix the values ​​of the hyperparameters; A second module (200) performs data interaction with the first module (100) and is used to solve the problem of joint scheduling of machines and AGVs in a flexible job shop by using the value of the hyperparameter; The third module (300) interacts with the second module (200) for data construction and is used to construct a decision-making process model with dynamically adjustable parameters. Based on the population fitness function, the third module (300) searches for the best solution with the best fitness value from the entire population to automatically adjust the joint scheduling parameters of the flexible workshop machine and AGV.

9. An electronic device for joint dispatching of machines and AGVs in a flexible workshop, characterized in that: include: Storage medium for storing computer programs; A processing unit exchanges data with the storage medium, and is used to execute the computer program through the processing unit when generating vehicle trajectory data, and perform the steps of the flexible job shop machine and AGV joint scheduling method as described in claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program; When the computer program is running, it executes the steps of the flexible job shop machine and AGV joint scheduling method as described in claims 1-7.

Images