Dual-resource constraint flexible job shop scheduling method for reducing worker load

Abstract

The invention relates to a dual-resource constraint flexible job shop scheduling method for reducing worker load, which comprises the following steps: S1, construction of a worker load model: dividing the worker load model into four conditions of light work, moderate work, micro-severe work and rest according to daily work arrangement of workers, and setting the maximum working time length for each worker, the calculation module is used for calculating extra workloads; in the scheduling method provided by the invention, three different initialization strategies are combined, and the proportion of the initialization strategies in a population is set, so that the diversity and quality of an initial solution are ensured, specifically, the used initialization strategies comprise random initialization, initialization according to the process completion time and initialization according to the process remaining time; under the random initialization strategy, the initial solution of the population has great diversity, which is helpful for avoiding the trouble of a local optimal solution.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of production scheduling, in particular to a double resource constraint flexible job shop scheduling method for reducing worker load. BACKGROUND

[0002] In the flexible job shop scheduling problem, a process needs to be processed on multiple optional machines. However, in some actual production environments, in addition to machine resources, worker resources must also be considered, that is, some processes not only require machines but also require workers with corresponding skills to complete. Traditional scheduling objectives are mostly to minimize the completion time or improve equipment utilization.

[0003] In the actual implementation process, there are still some problems:

[0004] 1. In the field of flexible job shop scheduling, how to effectively reduce worker load and optimize resource allocation has always been a hot research topic. Traditional shop scheduling methods mostly focus on the time sequence arrangement of processes, and less consider the impact of worker load, which may cause workers to face overloading work in some complex scheduling scenarios, thereby affecting production efficiency and worker health.

[0005] 2. In the shop scheduling, the arrangement of workers and machines is usually affected by many factors such as working time, processing time and task complexity. Traditional scheduling methods mostly use simple insertion decoding to arrange processes, ignoring the changes in worker load and their impact on production efficiency. Since the daily working time of each worker is limited, the arrangement of processes often cannot maximize the use of worker time without considering this factor, resulting in overloading work for workers, affecting production efficiency and even worker health. SUMMARY

[0006] (I) Technical problems to be solved

[0007] In order to solve the above problems of the prior art, the present application provides a double resource constraint flexible job shop scheduling method for reducing worker load, which solves the problems raised in the above background.

[0008] (II) Technical scheme

[0009] In order to achieve the above purpose, the main technical scheme adopted by the present application is:

[0010] A double resource constraint flexible job shop scheduling method for reducing worker load, comprising the following steps:

[0011] S1: Construction of worker load model, the worker load model is divided into four situations of light work, moderate work, mild heavy work and rest according to the daily work arrangement of workers, and the maximum working time for each worker is set to calculate the additional working load;

[0012] S2: Process resource combination selection, the resource combination is composed of machine tools and workers, machine tools include CNC milling machine, CNC lathe, machining center and drilling machine, workers include machine tool operators and maintenance workers, the time required for each process is not more than the maximum working time of the worker on the same day;

[0013] S3: Process scheduling sequence development, under the condition of meeting the worker load constraint, the scheduling rule based on the combination of shortest processing time priority and critical path method is used to determine the process sequence;

[0014] S4: Dynamic scheduling strategy design, the dynamic scheduling strategy is based on Q-learning algorithm and SARSA algorithm, and the scheduling effect is optimized by adaptive adjustment of the probability of crossover operator and mutation operator in evolutionary algorithm;

[0015] S5: Optimization target setting, set the minimum maximum completion time as the optimization target during scheduling, and avoid overloading work by introducing the fitness function based on worker load through genetic algorithm;

[0016] S6: Decoding mode design, when decoding process scheduling, left insertion decoding mode based on the maximum working time of workers is adopted, and the process is inserted into the time period not exceeding the maximum working time of workers per day 8 hours;

[0017] S7: Neighborhood structure optimization, the neighborhood structure is based on the critical path exchange structure, adjacent processes are selected on the critical path to exchange to reduce the bottleneck;

[0018] S8: Scheduling effect verification, use Matlab software to build a flexible job shop model for simulation experiment, and compare with genetic algorithm and tabu search algorithm.

[0019] A three-section code is used to represent the scheduling scheme, which includes:

[0020] The first section is the resource selection layer, which represents the machine and worker combination required for process processing, where machine types include CNC milling machine, CNC lathe, machining center and drilling machine, and workers are identified by number;

[0021] The second section is the process sorting layer, which uses process number arrangement, where the first appearance of the workpiece number represents the first process of the workpiece, and the subsequent ones correspond to the subsequent processes;

[0022] The third section is the work arrangement layer, which follows the worker number order, and every seven digits represent a week's work arrangement, where 0 represents rest, 1 represents light work, 2 represents moderate work, and 3 represents mild heavy work. The code is optimized by crossover and mutation of different layers through Q-learning and SARSA algorithm.

[0023] In the crossover and mutation operations of the genetic algorithm:

[0024] The resource selection layer adopts a multi-point crossover method;

[0025] The process sequencing layer adopts a crossover method based on workpiece position exchange;

[0026] The work arrangement layer divides the seven genes into a section and performs sectional crossover between the crossover points cp and cp;

[0027] The crossover rate Pc is adaptively adjusted between 0.6 and 0.9, the mutation rate Pm is adaptively adjusted between 0.01 and 0.1, and the crossover rate and the mutation rate are selected according to the Q table and the reward value in different evolution generations through the Q-learning and SARSA algorithms.

[0028] The decoding method is a left insertion decoding method based on the working time of workers, which specifically includes:

[0029] When there is a gap between processes, it is judged whether the gap start time, gap end time and process startable time are in the same working day, if the sum of the process processing time and the startable time exceeds the maximum working time of the worker on the same day or exceeds the gap end time, the insertion is not performed;

[0030] If it does not exceed, the process is inserted into the gap time period;

[0031] When there is no available gap, the process is arranged to the next day start time or other workable workers.

[0032] The initialization strategy includes:

[0033] Random initialization is used to improve the diversity of solutions, accounting for 50% of the initial population;

[0034] Initialization according to the earliest completion time of the process makes the initial solution closer to the shortest processing time, accounting for 25% of the initial population;

[0035] Initialization according to the remaining processing time of the process makes the initial solution more consistent with subsequent load balancing, accounting for 25% of the initial population;

[0036] Through the proportional combination of the above three methods, the diversity and quality of the initial population are improved.

[0037] In each scheduling cycle, a neighborhood structure search is adopted, and the neighborhood structure is a key block exchange structure based on the critical path, including:

[0038] When the critical path is the first key block, exchange the two processes at the end of the block;

[0039] When the critical path is the last key block, exchange the two processes at the beginning of the block;

[0040] When the critical path is in the middle position, the process of exchanging the head and tail of the critical block, or the process of exchanging the first two and the last two when the critical block length is greater than is performed;

[0041] And if a better scheduling solution is obtained after the exchange, the scheduling result is accepted.

[0042] In the process of sequence sorting, an adaptive neighborhood search strategy is adopted, and the neighborhood search includes:

[0043] A position-based crossover operator is used to randomly select the position of the workpiece for exchange;

[0044] A swap-based mutation operator is used to randomly exchange the positions of two processes;

[0045] And the selection probability of the crossover operator and the mutation operator is dynamically adjusted through the Q-learning and SARSA algorithms to maintain the diversity of the population in the early stage of the algorithm and guide the population to converge to a better solution in the later stage.

[0046] A 10*10 scale flexible job shop instance is constructed on the Matlab platform to perform simulation experiments, and the inverse generation distance IGD and the solution set coverage SC are used as performance evaluation indexes, and the method is compared with the standard genetic algorithm and the tabu search algorithm.

[0047] (Three) beneficial effects

[0048] The beneficial effects of the present application are:

[0049] 1. In this invention, the proposed scheduling method combines three different initialization strategies and sets the proportion of each strategy in the population, thereby ensuring the diversity and quality of the initial solutions. Specifically, the initialization strategies used include random initialization, initialization based on process completion time, and initialization based on the remaining time of the process. Under the random initialization strategy, the initial solutions of the population have greater diversity, which helps to avoid the problem of local optima. Initialization based on process completion time can reduce time waste during scheduling to a certain extent, making process arrangement more efficient. The initialization strategy based on the remaining time of the process can reduce resource waste and improve the balance of worker workload during later optimization. Combining initialization strategies can enhance the diversity of the initial population, thereby improving the search efficiency of the algorithm. This diversified initialization strategy can effectively avoid the insufficient global search capability caused by the low quality of the initial solution in traditional algorithms, thus improving the optimization effect of the entire scheduling process. In addition, this method introduces a dynamic adjustment mechanism of reinforcement learning, which enables the crossover rate and mutation rate to change adaptively during the iteration process. The introduction of this mechanism greatly enhances the flexibility and stability of the search. During the optimization process, the algorithm can maintain a high diversity in the early stage according to the current state of the population, and tend to focus more on the convergence of the optimal solution in the later stage, effectively avoiding premature convergence in the search process.

[0050] 2. The proposed left-insertion decoding method based on worker working hours effectively considers the maximum working time of workers and the gap time of processes, providing a more realistic and accurate solution to scheduling problems. In traditional decoding methods, process insertion usually ignores the limitations of worker load and available working time. However, the left-insertion decoding method, by analyzing each worker's daily working hours and remaining gap time, can ensure that the arrangement of each process is within the acceptable load range of workers, thereby avoiding worker overload. This decoding method not only improves the rationality of scheduling but also reduces production bottlenecks caused by unbalanced worker loads. In addition, the left-insertion decoding method provides a flexible process insertion strategy for solving process scheduling problems. Within limited working hours and space, it can dynamically adjust the start and end times of processes to maximize production efficiency within the worker's workday and ensure that the worker's workload does not increase excessively. This method makes the scheduling scheme more in line with actual production requirements, thereby improving the overall production efficiency of the workshop and worker job satisfaction. Through experimental data verification, the left-insertion decoding strategy, compared with other traditional scheduling methods, can significantly reduce worker load and optimize process completion time, thereby effectively improving the overall efficiency of workshop production. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the workflow of part of the present invention;

[0052] Figure 2 Part of the algorithm flowchart of the present application;

[0053] Figure 3 Part of the three-stage encoding of the present application;

[0054] Figure 4 DS layer chromosome crossover of the present application;

[0055] Figure 5 IGD value of each factor level of the DOE experiment of the present application;

[0056] Figure 6 IGD value box plot of each algorithm of the present application;

[0057] Figure 7 SC value box plot of the algorithm of the present application;

[0058] Figure 8 Frontier solution comparison of the four algorithms of the present application;

[0059] Figure 9 Convergence curve of the completion time of the WA01 example of the present application;

[0060] Figure 10 Convergence curve of the additional workload of the example of the present application;

[0061] Figure 11 Worker Gantt chart of the WA01 example of the present application;

[0062] Figure 12 Machine Gantt chart of the WA01 example of the present application;

[0063] Figure 13 Additional workload distribution generated by different algorithms of the WA01 example of the present application. DETAILED DESCRIPTION

[0064] In order to better explain the present application, so as to be understood, the present application is described in detail by specific embodiments, combined with the accompanying drawings.

[0065] Please refer to Figures 1 to 13 The worker load double resource constraint flexible job shop scheduling method of the present application includes the following steps:

[0066] S1: Construction of worker load model, the worker load model is divided into four situations of light work, moderate work, mild heavy work and rest according to the daily work arrangement of the worker, and the maximum working time is set for each worker, which is used to calculate the additional workload; It should be noted that the symbol explanation of the construction of the worker load model is shown in Table 1:

[0067] Table 1 Symbol explanation

[0068]

[0069] The optimization objectives herein are to minimize the maximum completion time and minimize the extra worker load.

[0070] (1) Minimizing the maximum completion time:

[0071] In the WWFJSP-DRC, the minimum maximum completion time refers to the end time of the latest completion of processing of all workpieces, which represents the time of processing of a workpiece, and the shorter the minimum maximum completion time, the higher the production efficiency of the workshop, wherein C max is a function as shown in the following formula:

[0072]

[0073] (2) Minimizing the extra worker load:

[0074] The WWFJSP-DRC needs to reduce the extra worker load while maintaining the minimum maximum completion time, wherein the calculation formula of the minimum extra worker load is shown in the following formula:

[0075]

[0076] wherein the constraint condition is shown in the following formula:

[0077]

[0078] indicates that a process can only be processed by one worker and one machine;

[0079]

[0080] indicates that the start time of the j+1th process of the workpiece i must be later than the jth process of the workpiece i;

[0081]

[0082] indicates that the process completion time should be before the maximum working time of the worker on the same day;

[0083]

[0084] provides that the worker is not allowed to operate multiple machines at the same time;

[0085]

[0086] provides that any process of a workpiece can only be processed on one machine. ​

[0087] And, it needs to be noted that: because there is no standard example setting for WWFJSP-DRC, according to the characteristics of the problem, the example WA01-10 conforming to WWFJSP-DRC is randomly generated according to the size of the example, and the related parameter range of the set example is shown in Table 2. In the experimental part, the optimal factor combination is first determined using Taguchi orthogonal experiment, and the factor combination is used for comparison with other algorithms to illustrate the effectiveness of the algorithm used in this paper.

[0088] Table 2: WA01-10 example setting

[0089]

[0090] And, this part will determine the parameters in the algorithm through experiment, learning rate determines the influence of the new value on the numerical value after updating; the discount factor has a decaying effect on the subsequent reward to prevent the agent from falling into a non-optimal crossover rate and mutation rate, and the initial greed factor has an important influence on the strategy of updating the Q table of the agent in the reinforcement learning part, so a three-factor four-level Taguchi orthogonal experiment is designed, and the level of each parameter is set to , and The L(16)4 table formed by DOE experiment is shown in Table 3, and when the experiment is performed, the example WA02 is used, and the average IGD value of the experiment is obtained through ten experiments. The optimal parameter combination obtained through Taguchi orthogonal experiment is: learning rate =0.7, discount factor =0.6, initial greed factor =0.7. The point-line graph of the three parameters and the IGD average is shown in Figure 5 .

[0091] Table 3: DOE experiment

[0092]

[0093] In order to verify the effectiveness of the QSNSGA algorithm proposed in this paper compared with other algorithms, three heuristic algorithms for solving multi-objective optimization are selected, and the selected comparison algorithms are: DQNSGA

[19] , IGA

[20] and MOEAD

[21] . In terms of the strategy of each comparison algorithm, the generation of initial solution, the crossover and mutation mode of chromosome and the decoding mode are set to be the same as the QSNSGA algorithm proposed in this paper; in terms of the parameter setting of each algorithm, the initial population size, iteration number and other general parameters are set to the same value; in order to reflect the effectiveness of the algorithm proposed in this paper compared with other algorithms.

[0094] In the comparative experiments, all four algorithms were calculated using the WA01-10 example, with each example calculated 10 times. The resulting Pareto solution sets were compared to obtain the IGD value for each solution set. Finally, the average of the ten IGD values ​​for each algorithm in each example was taken as the IGD value for that algorithm in that example. The comparison of IGD values ​​for each algorithm after the experiments is shown in Table 4, and the comparison of SC values ​​is shown in Table 5. The IGD value of QSNSGA is significantly higher than that of algorithms MOEAD and IGA, and slightly higher than that of DQNSGA. The numbers representing the best-performing values ​​in the algorithms are bolded. Figure 6 For comparison of IGD values ​​from different algorithms, Figure 7 Box plot comparison of SC values ​​between QSNSGA and other algorithms.

[0095] Table 4: Comparison of IGD Values

[0096]

[0097] Table 5: Comparison of SC values ​​for various algorithms

[0098]

[0099] It should be noted that, Figure 8 The non-dominated solutions to WA01 were calculated using four algorithms for comparison. The figure shows that QSNSGA has a higher quality solution set. To more intuitively compare the effectiveness of each algorithm and make the results more convincing, this paper selects WA01 as a representative example. Figure 9 and Figure 10 The convergence curves of the four algorithms are shown for two objectives. It is clear from the figure that QSNSGA outperforms the other two algorithms in terms of accuracy on both objectives.

[0100] S2: Process resource combination selection. The resource combination consists of machine tools and workers. Machine tools include CNC milling machines, CNC lathes, machining centers and drilling machines. Workers include machine tool operators and maintenance workers. The processing time required for each process shall not exceed the maximum working time of the worker on that day.

[0101] S3: Process scheduling sequence determination. Under the condition of satisfying worker load constraints, the process sequence is determined by a scheduling rule that combines the shortest processing time priority and the critical path method.

[0102] S4: Dynamic scheduling strategy design. The dynamic scheduling strategy is based on the Q-learning algorithm and the SARSA algorithm, and the scheduling effect is optimized by adaptively adjusting the probabilities of the crossover operator and mutation operator in the evolutionary algorithm.

[0103] It needs to be explained: reinforcement learning in machine learning is increasingly widely used in scheduling neighborhood, often used to dynamically change part of the parameters of the algorithm. Because the single genetic algorithm is easy to fall into local optimal solution, so choose Q-learning combined with SARAS algorithm to adaptively adjust the crossover rate Pc and mutation rate Pm of each generation of the algorithm. The dynamic change of crossover rate and mutation rate can make the algorithm focus on population diversity or convergence of optimal solution when selecting, and no longer let the population evolve in a single way.

[0104] In this paper, Q table based on crossover rate Pc and Q table based on mutation rate Pm are used, and the rows and columns of Q table represent the state and action of agent respectively. The update formula of Q table is shown in the following formula:

[0105]

[0106] Among them Learning rate, represents the influence of new value on the updated value;

[0107] Discount factor, this value has a decay effect on the subsequent reward, in order to prevent the agent from falling into the non optimal crossover rate and mutation rate.

[0108] And in reinforcement learning, the information in the state space is processed by the agent and makes decisions, in this paper, the fitness value of two targets is selected to be weighted and reduced to one-dimensional value as the state space.

[0109] The fitness value of the initial generated population and the population fitness value after crossover and mutation operation are converted to one-dimensional numerical value, which needs to get the ratio SM of the distance measurement value of the fitness of the two populations as shown in formula (1); the diversity ratio DV of the two populations as shown in formula (5); the coverage ratio Scr of the two populations as shown in formula (6):

[0110] (1)

[0111] (2)

[0112] (3)

[0113] (4)

[0114] (5)

[0115] (6)

[0116] Among them: represents the population size to be calculated, di(i) represents the minimum distance from each solution in the solution set of the population to other solutions, formula (2) is a population spacing measurement value calculation formula, SMt1 represents the initial population without crossing and mutation, SMt2 represents the population after crossing and mutation, Dim is the dimension of the target, dis(i) represents the Euclidean distance between adjacent solutions B and, the expression is shown in formula (2), di averge is the average distance between solutions, the expression is shown in formula (3), the expression of the diversity measurement DV is shown in formula (4), wherein represents the percentage of solutions in ai dominated by at least one solution in bi, represents the percentage of solutions in bi dominated by at least one solution in ai.

[0117] After obtaining SM, DV and SCr, according to the weight value The final state space information used for processing by the agent is obtained, the functional expression of the value F is shown in formula (7), wherein the weight value After the experiment, it is set to 0.2, 0.4, 0.4.

[0118] (7)

[0119] According to the value of , the state space is divided into 21 states, wherein , the length interval of each state is 0.05, and state 21 is >1.

[0120] Ten actions are set in this paper to meet the changes of the crossover rate and the mutation rate, wherein the detailed actions The values of the crossover rate and the mutation rate corresponding to the actions are set as shown in the following table.

[0121] Action space setting:

[0122]

[0123] After the agent performs an action, the crossover rate and the mutation rate obtained according to the action are used for population evolution, the fitness value is calculated, and positive or negative rewards are given according to the evaluation index of the above state space.

[0124] (8)

[0125] (9)

[0126] wherein R is the weighted sum of the ratio of population diversity DV and the ratio of population coverage SCr, and its functional expression is shown in equation (8). In the state space setting part, the weights of DV and SCr are approximately the same through experiments, so the weight coefficient in the reward mechanism part is set to 0.5, 0.5, as shown in equation (9). When the value of R is greater than 1, a positive reward value 1 is given; when the value of R is less than 1, a negative reward value -1 is given. and 0.5, 0.5, as shown in equation (9). When the value of R is greater than 1, a positive reward value 1 is given; when the value of R is less than 1, a negative reward value -1 is given.

[0127] wherein the action selection is a key step of Q-learning. If the agent can select a better action, it can obtain a positive reward in the next cycle, improve the learning efficiency of the agent, and also enable the algorithm to iterate at a higher crossover and mutation rate at different times, so as to find the optimal solution faster.

[0128] In this paper, Q-learning is combined with SARSA. First, an auxiliary variable is obtained according to the iteration algebra of the population The difference between the greed factor in the greedy strategy in SARSA and the auxiliary variable is compared with an interval range random number. If the random number is less than the difference, a random action is selected from the 10 actions; if the random number is greater than the difference, the action with the maximum Q value is selected according to the known strategy. Equation (10) is the calculation formula of the auxiliary variable .

[0129] (10)

[0130] wherein gen represents the current iteration number of the population, represents the maximum number of iterations set for the population, and the detailed steps of the action selection strategy are shown in the following algorithm:

[0131]

[0132] S5: optimization target setting. In the scheduling process, the minimum maximum completion time is set as the optimization target, and a fitness function based on worker load is introduced through genetic algorithm to avoid overloading work;

[0133] S6: decoding mode design. In the process scheduling decoding, a left insertion decoding mode based on the maximum working time of workers is adopted to insert the process into a time period that does not exceed the maximum working time of 8 hours per day of workers;

[0134] S7: neighborhood structure optimization. The neighborhood structure is based on the critical path exchange structure, and adjacent processes are selected on the critical path to exchange to reduce the bottleneck.

[0135] S8: Schedule effect verification, use Matlab software to construct flexible job shop model for simulation experiment, and compare with genetic algorithm and tabu search algorithm.

[0136] It should be noted that: Figure 11 、 Figure 12 is the worker and machine Gantt chart obtained by using QSNSGA algorithm for WA01, Figure 13 is the additional workload distribution of different algorithms under WA01 example, from the figure, it can be seen that the QSNSGA algorithm has the most no additional workload scheduling and the least high additional workload scheduling. In summary, the QSNSGA algorithm proposed in this paper is superior to DQNSGA, IGA, MOAED algorithm in solving WWFJSP-DRC problem.

[0137] Among them, in step S4, first, three different initialization strategies are used to generate initial solution to improve the diversity and quality of solution, then, after the initial solution is generated, Q-learning and SARSA algorithm are used to randomly select the value of crossover rate and mutation rate in a certain range, next, the initial population is crossed and mutated, and the target fitness value is calculated according to the decoding method of worker working time, then, the generated population is sorted according to the fitness value, neighborhood search is carried out, and the parent and child population are merged. On this basis, the reinforcement learning module is applied to evaluate the fitness of the merged population, so as to obtain the state of the current population, and the corresponding reward signal is determined according to the evaluation index, finally, all populations are non-dominated sorted to obtain the final non-dominated solution, wherein the algorithm flow chart is shown in Figure 2 .

[0138] And the application of reinforcement learning in scheduling neighborhood in machine learning is more and more widely used, which is often used to dynamically change part of the parameters of the algorithm. Because the single genetic algorithm has the problem of easy falling into local optimal solution, Q-learning combined with SARAS algorithm is selected to adaptively adjust the crossover rate Pc and mutation rate Pm of each generation of the algorithm. The dynamic change of crossover rate and mutation rate can let the algorithm choose when it needs to focus on population diversity or optimal solution convergence, instead of evolving the population in a single way.

[0139] This paper uses Q table based on crossover rate Pc and Q table based on mutation rate Pm, and the rows and columns of Q table represent the state and action of agent respectively. The update formula of Q table is shown in the following formula.

[0140]

[0141] Among them represents learning rate, and represents the influence size of new value on updated value. denotes the discount factor, which has a decay effect on the subsequent rewards, to prevent the agent from falling into non-optimal crossover rate and mutation rate.

[0142] After updating the Q table, the population is subjected to neighborhood search and non-dominated sorting in this paper, and the next action is selected according to the Q table.

[0143] Optionally, a three-segment code is used to represent the scheduling scheme, and the three-segment code includes:

[0144] The first segment is the resource selection layer, which represents the machine and worker combination required for process processing, where the machine type includes CNC milling machine, CNC lathe, machining center and drilling machine, and the worker is identified by number;

[0145] The second segment is the process sequencing layer, which uses process number arrangement, and the first appearance of the workpiece number represents the first process of the workpiece, and the subsequent ones correspond to the subsequent processes in turn;

[0146] The third segment is the work arrangement layer, which arranges the work according to the worker number in order, and every seven digits represent a week's work arrangement, where 0 represents rest, 1 represents light work, 2 represents moderate work, and 3 represents mild work. The code is optimized for crossover and mutation in different layers by Q-learning and SARS algorithm. In actual implementation, the three-segment code divides the scheduling problem into three independent and cooperative parts, the first segment (resource selection) is responsible for the selection of machine and worker combination required for process processing, which selects appropriate machine and worker according to the processing requirements of each process and the worker load, the second segment (process sequencing) arranges the start and end time of the process according to the processing sequence of the process to ensure that the priority and order of each process are satisfied, and the third segment (worker arrangement) arranges the work of the worker in a week through coding to ensure that the work time and rest time of the worker are reasonably arranged every day, each part is optimized using reinforcement learning algorithm, reinforcement learning trains agent model to make the coding method adapt to the actual production demand, thereby improving the efficiency and quality of scheduling, through three-segment coding, the scheduling problem is clearly divided into three independent modules of resource selection, process sequencing and worker arrangement, so that each decision in the scheduling process can be optimized, reducing the constraint conflict between each other, the introduction of reinforcement learning algorithm can adaptively adjust the scheduling scheme according to the state of worker load, process priority and machine resource, thereby significantly improving the overall scheduling efficiency, this method can effectively avoid the production bottleneck caused by resource conflict or uneven load in traditional scheduling method, and provides more efficient and flexible scheduling scheme.

[0147] It should be noted that the schematic diagram of the three-segment code is as follows: Figure 3As shown, taking workpieces J1 and J2 as examples, the RS layer chromosome below the figure shows that the processing machines and workers selected by the chromosome are marked with corresponding colors. The DS layer takes the work arrangement of a worker as an example, and the chromosome below the figure shows the chromosome numbers corresponding to different work arrangements and the extra load working time of the workers. The longer the time, the more the maximum processing time of the worker on the day, but the higher the extra load generated.

[0148] Optionally, in the crossover and mutation operations of the genetic algorithm:

[0149] The resource selection layer adopts a multi-point crossover mode;

[0150] The process sequencing layer adopts a crossover mode based on workpiece position exchange;

[0151] The work arrangement layer divides the seven genes into a segment and performs segmented crossover between the crossover points cp1 and cp2;

[0152] The crossover rate Pc is adaptively adjusted between 0.6 and 0.9, and the mutation rate Pm is adaptively adjusted between 0.01 and 0.1. The crossover rate and mutation rate are selected according to the Q table and reward value in different evolution generations through the Q-learning and SARSA algorithms. In actual implementation, the reinforcement learning algorithm combines Q-learning and SARSA algorithms, so that the crossover rate and mutation rate can be dynamically adjusted in the evolution process of the population. The algorithm evaluates the influence of different crossover rates and mutation rates on the evolution effect of the population through an adaptive mechanism. Q-learning is used to calculate the expected reward of the current crossover and mutation strategy, and SARSA is used to update the strategy according to the actual feedback. In each evolution generation, the reinforcement learning model adaptively selects the appropriate crossover and mutation rate according to the current population performance and the requirements of the objective function, ensuring that the search process can maintain the diversity of the population and accelerate the convergence to the optimal solution. Through this mechanism of dynamically adjusting the crossover rate and mutation rate, the premature convergence problem in traditional genetic algorithms is avoided, and the global search ability and local optimization ability of the algorithm are enhanced. In the early stage, a higher crossover rate enables the population to extensively explore the solution space, improving the diversity. In the later stage, increasing the mutation rate helps the algorithm jump out of the local optimal solution and more finely explore the solution space, finally converging to the global optimal solution. This adaptive adjustment can improve the stability and convergence speed of the scheduling algorithm, making the production scheduling of the workshop more efficient and ensuring the flexibility and adaptability in complex production environments.

[0153] Optionally, the decoding method is a left insertion decoding method based on worker working time, which specifically includes:

[0154] When there is a gap between processes, it is determined whether the gap start time, the gap end time and the process startable time are in the same working day, respectively, if the sum of the process processing time and the startable time exceeds the maximum working time of the worker on the same day or exceeds the gap end time, then the insertion is not performed;

[0155] If not, the process is inserted into the gap time period;

[0156] When there is no available gap, the process is arranged to the next day start time or other workable workers. In actual implementation, the left insertion decoding mode first calculates the maximum working time of each worker per day and the gap time between processes, and then determines the reasonable insertion position of the process according to these parameters, so as to ensure that the processing time of each process does not exceed the working time of the worker. If the current process cannot be completed within the remaining working time of the worker on the same day, the process is delayed to the next day or arranged to other workers for processing. This insertion strategy in the decoding process not only considers the sequence requirement of the process, but also ensures that the working load of the worker is within an acceptable range. The left insertion decoding mode makes the arrangement of each process more suitable for the actual production environment, avoids overloading work of the worker by reasonably allocating the working time of the worker, and improves the production efficiency. At the same time, this way makes the production scheduling more flexible, can dynamically adjust the time of the process and the allocation of the worker, reduces the waiting time and production bottleneck, significantly improves the production capacity of the workshop, in addition, the method optimizes the use of idle time, reduces resource waste, and can effectively avoid production quality problems caused by worker fatigue.

[0157] Optionally, the initialization strategy comprises:

[0158] Random initialization is used to improve the diversity of the solution, accounting for 50% of the initial population;

[0159] Initialization according to the earliest completion time of the process makes the initial solution closer to the shortest working period, accounting for 25% of the initial population;

[0160] Initialization according to the remaining processing time of the process makes the initial solution more consistent with the subsequent load balance, accounting for 25% of the initial population;

[0161] The diversity and quality of the initial population are improved by the combination of the above three ways. In the actual implementation process, multiple strategies are used for population initialization in the initialization stage. Random initialization can ensure the diversity of the initial population and avoid limiting the search space. Initialization according to process completion time can prioritize processes that need to be completed first, reducing waiting time in overall scheduling. The initialization strategy according to the remaining time of the process prioritizes processes with longer processing times, ensuring more balanced resource allocation in the production process. By combining these strategies, the algorithm can obtain a diverse population and increase the globality of the search. This multi-strategy initialization method can effectively improve the diversity of the population and avoid the early convergence problem caused by the over-simplicity of the population in traditional methods. The population generated in this way can better adapt to various production environments and maintain high flexibility in the subsequent optimization process. The quality and diversity of the initial population improve the efficiency of the overall scheduling, helping the algorithm to find the global optimal solution more quickly and ultimately optimize the scheduling effect.

[0162] Optionally, a neighborhood structure search is used in each scheduling period. The neighborhood structure is a key block exchange structure based on the critical path, including:

[0163] When the critical path is the first key block, the two processes at the end of the block are exchanged;

[0164] When the critical path is the last key block, the two processes at the beginning of the block are exchanged;

[0165] When the critical path is in the middle, the processes at the beginning and end of the key block are exchanged, or when the length of the key block is greater than 4, the first two processes and the last two processes are exchanged;

[0166] If a better scheduling solution is obtained after the exchange, it is accepted. In the actual implementation process, the neighborhood structure search first analyzes the critical path in the current scheduling scheme to find the longest process chain that has the greatest impact on the completion time. Then, the process in the critical path is exchanged to optimize its order, reduce the idle time between processes, and shorten the overall completion time. This process continuously adjusts the process order on the critical path to improve production efficiency. The neighborhood structure search can significantly optimize the bottleneck link in production scheduling, shorten the overall completion time, and improve production efficiency. By locally optimizing the critical path, the quality of the scheduling scheme can be quickly improved. In addition, the introduction of the neighborhood structure search allows for flexible adjustment of the process order during scheduling, thereby avoiding resource conflicts and production delays caused by unreasonable process arrangement, and improving the resource utilization rate and production capacity of the workshop.

[0167] It should be noted that the neighborhood structure allows the solution to evolve in a better direction while maintaining randomness and diversity. In this paper, two different optimization objectives of WWFJSP-DRC are used to design the neighborhood structure.

[0168] (1) For the neighborhood structure of minimizing the maximum completion time, first find the critical path of the individual to be searched in the neighborhood, the critical path is the longest path without idle time in the scheduling scheme generated by the feasible solution, the process of finding the critical path is shown in algorithm 2, after finding the critical path, the critical block is exchanged within the process, if it is the first critical block, only exchange the two processes connected at the end of the block; if it is the last critical block, only exchange the two processes connected at the beginning of the block; in the case of not being the first critical block or the last critical block, exchange the two processes at the beginning of the block and the two processes at the end of the block, the process is shown in algorithm 3.

[0169] (2) For the neighborhood structure of minimizing the additional worker load, search the work arrangement part of the chromosome to be searched in the neighborhood, find the workers with long daily working hours, adjust the work arrangement of the workers to reduce the long working hours of the additional worker load, and then decode the chromosome to check whether the maximum completion time is reduced or unchanged, if it is reduced or unchanged, it is accepted. The process of this neighborhood structure is shown in algorithm 4.

[0170] Among them, algorithm 2, algorithm 3 and algorithm 4 are as follows:

[0171]

[0172]

[0173] Optionally, in the process of process sorting, an adaptive neighborhood search strategy is adopted, and the neighborhood search includes:

[0174] The position-based crossover operator is used to randomly select the position of the workpiece for exchange;

[0175] The exchange-based mutation operator is used to randomly exchange the positions of two processes;

[0176] And through the Q-learning and SARSA algorithm dynamic adjustment of crossover operator and mutation operator selection probability, to maintain population diversity in the early stage of the algorithm, the latter guide the population to converge to a better solution. In the actual implementation process, the adaptive neighborhood search strategy will first adjust the current population through a variety of crossover and mutation strategies, according to the fitness of the current population, intelligent selection of appropriate crossover and mutation method, both can maintain the diversity of the population, but also can speed up the convergence of the population, eventually guide the population to the optimal solution direction development, the strategy in the process of continuous iteration dynamically selects the most suitable search method to ensure that the scheduling target can be effectively optimized, the adaptive neighborhood search strategy can flexibly cope with the complex constraints and objectives in the scheduling process, avoid the excessive concentration of search in the traditional method, ensure that the scheduling algorithm finds a balance between global and local, improve the convergence speed and quality of the population, in addition, the strategy can effectively improve the robustness and efficiency of production scheduling, ensure the reasonable allocation of worker load, reduce the waiting and conflict problems in production.

[0177] Among them, crossover and mutation play a crucial role in the process of simulating biological evolution. In order to find a better target solution faster, three different crossover methods are designed in this paper.

[0178] The crossover of the RS layer adopts multi-point crossover. The crossover process is: select the parent S1 and the parent S2 of the resource selection part, randomly generate a set of crossover points pos1, and then randomly generate the number of crossovers CN1. At each position of the crossover, exchange the gene values of the two parents to complete the crossover operation.

[0179] The OS layer selects the way of exchanging workpieces to cross, selects the parent S1 and the parent S2 of the process part, randomly selects the workpiece position of the crossover, and generates a new gene sequence by exchanging the positions of the workpiece parent individuals. For the remaining genes in the process, take out the exchanged workpieces from one parent and fill in the empty positions in the other parent, and finally update the genes after crossover.

[0180] The DS layer also selects multi-point crossover, selects the parent S1 and the parent S2 of the work arrangement part, divides them into multiple segments according to a worker's 7 genes, and the 7 genes are sequentially represented as Monday to Sunday of a week, randomly selects the crossover points cp1 and cp2, and exchanges the first five factors in each segment, and exchanges the last two elements in each segment. The crossover of the DS layer chromosome is shown in Figure 4 .

[0181] The purpose of mutation is to increase the randomness in the process of finding a better solution. The mutation method adopted in the resource selection part is to randomly change to other selectable resources, the process sequencing part is to randomly exchange the positions of two processes, and the work arrangement part is to randomly determine the point position and change to other selectable work duration.

[0182] Optionally, simulation experiments are conducted by building a 10x10 scale flexible job shop instance on the Matlab platform, and inverse generational distance IGD and solution coverage SC are used as performance evaluation indicators to compare the method with standard genetic algorithm and tabu search algorithm. In actual implementation, the proposed scheduling algorithm is tested through simulation experiments, and compared with traditional scheduling methods to analyze and evaluate the superiority in minimizing completion time and worker load. By setting different test scenarios and parameters, various possible situations in actual production environment of the workshop are simulated to verify the stability and applicability of the algorithm. Through comparison with traditional methods, the experimental results can verify the significant advantages of the proposed algorithm in worker load optimization and production efficiency improvement, further improving the applicability and flexibility of the algorithm in actual workshop environment, ensuring that the proposed method can effectively deal with multi-objective optimization problems in complex production tasks. The simulation experiment provides a reliable theoretical basis for the practical application of the algorithm and provides a feasible optimization scheme for future workshop scheduling.

[0183] Working principle: Firstly, the construction of the worker load model is the basis of this method. By considering the influence of different worker work arrangements on the load, workers are divided into four situations: light work, moderate work, mild heavy work and rest, and then the work load of each worker is accurately calculated. According to the maximum working time of workers per day, dynamically adjust the task allocation to ensure that the production task is completed without exceeding the maximum load of workers. In this process, the key role of the worker load model is to accurately assess the working time and rest time of each worker, maximize the reduction of worker overload, and improve the efficiency and accuracy of production task allocation;

[0184] After the construction of the worker load model, the next step is resource selection and scheduling. According to the processing requirements of each process, select appropriate machine and worker resources. The processing requirements of each process and the worker load model will be used as the basis for decision-making. By optimizing resource combination, ensure that the processing time of each process does not exceed the maximum working time of workers per day. Through this process, the arrangement of processes can be reasonably arranged according to the load of workers to avoid unnecessary waiting time and resource waste. In addition, the scheduling order of processes will be optimized according to the worker load limit to ensure that the completion time of processes will not exceed the specified time limit;

[0185] Then, the method introduces a dynamic scheduling strategy based on reinforcement learning. The application of reinforcement learning in scheduling problems can automatically adjust the crossover rate and mutation rate during the algorithm evolution process. By combining Q-learning and SARSA algorithms, the most suitable crossover rate and mutation rate can be adaptively selected in different evolution generations, thereby optimizing resource allocation and process sequencing during scheduling. During the optimization process, a higher crossover rate is used for extensive solution space exploration in the early stage, and a higher mutation rate is used to avoid falling into a local optimal solution in the later stage, thereby improving the quality and stability of the solution.

[0186] Meanwhile, the application also designs a left insertion decoding method based on the maximum working time of workers. Traditional scheduling methods often ignore the limitation of the maximum daily working time of workers. However, the method uses left insertion decoding to reasonably arrange the insertion position of the process by combining the maximum working time of workers with the gap time between processes. This decoding method ensures that the processing time of each process does not exceed the working time of workers on the same day, effectively avoiding the phenomenon of overloading work of workers, ensuring the smooth operation of production, and improving the production efficiency and the overall working environment of the workshop through this strategy.

[0187] In the neighborhood structure search phase of the optimization process, based on the optimization of process sequencing, the method identifies the processes that affect the completion time by analyzing the critical path and exchanges the processes in the critical path to reduce the bottleneck in production. Neighborhood structure optimization can gradually improve the quality of the production scheduling scheme by exchanging processes locally, especially in the production process. The optimization and adjustment of the critical path play a crucial role in improving the overall scheduling efficiency.

[0188] Finally, the effectiveness of the method is verified through simulation experiments, further verifying the advantages of the proposed method over traditional scheduling methods in multi-objective optimization. Through comparison with traditional scheduling methods, the experimental results show that the method can significantly reduce the workload of workers, improve the production efficiency of the workshop, and optimize the completion time while meeting the dual resource constraints. Through simulation data, the practical application effect of the proposed scheduling method is verified, proving its superiority in workshop production scheduling.

[0189] In summary, the dual resource constraint flexible job shop scheduling method with reduced worker load successfully solves the problems of uneven worker load and resource conflict in workshop scheduling by introducing innovative technologies such as worker load model, reinforcement learning adaptive adjustment, left insertion decoding, and neighborhood structure optimization. The method optimizes the production process and improves the production efficiency. At the same time, through experiments, the method has significant advantages in multi-objective optimization, minimization of worker load, and maximum completion time, and can be widely applied to complex production scheduling environments, with good practicality and generalizability.

[0190] The basic principles and main features of the present application and the advantages of the present application are shown and described above, and the standard parts used in the present application can be purchased from the market, and the special-shaped parts can be ordered according to the description and the drawings, the specific connection mode of each part adopts the conventional means such as bolt rivet, welding in the prior art, the mechanical, part and equipment adopt the conventional type in the prior art, and the circuit connection adopts the conventional connection mode in the prior art, which will not be described in detail here.

[0191] The above is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent transformation or direct or indirect application in the related technical field by using the content of the specification and drawings of the present application is also included in the patent protection scope of the present application.

Claims

1. A method for scheduling flexible workshops under dual resource constraints to reduce worker workload, characterized in that, Includes the following steps: S1: Construction of the worker workload model. The worker workload model is divided into four situations based on the daily work arrangement of workers: light work, moderate work, slightly heavy work and rest. A maximum working time is set for each worker to calculate the additional workload. S2: Process resource combination selection, wherein the resource combination consists of machine tools and workers. The machine tools include CNC milling machines, CNC lathes, machining centers and drilling machines, and the workers include machine tool operators and maintenance workers. The processing time required for each process shall not exceed the maximum working time of the worker on that day. S3: Process scheduling sequence determination. Under the condition of satisfying worker load constraints, the process sequence is determined by a scheduling rule that combines the shortest processing time priority and the critical path method. S4: Dynamic scheduling strategy design, the dynamic scheduling strategy is based on Q-learning algorithm and SARSA algorithm, and the scheduling effect is optimized by adaptively adjusting the probability of crossover operator and mutation operator in evolutionary algorithm; S5: Optimize target setting. During the scheduling process, set the minimum maximum completion time as the optimization target, and introduce a fitness function based on worker load through a genetic algorithm to avoid overwork. S6: Decoding method design: When decoding the process scheduling, a left-insertion decoding method based on the maximum working time of workers is adopted to insert the process into a time period that does not exceed the maximum working time of 8 hours per day for workers; S7: Neighborhood structure optimization, wherein the neighborhood structure is based on the critical path exchange structure, and adjacent processes on the critical path are selected for exchange to reduce bottlenecks; S8: Scheduling effect verification. A flexible workshop model was built using Matlab software for simulation experiments, and the results were compared with those of genetic algorithm and tabu search algorithm.

2. The method for scheduling flexible workshops under dual resource constraints to reduce worker workload according to claim 1, characterized in that, The scheduling scheme is represented by a three-segment code, which includes: The first section is the resource selection layer, which represents the combination of machines and workers required for the process. The machine tool models include CNC milling machines, CNC lathes, machining centers, and drilling machines, and the workers are identified by numbers. The second section is the process sequencing layer, which is arranged by process number. The first appearance of the workpiece number indicates the first process of that workpiece, and subsequent appearances correspond to the subsequent processes. The third segment is the work arrangement layer, which is arranged according to the worker's number. Every seven numbers represent the work arrangement for a week, where 0 represents rest, 1 represents light work, 2 represents moderate work, and 3 represents very heavy work. The encoding is optimized by Q-learning and the SARSA algorithm to optimize the crossover and mutation of different layers.

3. The method for scheduling flexible workshops under dual resource constraints to reduce worker workload according to claim 1, characterized in that, In the crossover and mutation operations of genetic algorithms: The resource selection layer adopts a multi-point cross-processing approach; The process sequencing layer adopts a cross-processing method based on workpiece position exchange; The work arrangement layer divides the seven genes into a segment and performs segmented crossover between the crossover points cp1 and cp2; The crossover rate Pc is adaptively adjusted between 0.6 and 0.9, and the mutation rate Pm is adaptively adjusted between 0.01 and 0.

1. The crossover rate and mutation rate are selected based on the Q table and reward value in different generations using Q-learning and the SARSA algorithm.

4. The method for scheduling flexible workshops under dual resource constraints to reduce worker workload according to claim 1, characterized in that, The decoding method is a left-insertion decoding method based on worker working hours, which specifically includes: When there is a gap between processes, determine whether the start time of the gap, the end time of the gap, and the start time of the process are on the same working day. If the sum of the processing time of the process and the start time exceeds the maximum working time of the worker on that day or exceeds the end time of the gap, no insertion is performed. If the time limit is not exceeded, the process will be inserted into the gap time period; When there are no available slots, schedule the process to start at the next day or to another available worker.

5. The method for scheduling flexible workshops under dual resource constraints to reduce worker workload according to claim 1, characterized in that, The initialization strategy includes: Random initialization, used to improve solution diversity, accounts for 50% of the initial population; Initialization based on the earliest completion time of each process makes the initial solution closer to the shortest duration, accounting for 25% of the initial population; Initialization based on the remaining processing time of each process makes the initial solution more consistent with subsequent load balancing, accounting for 25% of the initial population; By combining the above three methods in appropriate proportions, the diversity and quality of the initial population can be improved.

6. The method for scheduling flexible workshops under dual resource constraints to reduce worker workload according to claim 1, characterized in that, Neighborhood structure search is performed within each scheduling cycle. This neighborhood structure is a critical block exchange structure based on the critical path, including: When the critical path is the first critical block, swap the two operations at the end of the block. When the critical path is the last critical block, swap the two operations at the beginning of the block. When the critical path is in the middle, swap the first and last processes of the critical block, or swap the first two processes with the last two processes when the critical block length is greater than 4. If a better scheduling solution is obtained after the exchange, the scheduling result is accepted.

7. The method for scheduling flexible workshops under dual resource constraints to reduce worker workload according to claim 1, characterized in that, In the process of process sequencing, an adaptive neighborhood search strategy is adopted, wherein the neighborhood search includes: A position-based crossover operator is used to randomly select workpiece positions for exchange; A mutation operator based on exchange is used to randomly swap the positions of two processes; Furthermore, by using Q-learning and the SARSA algorithm, the selection probabilities of the crossover and mutation operators are dynamically adjusted to maintain population diversity in the early stages of the algorithm and guide the population to converge to a better solution in the later stages.

8. The method for scheduling flexible workshops under dual resource constraints to reduce worker workload according to claim 1, characterized in that, A simulation experiment was conducted by constructing a 10×10 flexible workshop instance on the Matlab platform. The inverse generation distance (IGD) and solution set coverage (SC) were used as performance evaluation indicators. The results of the proposed method were compared with those of the standard genetic algorithm and the tabu search algorithm.

Images