A human-robot collaborative assembly cell task intelligent scheduling method considering fatigue recovery

Abstract

The application provides a human-machine cooperation assembly unit task intelligent scheduling method considering fatigue recovery, belongs to the technical field of intelligent manufacturing and human-machine cooperation, and is characterized in that the method establishes a multi-objective optimization model integrating finished product time, load and dynamic fatigue, integrates a dynamic fatigue recovery mechanism based on a learning-forgetting-fatigue-recovery model, constructs a scheduling problem into a Markov decision process, and designs corresponding state, action space and reward function. Through training of a Dueling Double DQN intelligent agent integrating priority experience replay, the method realizes dynamic distribution of tasks according to real-time task progress and worker fatigue state, intelligently inserts a worker rest period, and enables a robot to continue working. The application can realize real-time sensing of a worker state, dynamic adjustment of a scheduling strategy, and an intelligent scheduling method with high convergence and robustness, is suitable for intelligent human-machine cooperation production lines in the fields of automobile assembly, electronic manufacturing, aerospace and the like, and supports intelligent manufacturing with a people-oriented concept.

Description

Technical Field

[0001] This invention belongs to the field of intelligent manufacturing and human-machine collaboration technology, specifically relating to an intelligent scheduling method for human-machine collaborative assembly unit tasks that takes fatigue recovery into account. Background Technology

[0002] With the development of Industry 4.0, human-robot collaborative hybrid assembly units can combine the advantages of both humans and robots to meet the requirements of small-batch customized production. In this model, workers and collaborative robots share the same workspace and must coordinate to complete the same assembly tasks. Currently, traditional task scheduling focuses on system efficiency, such as the shortest production cycle or maximum equipment utilization, and often uses mathematical programming or metaheuristic algorithms to solve problems, while neglecting the problem of physiological fatigue accumulation in workers who continuously perform the same work. In addition, since fatigue leads to a decline in worker work quality and safety, it will result in a decrease in overall productivity in the long run.

[0003] Some scholars, starting from the exploration of human comfort, have attempted to introduce ergonomic elements, or even simplify them to fatigue factors, into scheduling models. They treat fatigue as a fixed, discrete penalty term, failing to fully consider its dynamic accumulation over time and workload, and its diminishing effect due to rest. Traditional optimization algorithms lack the ability to adapt and correct in real-time to the dynamic nature of the scheduling environment and the multi-objective nature of scheduling problems. In recent years, some research has introduced deep reinforcement learning into the production scheduling problem; however, for solving multi-party collaborative optimization problems involving high-dimensional state information such as tasks, resources, and fatigue, and complex action spaces, standard algorithms generally suffer from stability problems and struggle to obtain excellent solutions, thus still having many shortcomings.

[0004] Therefore, there is an urgent need for an intelligent scheduling algorithm for human-machine collaborative assembly systems that can monitor human status, dynamically adjust scheduling strategies, and has high convergence and robustness, so as to continuously improve assembly quality and efficiency in the long term. Summary of the Invention

[0005] This invention provides an intelligent scheduling method for human-machine collaborative assembly unit tasks that considers fatigue recovery, addressing the difficulty in coordinating and optimizing production efficiency and worker fatigue in existing technologies. This method dynamically allocates assembly tasks and intelligently inserts rest periods, achieving shorter production cycles while effectively managing and controlling worker fatigue, thus achieving the dual optimization goals of production efficiency and worker health.

[0006] The technical solution of the present invention includes the following steps:

[0007] Step S10: Establish a multi-objective scheduling optimization model that integrates completion time, worker fatigue, and system load. This model is based on the assembly task set, task timing constraints, human and machine resources, and operation time, with the goal of minimizing the comprehensive index Q.

[0008] Step S20: Integrate a dynamic fatigue accumulation and recovery mechanism based on the learning-forgetting-fatigue-recovery model to calculate the accumulation of worker fatigue and rest recovery in real time during the work cycle;

[0009] Step S30: Construct a task scheduling environment based on Markov decision process, and define the state space, action space and reward function;

[0010] Step S40: Design and train a Dueling Double DQN agent that incorporates a priority experience replay mechanism;

[0011] Step S50: Based on the trained agent, perform online dynamic scheduling, select actions according to the real-time system status, execute task assignment or insert restorative breaks for workers, update the environmental status, until all tasks are completed, and output the final scheduling scheme.

[0012] Furthermore, in step S10, the construction of the multi-objective optimization model includes:

[0013] Step S101: Define the assembly task set Each task has predefined attributes, including H performed only by humans, R performed only by robots, H and R performed jointly by humans and robots, task types that can be performed by humans or robots, and worker operation time. Robot operation time and the physiological fatigue coefficient corresponding to the task ;

[0014] Step S102: Define the temporal constraint relationship matrix between tasks. ;

[0015] Step S103, define assembly operation cycle A, involving h workers and r robots;

[0016] Step S104: Establish a multi-objective optimization model that includes fatigue state, with the objective function as follows:

[0017]

[0018] in, To maximize the completion time, The total system load. Total worker fatigue These are the weighting coefficients for each objective. .

[0019] Furthermore, in step S20, the dynamic fatigue accumulation and recovery mechanism based on the learning-forgetting-fatigue-recovery model is as follows:

[0020] Step S201: Model the fatigue accumulation process of workers during continuous work as an exponential growth process, and calculate the duration of continuous work. The level of fatigue reached later :

[0021]

[0022] in, The fatigue coefficient for the current task. This is the initial fatigue value;

[0023] Step S202: Model the fatigue recovery process of workers during rest periods as an exponential decay process and calculate the duration of the worker's experience. Remaining fatigue after rest :

[0024]

[0025] in, The coefficient of recovery, This represents the fatigue level before rest.

[0026] Step S203: Embed the above dynamic fatigue accumulation and recovery model into each decision point of the scheduling process; predict the changes in the worker's fatigue state after selecting different actions based on the worker's current fatigue value, the fatigue coefficient of the task to be assigned, and the expected execution time.

[0027] Step S204: In each work cycle, the worker continuously performs a series of tasks and breaks, and their fatigue level is updated and passed on to the next cycle after performing steps S201 to S202 in sequence.

[0028] Furthermore, in step S30, the definition of the state space, action space, and reward function is as follows:

[0029] Step S301: The state space is represented by a multi-channel matrix, including a task allocation matrix, a task progress matrix, a real-time worker fatigue matrix, and a human-machine load matrix; all state features are normalized.

[0030] Step S302, the action space includes assigning tasks to available workers or robots, and inserting rest actions for workers. The action set at time step t is represented as:

[0031]

[0032]

[0033] in, This indicates that task J1 will be assigned to a worker. This indicates that task J1 will be assigned to the robot; and This indicates that no task will be assigned.

[0034] Step S303: The reward function adopts a step-by-step reward mechanism to distinguish between actions performed by humans and robots.

[0035]

[0036] in, Indicates the weighting coefficient; , , They represent the actions to be performed. The increments in post-completion time, workload, and worker fatigue;

[0037] Furthermore, in step S40, the training steps of the Dueling Double DQN agent with the fusion priority experience replay mechanism are as follows:

[0038] Step S401: Construct a Dueling Double DQN agent, using a Dueling structure to decompose the Q-value into a state value function. and action advantage function The target Q-value is calculated using the Double DQN method:

[0039]

[0040] in, , For the current network parameters, The target network parameters to be learned, This is the discount factor.

[0041] Step S402: In the experience replay pool, samples are assigned priority based on the absolute value of the time-series difference error, and samples with larger errors are sampled with higher probability.

[0042] In step S403, the agent performs trial-and-error training in a simulated environment, updates network parameters by minimizing the loss function, and learns a scheduling strategy that simultaneously optimizes production efficiency and fatigue state.

[0043] Furthermore, in step S50, the specific execution steps of the online dynamic scheduling decision are as follows:

[0044] Step S501: Input assembly task data, human-machine resource information, and worker initial status;

[0045] Step S502, at each decision moment Based on the current task completion status, resource availability, and real-time worker fatigue, a state feature matrix is ​​constructed. ;

[0046] Step S503, will Input the trained Dueling Double DQN agent and select the action with the highest Q value. ;

[0047] Step S504, execute the action; if For task assignment, the corresponding task will be assigned to the specified worker or robot, and the task progress and resource status will be updated; if To insert a rest period, the rest duration is calculated based on the learning-forgetting-fatigue-recovery model, allowing the designated worker to rest while the robot continues working, and the worker's fatigue status is updated.

[0048] Step S505, repeat steps S502 to S504 until all assembly tasks are completed;

[0049] Step S506: Generate a scheduling Gantt chart that includes task sequence, human-machine allocation, start and end times, and rest periods, and output performance indicators such as completion time and average fatigue level. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the overall system flow of the method of the present invention.

[0051] Figure 2 This is a diagram showing the sequence of assembly tasks in an embodiment of the present invention.

[0052] Figure 3 This is a comparison chart showing the impact of different hyperparameters on the training process in an embodiment of the present invention;

[0053] Figure 4 This is a comparison chart of the training convergence curves of five reinforcement learning algorithms in this embodiment of the invention;

[0054] Figure 5 A bar chart comparing the single scheduling solution time of different algorithms under different task sizes;

[0055] Figure 6 A bar chart comparing the final reward values ​​of scheduling schemes obtained by different algorithms under different task sizes;

[0056] Figure 7 A bar chart comparing the maximum completion time of scheduling schemes obtained by different algorithms under different task sizes;

[0057] Figure 8A bar chart comparing the total cumulative worker fatigue for scheduling schemes obtained by different algorithms under different task sizes;

[0058] Figure 9 This is a Gantt chart comparison between the traditional scheduling scheme and the scheduling scheme of the present invention in this embodiment of the invention. Detailed Implementation

[0059] The present invention will now be described in further detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit the invention itself.

[0060] like Figure 1 As shown, the specific implementation process of the intelligent scheduling method for human-machine collaborative assembly unit tasks considering fatigue recovery provided by the present invention is as follows:

[0061] Step S10: In this embodiment, the assembly process of a certain type of gearbox is used as an example. There are a total of 34 assembly tasks, and the sequence constraints between tasks 1 to 34 are as follows: Figure 2 As shown. The task annotation types (H, R, H&R, H / R), human-machine operation time, and fatigue coefficient are displayed. The system is configured with multiple human-machine collaboration scenarios P(2,2), P(2,3), P(3,2), and P(3,3), including single-cycle and dual-cycle modes.

[0062] Step S20: Utilize the dynamic fatigue accumulation and recovery mechanism based on the learning-forgetting-fatigue-recovery model, where the fatigue recovery model is set with a recovery coefficient μ=0.05 and the worker fatigue initialization value is 0.

[0063] Step S30: The state space adopts a 4-channel matrix form; the action space is responsible for assigning tasks to workers and robots, and inserting rest periods for workers; the weights of each objective in the reward function are set as follows: , , .

[0064] Step S40: Establish a Dueling Double DQN agent. The network consists of three fully connected layers plus a ReLU activation layer, and finally a Dueling layer is added to obtain the final Q-value. According to... Figure 3 The hyperparameters obtained using grid search and sensitivity analysis are shown in Table 1. The target network update frequency C = 300 steps; empirical replay pool size = 10000; batch size = 64; priority replay parameters... , The total number of training rounds is set to 100,000. Figure 4The training curves of the Dueling Double DQN algorithm are shown. It can be seen that the reward of Dueling Double DQN begins to increase after approximately 20,000 iterations, and converges to a relatively high value after 50,000 iterations. The overall performance of the Dueling Double DQN algorithm is higher than all baseline algorithms.

[0065] Table 1. Parameter settings for the Dueling Double DQN algorithm

[0066]

[0067] Step S50: Use the trained model in online scheduling and obtain the following results. Figure 9 (b) shows the scheduling Gantt chart. The solution of this invention, compared to... Figure 9 (a) only considers the algorithm with the minimum total completion time. It divides the rest time of each worker cycle into several short rest periods and inserts them into the work. The robot can continue to work during the worker's rest period. This can maintain high production efficiency and reduce worker fatigue.

[0068] To verify the superiority of the method of this invention, the standard DQN, DDQN, and Dueling DQN algorithms were run and compared under the same environment. Statistical indicators included solution time, overall reward value, maximum completion time, and total fatigue accumulation, such as... Figures 5-8 As shown, all deep reinforcement learning methods complete decisions in milliseconds to seconds, meeting real-time requirements. Under different task cycle sizes, Dueling Double DQN consistently achieves the highest final reward value. The Dueling Double DQN scheme significantly reduces worker fatigue, with a reduction of up to 30% or more.

Claims

1. A human-machine collaborative assembly unit task intelligent scheduling method considering fatigue recovery, characterized in that, Includes the following steps: Step S10: Establish a multi-objective scheduling optimization model that integrates completion time, worker fatigue, and system load. This model is based on the assembly task set, task timing constraints, human and machine resources, and operation time, with the objective of minimizing the comprehensive index Q. The comprehensive index Q is expressed as: ； in, To maximize the completion time, The total system load. Total worker fatigue These are the weighting coefficients for each objective. ; Step S20 integrates a dynamic fatigue accumulation and recovery mechanism based on a learning-forgetting-fatigue-recovery model to calculate worker fatigue accumulation and rest recovery in real time during the work cycle; step S20 includes: Step S201: Model the fatigue accumulation process of workers during continuous work as an exponential growth process, and calculate the duration of continuous work. The level of fatigue reached afterward : ； in, The fatigue coefficient for the current task. This is the initial fatigue value; Step S202: Model the fatigue recovery process of workers during rest periods as an exponential decay process and calculate the duration of the worker's experience. Remaining fatigue after rest : ； in, The coefficient of recovery, This represents the fatigue level before rest. Step S203: The dynamic fatigue accumulation and recovery mechanism is embedded into each decision point of the scheduling process. Based on the worker's current fatigue value, the fatigue coefficient of the task to be assigned, and the expected execution time, the changes in the worker's fatigue state after selecting different actions are predicted. In step S204, during each work cycle, the worker continuously performs a series of work and rest periods, and their fatigue level is updated and passed on to the next work cycle after performing operations S201~S202 in sequence. Step S30: Construct a task scheduling environment based on a Markov decision process, defining a state space, action space, and reward function. The state space is represented by a multi-channel matrix, including a task allocation matrix, a task progress matrix, a real-time worker fatigue matrix, and a human-machine load matrix. All state features are normalized. The action space includes assigning tasks to available workers or robots, and inserting rest actions for workers. The set of actions at each time step is represented as: ； ； in, This indicates that task J1 will be assigned to a worker. This indicates that task J1 will be assigned to the robot; and This indicates that no task is assigned; the reward function adopts a step-by-step reward mechanism to distinguish between actions performed by humans and robots, and the reward function is defined as follows: ； in, Indicates the weighting coefficient; , , They represent the actions to be performed. The increments in maximum completion time, total system load, and total worker fatigue; Step S40: Design and train a Dueling Double DQN agent that incorporates a priority experience replay mechanism; Step S40 includes: building a Dueling Double DQN agent and using a Dueling structure to decompose the Q-value into a state value function. and action advantage function Double DQN is used to calculate the target Q value: ； in, , For the current network parameters, The target network parameters to be learned, The discount factor is used; in the experience replay pool, samples are assigned priority based on the absolute value of the temporal difference error, and samples with larger errors are sampled with higher probability; the agent is trained through trial and error in a simulated environment, updates network parameters by minimizing the loss function, and learns a scheduling strategy that simultaneously optimizes production efficiency and fatigue state. Step S50 involves making online dynamic scheduling decisions based on the trained agent, selecting actions according to the real-time system state, performing task allocation or inserting rest periods for workers, updating the system state until all tasks are completed, and outputting the final scheduling scheme; step S50 includes: Step S501: Input assembly task data, human-machine resource information, and worker initial status; Step S502, at each decision moment Based on the current task completion status, resource availability, and real-time worker fatigue, a state feature matrix is ​​constructed. ; Step S503, the state feature matrix Input the trained Dueling Double DQN agent and select the action with the highest Q value. ; Step S504, execute the action; if For task assignment, the corresponding task will be assigned to the specified worker or robot, and the task progress and resource status will be updated; if To insert restorative breaks, the rest duration is calculated based on the learning-forgetting-fatigue-recovery model, allowing the designated worker to rest while the robot continues working, and the worker's fatigue status is updated. Step S505, repeat steps S502 to S504 until all assembly tasks are completed; Step S506: Generate a scheduling Gantt chart that includes task sequence, human-machine allocation, start and end times, and rest periods, and output the completion time and average fatigue performance index.

Images