Self-adaptive man-machine cooperation task scheduling method for intelligent manufacturing scene

Abstract

The invention belongs to the technical field of man-machine cooperation. The invention provides a self-adaptive man-machine cooperation task scheduling method for an intelligent manufacturing scene. According to the embodiment of the invention, an extensible context state encoder is designed, scene features, task dependency relationships, resource states and the like are fused based on a graph neural network, and multi-dimensional constraint state representation is generated; an experience-driven task distributor is constructed, a cross-scene cooperation mode is mined through historical scheduling knowledge, and an initial distribution scheme is generated to reduce the search space dimension; an autonomous learning optimizer based on element reinforcement learning is developed, and field differences and scale elastic changes are quickly adapted through multi-scene element strategy pre-training and online lightweight fine tuning.

Description

Technical Field

[0001] This disclosure relates to the field of human-machine collaboration technology, and in particular to an adaptive human-machine collaboration task scheduling method for intelligent manufacturing scenarios. Background Technology

[0002] As human-machine collaborative systems evolve towards openness and high dynamism, task scheduling faces the challenge of complex scenarios involving multi-dimensional constraints and frequent disturbances. In complex industrial scenarios such as automotive assembly lines and warehousing logistics, task scheduling in human-machine collaboration must simultaneously meet the synergistic optimization of time constraints, spatial constraints, and capability constraints, specifically involving key factors such as task deadlines, equipment layout limitations, and the complementarity of human and machine skills. Furthermore, the dynamism of scenarios is not only reflected in the differences across manufacturing and logistics fields, but also in the elastic changes in scale such as the increase or decrease in task volume, resource quantity, and equipment status fluctuations within a single scenario. Traditional scheduling methods rely on static rules or offline optimization models, which cannot perceive environmental changes in real time and dynamically adjust strategies, easily leading to a chain reaction of task backlog, resource idleness, and process blockage. Although existing reinforcement learning methods can respond dynamically, their policy networks are usually designed for a fixed scale and disturbance pattern specific to a particular scenario. When migrating across domains or dealing with temporary robot additions or a surge in tasks within a scenario, their generalization ability significantly decreases, making it difficult to maintain the global stability of the system.

[0003] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0004] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0005] The purpose of this disclosure is to provide an adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios, thereby overcoming, to at least to some extent, one or more problems caused by the limitations and defects of related technologies.

[0006] According to embodiments of this disclosure, an adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios is provided, including: Construct system models and resource performance models; Based on the system model and resource performance model, the context encoder dynamically aggregates task dependencies and human-machine resource states according to the graph attention mechanism to generate context embedding vectors; The context embedding vector is input into the experience-driven task allocator. The feedforward neural network and LSTM in the experience-driven task allocator are used to update the state embedding and agent embedding. The task-agent matching sequence is generated step by step through the cyclic task-agent matching mechanism to obtain the initial task matching scheme. Calculate the immediate reward based on the initial task matching scheme; Based on the initial network parameters of the immediate reward, context encoder, and task allocator, the autonomous learning optimizer uses the proximal policy optimization PPO algorithm for task-specific updates and the Reptile algorithm for meta-policy updates to generate the optimal task matching scheme.

[0007] Furthermore, the steps in constructing the system model and resource performance model include: Define the set of heterogeneous human-machine resources and constraints; wherein, the set of heterogeneous human-machine resources includes the set of human operators and the set of robot agents, and the constraints include time constraints, space constraints and process constraints; The human-machine collaborative task scheduling problem is modeled as a partially observable Markov decision process, namely the POMDP model; the POMDP model includes a state space, action space, state transition probabilities, reward function, observation space, and observation function. A resource performance model is constructed based on a deterministic machine performance model, a fatigue-effect coupled human learning model, and a human dynamic performance model tracker.

[0008] Furthermore, the expression for the reward function is:

[0009] in, To satisfy the current job's strict constraints on the feasible subset, An infeasible subset that violates constraints. For the first i One task, For the first j An intelligent agent. This is the infeasibility coefficient.

[0010] Furthermore, the step of the context encoder dynamically aggregating task dependencies and human-machine resource states based on the graph attention mechanism to generate context embedding vectors includes: Construct a heterogeneous graph based on a Simple Temporal Network (STN); the heterogeneous graph includes task type nodes, agent nodes, and various edge types, including time-constrained edges, task execution time edges, task time window edges, and agent cooperation edges. During the message passing phase, each edge type is associated with an independent weight matrix, which is then linearly transformed into the source node's features and stored in the target node's message mailbox. In the feature aggregation stage, attention coefficients related to edge types are calculated based on the attention mechanism, and the final feature vector of the node, i.e. the context embedding vector, is generated by weighted summation and activation function; among them, a multi-head attention mechanism is used to capture multi-dimensional semantics.

[0011] Further, the process of inputting the context embedding vector into an experience-driven task allocator, updating the state embedding and agent embedding using the feedforward neural network and LSTM in the experience-driven task allocator, and progressively generating a sequence of task-agent matching pairs through a cyclic task-agent matching mechanism to obtain the initial task matching scheme includes: Based on the context embedding vector and all agent embeddings, the probability of each agent being selected is calculated using a feedforward neural network, and one agent is selected. After selecting an agent, the agent embedding, the task embedding of all tasks to be assigned, and the global state embedding are combined to evaluate and select the most suitable task for matching to generate a matching pair. The state embedding and agent embedding are updated by LSTM, and the next pair of matches is performed in a loop until all tasks are assigned to generate an initial task matching scheme.

[0012] Furthermore, the steps of matching initial tasks and calculating immediate rewards include: Execute the initial task matching scheme and record the actual task type and quantity assigned to each human operator; The dynamic human performance tracker updates the prediction of the actual task completion time of human operators based on new observation data and combined with Gaussian distribution and exponential decay mechanism, and combines the completion time of the tasks assigned to each machine resource to obtain the total completion time of this scheduling cycle. Based on the total completion time and constraint violations of this scheduling cycle, calculate the immediate reward for this initial task matching scheme.

[0013] Furthermore, based on the initial network parameters of the immediate reward, context encoder, and task allocator, the autonomous learning optimizer employs the proximal policy optimization PPO algorithm for task-specific updates and the Reptile algorithm for meta-policy updates to generate the optimal task matching scheme. This process includes: Define a task difficulty metric and a sample weight vector, and use a proximal strategy to optimize the PPO gradient and update the model parameters. Samples are taken from various scheduling scenarios. After each inner loop update, the Reptile algorithm is used to fuse the policy update directions learned from each scenario and fine-tune the initial parameters of the context encoder and task allocator to obtain the optimal network parameters of the context encoder and task allocator. Based on the optimal network parameters of the context encoder and task allocator, an optimal task matching scheme is generated.

[0014] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In the embodiments of this disclosure, the adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios described above, on the one hand, firstly, designs an scalable context state encoder, which generates a multi-dimensional constraint state representation based on graph neural networks that fuse scene features, task dependencies, and resource states; secondly, constructs an experience-driven task allocator, which mines cross-scene collaboration patterns through historical scheduling knowledge to generate initial allocation schemes to reduce the dimensionality of the search space; finally, develops an autonomous learning optimizer based on meta-reinforcement learning, which quickly adapts to domain differences and scale elasticity changes through multi-scene meta-policy pre-training and online lightweight fine-tuning. On the other hand, this application integrates multimodal information to construct a context-aware model, dynamically integrating physical constraints and resource states through a graph attention mechanism to improve the understanding and modeling capabilities of complex production line environments, achieving accurate state representation with spatiotemporal semantics. This application introduces an experience-driven task allocation mechanism, which can mine human-machine collaboration patterns from historical scheduling, generate high-quality initial schemes through reasoning, effectively reduce online decision-making costs, and improve scheduling efficiency. This application designs an adaptive optimizer based on meta-reinforcement learning, which adjusts model parameters in real time through meta-policies to cope with macro-level disturbances such as task reorganization and resource changes, significantly improving the system's response speed and adaptability in dynamic environments. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0016] Figure 1 The diagram illustrates the steps of an adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios in an exemplary embodiment of this disclosure. Figure 2 This diagram illustrates human-machine collaboration in a smart manufacturing scenario according to an exemplary embodiment of this disclosure. Figure 3 This illustration shows schematic diagrams of task structures of different scales in intelligent manufacturing according to exemplary embodiments of this disclosure; Figure 4 This invention illustrates the efficiency analysis results of existing solutions in terms of feasible allocation ratio and training cost in exemplary embodiments of the present disclosure; Figure 5 This diagram illustrates the overall framework of the COACH method in an exemplary embodiment of this disclosure. Figure 6 A schematic diagram of an HGA-based context encoder in an exemplary embodiment of this disclosure is shown; Figure 7 A schematic diagram of a course-based learning strategy adapter is shown in an exemplary embodiment of this disclosure; Figure 8 The embodiments of this disclosure are shown to demonstrate the performance of different methods in small-to-medium-sized and large-scale operation scenarios in terms of completion time, feasibility ratio, training time, and inference time. Detailed Implementation

[0017] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0018] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0019] This example implementation provides an adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios. (Reference) Figure 1 As shown, this adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios may include: Step S101: Construct the system model and resource performance model; Step S102: Based on the system model and resource performance model, the context encoder dynamically aggregates task dependencies and human-machine resource states according to the graph attention mechanism to generate context embedding vectors; Step S103: Input the context embedding vector into the experience-driven task allocator, use the feedforward neural network and LSTM in the experience-driven task allocator to update the state embedding and agent embedding, and generate a sequence of matching pairs of tasks and agents step by step through the cyclic task-agent matching mechanism to obtain the initial task matching scheme. Step S104: Calculate the immediate reward based on the initial task matching scheme; Step S105: Based on the initial network parameters of the immediate reward, context encoder, and task allocator, the autonomous learning optimizer uses the proximal policy optimization PPO algorithm for task-specific updates and the Reptile algorithm for meta-policy updates to generate the optimal task matching scheme.

[0020] The aforementioned adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios addresses several key aspects. First, it designs a scalable context state encoder, generating multi-dimensional constraint state representations based on graph neural networks that integrate scene features, task dependencies, and resource states. Second, it constructs an experience-driven task allocator, mining cross-scene collaboration patterns through historical scheduling knowledge to generate initial allocation schemes and reduce the dimensionality of the search space. Finally, it develops a self-learning optimizer based on meta-reinforcement learning, rapidly adapting to domain differences and scale elasticity changes through multi-scene meta-policy pre-training and online lightweight fine-tuning. Second, it integrates multimodal information to construct a context-aware model, dynamically integrating physical constraints and resource states through graph attention mechanisms to enhance understanding and modeling capabilities of complex production line environments, achieving accurate state representation with spatiotemporal semantics. The experience-driven task allocation mechanism introduced in this application can mine human-machine collaboration patterns from historical scheduling, generating high-quality initial schemes through reasoning, effectively reducing online decision-making costs and improving scheduling efficiency. The adaptive optimizer based on meta-reinforcement learning adjusts model parameters in real time through meta-policies to cope with macro-level disturbances such as task reorganization and resource changes, significantly improving the system's response speed and adaptability in dynamic environments.

[0021] Below, we will refer to Figures 1 to 8 The steps of the adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios described in this example embodiment will be explained in more detail.

[0022] In one embodiment, this application focuses on the multi-constraint coupling problem faced by human-machine collaborative task scheduling in intelligent manufacturing scenarios, systematically constructing a task scheduling model and analyzing its core challenges. First, based on typical scenarios such as automotive assembly lines and intelligent logistics sorting, it elucidates the characteristics of human-machine collaborative intelligent manufacturing tasks, including requirements such as multi-process dependence, spatial layout constraints, and human-machine dynamic capability adaptation, revealing its differences from traditional industrial scenarios. Second, it analyzes the root causes of the complexity of multi-constraint scheduling problems from three dimensions: dynamic uncertainty (such as equipment failure and sudden changes in task priority), resource heterogeneity (human skill degradation and robot functional limitations), and multi-objective conflict. Finally, by formally defining the task set, resource state space, and constraints, it constructs an optimization model with the objective of minimizing task completion time, and introduces partially observable Markov decision processes to transform the problem model.

[0023] 1. Description of a human-machine collaboration-driven intelligent manufacturing scenario: As a core practical carrier of the Industry 4.0 strategy, intelligent manufacturing is leading a profound transformation in the global manufacturing industry. This new production paradigm, through the deep integration of cutting-edge digital technologies such as the Industrial Internet of Things (IIoT), AI, digital twins, and robotics, constructs a new generation of intelligent production systems with self-sensing, self-decision-making, and self-execution capabilities. From a technical architecture perspective, intelligent manufacturing systems typically comprise three key layers: intelligent equipment at the physical layer (such as collaborative robots and AGVs), an IIoT platform at the network layer, and an intelligent decision-making system at the application layer. These three layers work together to achieve the digitization and intelligentization of processes. In such systems, the intelligent scheduling controller plays a central role as the "manufacturing brain," and its function has evolved from traditional static scheduling to dynamic optimization decision-making. On the one hand, it needs to coordinate the spatiotemporal relationship between production processes and heterogeneous human-machine resources in real time; on the other hand, it must respond quickly to uncertainties such as environmental fluctuations, sudden equipment failures, and temporary resource shortages. This intelligent production model not only improves the flexibility and responsiveness of manufacturing systems, but more importantly, it reconstructs the value chain of traditional manufacturing through a data-driven approach, providing key technical support and implementation paths for industrial transformation and upgrading. Currently, national strategies such as Germany's "Industry 4.0", the United States' "Industrial Internet", and China's "Made in China 2025" all regard intelligent scheduling systems as a core technological breakthrough point, fully demonstrating their strategic position in modern manufacturing.

[0024] Against this backdrop, human-robot collaboration (HRC) has become a key paradigm for improving the operational flexibility and production efficiency of manufacturing systems. For example... Figure 2 The diagram illustrates human-machine collaboration in a smart manufacturing scenario. Taking the assembly process of an automotive door panel as an example, this process involves a series of interconnected tasks, each requiring the participation of a specific human worker or auxiliary material handling equipment. Tasks requiring precise assembly and complex decision-making (such as sealing strip installation and hinge adjustment) are typically assigned to dexterous and adaptable human operators; while tasks involving repetitive movements and heavy object handling (such as component transfer and spot welding) are handled by robotic arms or mobile robots. This task allocation mechanism, based on the complementary advantages of human and machine resources, not only fully leverages the cognitive advantages of humans in unstructured environments but also utilizes the strengths of robots in terms of precision, power, and endurance, achieving optimal allocation of manufacturing resources.

[0025] To systematically describe the correlation characteristics between complex tasks in intelligent manufacturing scenarios, this application proposes a formal modeling method based on Directed Acyclic Graphs (DAGs). For example... Figure 3The diagram shows the task structure of different scales in intelligent manufacturing. The nodes in the DAG model represent atomic operation units in the production process (such as tightening bolts, welding components, etc.), and the edges represent the temporal dependencies between operations (such as preconditions, resource constraints, etc.). This modeling method has the following advantages: (1) It can intuitively show the topological structure of the process flow; (2) It clearly constrains the execution order of each operation; (3) It provides a computable dependency matrix for subsequent task allocation. In the actual scheduling process, the system will dynamically allocate each atomic task to the most suitable execution subject according to the complexity (such as operation accuracy requirements) and resource requirements (such as the degree of dependence on special tools). Unstructured tasks that require flexible judgment (such as quality inspection) are given priority to human operators, while structured tasks with high repetition and high precision (such as parts assembly) are allocated to industrial robots. Further analysis shows that there are significant dynamic differences in the task scale of modern intelligent manufacturing systems. These differences are mainly reflected in three dimensions: (1) the change in the order of magnitude of atomic tasks, where small tasks may contain only 5 to 10 operation units, while large tasks can contain 50 to 100; (2) the non-linear growth of task complexity, such as Figure 3 The complexity of the door assembly task is exponentially higher than that of the door panel assembly; (3) the diversity of resource requirements, large tasks often require the coordination of multiple types of resources (such as multi-skilled workers and special equipment). Based on examples of typical scenarios such as automobile manufacturing, this application proposes a three-level scale classification standard: small tasks, medium tasks and large tasks. Each scale corresponds to a specific DAG topology, which provides an important theoretical basis for the design of adaptive scheduling algorithms in subsequent chapters.

[0026] Challenges of Task Scheduling: Multi-agent Task Dispatching (MaTD) in a human-robot collaborative environment aims to allocate production tasks to heterogeneous teams composed of humans and robots, considering factors such as time constraints, spatial layout, and process dependencies. The inherent diversity of these teams in terms of capabilities, skills, and working conditions is a key factor contributing to the complexity of task allocation. Specifically, the MaTD problem faces the following three core challenges: (1) Complexity of constraints: The manufacturing process must satisfy multiple constraints inherent in the process, including spatial constraints, temporal constraints, and dependency constraints. Spatial constraints require that specific tasks must be completed by specific types of operators on a designated production line; temporal constraints specify the deadline for task execution; and dependency constraints define the execution order between consecutive tasks. These constraints are intertwined, forming a high-dimensional optimization space, which significantly increases the complexity of the scheduling problem. For example, in the automotive assembly scenario, door installation (spatial constraint) must be carried out after the painting process (temporal constraint) is completed, and must be performed by human technicians with specific skills (dependency constraint).

[0027] (2) Resource and Task Heterogeneity: Heterogeneity is reflected in two dimensions: On the one hand, there are significant differences between human workers and robots in terms of proficiency, operating costs, and error rates. For example, humans perform well in complex decision-making tasks but are susceptible to fatigue, while robots are highly accurate in repetitive tasks but lack flexibility. On the other hand, the tasks themselves also differ significantly in terms of scale and complexity (e.g., Figure 3 (As shown). This dual heterogeneity makes traditional scheduling strategies difficult to apply directly, necessitating the development of new adaptive allocation algorithms.

[0028] (3) Dynamic uncertainty: There are multiple uncertainties in the operation of manufacturing systems: human workers may experience efficiency fluctuations due to fatigue accumulation, and such changes are real-time and unpredictable; at the same time, adjustments to the manufacturing targets and budgets of the production line may lead to sudden changes in the scale of tasks. For example, when orders surge, the workload of an electronic assembly line may suddenly increase by more than 50%. These dynamic factors require the system to have real-time adaptive capabilities and be able to adjust scheduling strategies without interrupting the production process.

[0029] To thoroughly evaluate the practical performance of existing task scheduling methods in modern intelligent manufacturing production lines, this application designed motivational experiments covering three job sizes. Four benchmark methods were selected for comparative analysis, including the heuristic rule-based EDF scheduler and three learning-based schedulers. To comprehensively assess the challenges faced by the MaTD benchmark methods, this application adopted two key metrics: 1) Feasible Ratio (FR), defined as the proportion of jobs successfully scheduled, which is the core basis for measuring the effectiveness of the scheduling scheme. The FR value is affected by multiple factors such as job size, constraint complexity, and the amount of training data; 2) Training Cost (TC), quantified by model training time, reflecting both the overhead of algorithm retraining and computational complexity characteristics, providing an important reference for evaluating the scalability of the scheme. Based on the above-defined evaluation metrics, the experimental results are as follows: Figure 4The figure shows the efficiency analysis results of the existing scheme in terms of feasible allocation ratio and training cost.

[0030] from Figure 4 As shown in the left figure, when the task size shifts from small to large scale, the average performance recovery (FR) of all existing solutions drops sharply (average 14.9%). These results indicate that existing methods are ill-suited to the scheduling requirements of large-scale manufacturing operations. More importantly, in scenarios where production lines dynamically adjust their size, these methods exhibit significant performance fluctuations and fail to provide stable job completion guarantees. Furthermore, further analysis... Figure 4 As shown in the training curve of the HybridNet method on a small-scale job in the right figure, the model requires up to 100 hours of training time on an NVIDIA RTX 3090 GPU to achieve basic convergence. This result illustrates that when faced with dynamic runtime factors (such as fluctuations in operator performance and changes in job size), learning-based methods lack engineering practicality due to the high cost of retraining. These findings provide a key motivation for developing a novel adaptive scheduling algorithm in this application.

[0031] In step S101, a system model and a resource performance model are constructed.

[0032] Specifically, 1. System Model and Problem Modeling (1) System Model This application explores the MaTD problem with spatiotemporal constraints in heterogeneous human-machine collaborative work environments. An experimental system (i.e., a system model) is constructed based on a multi-round coordination (MRC) environment. This system can simulate the dynamic collaborative behavior of heterogeneous intelligent agents in a multi-round manufacturing process, taking into account the changes in task execution capabilities of human operators based on historical round performance. Figure 2 Taking the car door assembly process as an example: each round represents a complete door assembly process. The feasibility assessment of the scheduling scheme requires comprehensive consideration of all task-agent pairings in the current round. After all tasks are reasonably allocated to human and machine resources, the system will evaluate the feasibility of the scheduling scheme based on strict constraints and calculate the total task completion time under feasible schemes. It should be emphasized that the task completion time of human operators in a new round will be affected by their historical task execution records. This dynamic capability modeling significantly improves the realism of the system simulation.

[0033] The set of heterogeneous human-machine resources in the system is defined as ,in, Represents the set of human operators. The number of available manpower; Represents a set of robot agents. The number of robots acting as agents. Multiple batches of operations exist in the manufacturing environment. Each assignment It contains a set of several tasks with dependencies. And subject to the following constraints: 1) Time Constraints: Let... For the task The deadline must be met, and the completion time must satisfy the following: .

[0034] 2) Spatial Constraints: Definition To ensure a safe distance for collaborative operations, the following conditions must be met when agents execute adjacent subtasks. For example, welding robots and quality inspectors must maintain a minimum distance of 1.5 meters.

[0035] 3) Process constraints: Use a directed acyclic graph. Represents the predecessor relationship between tasks, edge weight Indicates task and The minimum waiting interval. Taking the riveting process as an example, it must wait for the preceding drilling process to complete and for the cooling time to pass. It can only be started after that.

[0036] (2) Markov decision process modeling of task scheduling problem Based on the above problem description and constraints, this application models the MaTD problem as a heterogeneous graph structure. In this graph, the set of nodes It contains two types of entities: task nodes Represents all tasks and processes in a specific production process; intelligent agent nodes. This represents the collection of various resources within a human-machine collaborative team. (Edge set) This includes three types of topological relationships: inter-agent cooperation relationships. Task-agent allocation relationship and the temporal relationship between tasks This graph-based modeling method can systematically represent the complex interactions between multiple elements in intelligent manufacturing scenarios. The optimization objective remains minimizing the maximum completion time (Makespan), i.e., the maximum value of the completion time of all tasks. Within the graph model framework, the objective function... It can be formalized as: (1) In the formula, Indicates task The completion time is determined by the scheduling scheme. This optimization objective ensures that the scheduling scheme minimizes the overall execution cycle of all tasks in the graph structure. To ensure the uniqueness of task allocation, the following constraints are introduced: (2) In the formula, Let be a binary decision variable, representing the task. Whether to assign to an agent ; Represents intelligent agents Execute the task The time required. This constraint ensures that each task is assigned to only one human or robotic agent, and the duration of the task is determined by the capabilities of the executing agent.

[0037] Furthermore, timing constraints are modeled using the connection edges between tasks to ensure that subsequent tasks can only start after the predecessor task is completed. Task deadlines are also considered. and waiting time Establish the following constraint group: (3) In the formula, This indicates the scheduling start time. To handle process dependencies and spatial conflicts in the production flow, additional priority constraints and spatial mutual exclusion constraints are introduced: (4) In the formula, As an indicator variable for priority relations, This represents a set of task pairs with process dependencies. This constraint ensures that: ① Tasks on the same agent are executed sequentially; ② Tasks with process dependencies are executed strictly in process order; ③ Only one heterogeneous agent is allowed to operate at each spatial location at a time.

[0038] Based on the aforementioned goal optimization modeling, to effectively address time-varying and unobservable human factors during task scheduling (such as dynamic changes in operator cognitive load), this application transforms the modeled MaTD goal optimization problem into a Partially Observable Markov Decision Process (POMDP). This POMDP model utilizes a six-tuple... The formal definition is as follows: 1) State Space It includes the dynamic state of all intelligent agents in a human-machine collaborative environment. The state of human intelligent agents involves implicit indicators such as real-time workload and skill level decay coefficient, while the state of machine intelligent agents covers explicit parameters such as device location and processing time.

[0039] 2) Action Space The task-agent pairing set generated by the scheduling decision. In the... In round-robin scheduling, the set of actions can be represented as Each pair corresponds to a specific task allocation decision.

[0040] 3) State transition probability : Describe the environmental state arrive The probability distribution of the transfer process is difficult to model accurately due to uncertainties such as equipment operating noise and fluctuations in human performance.

[0041] 4) Reward Function Based on the objective function Instant rewards for design It is used to quantify the quality of scheduling decisions, and its calculation mechanism will be detailed below.

[0042] 5) Observation space : direct observations by machine intelligence agents (Such as sensor data) and estimates by human intelligent agents (Such as performance forecasting) together constitute, that is .

[0043] 6) Observation function This is achieved through a human dynamic performance tracker, which performs probabilistic inferences about human states.

[0044] In the MRC environment During a round-robin scheduling cycle, the set of output actions is determined. It contains a sequence of task-agent pairings to be executed sequentially. This set can be divided into two subsets: a feasible subset that satisfies the strict constraints of the current task. and infeasible subsets that violate constraints. ,Right now Reward function The calculation takes into account the contributions of both subsets: (5) Among them, the infeasibility coefficient This design adjusts the intensity of penalties for constraint violations. It guides agent behavior through a dual mechanism: firstly, the reward for feasible allocation is negatively correlated with the total completion time, meaning a shorter total completion time corresponds to a higher reward; secondly, the penalty for infeasible allocation is adjusted through worst-case estimation (assigning unfinished tasks to the agent with the longest processing time) and coefficient adjustment, thus progressively suppressing constraint violations. This reward mechanism enables the system to achieve a dynamic balance between exploring efficient scheduling strategies and adhering to job constraints.

[0045] 2. Resource Model and Dynamic Human Performance Tracker In the intelligent manufacturing human-robot collaborative MRC environment, human operators and industrial robots are the core task executors, and their accurate performance modeling is the foundation for task scheduling optimization. This section constructs an environmentally adaptable bimodal resource performance representation system by dynamically tracking the agent's state and task completion rate, updating the parameters of the heterogeneous resource model after each scheduling cycle. Specifically, the robot performance model runs stably according to a preset program, while the human model needs to consider the dynamic coupling relationship between learning effects and fatigue accumulation. Iterative optimization is performed using historical task-agent matching data to achieve online evolution of the resource model.

[0046] (1) Deterministic machine performance model Based on the high repeatability of industrial robot execution processes, their task processing time can be characterized by a deterministic model. This application uses a uniform distribution to generate initial task execution time parameters: for the robot... Its mission The time consumption follows a uniform distribution, which aligns with the time consumption characteristics of standardized work processes in industrial scenarios (e.g., a welding robot's single-point welding time is 2.8 ± 0.3 minutes). Robot operational capability coverage. This reflects that current industrial robots are limited by programmed operation modes and can only handle specific processes with a high degree of structure.

[0047] (2) Human learning model coupled with fatigue effect The dynamic performance evolution of human operators is a key factor affecting the effectiveness of MRC systems. It is known that humans exhibit a U-shaped learning curve in repetitive tasks, with initial efficiency gains followed by efficiency decline. This is the result of a dynamic game between the learning effect and physical fatigue. This application proposes a coupled model based on improvements from existing literature: (6) In the formula, The number of times the task is repeated. The learning rate parameter represents the rate at which skill proficiency improves. The fatigue accumulation coefficient reflects the degree to which work intensity reduces efficiency; the model achieves accurate prediction through a two-stage dynamic adjustment mechanism: when At that time, the learning effect was dominant. It declines exponentially; when At that time, fatigue effect was dominant. It then transitions to a linear upward trend. This model provides a dynamic capability assessment benchmark for the scheduling system, supporting the optimization of task-human resource adaptability.

[0048] (3) Human dynamic performance model tracker Human performance modeling research indicates that learning effects and fatigue accumulation are key factors affecting human work efficiency in repetitive tasks within manufacturing environments. However, the complexity of real-world industrial scenarios means that other potential human factors (such as pressure from urgent orders and interference from multitasking) can also significantly impact work efficiency. Therefore, task allocation systems, when making scheduling decisions, need to quantify the dynamic impact of known explicit factors and consider the combined effects of unobservable implicit factors. This application proposes a dynamic human performance tracking mechanism that constructs a time-series-based forward performance prediction model by analyzing historical task-person pairing data in an MRC environment. This mechanism employs black-box modeling methods (such as Hidden Markov Models) to dynamically evaluate human performance. Specifically, during multiple rounds of task execution by human operators, the system continuously records their task time data and, based on the observation function set... The subsequent time consumption of similar tasks is gradually adjusted.

[0049] The core mechanism of this dynamic performance tracker lies in treating the actual task time as the mean of a Gaussian distribution and simulating the noise reduction effect brought about by increased task proficiency using an exponential decay function. Its mathematical model can be expressed as: (7) In the formula, Indicates the number of repetitions of a specific task. and The empirical attenuation coefficient is... This is the estimated task time corrected for the tracker. For example, in car assembly, the time it takes for a worker to perform a dashboard installation task for the first time. At the minute mark, the system will record this data and generate... Initial noise for minutes; after three repetitions (i.e., The noise level will drop to The minutes reflect the increased efficiency and stability resulting from improved worker skills.

[0050] In steps S102 to S105, based on the system model and resource performance model, the context encoder dynamically aggregates task dependencies and human-machine resource states according to the graph attention mechanism to generate a context embedding vector. The context embedding vector is input to the experience-driven task allocator, which uses the feedforward neural network and LSTM in the experience-driven task allocator to update the state embedding and agent embedding. The task-agent matching mechanism is used to gradually generate a sequence of matching pairs between tasks and agents to obtain an initial task matching scheme. Based on the initial task matching scheme, the immediate reward is calculated. Based on the immediate reward, the initial network parameters of the context encoder and the task allocator, the autonomous learning optimizer uses the proximal policy optimization PPO for task-specific updates and the Reptile algorithm for meta-policy updates to generate the optimal task matching scheme.

[0051] Specifically, a constraint-aware adaptive human-computer collaborative task scheduling algorithm To address the challenge of task scheduling in complex scenarios characterized by multiple constraints and intertwined dynamic uncertainties, this application proposes a constraint-aware COACH method. Its core architecture comprises four key components: First, the overall algorithm framework achieves closed-loop optimization of the entire task scheduling process through heterogeneous information fusion and decision-making mechanisms. Second, a scalable context encoder dynamically aggregates multi-source heterogeneous data, such as task dependencies and human-machine resource states, based on a graph attention mechanism, generating feature representations with spatiotemporal awareness. Third, an experience-driven task allocator mines human-machine collaboration patterns through historical scheduling knowledge to generate candidate allocation schemes, reducing real-time decision complexity. Finally, an uncertainty-aware optimizer, combined with a meta-reinforcement learning framework, adaptively adjusts policy network parameters to cope with dynamic disturbances such as task surges and resource shortages. These components form a collaborative interaction mechanism, creating a decision-making closed loop of "feature extraction - experience guidance - dynamic optimization," significantly improving scheduling efficiency and system robustness in complex manufacturing scenarios while ensuring multiple constraints are satisfied.

[0052] (1) Overview of the overall algorithm framework like Figure 5 The diagram shows the overall framework of the COACH method. The COACH framework employs a hierarchical collaborative architecture to achieve adaptive task scheduling and allocation in intelligent manufacturing scenarios. Its core consists of four key modules: 1) Production Line Environment Modeling Module: This module provides a formal description of multi-constraint tasks and heterogeneous resources in intelligent manufacturing scenarios. The task set includes manufacturing processes with strict spatiotemporal constraints (such as process deadlines and waiting times) (e.g., the chained task of "engine installation → body welding → circuit testing" in automobile assembly), while heterogeneous resources encompass industrial robots and human operators with differentiated skills. 2) Production Line State Encoding Module: A basic constraint model for task dependencies is constructed based on a Simple Temporal Network (STN). By introducing heterogeneous agent nodes and their attribute weights, a complete spatiotemporal heterogeneous graph structure is formed. On this basis, a Heterogeneous Graph Attention Network (HGA) is used to extract contextual embedding features specific to each job instance. For example, the collaborative requirements of an assembly task for a "high-precision welding robot" and an "experienced quality inspector" will be used as input to the scheduler. 3) Experience-driven task allocation module: This module dynamically updates feature embeddings through a recurrent neural network module with memory capabilities and generates candidate "task-agent" matching pairs. Its core mechanism includes two levels: first, the generation of an immediate matching strategy based on attention weights; and second, the retention of long-term effective scheduling strategies through reinforcement learning. 4) Uncertainty-Aware Policy Adaptation Module: This module adjusts the scheduler's behavior through a meta-reinforcement learning framework, guiding the system to quickly explore the optimal task allocation scheme under dynamic perturbations. Specifically, when encountering equipment failure or sudden changes in task priority, this module adjusts the policy network parameters in real time based on contextual uncertainty metrics, ensuring that the scheduling scheme simultaneously meets the requirements of robustness and agility. All modules collaborate through a closed-loop chain of "environmental awareness → feature extraction → experience guidance → dynamic optimization," providing adaptive learning and dynamic scheduling capabilities for complex scenarios.

[0053] (2) Extensible context encoder 1) Construction of heterogeneous graphs based on STN In response to Human Operators and A multi-agent collaborative team composed of mobile robots needs to work together to complete tasks including In this application, a heterogeneous graph is constructed using a Simple Temporal Network (STN) to capture the implicit relationships between nodes. The advantage of STN lies in its ability to efficiently verify the consistency of temporal constraints (such as upper / lower bound constraints) with polynomial time complexity, making it particularly suitable for scenarios involving multi-agent spatiotemporal collaboration. When task scheduling involves multiple agents and physical constraints, potential disjunctive variables need to be considered, such as constraints that a single agent cannot execute multiple tasks in parallel or that a single task position cannot be occupied by multiple agents simultaneously. Given the graph structure characteristics of STN, Graph Neural Networks (GNNs) are naturally suitable for reasoning and modeling such relationships. Based on existing work, this application expands the representational dimension of the graph to simultaneously characterize spatiotemporal disjunctive constraints and directed dependencies in multi-agent collaboration.

[0054] In the reconstructed heterogeneous graph model, task type nodes are accessed through... Edge type connections are used to characterize the time constraints of STN encoding; two new types of edges are further introduced: Indicates the time required for the subject to perform the task and Indicates the main time window occupied by the task; and defines... Edge types are associated with tasks and their initial physical locations. To facilitate information exchange among multiple agents, all human-computer agents communicate through... Edge-type interconnection. Furthermore, through... and Edge types connect the state summary node to the task node and the main node respectively, forming a global state awareness path. A heterogeneous graph attention network (HGA) is used to learn the embedding of the relationship graph, specifically through a triplet format. Define the association paths between nodes. This architecture dynamically aggregates multimodal edge features through a multi-head attention mechanism, for example, in automotive assembly scenarios. The time consumption characteristics of a robotic arm tightening bolts can be encoded, while The edges reflect the real-time distance information between the AGV and the assembly station, ultimately generating a contextual embedding representation with both spatiotemporal semantics, providing interpretable feature support for downstream task allocation.

[0055] 2) Adaptive context coding The feature update process of heterogeneous graph attention networks consists of a two-step collaborative mechanism: edge-type-based message passing and node-type-based feature aggregation. For example... Figure 6 As shown, this is a context encoder based on HGA; this mechanism achieves the representation and relationship extraction of multi-constraint task features and heterogeneous resource features in the scheduling environment by dynamically perceiving contextual relationships.

[0056] ① Message passing phase: Each edge type (such as task-resource dependency edge, human-computer collaboration edge) is associated with an independent weight matrix. (in Input the feature dimension to the source node. The feature dimension is output for the target node, and the source node features are linearly transformed and stored in the target node's message mailbox. This design enables different relationship types (such as process sequence constraints and spatial reachability constraints) to learn independent evolutionary features through differentiated parameters.

[0057] ② Feature Aggregation Stage: The HGA layer performs multi-granularity fusion of message mailbox content based on an attention mechanism. Specifically, it first calculates the attention coefficients related to edge types. : Through learnable weight vectors Features of source nodes Target node features and edge attributes After performing splicing projection (Equation 8) and then LeakyReLU activation and softmax normalization, attention weights that are relevant to features and decoupled from the graph structure are generated: (8) Next, the final feature vector of the node is generated by weighted summation and ReLU activation (Equation 9). Wherein, Represents a node The set of neighbors (including self-loop connections). and The linear transformation results of neighbor node features and edge features are represented respectively. This process dynamically adjusts the contribution of multiple constraints through attention weights: (9) To enhance model robustness, this application employs a multi-head attention mechanism, executing the HGA process independently multiple times and concatenating the feature vectors output from each iteration to form a higher-order representation (the final layer uses mean aggregation). It is important to emphasize that the above method captures the multi-dimensional semantics of constraint relationships (such as temporal urgency and spatial conflict intensity) through multi-head parallel computation, ultimately generating context embedding vectors for task nodes, heterogeneous agent nodes, and global state nodes.

[0058] (3) Experience-driven task assigner 1) Embedded update mechanism The context encoder generates task embeddings, agent embeddings, and global state summary embeddings by fusing agent heterogeneity, spatiotemporal constraints, and inter-task dependencies. If only the HGA encoder is relied upon to infer the complete task-agent allocation scheme, the next allocation relationship must be determined through iterative interaction between the current task-agent pair and the multi-resource-constrained scheduling environment. This leads to a surge in computational complexity and significant time consumption in the scheduling process. Therefore, this application employs an experience-driven task allocator, which reconstructs environmental information through a partial task-agent allocation relationship propagation mechanism, eliminating dependence on the forward HGA network and significantly reducing the computational complexity of scheduling scheme generation. The core process of this allocator is as follows: first, the task embedding, state embedding, and agent embedding output by the encoder are used as input; then, based on the cyclic update mechanism of LSTM (Long Short-Term Memory), task-agent matching pairs are gradually generated.

[0059] Specifically, after each "task-agent" match is generated, the system updates the state embedding and agent embedding respectively using the state LSTM and agent LSTM models, achieving next-stage state prediction and potential matching pair generation without environmental interaction. These two LSTM models continuously maintain a memory unit of "hidden state + cell state" accumulated from historical allocation steps and perform prediction for the next time step based on the updated embedding. The update process follows this formula: (10) In the formula, and Representing time steps Updated hidden state and cell state. This serves as the input feature for the current step. Through this iterative update mechanism, the model can effectively capture temporal dependency features in the task-agent interaction process. For example, in an automotive assembly scenario, after a robot at a certain workstation completes the bolt tightening task, the LSTM will predict the types of parts most likely to be transported in the next stage based on the updated state embedding, thereby planning resources in advance. This progressive allocation strategy based on historical experience reduces computational complexity from... Down to .

[0060] 2) Looping task-agent matching mechanism The embedding update process generates task / agent-related embedding vectors at each time step, and the scheduler uses these vectors to make task allocation decisions for the current time step. This application will implement a scheduling strategy. The process is decomposed into independent agent selectors and task selectors, and its mathematical formal definition is as follows: (11) This decomposition method effectively captures the conditional dependency of task assignment decisions: the task assignment result depends on the current state of the selected agent. The agent selector dynamically filters agents to be matched based on state information and agent features. Specifically, the concatenated state-agent embedding vector is input into a feedforward neural network. Calculate the likelihood probability that each agent will be selected as the next task executor: (12) in, The function transforms the original predictions into a probability distribution. The task selector, after selecting an agent, then bases the selection on the task embedding. Intelligent agent embedding and global state embedding The system uses a joint representation to assess the fit between the task to be assigned and the agent. Then, it updates only the embedding vector of the selected agent, dynamically adjusting the embedding space to reflect changes in the agent's workload. For example, in automobile assembly, when a worker continuously handles highly complex tasks, the "fatigue" dimension value in their embedding vector will increase significantly, automatically reducing their probability of being selected for subsequent task assignments.

[0061] (4) Learning adapter for uncertainty perception 1) Automatic Course Learning The multi-scale variability and uncertainty of pre-sampled task instances pose significant challenges to traditional sampling training strategies. In such scenarios, random sampling tasks often exhibit a polarization in difficulty: overly difficult tasks can lead to agent training failure, while overly easy tasks fail to effectively expand policy boundaries. To address this contradiction, this application introduces an automatic task sampling mechanism based on capability boundary detection, prioritizing "heuristic" tasks (i.e., challenging tasks that can be completed with appropriate guidance) that fall within the scheduler's current capability threshold. This mechanism is inspired by the Zone of Proximal Development (ZPD) theory in human cognitive development, achieving progressive learning by dynamically adjusting the task difficulty curve. For example, in medical image analysis, the system can autonomously identify the accuracy threshold of the current model for early lung nodule detection, and then select difficult cases with a false negative rate of around 20% as training samples.

[0062] ① Self-paced filtering: To address the model adaptation problem for multi-scale (small / medium / large-scale) tasks, this paper proposes a dynamic difficulty-aware self-paced filtering mechanism. Given an unlabeled dataset... , Build optimization goals: ,in Characterizing task difficulty measurement For the sample weight vector, To suppress the regularization term of weight bias (this application employs entropy regularization) ), These are the balance coefficients. They are determined by defining a weighting function. ( (Using a sigmoid activation function), the model parameters are updated according to the gradient direction: This mechanism allows the policy adapter to gradually focus on complex tasks during training. For example... Figure 6 As shown, in automobile assembly, the system initially learns basic tasks such as bolt tightening. As training progresses, the sampling weight of higher-order tasks such as complex debugging is gradually increased, ultimately improving the task success rate.

[0063] 2) Robust training based on meta-reinforcement learning To improve the robustness of the scheduler in dynamic and uncertain environments, this application proposes a policy adaptation mechanism based on meta-reinforcement learning (Meta-RL). This method employs a two-layer optimization architecture, such as... Figure 7 As shown, this is a policy adapter based on course learning: the inner layer achieves task-specific policy learning through proximal policy optimization (PPO), and the outer layer achieves cross-task meta-policy optimization based on the Reptile algorithm, thereby ensuring the policy's ability to adapt quickly in unseen task scenarios.

[0064] ① Task-Specific Update Based on PPO: The inner loop performs policy gradient optimization using the PPO algorithm for each specific task environment. Given a dynamic model of the task environment, the objective function for updating the policy parameters is defined as: (13) In the formula, This represents the ratio of the probability of actions performed by the current strategy to that of historical strategies. This is the estimated value of the dominance function. This is the shear coefficient (usually taken as 0.2). Parameter updates follow... ,in This is a task-specific learning rate. This mechanism effectively avoids the training instability problem in traditional policy gradient methods by limiting the policy update magnitude, making it particularly suitable for high-variance manufacturing scenarios.

[0065] ② Meta-policy update based on Reptile: The outer loop adopts the Reptile meta-learning framework to optimize the initial policy parameters through cross-task collaborative optimization. Its core objective is to extract transferable meta-knowledge from diverse task distributions. The parameter update rule is defined as follows: .in These are the policy parameters of the inner PPO after adaptation in a single-task scenario. The meta-learning rate controls the strength of cross-task knowledge fusion. (Difference term) This reflects the direction of task-specific adaptation in correcting the initial policy. By continuously accumulating such correction vectors in multi-task scenarios, the initial policy parameters gradually shift towards the "shared optimal solution" direction of the task distribution. This process is equivalent to performing implicit gradient descent in the expected gradient direction of the task distribution, enabling the policy parameters to quickly adapt to unknown tasks. To improve the generalization ability of the meta-policy, this application adopts a phased optimization strategy: firstly, a meta-policy base model covering typical failure modes is pre-trained in a simulation environment in an offline phase; then, in the online phase, lightweight fine-tuning is used to achieve rapid adaptation to new tasks. Furthermore, Reptile simplifies second-order gradient calculation, reducing the time consumption of single-dimensional updates, making it more suitable for manufacturing scenarios with stringent real-time requirements.

[0066] In a specific embodiment, as a foundational step in experimental verification, this experiment constructed a Multi-Constraint Task Scheduling (MaTD) benchmark dataset for intelligent manufacturing. First, task instances with differentiated constraint characteristics were created using a parameterized generator: approximately 25% of each task were subject to time window constraints (range [1, 5N], where N is the total number of tasks), and another 25% of tasks needed to satisfy process dependency constraints (inter-task waiting times were uniformly sampled within [1, 10]), with task durations uniformly set to 10-100 time units. To comprehensively evaluate algorithm performance, the dataset was divided into three levels based on task size: small task sets (9-11 tasks, simulating simple assembly scenarios at a single workstation), medium task sets (18-22 tasks, corresponding to modular assembly scenarios at multiple workstations), and large task sets (38-42 tasks, mapping complex scenarios such as vehicle integration). A total of 6,000 task instances were generated (each instance containing 2-4 human-machine collaborative agents), with 200 instances from each size retained as independent test sets to verify the model's generalization ability.

[0067] Regarding the experimental environment configuration, the multi-round scheduling MRC environment is set as follows: infeasible scheduling penalty coefficient. Set to 2.0, the scheduling rounds are fixed at 4 rounds to balance the optimization goals of exploration and utilization. In order to truly reflect the dynamic characteristics of human-machine collaboration scenarios, this paper introduces Gaussian noise perturbation into the human intelligent agent learning model. Based on the parameter natural distribution assumption of formula (7), the normal distribution with a standard deviation of 0.15 is used to simulate the skill fluctuations of human operators (such as efficiency decline caused by fatigue, skill proficiency differences, etc.).

[0068] To comprehensively evaluate the performance advantages of the proposed method, this experiment selects five representative baseline methods for comparative analysis, covering traditional heuristic algorithms, evolutionary algorithms, deep reinforcement learning methods, and online learning paradigms: 1) Earliest Deadline First (EDF): A classic scheduling method based on static priority rules, it selects the task with the earliest deadline from the list of pending tasks and assigns it to the first available agent. This method has low computational overhead in simple scenarios, but it is difficult to handle complex and coupled scenarios. 2) Genetic Algorithm (GA): Optimizes the initial EDF scheduling scheme through iterative mutation (including task allocation swapping and execution order exchange). Each generation selects the 10 schemes with the best feasibility and total completion time from 90 baseline schemes, and generates a new population by combining 10 task allocation mutations and 10 task order mutations. It has strong global search capabilities but high computational cost. 3) HybridNet: An advanced method based on reinforcement learning, it uses a joint architecture of graph attention encoder and recurrent scheduling propagator to quickly generate scheduling schemes without environmental interaction between task-agent pairing and selection. Its characteristics are suitable for medium-scale scheduling scenarios, but it faces the risk of memory explosion in ultra-large-scale problems. 4) Hierarchical Reinforcement Learning (AeHRL): This method employs a hierarchical reinforcement learning framework to decompose the complex long-term task assignment problem into structured sub-problems, capturing dependencies through a hierarchical attention mechanism across human-machine-task attributes. This approach performs well in process-dependent scenarios but is less adaptable to dynamic perturbations. 5) Linear Confidence Upper Bound (LinUCB): A context-based online learning method that estimates expected reward using a linear model and ridge regression, dynamically adjusting action selection to balance exploration and exploitation. Its lightweight nature makes it suitable for real-time scheduling, but the model's expressive power is limited by the linear assumption.

[0069] Through multidimensional comparisons of the aforementioned baselines, the experiments will focus on verifying the breakthrough advantages of the COACH method in terms of efficiency in handling complex constraints, dynamic adaptability, and computational scalability. In particular, HybridNet and AeHRL are selected as representative methods of deep reinforcement learning to effectively evaluate the performance gains of meta-learning mechanisms; while the comparison with LinUCB highlights the necessity of neural network feature extraction capabilities.

[0070] like Figure 8 The results (a)-(d) in the figure compare the performance of different methods in terms of completion time (makespan), feasible ratio, training time, and inference time in small-to-medium-sized and large-scale job scenarios. Among them, (a) shows the performance of different methods in terms of completion time in small-to-medium-sized and large-scale job scenarios; (b) shows the performance of different methods in terms of feasible ratio in small-to-medium-sized and large-scale job scenarios; (c) shows the performance of different methods in terms of training time in small-to-medium-sized and large-scale job scenarios; and (d) shows the performance of different methods in terms of training time and inference time in small-to-medium-sized and large-scale job scenarios.

[0071] like Figure 8As shown in (a) and (b), experimental results demonstrate that precise methods (such as EDF and genetic algorithms) perform the worst on both metrics, and their performance drops sharply as the task size increases. When the task size increases from 10 to 40, the feasibility rate of the genetic algorithm decreases from 68.3% to 12.4%, due to the exponential expansion of the solution space and the nonlinear increase in task complexity leading to a decrease in the density of feasible solutions. In contrast, reinforcement learning-based methods (training batch size labeled "#Batch=8" indicates a training batch size of 8) exhibit significant advantages: the median feasibility rates of HybridNet and AeHRL reach 81.5% and 76.2%, respectively. HybridNet maintains suboptimal performance across all scale scenarios, but its completion time is on average 23.6% longer than the method proposed in this application (red bars). These results illustrate the technical potential of reinforcement learning-based scheduling methods with sufficient training data.

[0072] The online scheduling method LinUCB performs exceptionally well in small- to medium-scale scenarios, but its advantage diminishes as the scale increases. Its feasibility drops significantly when the task load increases to a large scale. This is attributed to the difficulty of its linear model in effectively balancing exploration and exploitation: in complex task scheduling decision-making scenarios, the limitations of linear representation lead to an increased probability of suboptimal action selection. In stark contrast, the method proposed in this application maintains optimal performance across different scale scenarios through dynamic human factor performance modeling and a meta-policy adaptation mechanism. Specifically, the allocation feasibility of the proposed method is improved by 4.60%, 14.02%, and 30.75% compared to HybridNet in small- to medium-sized, and large-scale task scenarios, respectively. Furthermore, it should be noted that the completion time of the proposed method is slightly higher than that of the online decision-making LinUCB method in small- to medium-scale scenarios. This is related to its high feasibility characteristic; the scheduling and execution of more feasible tasks leads to a moderate increase in average completion time, but this characteristic translates into a significant advantage in large-scale scenarios. This scale robustness verifies the synergistic effect of the constraint-aware mechanism and the meta-reinforcement learning framework in the proposed method. Overall, the method in this application improves task completion time and allocation feasibility by 13.9% and 27.4% respectively, verifying the high adaptability of the method in complex scenarios.

[0073] To quantify the computational efficiency advantage of the method in this application, experiments were conducted to compare and analyze the computational overhead of baseline methods under different task scales using a dual-dimensional index of training time and inference time. The results are as follows: Figure 8As shown in (c) and (d), methods such as EDF, genetic algorithms, and LinUCB, which do not require model training, were not included in the training time comparison. As can be seen from Figure (c), compared to HybridNet and AeHRL, COACH exhibits the lowest single-task instance training time across all three task scales: in small, medium, and large scenarios, its training times are 18.3 minutes, 42.7 minutes, and 96.5 minutes, respectively, representing reductions of 23.6%, 34.1%, and 48.9% compared to the next best methods. This advantage becomes increasingly significant as the task scale increases, demonstrating the effectiveness of COACH's design in reducing computational load through meta-knowledge transfer. The inference time comparison in Figure (d) further verifies the real-time advantage of this method: the single-task inference time of this application is 2.1 seconds, 2.5 seconds, and 5.5 seconds across the three scales, showing a significant advantage over reinforcement learning-based HybridNet and AeHRL. In large-scale operations, COACH achieves a 55.6% improvement in inference efficiency compared to HybridNet. This is attributed to its innovative design of an experience-driven task allocator, which iteratively infers the optimal allocation scheme through an RNN model, avoiding the complex real-time graph encoding computations of traditional methods. In contrast, the LinUCB method suffers from inference time that increases exponentially with scale due to the difficulty of its linear model in handling high-dimensional task characteristics, severely limiting its practical deployment feasibility. Experimental results demonstrate that COACH, through its hierarchical optimized architecture, achieves a significant improvement in computational efficiency while maintaining scheduling quality, making it suitable for intelligent manufacturing scenarios requiring rapid response.

[0074] The aforementioned adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios addresses several key aspects. First, it designs a scalable context state encoder, generating multi-dimensional constraint state representations based on graph neural networks that integrate scene features, task dependencies, and resource states. Second, it constructs an experience-driven task allocator, mining cross-scene collaboration patterns through historical scheduling knowledge to generate initial allocation schemes and reduce the dimensionality of the search space. Finally, it develops a self-learning optimizer based on meta-reinforcement learning, rapidly adapting to domain differences and scale elasticity changes through multi-scene meta-policy pre-training and online lightweight fine-tuning. Second, it integrates multimodal information to construct a context-aware model, dynamically integrating physical constraints and resource states through graph attention mechanisms to enhance understanding and modeling capabilities of complex production line environments, achieving accurate state representation with spatiotemporal semantics. The experience-driven task allocation mechanism introduced in this application can mine human-machine collaboration patterns from historical scheduling, generating high-quality initial schemes through reasoning, effectively reducing online decision-making costs and improving scheduling efficiency. The adaptive optimizer based on meta-reinforcement learning adjusts model parameters in real time through meta-policies to cope with macro-level disturbances such as task reorganization and resource changes, significantly improving the system's response speed and adaptability in dynamic environments.

[0075] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0076] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. An adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios, characterized in that, include: Construct system models and resource performance models; Based on the system model and resource performance model, the context encoder dynamically aggregates task dependencies and human-machine resource states according to the graph attention mechanism to generate context embedding vectors; The context embedding vector is input into the experience-driven task allocator. The feedforward neural network and LSTM in the experience-driven task allocator are used to update the state embedding and agent embedding. The task-agent matching sequence is generated step by step through the cyclic task-agent matching mechanism to obtain the initial task matching scheme. Calculate the immediate reward based on the initial task matching scheme; Based on the initial network parameters of the immediate reward, context encoder, and task allocator, the autonomous learning optimizer uses the proximal policy optimization PPO algorithm for task-specific updates and the Reptile algorithm for meta-policy updates to generate the optimal task matching scheme.

2. The adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios according to claim 1, characterized in that, The steps involved in constructing the system model and resource performance model include: Define the set of heterogeneous human-machine resources and constraints; wherein, the set of heterogeneous human-machine resources includes the set of human operators and the set of robot agents, and the constraints include time constraints, space constraints and process constraints; The human-machine collaborative task scheduling problem is modeled as a partially observable Markov decision process, namely the POMDP model; the POMDP model includes a state space, action space, state transition probabilities, reward function, observation space, and observation function. A resource performance model is constructed based on a deterministic machine performance model, a fatigue-effect coupled human learning model, and a human dynamic performance model tracker.

3. The adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios according to claim 2, characterized in that, The expression for the reward function is: in, To satisfy the current job's strict constraints on the feasible subset, An infeasible subset that violates constraints. For the first i One task, For the first j An intelligent agent. The infeasibility coefficient.

4. The adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios according to claim 3, characterized in that, The step of the context encoder dynamically aggregating task dependencies and human-machine resource states based on the graph attention mechanism to generate context embedding vectors includes: Construct a heterogeneous graph based on a Simple Temporal Network (STN); the heterogeneous graph includes task type nodes, agent nodes, and various edge types, including time-constrained edges, task execution time edges, task time window edges, and agent cooperation edges. During the message passing phase, each edge type is associated with an independent weight matrix, which is then linearly transformed into the source node's features and stored in the target node's message mailbox. In the feature aggregation stage, attention coefficients related to edge types are calculated based on the attention mechanism, and the final feature vector of the node, i.e. the context embedding vector, is generated by weighted summation and activation function; among them, a multi-head attention mechanism is used to capture multi-dimensional semantics.

5. The adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios according to claim 4, characterized in that, The steps of inputting the context embedding vector into an experience-driven task allocator, updating the state embedding and agent embedding using the feedforward neural network and LSTM in the experience-driven task allocator, and progressively generating a sequence of task-agent matching pairs through a cyclic task-agent matching mechanism to obtain the initial task matching scheme include: Based on the context embedding vector and all agent embeddings, the probability of each agent being selected is calculated using a feedforward neural network, and one agent is selected. After selecting an agent, the agent embedding, the task embedding of all tasks to be assigned, and the global state embedding are combined to evaluate and select the most suitable task for matching to generate a matching pair. The state embedding and agent embedding are updated by LSTM, and the next pair of matches is performed in a loop until all tasks are assigned to generate an initial task matching scheme.

6. The adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios according to claim 5, characterized in that, The steps of matching tasks according to the initial task matching scheme and calculating the instant reward include: Execute the initial task matching scheme and record the actual task type and quantity assigned to each human operator; The dynamic human performance tracker updates the prediction of the actual task completion time of human operators based on new observation data and combined with Gaussian distribution and exponential decay mechanism, and combines the completion time of the tasks assigned to each machine resource to obtain the total completion time of this scheduling cycle. Based on the total completion time and constraint violations of this scheduling cycle, calculate the immediate reward for this initial task matching scheme.

7. The adaptive human-machine collaborative task scheduling method for intelligent manufacturing scenarios according to claim 6, characterized in that, Based on the initial network parameters of the immediate reward, context encoder, and task allocator, the autonomous learning optimizer employs the proximal policy optimization PPO algorithm for task-specific updates and the Reptile algorithm for meta-policy updates to generate the optimal task matching scheme. The steps include: Define a task difficulty metric and a sample weight vector, and use a proximal strategy to optimize the PPO gradient and update the model parameters. Samples are taken from various scheduling scenarios. After each inner loop update, the Reptile algorithm is used to fuse the policy update directions learned from each scenario and fine-tune the initial parameters of the context encoder and task allocator to obtain the optimal network parameters of the context encoder and task allocator. Based on the optimal network parameters of the context encoder and task allocator, an optimal task matching scheme is generated.

Images