A multi-agent based scheduling method

Abstract

The present application relates to artificial intelligence and human resource management technical field, disclose a kind of scheduling method based on multi-agent, comprising: obtaining the scheduling parameter of the scheduling period to be arranged, the scheduling period to be arranged is dispersed into multiple time slices, according to scheduling parameter and time slice, the state data of the association between the representation worker, post and time is constructed, according to scheduling constraint condition, the action mask for shielding the action of violating hard constraint is generated, state data and action mask are input into multi-agent reinforcement learning model, obtain scheduling decision information, according to scheduling decision information and action mask, joint scheduling action is formed by joint action generator or local constraint solver, obtain first scheduling scheme, the present application solves the problem that single agent is difficult to solve the collaborative relationship between worker, post and business demand, speeds up the scheduling generation speed, with good scene adaptability.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and human resource management technology, and in particular to a scheduling method based on multiple agents. Background Technology

[0002] In operational scenarios such as retail stores, restaurants, manufacturing workshops, customer service centers, healthcare, property security, and airport ground handling, scheduling management is a crucial link in ensuring the smooth operation of business. Managers need to rationally arrange employee work positions and hours within multiple constraints, including labor laws, job skill requirements, employee available time, and cost budgets. With the expansion of business scale and the increase in employment flexibility, the complexity of scheduling issues has significantly increased, and traditional scheduling methods relying on human experience are insufficient to balance efficiency, fairness, and service quality.

[0003] Related technologies typically employ rule-based scheduling methods, mathematical programming methods, or single-agent reinforcement learning methods. Rule-based scheduling methods rely on manually preset job priorities and work hour rules to generate schedules according to fixed logic. Mathematical programming methods model the scheduling problem as a linear or integer programming model, obtaining the optimal solution that satisfies the constraints through a solver. Single-agent reinforcement learning methods incorporate all employees, jobs, and time periods into a unified action space, training a single agent to output scheduling actions. However, mathematical programming methods suffer from a rapidly expanding combination space at large-scale employee, job, and time granularity, resulting in long solution times and difficulty in meeting real-time scheduling requirements. Single-agent reinforcement learning methods have excessively large action space dimensions, slow training convergence, and difficulty in expressing the local collaborative relationships between employees, jobs, and business needs.

[0004] Therefore, there is an urgent need for a multi-agent-based scheduling method to achieve effective integration of multi-agent decision-making information and flexible generation of joint scheduling actions, thereby improving the efficiency of scheduling scheme generation and scenario adaptability. Summary of the Invention

[0005] This invention provides a multi-agent scheduling method to solve at least one problem existing in related technologies.

[0006] In a first aspect, the present invention provides a scheduling method based on multiple agents, the method comprising the following steps: Obtain the scheduling parameters for the pending scheduling period; The scheduling period is discretized into multiple time slices, and state data representing the relationship between employees, positions and time is constructed based on the scheduling parameters and the time slices. An action mask is generated based on the scheduling constraints, and the action mask is used to filter out candidate actions that violate the hard constraints; The state data and the action mask are input into the multi-agent reinforcement learning model to obtain the scheduling decision information output by the multi-agent reinforcement learning model. The joint action generator or the local constraint solver generates joint scheduling actions based on the scheduling decision information and the action mask to obtain the first scheduling scheme.

[0007] In an optional implementation, the method further includes: When a dynamic event is detected, an event impact subgraph is determined, the shift status outside the event impact subgraph is kept fixed, the scheduling actions within the event impact subgraph are re-determined, and a second scheduling plan is generated. The event impact subgraph is a local scheduling range related to the dynamic event. Based on the second scheduling plan, determine the reward value and disturbance cost; Whether to issue the second scheduling scheme is determined based on the reward value and disturbance cost.

[0008] In one optional implementation, the action mask is a binary action mask, denoted as M(e,p,t,a)∈{0,1}, where e represents an employee, p represents a job position, t represents a time slice, a represents a candidate scheduling action, M=1 indicates that the candidate scheduling action is allowed to enter the candidate action pool, and M=0 indicates that the candidate scheduling action is blocked due to violation of hard constraints. The hard constraints include at least one of the following: employee unavailable time, job skill mismatch, maximum continuous working time exceeding the limit, minimum rest interval insufficient, working hours exceeding the limit, executed shifts, and locked shifts.

[0009] In one optional implementation, the multi-agent reinforcement learning model includes an employee agent, a job agent, and a demand agent. The step of inputting the state data and the action mask into the multi-agent reinforcement learning model to obtain the scheduling decision information output by the multi-agent reinforcement learning model includes: The state data and the action mask are input into the employee agent to obtain the employee acceptability score and candidate action probability output by the employee agent. The status data and the action mask are input into the job agent to obtain the job coverage gap and job priority output by the job agent. The state data and the action mask are input into the demand agent to obtain the demand change intensity output by the demand agent.

[0010] In conjunction with the first aspect or its corresponding implementation, in one implementation, the step of forming joint scheduling actions by the joint action generator or local constraint solver based on the scheduling decision information and the action mask includes: Based on the aforementioned job coverage gaps, target jobs and target time slots are determined; Based on the employee acceptability score and the intensity of demand change, a candidate action pool is formed for the target position and the target time slice; The candidate scheduling actions that violate hard constraints in the candidate action pool are eliminated using the action mask to obtain candidate action combinations. The joint scheduling actions are determined from the candidate action combinations with the objectives of job coverage, labor costs, employee preferences, work hour fairness and schedule disturbance costs.

[0011] In conjunction with the first aspect or its corresponding implementation, in one implementation, the step of forming joint scheduling actions by the joint action generator or local constraint solver based on the scheduling decision information and the action mask specifically includes: The generation process of the joint scheduling action is decomposed into an abstract shift construction stage and a specific personnel assignment stage by adopting a spatiotemporal decoupling strategy. In the abstract shift construction stage, based on the job coverage gap and the intensity of demand change, the abstract shift segments to be adjusted are determined, and the abstract shift segments include job attributes, time attributes and skill attributes; During the specific personnel assignment phase, based on the employee acceptability score and the action mask, the abstract shift fragment is mapped to the target employee node to generate a joint scheduling action containing the specific employee identifier.

[0012] In conjunction with the first aspect or its corresponding implementation, in one implementation, the reward value is calculated using a multi-objective reward function, which employs the following formula: Wherein, COV represents job coverage or demand fulfillment rate, COST represents labor cost deviation, VIOL represents the number of hard constraint violations or the degree of violation, PREF represents employee preference fulfillment rate, FAIR represents work hour fairness, STAB represents schedule stability, QUAL represents service quality indicators, and DIST represents schedule disturbance cost; α, β, γ, δ, ε, η, θ, and λ are weighting parameters.

[0013] In one implementation, the state data includes an employee-position-time state tensor and a scheduling relationship diagram. The employee-position-time state tensor is a three-dimensional tensor, wherein the employee dimension records at least one of the following employee characteristics: availability identifier, scheduled working hours, continuous working hours, minimum rest interval satisfaction identifier, employee preference matching value, historical refusal probability, and historical lateness probability; the position dimension records at least one of the following position characteristics: position demand gap, position priority, and unit labor cost; the time dimension records at least one of the following time slice characteristics: time period type and peak period identifier; each element in the tensor records a matching feature, which includes at least one of skill matching degree and position qualification level; the scheduling relationship diagram consists of employee nodes, position nodes, and demand nodes; wherein, employee nodes record employee skills, available time, employee preferences, historical working hours, and attendance feedback; position nodes record position headcount requirements, skill qualification requirements, position priority, and position opening time; and demand nodes record predicted customer flow, order volume, call volume, task volume, and service level targets.

[0014] In conjunction with the first aspect or its corresponding implementation, in one implementation, determining whether to issue the second scheduling plan based on the reward value and the disturbance cost specifically includes: Extract the specific adjustment instructions made in response to the dynamic event from the second scheduling scheme as the target scheduling action. The specific adjustment instructions include personnel assignment instructions, shift cancellation instructions or time change instructions. Based on the current environmental state of the event influence subgraph, the target scheduling action is replaced with the baseline default action before adjustment, and a comparative scheduling result is generated by simulation. Calculate the difference in business indicators between the second scheduling plan and the comparison scheduling result, and output the difference in business indicators as the basis for adjusting the target scheduling action.

[0015] In conjunction with the first aspect or its corresponding implementation, in one implementation, the event impact subgraph includes an affected time window, affected positions, skill groups related to the affected positions, currently scheduled employees, alternative candidate employees, relevant shift nodes, and at least one of the following relationships among the relevant shift nodes: skill matching relationship, time availability relationship, position demand relationship, and shift replacement relationship. Determining the event impact subgraph specifically includes: Obtain the direct impact intensity of the dynamic event on the initially affected positions; Based on a preset job skill association map, the propagation impact intensity of the direct impact intensity to the associated skill group is calculated, wherein the associated skill group shares the same skill resources or has a job collaboration relationship with the initially affected job. If the propagation impact intensity is greater than the preset disturbance threshold, the associated skill group will be included in the event impact subgraph, and the existing shifts within the associated skill group will be frozen.

[0016] The method provided by this invention constructs state data representing the relationship between employees, positions, and time, and generates action masks to filter out candidate actions that violate hard constraints. This allows for the pre-filtering of illegal actions before scheduling decisions are made, improving scheduling efficiency. A multi-agent reinforcement learning model is used to output scheduling decision information, which is then used by a joint action generator or local constraint solver to form joint scheduling actions based on the decision information and action masks. This achieves the fusion of multi-agent decision information and the generation of joint actions, solving the problem that a single agent cannot effectively address the collaborative relationship between employees, positions, and business needs. This accelerates scheduling speed and demonstrates good scenario adaptability. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a multi-agent scheduling method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a multi-agent collaborative decision-making model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the static scheduling and dynamic incremental rescheduling branch process according to an embodiment of the present invention; Figure 4 This is a schematic diagram of dynamic event influence subgraph determination and frozen rearrangement according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the deployment and computer device hardware structure according to an embodiment of the present invention; Figure 6 This is a structural block diagram of a multi-agent scheduling device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] According to an embodiment of the present invention, a scheduling method based on multiple agents is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] This embodiment provides a multi-agent-based scheduling method. Figure 1 This is a flowchart of a multi-agent scheduling method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step 101: Obtain the scheduling parameters for the scheduled period; Step 102: Discretize the shift scheduling period into multiple time slices, and construct state data representing the relationship between employees, positions and time based on the scheduling parameters and the time slices; Step 103: Generate an action mask based on the scheduling constraints. The action mask is used to mask candidate actions that violate hard constraints. Step 104: Input the state data and the action mask into the multi-agent reinforcement learning model to obtain the scheduling decision information output by the multi-agent reinforcement learning model; Step 105: The joint action generator or the local constraint solver generates joint scheduling actions based on the scheduling decision information and the action mask to obtain the first scheduling scheme.

[0024] The method provided by this invention constructs state data representing the relationship between employees, positions, and time, and generates action masks to filter out candidate actions that violate hard constraints. This allows for the pre-filtering of illegal actions before scheduling decisions are made, improving scheduling efficiency. A multi-agent reinforcement learning model is used to output scheduling decision information, which is then used by a joint action generator or local constraint solver to form joint scheduling actions based on the decision information and action masks. This achieves the fusion of multi-agent decision information and the generation of joint actions, solving the problem that a single agent cannot effectively address the collaborative relationship between employees, positions, and business needs. This accelerates scheduling speed and demonstrates good scenario adaptability.

[0025] The steps described above will be explained in detail below.

[0026] In step 101, the scheduling parameters for the scheduled period are obtained.

[0027] Among them, the scheduling parameters are a set of input information describing the various basic data and constraints required for scheduling within the scheduling period.

[0028] In a specific implementation of this invention, scheduling parameters may include basic employee parameters, employee availability parameters, employee skill parameters, job requirement parameters, business forecast parameters, labor rule parameters, cost budget parameters, historical scheduling parameters, historical execution feedback parameters, and current scheduling environment parameters. Basic employee parameters record employee identity information, job qualifications, skill levels, and working hours. Employee availability parameters record employee leave times, unavailable periods, and locked shifts. Employee skill parameters record the job skills and skill levels possessed by employees. Job requirement parameters record the minimum and target number of employees required for each job at different times. Business forecast parameters record predicted customer flow, predicted order volume, or predicted call volume. Labor rule parameters record the maximum continuous working hours, minimum rest interval, daily work hour limit, and weekly work hour limit. Cost budget parameters record the upper limit of labor cost budget and unit hourly cost. Historical scheduling parameters record scheduling plans for historical periods. Historical execution feedback parameters record employee actual attendance, refusal records, manual modification records, and service quality data for historical periods. Current scheduling environment parameters record weather information, promotional activities, and special events.

[0029] Obtaining the scheduling parameters for the upcoming scheduling period is the initial step in the entire scheduling optimization method. This involves collecting all the raw data required for scheduling from external business systems, databases, or management input interfaces. As mentioned earlier, scheduling parameters encompass employee availability and skill information, job requirement information, business forecasting information, rule-based constraint information, and historical reference information. These parameters are subsequently used to construct state data, generate action masks, and train and invoke multi-agent reinforcement learning models. Through this data collection step, the scheduling system can obtain the complete business information for the current scheduling period, providing a data foundation for automated scheduling.

[0030] For example, customer flow and order data from the same week in the past 8 weeks can be read as historical scheduling parameters and historical execution feedback parameters; the weather forecast and business district activity information for the next day can be read as current scheduling environment parameters; leave records, skill lists and scheduled working hours of employees A to L can be read as employee availability parameters and employee skill parameters; the job requirement configuration of the store from 10:00 to 22:00 the next day can be read as job requirement parameters; and the maximum continuous working time of no more than 6 hours, the minimum rest interval of no less than 30 minutes, and the weekly working hour limit of 40 hours can be read as labor rule parameters.

[0031] In step 102, the schedule to be scheduled is discretized into multiple time slices, and state data representing the relationship between employees, positions and time is constructed based on the scheduling parameters and time slices.

[0032] In one embodiment, the state data includes an employee-position-time state tensor and a scheduling relationship diagram. The employee-position-time state tensor is a three-dimensional tensor, wherein the employee dimension records at least one of the following employee characteristics: availability identifier, scheduled working hours, continuous working hours, minimum rest interval satisfaction identifier, employee preference matching value, historical refusal probability, and historical lateness probability; the position dimension records at least one of the following position characteristics: position demand gap, position priority, and unit labor cost; the time dimension records at least one of the following time slice characteristics: time period type and peak period identifier; each element in the tensor records a matching feature, which includes at least one of skill matching degree and position qualification level; the scheduling relationship diagram consists of employee nodes, position nodes, and demand nodes; wherein, the employee nodes record employee skills, available time, employee preferences, historical working hours, and attendance feedback; the position nodes record the position headcount requirement, skill qualification requirements, position priority, and position opening time; and the demand nodes record predicted customer flow, order volume, call volume, task volume, and service level targets.

[0033] A time slice is the smallest unit of scheduling time obtained by dividing the to-be-scheduled period according to a preset time granularity. State data is a set of state information representing the relationships between employees, positions, and time, including an employee-position-time state tensor and a scheduling relationship graph. The employee-position-time state tensor is a three-dimensional tensor. The employee dimension records the employee's own state characteristics, the position dimension records the position's requirement characteristics, and the time dimension records the time slice's own attribute characteristics. Each element in the tensor records the matching characteristics between the employee and the position within that time slice. The scheduling relationship graph is a graph structure composed of employee nodes, position nodes, and requirement nodes. The edges between employee nodes and position nodes represent the skill matching relationship between the employee and the position, and the edges between position nodes and requirement nodes represent the position's contribution to business needs.

[0034] In a specific implementation of this invention, continuous business hours are discretized into several time slices of equal length, for example, 30 minutes per slice, enabling scheduling decisions to be made at discrete time units. Based on this, this step constructs two forms of state data. An employee-position-time state tensor records scheduling-related feature information in a three-dimensional structure. The employee dimension stores employee availability and working hours, the position dimension stores demand gaps and cost characteristics for each position in different time slices, and the time dimension stores attribute features such as time slot type and peak indicator for each time slice. Each element in the tensor stores the skill matching degree and qualification level between the employee and position in that time slice. A scheduling relationship graph records the relationships between employee nodes, position nodes, and demand nodes in a graph structure, facilitating subsequent determination of event impact subgraphs and impact propagation analysis. Both types of state data provide structured environmental representation input for the multi-agent reinforcement learning model.

[0035] Suppose a chain restaurant needs to schedule shifts for the next day. The operating hours from 10:00 AM to 10:00 PM can be discretized into 30-minute time slices, generating 24 time slices in total. First, construct an employee-position-time state tensor. In the employee dimension, record that employee A is on leave, therefore availability is marked as unavailable; record that employee C has already scheduled 38 hours this week, therefore scheduled work hours are 38 hours. In the position dimension, record that the front-of-house position has a demand gap of 2 people in the lunch peak time slice, and the cashier position has a demand gap of 1 person in the evening peak time slice. In the time dimension, record that the time slice from 12:00 PM to 2:00 PM is the lunch peak time slice, and the time slice from 6:00 PM to 8:00 PM is the evening peak time slice. In each element of the tensor, record that employee B has cashier skills, therefore skill matching is high; employee D does not have takeout packaging qualifications, therefore position qualification level is unsatisfactory. At the same time, a scheduling relationship diagram is constructed. The edge weight between employee nodes and job nodes is determined by the skill matching degree and time overlap degree, and the edge weight between job nodes and demand nodes is determined by the contribution of the job to service demand. The contribution of front-of-house positions to customer flow demand is higher than that of back-of-house positions.

[0036] This embodiment uses a three-dimensional state tensor to hierarchically store employee characteristics, job characteristics, time slice characteristics, and matching characteristics, supporting rapid indexing of any combination of state information through employee identifiers, job identifiers, and time slice identifiers. The scheduling relationship diagram expresses the multi-dimensional relationships between employees, jobs, and needs, providing a clear and complete data foundation for subsequent multi-agent collaborative decision-making and dynamic event impact range analysis.

[0037] In step 103, an action mask is generated based on the scheduling constraints. The action mask is used to mask candidate actions that violate hard constraints.

[0038] In one embodiment, the action mask is a binary action mask, denoted as M(e,p,t,a)∈{0,1}, where e represents an employee, p represents a job position, t represents a time slice, a represents a candidate scheduling action, M=1 indicates that the candidate scheduling action is allowed to enter the candidate action pool, and M=0 indicates that the candidate scheduling action is blocked due to violation of hard constraints. The hard constraints include at least one of the following: unavailable employee time, job skill mismatch, maximum continuous working time exceeding the limit, insufficient minimum rest interval, working hours exceeding the limit, executed shifts, and locked shifts.

[0039] The action mask is a binary mask matrix denoted as M(e,p,t,a)∈{0,1}, where e represents the employee, p represents the job position, t represents the time slice, and a represents the candidate scheduling action. M=1 indicates that the candidate scheduling action is allowed to enter the candidate action pool, and M=0 indicates that the candidate scheduling action is blocked due to violation of hard constraints. Hard constraints are mandatory restrictions that cannot be violated during the scheduling process, including employee unavailability time, job skill mismatch, exceeding the maximum continuous working time limit, insufficient minimum rest interval, working hours exceeding the hard limit, already executed shifts, and locked shifts. Candidate scheduling actions are optional atomic operation units in the scheduling decision, including at least one of the following: assigning shifts, canceling shifts, job replacement, adjusting start time, adjusting end time, filling in spare personnel, inserting rest periods, and maintaining the original shift.

[0040] In a specific implementation of this invention, an action mask is used to filter the candidate action space before scheduling decisions are made, eliminating illegal actions that violate hard constraints in advance. For hard constraints such as employee unavailability time, job skill requirements, and labor regulations, which do not rely on cumulative results, the current state is used to determine whether to set them to zero. For global constraints such as budget limits and work-hour balance, which rely on the cumulative results of multiple shifts, the current cumulative cost, the incremental cost of candidate actions, and the remaining budget are dynamically calculated to determine whether to set them to zero. After the action mask is generated, it will be used as input to the subsequent multi-agent reinforcement learning model and joint action generator, ensuring that only candidate actions that satisfy the hard constraints enter the subsequent decision-making process.

[0041] For example, suppose employee A takes temporary leave between 18:00 and 22:00. Therefore, all candidate actions for assigning employee A to a shift within this time window are set to M=0. Employee C has already scheduled 38 hours this week, and the weekly work hour cap is 40 hours. Therefore, all candidate actions for extending employee C's work hours by more than 2 hours are set to M=0. Employee B has cashier skills and is available during this time period. Therefore, the candidate action for assigning employee B to the cashier position is set to M=1. Employee D does not have the required qualification level for takeout packaging. Therefore, all candidate actions for assigning employee D to the takeout packaging position are set to M=0. The front-of-house position requires 2 people during the lunch peak but is currently unoccupied. Therefore, the candidate action for assigning an additional employee to the front-of-house position is set to M=1.

[0042] In step 104, the state data and action mask are input into the multi-agent reinforcement learning model to obtain the scheduling decision information output by the multi-agent reinforcement learning model.

[0043] In one embodiment, the multi-agent reinforcement learning model includes an employee agent, a job agent, and a demand agent. State data and action masks are input into the multi-agent reinforcement learning model to obtain scheduling decision information output by the model, including: Input the state data and action mask into the employee agent to obtain the employee acceptability score and candidate action probability output by the employee agent. Input the state data and action mask into the job agent to obtain the job coverage gap and job priority output by the job agent. Input the state data and action mask into the demand agent to obtain the demand change intensity output by the demand agent.

[0044] The multi-agent reinforcement learning model is a decision-making model trained using a centralized training and distributed execution framework, consisting of an employee agent, a job agent, and a demand agent. The employee agent outputs employee acceptability scores and candidate action probabilities based on employee available time, skills, preferences, historical work hours, and refusal records. The job agent outputs job coverage gaps and job priorities based on minimum number of employees, target number of employees, skill requirements, current coverage, and job opening hours. The demand agent outputs the intensity of demand change based on predicted customer flow, order volume, call volume, task volume, or service level targets. The scheduling decision information is the set of decision results output by the multi-agent reinforcement learning model, including employee acceptability scores and candidate action probabilities output by the employee agent, job coverage gaps and job priorities output by the job agent, and the intensity of demand change output by the demand agent.

[0045] In a specific implementation of this invention, the state data constructed in step 102 and the action mask generated in step 103 are jointly input into a pre-trained multi-agent reinforcement learning model. Each agent in the model outputs corresponding scheduling decision information based on its local observations. The employee agent, using state data and the action mask as input, outputs an employee's acceptability score and candidate action probability for each candidate shift; a higher score indicates that the employee is more likely to accept the shift arrangement. The job agent, using state data and the action mask as input, outputs the coverage gap and job priority for each job in different time slices; a positive gap indicates a need for additional personnel, while a negative gap indicates a surplus of personnel. The demand agent, using state data and the action mask as input, outputs the intensity of demand change, representing the degree of fluctuation in the current predicted demand compared to the baseline level. Figure 2 The diagram shown is a schematic diagram of the multi-agent collaborative decision-making model structure in an embodiment of the present invention. The outputs of the three types of agents together constitute the scheduling decision information. It should be noted that the outputs of each agent are constrained by the action mask and are combined by the joint action generator, the local constraint solver or the rule engine to form joint scheduling actions such as assigning shifts, canceling shifts, adjusting start times, replacing positions, filling in spare personnel, and maintaining the original shifts.

[0046] Assuming the state tensor, scheduling graph, and action mask are input into the multi-agent reinforcement learning model, the employee agent outputs an acceptability score of 0.9 for employee B taking on the cashier shift, 0.2 for employee C extending their kitchen shift, and 0.7 for employee D taking on the takeout packaging shift. The job agent outputs a shortage of 1 person for cashier positions, 1 person for takeout packaging positions, 0 people for front-of-house positions, and 0 people for kitchen positions from 18:00 to 20:00. The demand agent outputs, based on evening peak order forecast data, an increase of 0.8 in demand for takeout packaging positions, an increase of 0.1 in demand for front-of-house positions, and an increase of 0.2 in demand for kitchen positions. These outputs collectively constitute scheduling decision information for use by the subsequent joint action generator.

[0047] In step 105, the joint action generator or the local constraint solver generates joint scheduling actions based on the scheduling decision information and the action mask to obtain the first scheduling scheme.

[0048] In one embodiment, "forming joint scheduling actions by a joint action generator or a local constraint solver based on scheduling decision information and action masks" may specifically include the following steps: Determine target positions and target time slots based on the job coverage gaps; Based on employee acceptability scores and the intensity of demand changes, a pool of candidate actions is created for the target positions and target time slots. The candidate scheduling actions that violate hard constraints are removed from the candidate action pool using action masks to obtain candidate action combinations; Based on the objectives of job coverage, labor costs, employee preferences, work hour fairness, and schedule disturbance costs, joint scheduling actions are determined from candidate action combinations.

[0049] The joint action generator refers to converting scheduling decision information into joint scheduling actions using scoring and ranking. The local constraint solver refers to converting scheduling decision information into joint scheduling actions using constraint programming. The candidate action pool is a collection of candidate scheduling actions, each of which includes at least one of the following action types: assigning shifts, canceling shifts, job replacement, adjusting start time, adjusting end time, filling in spare personnel, inserting rest periods, or maintaining the original shift. The candidate action combination is a set of non-conflicting candidate scheduling actions selected from the candidate action pool. Non-conflict includes: the same employee not being assigned to multiple jobs in the same time slot; the number of employees in the same job not being less than the minimum requirement and not exceeding the configuration limit; the same locked shift not being modified repeatedly; and no new rest interval violations or continuous work time violations arising between candidate actions.

[0050] In a specific implementation of this invention, firstly, based on the job coverage gaps output by the job agent, it is determined which jobs require additional, reduced, or replaced personnel in which time slices. Then, based on the employee acceptability score output by the employee agent and the demand change intensity output by the demand agent, a candidate action pool is constructed for the identified target jobs and target time slices. Next, the action mask generated in step 103 is used to eliminate illegal candidate actions that violate hard constraints from the candidate action pool, resulting in a combination of candidate actions that meet the hard constraint requirements. Finally, with job coverage improvement, labor cost control, employee preference satisfaction, work hour fairness maintenance, and schedule disturbance cost control as optimization objectives, the final joint scheduling action to be executed is determined from the candidate action combination. In one embodiment of this invention, with job coverage improvement, labor cost control, employee preference satisfaction, work hour fairness maintenance, and schedule disturbance cost control as optimization objectives, a set of non-conflicting candidate actions is selected from the candidate action set using at least one of the following methods: greedy search, bundle search, local search, constraint programming, or mixed integer programming.

[0051] Continuing with the previous example, based on the job coverage gaps output by the job agent, the target jobs are identified as cashier and takeout packaging jobs, with the target time slot being 18:00 to 20:00. Based on the employee acceptability score output by the employee agent and the demand change intensity output by the demand agent, a candidate action pool is created for the cashier job during the 18:00 to 20:00 period, including employee B and employee E taking on cashier shifts; a candidate action pool is also created for the takeout packaging job during the same period, including employee D and employee G taking on takeout packaging shifts. Using action masks, employee E lacks cashier skills, so their corresponding candidate action is masked; employee D's continuous working hours will exceed the limit, so their corresponding candidate action is masked. The remaining candidate actions, employee B taking on cashier shifts and employee G taking on takeout packaging shifts, are then added to the candidate action pool. With the goals of improving job coverage, reducing labor costs, optimizing employee preferences, ensuring fairness in working hours, and minimizing the cost of shift disruptions, the joint scheduling action was determined from the candidate action combinations as follows: employee B will work as a cashier from 18:00 to 22:00, and employee G will work as a takeout packer from 18:30 to 20:30.

[0052] This embodiment efficiently integrates the dispersed outputs of employee agents, job agents, and demand agents into a consistent joint scheduling action through a four-stage process: first, determining the required positions and time slices; second, forming a candidate action pool; third, eliminating illegal actions; and finally, multi-objective optimization. The introduction of action masks ensures that the final output joint scheduling action meets all hard constraints. The multi-objective optimization mechanism balances multiple conflicting optimization objectives such as labor costs, employee satisfaction, work hour fairness, and schedule stability, thereby improving the practicality and acceptability of the generated scheduling plan.

[0053] In one embodiment, "forming joint scheduling actions by a joint action generator or a local constraint solver based on scheduling decision information and action masks" may specifically include the following steps: The generation process of the joint scheduling action is decomposed into an abstract shift construction stage and a specific personnel assignment stage by adopting a spatiotemporal decoupling strategy. In the abstract shift construction stage, based on the job coverage gap and the intensity of demand change, the abstract shift segments to be adjusted are determined, and the abstract shift segments include job attributes, time attributes and skill attributes; During the specific personnel assignment phase, based on the employee acceptability score and the action mask, the abstract shift fragment is mapped to the target employee node to generate a joint scheduling action containing the specific employee identifier.

[0054] The spatiotemporal decoupling strategy decomposes the generation process of joint scheduling actions into two independent stages. The first stage focuses on shift requirements, and the second stage focuses on matching and assigning specific personnel. An abstract shift fragment is a unit of shift requirement description that does not contain specific employee information, but includes job attributes, time attributes, and skill attributes. The job attribute specifies which job position the shift requires, the time attribute specifies the time window the shift needs to cover, and the skill attribute specifies the required skill level or certification for the employees. The personnel assignment stage is responsible for matching the abstract shift fragment with eligible employees, generating executable joint scheduling actions that include specific employee identifiers.

[0055] In a specific implementation of this invention, the complex problem of generating joint scheduling actions is decomposed into two stages. In the abstract shift construction stage, only the job coverage gap and the intensity of demand changes determine which shift segments need to be created, adjusted, or canceled. Each abstract shift segment clearly defines job requirements, time windows, and skill thresholds. In the specific personnel assignment stage, the abstract shift segments generated in the first stage are matched with the employee acceptability scores output by the employee agent, while undergoing legality filtering via action masks. Each abstract shift segment is then mapped to a specific target employee node, ultimately generating an executable joint scheduling action containing employee identifiers.

[0056] For example, continuing the next-day scheduling for chain restaurant stores, this step employs a spatiotemporal decoupling strategy to handle the scheduling needs during the evening peak. In the abstract shift construction phase, firstly, based on the job coverage gap output by the job agent, a shift segment requiring an additional takeout packaging position is determined. Based on the intensity of demand change output by the demand agent, this shift segment is prioritized as high. The constructed abstract shift segment includes the job attribute of takeout packaging, the time attribute of 18:30 to 20:30, and the skill attribute of possessing takeout packaging qualifications. In the specific personnel assignment phase, candidate employees are ranked according to the employee acceptability score output by the employee agent. Employee G has an acceptability score of 0.9, employee H has an acceptability score of 0.6, and employee K is marked as unavailable in the action mask because their work hours for the day are already full. The abstract shift segment is mapped to the node of employee G with the highest acceptability score, generating a joint scheduling action containing the employee G identifier, meaning employee G will work as a takeout packaging employee from 18:30 to 20:30.

[0057] In one embodiment of the present invention, the above method may further include: When a dynamic event is detected, the event impact subgraph is determined, the shift status outside the event impact subgraph is kept fixed, the scheduling actions within the event impact subgraph are re-determined, and a second scheduling plan is generated. The event impact subgraph is the local scheduling range related to the dynamic event. Based on the second scheduling plan, determine the reward value and disturbance cost; Whether to issue a second scheduling plan is determined based on the reward value and the disturbance cost.

[0058] Dynamic events are sudden or abnormal situations that occur during the execution of the first scheduling plan and require adjustments to the plan. These include temporary leave, lateness, sudden order surges, abnormal customer flow, equipment malfunctions, and changes in business tasks. The event impact sub-graph is a partial graph structure related to dynamic events, extracted from the scheduling relationship graph corresponding to the first scheduling plan. It includes the affected time window, affected positions, skill groups related to the affected positions, currently scheduled employees, potential replacement employees, relevant shift nodes, and the skill matching relationships, time availability relationships, job demand relationships, and shift replacement relationships between these nodes. Maintaining fixed shift status means that shifts outside the event impact sub-graph are considered immutable and locked during the rescheduling process, and do not participate in any rescheduling decisions. The reward value is used to evaluate the quality of the second scheduling plan; a higher reward value indicates a better overall performance. Disturbance cost is the cost of changes in shift arrangements incurred by the second scheduling plan relative to the first scheduling plan or relative to the baseline schedule before adjustment.

[0059] In a specific implementation of this invention, upon detecting a dynamic event, the scope of its impact, i.e., the event impact subgraph, is first determined. This determination process includes identifying the impact time window based on the event's occurrence time and duration, determining the set of affected positions based on the corresponding job position, identifying relevant skill groups based on job skill requirements, and determining the set of candidate employees based on their availability, skill matching, work hour rules, and rest interval rules. After determining the event impact subgraph, all executed shifts, manually locked shifts, and shifts not included in the event impact subgraph are added to a frozen shift set, keeping their status unchanged. Only the scheduling actions within the event impact subgraph are re-determined, generating a second scheduling plan. It should be noted that the second scheduling plan is a partial modification of the first scheduling plan, not a global replacement. Figure 3The diagram illustrates the static scheduling and dynamic incremental rescheduling branch processes of an embodiment of the present invention. When no dynamic events occur, the system enters the static initial scheduling process, sequentially executing tasks such as inputting scheduling status data, generating the first scheduling plan, constraint verification and conflict resolution, and publishing the schedule. When dynamic events such as temporary leave, lateness, or a sudden surge in orders occur, the system enters the dynamic execution process. It inputs the current scheduling status with an event identifier, determines the event impact subgraph, freezes unaffected shifts, determines whether rescheduling is necessary, and, if needed, calls the incremental rescheduling model to generate a second scheduling plan. It should be noted that the incremental rescheduling model uses the same working principle as the joint action generator or local constraint solver in the static initial scheduling; the only difference lies in the configuration of input parameters and constraints. In the incremental rescheduling scenario, the input status data is a subset within the event impact subgraph, including only the affected time window, affected positions, skill groups related to the affected positions, currently scheduled employees, and alternative candidate employees. The candidate employee set is limited to employees within the event impact subgraph who can participate in the rescheduling; employees corresponding to frozen shifts are not included in the candidate range. The candidate action pool is limited to candidate actions built for positions and time slices within the event-affected subgraph, excluding shifts outside the subgraph. An additional masking rule for frozen shifts is added to the action mask to ensure that shifts outside the event-affected subgraph remain unchanged during the rescheduling process. Through differentiated configuration of the above parameters and constraints, the incremental rescheduling model can quickly reschedule only the localized area affected by dynamic events, generating a second scheduling scheme while maintaining the overall stability of the global scheduling plan.

[0060] In one embodiment of the present invention, the event impact subgraph includes the affected time window, the affected job position, the skill group related to the affected job position, the currently scheduled employees, the alternative candidate employees, the relevant shift nodes, and at least one of the following relationships among the relevant shift nodes: skill matching relationship, time availability relationship, job demand relationship, and shift replacement relationship. The "determining the event impact subgraph" in the aforementioned step may specifically include: Obtain the direct impact intensity of dynamic events on initially affected positions; Based on a preset job skill association map, the propagation impact intensity of the direct impact intensity to the associated skill group is calculated, wherein the associated skill group shares the same skill resources or has a job collaboration relationship with the initially affected job. If the propagation impact intensity is greater than the preset disturbance threshold, the associated skill group will be included in the event impact subgraph, and the existing shifts within the associated skill group will be frozen.

[0061] The direct impact intensity is a quantified value of the direct impact of a dynamic event on the initially affected job. For example, a 50% surge in orders would have a direct impact intensity of 0.5 on the takeout packaging job. The job skill association graph is a graph structure recording skill sharing and collaboration relationships between different jobs. Nodes represent jobs, and edges represent shared skill resource requirements or collaborative relationships in business processes between two jobs. The propagation attenuation coefficient refers to the attenuation ratio of the direct impact intensity as it propagates along the job skill association graph from the initially affected job to adjacent related skill groups. It ranges from 0 to 1; a larger attenuation coefficient indicates a more significant impact. For example, jobs sharing the same skill resources have a larger attenuation coefficient, while jobs with only collaboration relationships but no skill sharing have a smaller attenuation coefficient. The disturbance threshold is the critical value used to determine whether the propagation of the impact is sufficient to include the related skill group in the event impact subgraph. When the propagation attenuation coefficient exceeds this threshold, the related skill group is included in the event impact subgraph.

[0062] In a specific implementation of this invention, when a dynamic event occurs, the direct impact intensity of the event on the initially affected positions is first assessed. For example, the impact of a sudden increase in orders on the packaging position, or the impact of equipment failure on the operation position. The degree of direct impact can be determined based on the increase in orders or the proportion of equipment failure. Then, based on the job skill association graph, the direct impact intensity is propagated along the edges of the graph to adjacent associated skill groups. The impact intensity is multiplied by a propagation attenuation coefficient after each edge to obtain the propagated impact intensity value, i.e., the propagated impact intensity. Associated skill groups that still exceed the disturbance threshold after propagation are included in the event impact subgraph, and existing shifts within these skill groups are frozen to ensure that these shifts remain unchanged or participate in the reshuffling process.

[0063] For example, at 18:20, takeout orders suddenly increase by 50%, corresponding to a direct impact intensity of 0.5. The initially affected position is the takeout packaging position. The job skill association graph shows that the takeout packaging position shares food preparation skills with the kitchen positions, indicating a close association, with a propagation attenuation coefficient preset to 0.8; the takeout packaging position collaborates with the front-of-house positions but does not share skills, with a propagation attenuation coefficient preset to 0.3. The propagation impact intensity from the takeout packaging position to the kitchen position is calculated as 0.5 multiplied by 0.8, which equals 0.4. Since the propagation impact intensity of 0.4 to the kitchen position is greater than the disturbance threshold of 0.3, the kitchen position is included in the event impact subgraph. The propagation impact intensity from the takeout packaging position to the front-of-house position is calculated as 0.5 multiplied by 0.3, which equals 0.15. Since 0.15 is less than the disturbance threshold of 0.3, the front-of-house position is not included in the event impact subgraph. The existing shifts within the kitchen positions from 18:30 to 20:30 are frozen, keeping their status unchanged.

[0064] This embodiment dynamically determines the event impact subgraph through an impact propagation mechanism, avoiding the inefficient practice of including the entire shift relationship graph in the rescheduling scope. The propagation attenuation coefficient based on the job skill association graph accurately reflects the degree of association between different jobs; the higher the degree of skill sharing, the larger the propagation attenuation coefficient and the wider the impact range. Jobs with only collaborative relationships but no skill sharing have limited impact. The perturbation threshold setting allows the system to flexibly control the expansion of the impact range according to business needs; the higher the perturbation threshold, the smaller the impact subgraph range, and the lower the perturbation threshold, the larger the impact subgraph range. This dynamic method of determining the impact range ensures that incremental rescheduling covers the truly affected jobs without excessively expanding the rescheduling scope and causing unnecessary shift disruptions.

[0065] like Figure 4 The diagram illustrates the determination and freezing / rearrangement of the dynamic event impact subgraph according to an embodiment of the present invention. The impact time window is determined based on the occurrence time and duration of the dynamic event; the set of affected positions is determined based on the corresponding position; the relevant skill groups are determined based on the skill requirements of the positions; and the set of candidate employees is determined based on the availability, skill matching, work hour rules, and rest interval rules. Executed shifts, manually locked shifts, and shifts not included in the event impact subgraph are added to the frozen shift set. Only the affected area is partially rearranged, meaning the scheduling actions within the event impact subgraph are re-decided, and a second scheduling plan or a manually confirmed item is output.

[0066] Continuing the example above, the event impact subgraph is determined to include the affected time window of 18:30 to 20:30, the affected positions being takeout packaging and kitchen positions, and the associated skill groups being employees with packaging and food preparation qualifications. First, the shift status outside the event impact subgraph is kept fixed: the shifts for front-of-house employees B and E from 18:30 to 20:30 remain unchanged, the shift for cashier employee C from 18:30 to 20:30 remains unchanged, and the shifts executed before 18:20 remain unchanged. Only the scheduling actions within the event impact subgraph are re-decided. For the takeout packaging position, three candidate actions are considered: employee G arriving early, employee H working extended hours, and employee K being replaced. For the kitchen position, whether additional food preparation staff are needed is considered. After decision-making, employee G is selected to arrive early to take on the takeout packaging position from 18:30 to 20:30, while the kitchen positions remain unchanged, generating a second scheduling plan.

[0067] In a specific implementation of the present invention, after generating the second scheduling plan, it is not directly released. Instead, the reward value of the second scheduling plan is determined, and the release of the second scheduling plan is determined based on the reward value.

[0068] Specifically, the reward value can be calculated using a multi-objective reward function. A higher reward value indicates that the scheduling scheme performs better in terms of job coverage, labor cost control, constraint satisfaction, employee preference satisfaction, work hour fairness, schedule stability, and service quality. The calculation of the reward value can also cover the schedule disturbance cost, which can include the number of employee shift changes, the extent of job adjustments, the degree of decline in employee satisfaction, and the indirect operating costs caused by the adjustments.

[0069] In one embodiment of the present invention, the reward value is calculated using a multi-objective reward function, which employs the following formula: Wherein, COV represents job coverage or demand fulfillment rate, COST represents labor cost deviation, VIOL represents the number of hard constraint violations or the degree of violation, PREF represents employee preference fulfillment rate, FAIR represents work hour fairness, STAB represents schedule stability, QUAL represents service quality indicators, and DIST represents schedule disturbance cost; α, β, γ, δ, ε, η, θ, and λ are weighting parameters.

[0070] In a specific implementation of the present invention, the multi-objective reward function adopts the above-mentioned weighted summation form, which integrates multiple indicators such as job coverage rate, human resource cost deviation, degree of violation of hard constraints, employee preference satisfaction rate, working hour fairness, schedule stability, service quality indicators and schedule disturbance cost into a single reward value.

[0071] Continuing with the previous example, a second scheduling plan is generated. Employee G arrives early to take on the takeout packaging position from 18:30 to 20:30, and the reward value for this plan is calculated. A COV of 1.0 indicates full coverage of the takeout packaging position; a COST of 50 yuan indicates an increase in cost of 50 yuan; a VIOL of 0 indicates no violations; a PREF of 0.6 indicates that employee G's preferences are partially satisfied; a FAIR of 0.7 indicates slight fluctuations in work hour allocation; a STAB of 0.8 indicates minor adjustments to the schedule; a QUAL of 0.8 indicates a shortened expected food preparation time; and a DIST of 30 indicates the cost of the change. The DIST can be calculated based on at least one of the following factors: the number of employees affected, the number of shifts affected, the time offset of the change, the decrease in employee preference matching value, the weighted value of the employee's historical refusal probability, and the management time cost incurred by manual confirmation. For example, if employee G's shift is moved from the originally planned 20:00 to 18:30, the time offset is 1.5 hours. The disturbance cost = time offset × unit disturbance coefficient 10 = 15. Simultaneously, the supervisor needs to notify employee G additionally, with a management cost of 15. The total disturbance cost is 30. Therefore, the disturbance cost is 30. The weight parameters are configured as α=10, β=0.1, γ=5, δ=1, ε=0.5, η=1, θ=5, λ=0.2. Based on this, the reward value for the second scheduling scheme is calculated to be 4.75. Assuming the reward value for the first scheduling scheme is -0.05, and the reward value for the second scheduling scheme is higher than that of the first, the second scheduling scheme is issued. In one embodiment of the present invention, when multiple candidate second scheduling schemes exist, the weight of the timetable disturbance cost DIST in the multi-objective reward function is increased, tending to select the adjustment scheme with the smallest change among multiple candidate schemes.

[0072] In one embodiment of the present invention, determining whether to issue a second scheduling scheme based on the reward value and the disturbance cost specifically includes: Extract specific adjustment instructions for dynamic events from the second scheduling plan as target scheduling actions. Specific adjustment instructions include personnel assignment instructions, shift cancellation instructions or time change instructions. Based on the current environmental state of the event impact subgraph, the target scheduling action is replaced with the baseline default action before adjustment, and the comparison scheduling results are simulated and generated. Calculate the difference in business indicators between the second scheduling plan and the comparison scheduling results, and output the difference in business indicators as the basis for adjusting the target scheduling action.

[0073] The target scheduling action is a specific adjustment instruction extracted from the second scheduling plan in response to a dynamic event. It includes personnel assignment instructions, shift cancellation instructions, and time change instructions. Personnel assignment specifies the time slot for assigning an employee to a specific position. Shift cancellation instructions cancel an employee's existing shift in a specific time slot. Time change instructions advance or postpone the start or end time of an employee's shift. The baseline default action is the original scheduling action before any adjustments were made for the dynamic event, such as the shift arrangements in the first scheduling plan determined before the dynamic event occurred. The comparative scheduling result is a simulated scheduling plan generated after replacing the target scheduling action with the baseline default action. It can be compared with the second scheduling plan. The business indicator difference value is the difference between the second scheduling plan and the comparative scheduling result in business indicators such as job coverage, labor costs, service quality, and employee working hours. It serves as the basis for adjustment interpretation and is output to management or employees.

[0074] In a specific implementation of this invention, firstly, specific adjustment instructions for dynamic events are extracted from the second scheduling scheme, filtering out original shift information unrelated to the dynamic events and retaining only the core actions of this adjustment. Then, based on the current environmental state of the event influence subgraph, the extracted target scheduling actions are replaced with the baseline default actions before adjustment, keeping other environmental states completely unchanged, and a comparative scheduling result is simulated and generated. The comparative scheduling result represents the hypothetical scenario of "what the scheduling scheme would be like without this adjustment." The differences between the second scheduling scheme and the comparative scheduling result in various business indicators are calculated, such as how much the job coverage rate has increased, how much labor costs have increased, and how much service quality has improved. These differences in business indicators serve as the basis for explaining the adjustment of the target scheduling actions and are output to managers or relevant employees in the form of text descriptions, key indicator changes, or comparison views.

[0075] For example, assume the second scheduling plan includes a target scheduling action: employee G arrives early to work at the takeout packaging position from 18:30 to 20:30. First, extract this target scheduling action, the action type being personnel assignment instruction, specifically assigning employee G to the takeout packaging position within the time window of 18:30 to 20:30. Based on the current environment state of the event impact subgraph, replace employee G's early arrival action with the baseline default action, i.e., employee G does not arrive early, generating a comparative scheduling result. Calculate the differences in business metrics between the second scheduling plan and the comparative scheduling result: job coverage increases from 67% to 100%, a difference of 33%; labor costs increase by 50 yuan due to employee G working 2 hours of overtime, a difference of 50 yuan; the estimated average food preparation time decreases from 15 minutes to 10 minutes, a difference of 5 minutes. It can output the difference values ​​of indicators and their corresponding explanations: the order volume exceeds the forecast by 35%, there is a shortage of 1 person in the takeaway packaging position, employee G has the corresponding skills and is available, this adjustment will achieve full coverage of the takeaway packaging position, the food preparation time is expected to be shortened by 5 minutes, and the labor cost will increase by 50 yuan within the budget. These differences of business indicators will be used as the basis for the explanation of the adjustment of the target scheduling action.

[0076] In some embodiments of the present invention, feedback learning can also be performed after the scheduling action is completed. For example, after business hours, actual order volume, employee attendance, manual modification records, and customer waiting time are collected to calculate the actual reward value. If employee G arrives early, reducing the average food preparation time and increasing labor costs within budget, this adjustment action is written as a positive sample into the experience replay library of the database server; if an employee refuses a temporary shift change, their preference vector and refusal record are updated. The above feedback samples first enter the offline training process, undergoing data cleaning, abnormal sample filtering, validation set evaluation, historical scene replay, or gray-scale operation, before being used to update the network parameters of the multi-agent reinforcement learning model to avoid unstable adjustments caused by a single abnormal event. The updated multi-agent reinforcement learning model replaces the online model after meeting preset online conditions and is used for the state input and decision output of subsequent scheduling cycles.

[0077] like Figure 5The diagram illustrates the deployment and hardware structure of the computer device according to an embodiment of the present invention. The computer device includes a processor, a memory, a storage medium, a network communication interface, a database server, and an optional training server. The processor performs computational steps such as state construction, constraint action mask generation, multi-agent policy reasoning, joint action generation, local constraint solving, rule constraint verification, determination of dynamic event influence subgraphs, and schedule publishing. The memory temporarily stores the scheduling state tensor, scheduling relationship graph, action mask, candidate actions, candidate scores, and intermediate calculation results. The storage medium stores the computer program, scheduling rule configuration, policy model parameters, incremental reordering model parameters, and model verification configuration. The network communication interface connects to the supervisor's end, employee's end, attendance system, POS system, order system, customer service system, and other business systems to receive scheduling parameters, dynamic events, and execution feedback. Supervisors and employees access the application server via Web, mini-programs, mobile apps, or enterprise instant messaging tools; attendance machines, store POS systems, order systems, customer service systems, and business systems connect to the network communication interface via API or message queues; the application server calls the online inference module, rule constraint engine, and shift schedule publishing module to complete daily shift scheduling; the database server stores employee information, job information, historical shift schedules, execution feedback, shift scheduling database, and experience replay library; the training server is configured with GPU or AI accelerator cards, reads the experience replay library according to preset cycles or trigger conditions, updates the multi-agent shift scheduling strategy model and incremental rescheduling strategy model, and then provides the updated model parameters to the application server for invocation.

[0078] Figure 5 The data flow relationship between the various components is as follows: external business systems and terminals send scheduling parameters and dynamic events to the network communication interface; the network communication interface writes the data to the application server and the database server; the application server reads historical scheduling, rule configuration and model parameters from the database server and outputs the first scheduling scheme or the second scheduling scheme; the execution feedback is written back to the experience playback library, the training server updates the model based on the feedback samples, and after being verified by the model management and verification module, it is provided to the application server for calling.

[0079] The method provided by this invention constructs state data representing the relationship between employees, positions, and time, and generates action masks to filter out candidate actions that violate hard constraints. This allows for the pre-filtering of illegal actions before scheduling decisions are made, improving scheduling efficiency. A multi-agent reinforcement learning model is used to output scheduling decision information, which is then used by a joint action generator or local constraint solver to form joint scheduling actions based on the decision information and action masks. This achieves the fusion of multi-agent decision information and the generation of joint actions, solving the problem that a single agent cannot effectively address the collaborative relationship between employees, positions, and business needs. This accelerates scheduling speed and demonstrates good scenario adaptability.

[0080] This embodiment also provides a multi-agent scheduling device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0081] This embodiment provides a multi-agent scheduling device, such as... Figure 6 As shown, it includes: The parameter acquisition module 601 is used to acquire the scheduling parameters for the scheduled period. Data construction module 602 is used to discretize the shift scheduling period into multiple time slices, and construct state data representing the relationship between employees, positions and time based on the scheduling parameters and the time slices; The mask generation module 603 is used to generate an action mask based on the scheduling constraints, and the action mask is used to block candidate actions that violate hard constraints. The scheduling decision module 604 is used to input the state data and the action mask into the multi-agent reinforcement learning model to obtain the scheduling decision information output by the multi-agent reinforcement learning model. The scheme generation module 605 is used to generate joint scheduling actions by a joint action generator or a local constraint solver based on the scheduling decision information and the action mask, thereby obtaining a first scheduling scheme.

[0082] The multi-agent-based scheduling device provided in this embodiment of the invention can execute a multi-agent-based scheduling method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the various modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0083] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, it implements the multi-agent-based scheduling method shown in the above embodiments.

[0084] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0085] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A multi-agent scheduling method, characterized in that, include: Obtain the scheduling parameters for the pending scheduling period; The scheduling period is discretized into multiple time slices, and state data representing the relationship between employees, positions and time is constructed based on the scheduling parameters and the time slices. An action mask is generated based on the scheduling constraints, and the action mask is used to filter out candidate actions that violate the hard constraints. The state data and the action mask are input into the multi-agent reinforcement learning model to obtain the scheduling decision information output by the multi-agent reinforcement learning model. The joint action generator or the local constraint solver generates joint scheduling actions based on the scheduling decision information and the action mask to obtain the first scheduling scheme.

2. The method according to claim 1, characterized in that, The method further includes: When a dynamic event is detected, an event impact subgraph is determined, the shift status outside the event impact subgraph is kept fixed, the scheduling actions within the event impact subgraph are re-determined, and a second scheduling plan is generated. The event impact subgraph is a local scheduling range related to the dynamic event. The reward value is determined based on the second scheduling plan; The decision on whether to issue the second scheduling plan is based on the reward value.

3. The method according to claim 1, characterized in that, The action mask is a binary action mask, denoted as M(e,p,t,a)∈{0,1}, where e represents the employee, p represents the job position, t represents the time slice, and a represents the candidate scheduling action. M=1 indicates that the candidate scheduling action is allowed to enter the candidate action pool, and M=0 indicates that the candidate scheduling action is blocked due to violation of hard constraints. The hard constraints include at least one of the following: employee unavailable time, job skill mismatch, maximum continuous working time exceeding the limit, minimum rest interval insufficient, working hours exceeding the limit, executed shifts, and locked shifts.

4. The method according to claim 1, characterized in that, The multi-agent reinforcement learning model includes an employee agent, a job agent, and a demand agent. The process of inputting the state data and the action mask into the multi-agent reinforcement learning model to obtain the scheduling decision information output by the model includes: The state data and the action mask are input into the employee agent to obtain the employee acceptability score and candidate action probability output by the employee agent. The status data and the action mask are input into the job agent to obtain the job coverage gap and job priority output by the job agent. The state data and the action mask are input into the demand agent to obtain the demand change intensity output by the demand agent.

5. The method according to claim 4, characterized in that, The process of generating joint scheduling actions by a joint action generator or a local constraint solver based on the scheduling decision information and the action mask includes: Based on the aforementioned job coverage gaps, target jobs and target time slots are determined; Based on the employee acceptability score and the intensity of demand change, a candidate action pool is formed for the target position and the target time slice; The candidate scheduling actions that violate hard constraints in the candidate action pool are eliminated using the action mask to obtain candidate action combinations. The joint scheduling actions are determined from the candidate action combinations with the objectives of job coverage, labor costs, employee preferences, work hour fairness and schedule disturbance costs.

6. The method according to claim 4, characterized in that, The process of generating joint scheduling actions by a joint action generator or a local constraint solver based on the scheduling decision information and the action mask specifically includes: The generation process of the joint scheduling action is decomposed into an abstract shift construction stage and a specific personnel assignment stage by adopting a spatiotemporal decoupling strategy. In the abstract shift construction stage, based on the job coverage gap and the intensity of demand change, the abstract shift segments to be adjusted are determined, and the abstract shift segments include job attributes, time attributes and skill attributes; During the specific personnel assignment phase, based on the employee acceptability score and the action mask, the abstract shift fragment is mapped to the target employee node to generate a joint scheduling action containing the specific employee identifier.

7. The method according to claim 2, characterized in that, The reward value is calculated using a multi-objective reward function, which employs the following formula: Wherein, COV represents job coverage or demand fulfillment rate, COST represents labor cost deviation, VIOL represents the number of hard constraint violations or the degree of violation, PREF represents employee preference fulfillment rate, FAIR represents work hour fairness, STAB represents schedule stability, QUAL represents service quality indicators, and DIST represents schedule disturbance cost; α, β, γ, δ, ε, η, θ, and λ are weighting parameters.

8. The method according to claim 1, characterized in that, The status data includes an employee-position-time status tensor and a scheduling relationship diagram. The employee-position-time status tensor is a three-dimensional tensor, wherein the employee dimension records at least one of the following employee characteristics: availability identifier, scheduled work hours, continuous working hours, minimum rest interval satisfaction identifier, employee preference matching value, historical refusal probability, and historical lateness probability; the position dimension records at least one of the following position characteristics: position demand gap, position priority, and unit labor cost; the time dimension records at least one of the following time slice characteristics: time period type and peak period identifier; each element in the tensor records a matching feature, which includes at least one of skill matching degree and position qualification level; the scheduling relationship diagram consists of employee nodes, position nodes, and demand nodes; wherein, employee nodes record employee skills, available time, employee preferences, historical work hours, and attendance feedback; position nodes record position headcount requirements, skill qualification requirements, position priority, and position opening time; and demand nodes record predicted customer flow, order volume, call volume, task volume, and service level targets.

9. The method according to claim 2, characterized in that, The step of determining whether to release the second scheduling plan based on the reward value and the disturbance cost specifically includes: Extract the specific adjustment instructions made in response to the dynamic event from the second scheduling scheme as the target scheduling action. The specific adjustment instructions include personnel assignment instructions, shift cancellation instructions or time change instructions. Based on the current environmental state of the event influence subgraph, the target scheduling action is replaced with the baseline default action before adjustment, and a comparative scheduling result is generated by simulation. Calculate the difference in business indicators between the second scheduling plan and the comparison scheduling result, and output the difference in business indicators as the basis for adjusting the target scheduling action.

10. The method according to claim 2, characterized in that, The event impact subgraph includes the affected time window, affected positions, skill groups related to the affected positions, currently scheduled employees, alternative candidate employees, relevant shift nodes, and at least one of the following relationships among the relevant shift nodes: skill matching relationship, time availability relationship, position demand relationship, and shift replacement relationship. Determining the event impact subgraph specifically includes: Obtain the direct impact intensity of the dynamic event on the initially affected positions; Based on a preset job skill association map, the propagation impact intensity of the direct impact intensity to the associated skill group is calculated, wherein the associated skill group shares the same skill resources or has a job collaboration relationship with the initially affected job. If the propagation impact intensity is greater than the preset disturbance threshold, the associated skill group will be included in the event impact subgraph, and the existing shifts within the associated skill group will be frozen.

Images