A government work order scheduling system based on deep reinforcement learning

Abstract

The present invention relates to the field of artificial intelligence and government service technology, and discloses a government work order scheduling system based on deep reinforcement learning, including: an intelligent agent scheduling module, which configures deep reinforcement learning scheduling intelligent agents for multiple government service areas, outputs execution work order allocation and priority ranking; a fairness monitoring module, which defines and monitors fairness indicators; a bias analysis module analyzes attention patterns in scheduling decisions, identifies potential bias features associated with changes in fairness indicators, and quantifies the impact of potential bias features on scheduling fairness; an intelligent agent optimization module, which optimizes the scheduling intelligent agent; a collaborative scheduling and auditing module, which builds a collaborative scheduling system and records the scheduling process to form an auditable log; the present invention makes the originally opaque deep reinforcement learning decision-making process understandable by integrating an explainable artificial intelligence module, thereby improving the transparency of scheduling decisions.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and government service technology, and more specifically, to a government work order scheduling system based on deep reinforcement learning. Background Art

[0002] With the advancement of smart cities and digital government, intelligent dispatching of government service work orders has become a key component in improving urban management efficiency and public service quality. Currently, traditional methods for dispatching government service work orders rely primarily on manual experience or simple rule-based systems, resulting in low efficiency, difficulty coping with complex scenarios, and a lack of dynamic adjustment capabilities.

[0003] Deep reinforcement learning, a cutting-edge technology in artificial intelligence, is being increasingly applied to government work order scheduling due to its autonomous learning and optimized decision-making capabilities. However, existing scheduling systems based on deep reinforcement learning still face the following technical challenges: Deep reinforcement learning models are often viewed as "black boxes," and their internal decision-making processes are opaque and difficult for humans to understand and explain. This leads to a lack of trust in scheduling decisions by government officials and the public, and increases the difficulty of system maintenance and optimization. Existing scheduling systems often lack a clear definition and quantitative assessment of fairness, which can easily lead to scheduling biases against specific regions or groups, making them unable to effectively ensure the fairness of government services, especially in situations of uneven resource allocation and complex work order types. Traditional scheduling systems struggle to balance group fairness with individual urgent needs and lack flexible negotiation mechanisms to respond to emergencies or special circumstances, often resulting in either sacrificing overall fairness or ignoring individual urgent needs. Existing systems also lack a comprehensive audit mechanism, making it difficult to comprehensively record and analyze the scheduling process, hindering problem tracing and continuous system optimization.

[0004] Therefore, there is an urgent need for a government work order scheduling method that can simultaneously solve technical problems such as decision transparency, scheduling fairness, multi-level demand balance, and auditable results. Summary of the Invention

[0005] The present invention provides a government work order scheduling system based on deep reinforcement learning, which solves the technical problems in related technologies such as the opaque decision-making of deep reinforcement learning models, the difficulty in ensuring scheduling fairness, the difficulty in balancing group and individual needs, and the difficulty in auditing the scheduling process.

[0006] The present invention provides a government work order scheduling system based on deep reinforcement learning, comprising:

[0007] The agent scheduling module configures deep reinforcement learning scheduling agents for multiple government service areas, outputs execution work order allocation and priority ranking, and integrates an explainable artificial intelligence module in each agent to provide decision explanations;

[0008] The fairness monitoring module defines and monitors fairness indicators based on the results of executing work order allocation and prioritization, measuring the fairness of service between regions and the fairness of handling specific types of work orders;

[0009] The bias analysis module analyzes attention patterns in scheduling decisions based on fairness metric monitoring results, identifies potential bias features associated with changes in fairness metrics, and quantifies the impact of potential bias features on scheduling fairness.

[0010] The agent optimization module optimizes the scheduling agent based on the results of quantifying potential bias characteristics;

[0011] The collaborative scheduling and auditing module builds a collaborative scheduling system based on the optimization results of the scheduling agent and records the scheduling process to form an auditable log.

[0012] Furthermore, the deep reinforcement learning scheduling agent adopts an ActorCritic architecture, where the Actor network outputs scheduling actions and the Critic network evaluates the value of actions.

[0013] Furthermore, the explainable artificial intelligence module is implemented based on the attention mechanism, which provides an explanation of the decision-making process by analyzing the attention weight distribution of the scheduling agent on the input features.

[0014] Furthermore, fairness indicators include macro fairness indicators and micro fairness indicators. Macro fairness indicators include:

[0015] Average response time difference, used to measure the degree of difference in work order response time between different regions;

[0016] Resource allocation balance is used to measure the degree of balance among the processing personnel's workload.

[0017] Furthermore, the micro-fairness indicators include:

[0018] The timely processing rate of high-priority work orders is used to measure the proportion of high-priority work orders that are completed within the specified time limit;

[0019] The personalized satisfaction score of special needs tickets is used to measure the satisfaction level of resolving special needs tickets.

[0020] Furthermore, the optimization of the scheduling agent includes:

[0021] Adjust feature weights or inputs to reduce bias through feature transformation, feature weighting, and feature masking;

[0022] Optimize the reward function and introduce fairness-related components into the original reward function to form a comprehensive reward function that includes efficiency and fairness;

[0023] Data augmentation or resampling balances the training data distribution through sampling techniques or synthetic sample generation techniques.

[0024] Furthermore, the collaborative scheduling system includes a meta-coordinator, a negotiation model and a reputation scoring model. The meta-coordinator is an upper-level reinforcement learning agent. Its input includes the macro-fairness indicators and micro-fairness indicators of each regional agent and the resource status. Its output includes the reward function adjustment parameters and resource allocation suggestions for the lower-level agents.

[0025] Furthermore, the negotiation model defines a collaborative request and response protocol between regional agents;

[0026] The reputation scoring model dynamically updates the reputation score of the regional agent according to the collaborative behavior of the regional agent, and is used to adjust the weight of the regional agent in the negotiation.

[0027] Furthermore, the auditable log records scheduling decisions, decision explanations, fairness indicator changes, meta-coordinator guidance parameters, negotiation processes, and reputation score changes, and uses a multi-level record structure to ensure data integrity and searchability.

[0028] The present invention provides a storage medium comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned government work order scheduling system based on deep reinforcement learning.

[0029] The beneficial effects of this invention are as follows: by integrating an explainable artificial intelligence module, especially an explanation method based on the attention mechanism, the originally opaque deep reinforcement learning decision-making process becomes understandable, thereby improving the transparency of scheduling decisions and enhancing the trust of government officials and the public in the system;

[0030] By establishing a fairness indicator system at both the macro and micro levels, we achieved objective quantification and continuous monitoring of scheduling fairness. Furthermore, by optimizing the reward function and other methods, we significantly improved processing fairness across regions and between different work order types.

[0031] By analyzing decision-making attention patterns and correlating them with changes in fairness metrics, it can effectively identify and mitigate potential scheduling biases, solving the technical problem of traditional systems that make it difficult to detect and eliminate implicit biases.

[0032] By building a collaborative scheduling system that includes a meta-coordinator, a negotiation model, and a reputation scoring model, a flexible balance between group fairness and individual emergency needs is achieved, enabling the system to maintain overall fairness while efficiently responding to emergencies and special individual needs.

[0033] By establishing an auditable log system with a multi-level record structure, comprehensive data support and traceability capabilities are provided for the scheduling process, which helps to locate problems, hold people accountable and continuously optimize the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a module diagram of a government work order scheduling system based on deep reinforcement learning in the present invention;

[0035] Figure 2 is a flow chart of the agent scheduling module in the present invention;

[0036] Figure 3 is a flow chart of the fairness monitoring module in the present invention;

[0037] Figure 4 is a flow chart of the bias analysis module of the present invention;

[0038] Figure 5 is a flow chart of the agent optimization module in the present invention;

[0039] Figure 6 It is a flow chart of the collaborative scheduling and auditing module in the present invention. DETAILED DESCRIPTION

[0040] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0041] At least one embodiment of the present invention discloses a government work order scheduling system based on deep reinforcement learning, such as Figures 1 to 6 Shown, including:

[0042] The agent scheduling module configures deep reinforcement learning scheduling agents for multiple government service areas, outputs execution work order allocation and priority ranking, and integrates an explainable artificial intelligence module in each agent to provide decision explanations;

[0043] It should be noted that, given the regional nature of government work orders, the embodiments of this application propose configuring a deep reinforcement learning scheduling agent for each government service area. The deep reinforcement learning scheduling agent is used to autonomously learn and execute the allocation and priority sorting of work orders based on the real-time work order data, available resource data, and preset scheduling goals of the area in which it is located. Specifically, the following steps are included:

[0044] Step 1.1: Build a deep reinforcement learning scheduling agent.

[0045] Specifically, an independent deep reinforcement learning agent is configured for each government area. Optionally, an ActorCritic-based reinforcement learning algorithm is adopted; the Actor network is used to output scheduling actions (such as assigning work orders to specific processing personnel or adjusting work order priorities), and the Critic network is used to evaluate the value of the actions output by the Actor network.

[0046] The state of the agent This may include information such as the characteristics of the current work order queue (e.g., the number of work orders, type distribution, average waiting time), processing resource status (e.g., the load of each processing staff, skill matching), etc. is a specific scheduling decision. The agent learns by interacting with the environment to maximize the cumulative reward:

[0047] ;

[0048] in represents the cumulative discounted reward (the sum of all possible rewards that the agent may obtain in the future under the current state), For the agent at time step The reward value obtained after performing the action, is a discount factor used to balance the importance of immediate rewards and future rewards, Indicates that the distance from the current time The reward obtained after the step is discounted by the factor Exponential decay is performed, that is, the weight of the reward is multiplied by one for each step away from the current moment. ; is the time step increment, represents the sum operation, represents the cumulative reward considering theoretically infinite time steps.

[0049] Step 1.2, integrating the explainable artificial intelligence module;

[0050] In addition, an XAI module is integrated into each deep reinforcement learning scheduling agent. It should be noted that the XAI module is intended to provide insights into the agent's decision-making process. In one embodiment of the present application, for an agent based on ActorCritic, an XAI module based on an attention mechanism can be optionally constructed. The core of the attention mechanism is to analyze the Actor network's response to a certain input state vector. (including work order characteristics, resource status, etc.) and decides to output a certain scheduling action (For example, when work order A is assigned to processing person B), the input status The attention weight of each specific feature (such as the urgency of the work order, the current load of the handler B, the skill matching degree between work order A and handler B, etc.) Here, the sum of the attention weights of all features is 1, which is expressed as:

[0051] ;

[0052] in is the dimension of the feature vector (the number of features contained in the input state), For the The attention weight of each feature (indicating the importance of the feature to the decision), Represents a sum operation.

[0053] Generally speaking, features with high attention weights can be considered key factors that significantly influence the agent's specific dispatch decision. For example, in an urban management work order dispatch scenario, when a high-urgency work order for a collapsed main road is assigned to a worker whose workload is nearing saturation but who possesses specialized road repair skills, the attention XAI module might indicate that the features "work order urgency" (weight 0.45) and "skill match" (weight 0.35) have high attention weights, while "personnel's current workload" (weight 0.10) has a relatively low weight, explaining how this seemingly unbalanced dispatch decision actually prioritizes road safety and specialized repair needs.

[0054] In one exemplary implementation, the attention weights may be calculated as follows:

[0055] ;

[0056] in and are the weight matrices used to calculate the attention score and the transition state, is the normalization function, is the hyperbolic tangent activation function, is the time step The state vector of is the calculated attention weight vector.

[0057] Optionally, the XAI module can also adopt an interpretation method based on SHAP (SHapley Additive exPlanations) values, which is based on Shapley values ​​in game theory and can assign each input feature its contribution to the current decision.

[0058] The final output is a set of key influencing features and their corresponding attention weights for each dispatch decision, forming an explainable explanation of the decision. In practice, these explanations can be presented to dispatch managers through intuitive visualization interfaces, such as bar charts or heat maps showing the weight distribution of each feature, or using natural language generation technology to convert the weights into easy-to-understand text descriptions, such as "The main reasons for prioritizing this work order are its extremely high urgency (45% influence) and its high match with the processing staff's professional skills (35% influence)."

[0059] The fairness monitoring module defines and monitors fairness indicators based on the results of executing work order allocation and prioritization, measuring the fairness of service between regions and the fairness of handling specific types of work orders;

[0060] To comprehensively evaluate and optimize the fairness of scheduling strategies, a fairness indicator system covering both macro and micro levels is established, and these indicators are continuously monitored online using the output of the XAI module in the agent scheduling module. Specifically, the following steps are involved:

[0061] Step 2.1, define macro fairness indicators;

[0062] Macro fairness indicators are used to measure service fairness across different regions. The input data is the historical scheduling data and work order data of agents in each region. Specific indicators may include, but are not limited to:

[0063] Average response time difference , characterized by the standard deviation of the average response time of each region.

[0064] ;

[0065] in represents the average response time difference, is the total number of government service areas, For the region The average response time, is the average response time of all regions, represents the sum operation, Represents the square root operation.

[0066] Lower A value of indicates smaller differences in ticket response times across regions, i.e., higher inter-region fairness.

[0067] Resource allocation balance , measured by the Gini coefficient of the workload distribution of the processing personnel:

[0068] ;

[0069] in Indicates the balance of resource allocation, Indicates the total number of processing personnel, Indicates the The workload of the processor, Indicates the The workload of the processor, represents the absolute value of the difference between the workloads of two processors, represents a double summation over all handlers, represents the average workload of all processing personnel, represents the normalization factor.

[0070] Optionally, you can also define regional service quality difference indicators This indicator is calculated based on the service satisfaction score or complaint rate of each region to reflect the differences in service quality actually felt by residents in different regions.

[0071] The output is the quantitative value of each macro fairness indicator, for example, (Unit: minutes), (The maximum score is 1). These values ​​can be used for subsequent monitoring and adjustment of scheduling strategies.

[0072] Step 2.2, define micro-fairness indicators;

[0073] The micro-fairness indicators focus on the satisfaction of specific types of work orders or individual needs. The input data is the work order processing history and the handling status of specific types of work orders (such as high priority). Specific indicators include, but are not limited to:

[0074] Timely processing rate of high-priority work orders , calculate the proportion of high-priority work orders that are processed within the specified time limit:

[0075] ;

[0076] in Indicates the timely processing rate of high-priority work orders, The number of high-priority work orders processed on time, The total number of high-priority work orders.

[0077] Personalized satisfaction scores for special needs tickets , obtained through subsequent return visits or feedback, or automatically evaluated based on the degree of match between the work order solution and the applicant's specific needs. The score range can be set to 0-100.

[0078] Optionally, you can also define priority indicators for handling work orders for vulnerable groups This indicator measures the degree of priority the system gives to government work orders submitted by vulnerable groups such as the elderly and people with disabilities. It can be calculated by comparing the average response time ratio of such work orders with that of ordinary work orders.

[0079] The output is the quantitative value of each micro-fairness indicator, for example, (meaning 92% of high-priority tickets were processed within the specified time limit), (A full score of 100 indicates a high overall satisfaction level for special demand work orders.) These indicators can be used to evaluate the fairness of the scheduling system's response to different types of individual demands.

[0080] Step 2.3, monitor fairness indicators;

[0081] Using the scheduling decision explanations output by the XAI module in step 1.2, combined with the actual results of processing specific work orders, we continuously calculate and record the dynamic changes of the aforementioned macro and micro fairness indicators. The output is a monitoring report containing the time series of each fairness indicator.

[0082] Specific types of work orders include:

[0083] High-priority work orders, including those related to public safety emergencies such as arterial road pavement collapse, major water supply network leaks, and public facility safety hazards, are marked as high priority due to their high urgency, wide impact, or high potential risk.

[0084] Special needs work orders, including applications for barrier-free facilities for people with disabilities, exclusive services for the elderly, and special medical needs. Applicants for these work orders have special service needs and usually require personalized solutions;

[0085] Work orders from vulnerable groups, including government service application orders from vulnerable groups in society such as low-income families, the elderly, people with disabilities, migrant workers, etc., need to ensure that they have fair access to services.

[0086] Cross-departmental collaborative work orders, which are complex work orders that require collaboration among multiple government departments or service areas, such as comprehensive work orders involving urban planning, environmental protection, and transportation;

[0087] Work orders with high sensitivity to people's livelihood are work orders that are closely related to people's livelihood and have high social attention, such as work orders for application for heating, water supply, medical care and other services that are related to the basic living security of a large number of residents.

[0088] The bias analysis module analyzes attention patterns in scheduling decisions based on fairness metric monitoring results, identifies potential bias features associated with changes in fairness metrics, and quantifies the impact of potential bias features on scheduling fairness.

[0089] Use XAI tools to analyze patterns in the agent's historical decisions and combine them with the monitoring results of fairness indicators to identify and quantify potential sources of bias that lead to unfair scheduling. The specific steps include:

[0090] Step 3.1, analyzing decision attention patterns;

[0091] Based on the attention weights provided by the XAI module for each scheduling decision in step 1.2, analyze the distribution pattern of attention paid by each regional agent to different features during the decision-making process. Optionally, the analysis can be performed in the following ways:

[0092] Statistical analysis: Calculate the average attention weight of each regional agent on each input feature over a period of time (e.g., the last 30 days) and compared with the expected reasonable distribution.

[0093] Cluster analysis: Cluster the attention weight patterns of agents in each region to identify groups of agents with similar preferences or biases.

[0094] Time series analysis: Tracking the evolution of attention weights over time to detect the emergence of emergent or systematic biases.

[0095] For example, in real-world applications, analysis may reveal that a regional agent assigns an unreasonably high weight to the "applicant's street" feature when processing work orders from different streets (an average of 0.32, compared to around 0.08 in other regions). This may indicate regional bias in the agent. The input data here is the decision explanation data provided by the XAI module, specifically the attention weights. The corresponding output is a report on the attention distribution characteristics of each regional agent for each decision feature, including unusual patterns that may indicate bias.

[0096] Step 3.2, correlation analysis of fairness indicator changes;

[0097] The fairness indicator deterioration events detected in step 2.3 (for example, A significant increase or a specific type of work order Perform correlation analysis with the high or disproportionately low attention features identified in step 3.1. You can choose the following methods for correlation analysis:

[0098] Correlation analysis: Calculate the Pearson correlation coefficient or Spearman rank correlation coefficient between the attention weight of each feature and the fairness indicator.

[0099] Causal Inference: Optionally, employ counterfactual analysis or intervention methods to assess the causal impact of changes in feature-specific attention weights on fairness metrics.

[0100] Event detection: Identify the time points when fairness metrics suddenly deteriorate and analyze significant changes in attention patterns before and after.

[0101] For example, the analysis might find that an increased attention weight for the “applicant socioeconomic status” feature is associated with lower income areas. There is a positive correlation of 0.78 between the two, indicating that this feature may be a key factor in unfair scheduling between regions. The input data here is the time series data of fairness indicators and the analysis results of the attention patterns of agents in each region. The corresponding output is a list of potential bias features and a quantification of their correlation strength with changes in fairness indicators.

[0102] Step 3.3, quantify the impact of bias;

[0103] For the potential bias features identified in step 3.2, quantitatively evaluate their impact on scheduling fairness. The following methods can be used for quantification:

[0104] Counterfactual analysis: This involves temporarily modifying the weights or values ​​of specific characteristics in a simulated environment to assess the impact on fairness indicators. For example, the weight of the "applicant's socioeconomic status" characteristic can be artificially set to zero, and then the changes in fairness indicators before and after the modification can be compared.

[0105] Sensitivity analysis: Optionally, systematically vary the weights of potentially biased features and plot the changes in weights against fairness metrics to quantify sensitivity.

[0106] A / B testing: In an isolated environment, compare the original model with a model that has had a specific bias removed to evaluate the difference in fairness metrics.

[0107] For example, in a real-world application scenario, counterfactual analysis might show that reducing the attention weight of the “applicant’s street” feature from 0.32 to the system average of 0.08 would increase the probability of the applicant’s street being compared to other regions. A 41% reduction (from 18.5 minutes to 10.9 minutes). The input data here is a list of potential bias characteristics and the current state of the scheduling environment. The corresponding output is a quantified value of the impact of each bias characteristic on the fairness indicator, as well as an estimate of the fairness improvement that would be achieved by eliminating or mitigating these biases.

[0108] The agent optimization module optimizes the scheduling agent based on the results of quantifying potential bias characteristics;

[0109] The embodiments of this application further intervene and optimize the learning process and strategy of each regional deep reinforcement learning scheduling agent to improve scheduling fairness. Specifically, the following steps are included:

[0110] Step 4.1, adjust feature weights or inputs;

[0111] Based on the analysis results of the bias analysis module, for scheduling inequities caused by improper weighting of specific input features (for example, "regional economic level") or biased information in the features themselves, the following methods can be optionally used to adjust feature processing:

[0112] Feature transformation: Sensitive features are transformed through data preprocessing techniques. For example, the feature “the street where the applicant belongs” is desensitized and encoded into a form that does not directly reflect the identity of a specific street.

[0113] Feature weighting: Apply different initial weights to different features at the model input layer to reduce the impact of identified high-bias features.

[0114] Feature masking: Certain sensitive features can be temporarily or permanently removed from the agent's input to evaluate the changes in scheduling fairness before and after the removal.

[0115] For example, in urban management work order scheduling, if the "regional population density" feature is found to reduce service quality in high-density, older neighborhoods, this feature can be combined with other regional characteristics (such as infrastructure age) into a composite indicator to reduce the direct impact of a single factor. The input data here is the original work order feature data and the bias feature analysis results output by the bias analysis module. The output is the adjusted training dataset or model input configuration after feature engineering.

[0116] Step 4.2, optimize the reward function;

[0117] Optionally, fairness-related components can be integrated into the reward function of the reinforcement learning agent to guide the agent to learn a fairer scheduling strategy. The reward function can be modified in the following ways:

[0118] Linear combination: a reward function that directs raw efficiency towards and fairness indicators Linear combination:

[0119] ;

[0120] in is the modified reward function value, is the original efficiency-oriented reward function value, is the fairness indicator function value, is an adjustable fairness weight parameter, is the weight of efficiency reward.

[0121] Multi-objective optimization: The fairness indicator is used as an independent optimization objective, and a multi-objective reinforcement learning method is used to simultaneously optimize efficiency and fairness.

[0122] Constraints: Set the fairness metric as a constraint so that efficiency rewards are only counted if a certain fairness threshold is met.

[0123] The modified reward function is:

[0124] ;

[0125] in is the modified total reward value, is the original efficiency bonus value, is the average response time difference index value, is the maximum allowable difference in average response time, Indicates the ratio of the current difference to the maximum allowable difference, The normalized value representing the fairness of response time between regions, The timely processing rate of high-priority work orders.

[0126] Step 4.3, data augmentation or resampling;

[0127] In addition, if the bias is related to the underrepresentation of samples of certain groups or scenarios in the training data, the following methods can be used to improve data balance:

[0128] Oversampling technology: Oversample the work order data of underrepresented groups or scenarios to increase their proportion in the training data.

[0129] Synthetic sample generation: Generate new synthetic training samples through data augmentation algorithms (e.g., SMOTE algorithm). SMOTE algorithm creates new artificial samples by interpolating between minority class samples, thereby increasing the representation of the minority class in the training data.

[0130] Data weight adjustment: Assign different weights to different types of samples during training to increase the learning importance of underrepresented samples.

[0131] For example, in an urban management scenario, if maintenance work orders from older residential areas are found to represent a disproportionately low proportion of a dataset (only 5%), the SMOTE algorithm can be used to generate additional synthetic samples to increase their proportion to a level comparable to other types of work orders (e.g., 15% to 20%). The input data here is the original training dataset and the results of the sample distribution analysis. The output is a balanced, enhanced training dataset, in which the proportion of underrepresented work order types is effectively increased.

[0132] The collaborative scheduling and auditing module builds a collaborative scheduling system based on the optimization results of the scheduling agent and records the scheduling process to form an auditable log;

[0133] In order to effectively balance the overall fairness between regions and the processing needs of individual emergency work orders within the region, and to achieve the auditability of the scheduling process, the embodiment of this application builds a collaborative scheduling system that includes a meta-coordinator, a negotiation model, and a reputation scoring model, and records all relevant data. Specifically, it includes the following steps:

[0134] Step 5.1, build the meta-coordinator;

[0135] It should be understood that the meta-coordinator is responsible for monitoring and coordinating the behavior of each regional agent from a global perspective to balance overall efficiency and multi-level fairness. The meta-coordinator can be constructed as an upper-level reinforcement learning agent. The state input of the upper-level agent may include the macro-fairness indicators of each regional agent (such as the one from step 2.1). and ), micro-level fairness indicators (such as and ) as well as resource utilization and work order backlog in each area.

[0136] Optionally, the action space of the meta-coordinator may include one or more of the following:

[0137] Parameter adjustment: Modify the reward function of each region agent in the lower layer in step 4.2 The weight coefficient (such as );

[0138] Resource Recommendation: Issue resource allocation recommendations to agents in specific areas (for example, recommend that Area A support certain processing personnel in Area B);

[0139] Priority adjustment: Temporarily increase or decrease the processing priority of a certain type of work order in a specific area.

[0140] The reward function of the meta-coordinator is expressed as:

[0141] ;

[0142] in represents the global reward function value of the meta-coordinator, represents the global efficiency index value, represents the macro fairness indicator function, represents the micro fairness indicator function, Indicates all The average response time difference indicator set for each region, Indicates all A set of indicators for timely processing rate of high-priority work orders in each region, 、 and are the weight coefficients of global efficiency, macro fairness, and micro fairness, which can be dynamically adjusted according to actual scheduling needs; Represents the region number in the reward function of the meta-coordinator.

[0143] For example, in the city management work order scheduling scenario, when the meta-coordinator detects the It is much lower than other areas (e.g. only 8 minutes, while the system average is 20 minutes), but the high priority ticket processing rate in area B is When the fairness weight of the agent in region A is not good (e.g. only 75%, lower than the target value of 90%), it may Raise the ratio from 0.2 to 0.4, or directly issue a directive to encourage Area A to temporarily share three processing personnel to support Area B. This will improve Area B's emergency work order processing capacity and overall fairness indicators while sacrificing a small amount of efficiency in Area A. The output is the specific guidance parameters or collaborative scheduling instructions generated by the meta-coordinator to the agents in each area.

[0144] Step 5.2: Establish a negotiation and reputation scoring mechanism;

[0145] When regional agents handle urgent or special types of individual work orders, they may need to sacrifice a certain degree of macro-fairness to meet specific needs. To this end, a negotiation mechanism between regional agents needs to be established.

[0146] Negotiation Model: This model defines the interaction protocol between regional agents for requesting and responding to collaborative tasks. Negotiation requests can contain the following elements:

[0147] Request type: such as resource borrowing, work order transfer, emergency support, etc.

[0148] Resource requirements: type, quantity, expected duration;

[0149] Priority indicator: indicates the urgency of the request;

[0150] Credit payment: the credit points you are willing to spend for this

[0151] Reputation scoring model, maintaining a dynamic reputation score for each regional agent , updated according to the following rules:

[0152] Active collaboration rewards: If the area Respond to and complete collaborative requests from other regions, then

[0153] ;

[0154] in Indicates area At the time point Reputation score for active collaboration, Indicates area At the time point The reputation score, Represents the positive reward increment for active collaboration.

[0155] Penalty for refusing reasonable request: If you refuse to cooperate without sufficient reason,

[0156] ;

[0157] in Indicates area At the time point Reputation score for rejecting reasonable requests, Indicates area At the time point The reputation score, Indicates the negative penalty for rejecting a reasonable request.

[0158] Resource consumption: When dealing with internal urgent needs,

[0159] ;

[0160] in Indicates area At the time point The reputation score of resource consumption, Indicates area At the time point The reputation score, Indicates the credit deduction for resource consumption.

[0161] For example, in an urban management scenario, if Region A faces a large number of flooding work orders due to a sudden downpour and lacks resources to handle them, it can request the temporary loan of three road management personnel from neighboring Region B for 24 hours, offering a payment of 10 credit points. Region B will then assess its own workload, Region A's credit score (assuming it's 85, which is high), and the level of urgency, and may agree to provide two personnel for 18 hours. Upon successful negotiation, Region A's credit score decreases by 8 points (based on the actual resources received), while Region B's credit score increases by 8 points after the task is completed. The input data here is each region's negotiation request, current credit score, and internal state. The output is the specific coordinated action after negotiation and the updated credit score.

[0162] Step 5.3, Recording and Auditing;

[0163] It should be noted that all regional agent scheduling decisions, decision explanations output by the XAI module, fairness indicator changes, meta-coordinator guidance parameters, negotiation process records, and reputation score changes can be recorded in the log system. This log system can have the following features:

[0164] Data integrity assurance: Optionally, hash chain or similar technology is used to ensure that log data is not tampered with. Each record contains the hash value of the previous record, forming a complete audit chain.

[0165] Multi-level record structure, including:

[0166] Basic layer: original scheduling decision and result data;

[0167] Explanation layer: decision explanation information generated by the XAI module;

[0168] Indicator layer: real-time values ​​and changing trends of various fairness indicators;

[0169] Coordination layer: Meta-coordinator’s intervention instructions and inter-regional negotiation records.

[0170] Retrieval and analysis interface: Optionally, provide an efficient retrieval mechanism that supports multi-dimensional queries and statistical analysis by time, region, decision type, fairness indicators, etc.

[0171] For example, in actual applications, when residents in area A complain that their work order processing time is too long, auditors can retrieve relevant records through the log system to analyze the area's This system comprehensively assesses the fairness of the system and identifies potential improvements by taking into account historical values, the decision patterns of the scheduling agents, and the interventions of the meta-coordinator. The input data here is all the data generated by each of the above steps. The output is a structured, searchable audit log database, which supports comprehensive post-event analysis of the scheduling system's operating status, fairness performance, and specific decisions.

[0172] A storage medium includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the above-mentioned government work order scheduling system based on deep reinforcement learning.

[0173] Here, the present invention provides an implementation example:

[0174] The proposed method was piloted at a city's government service center. The city is divided into four administrative regions, A, B, C, and D, and handles approximately 600 to 800 government work orders daily. These include matters related to municipal facility maintenance, environmental sanitation, community services, and traffic management. The urgency and complexity of these different work orders vary significantly.

[0175] A deep reinforcement learning scheduling agent based on the ActorCritic architecture was deployed for each of the four administrative regions. Each agent's state representation includes work order characteristics (including work order type, urgency, submission time, and geographic location) and resource state characteristics (number of processing personnel, professional skills, and current load). The application uses an attention mechanism as the XAI module, which calculates the influence of each state feature on the decision and generates a visual explanation.

[0176] For example, for a work order regarding "serious flooding on a main road," the dispatch agent's decision to assign it to a specific professional handler is explained by the following factors: work order urgency (weight 0.42), professional skill matching (weight 0.38), geographic proximity (weight 0.12), and the handler's current workload (weight 0.08). This allows managers to intuitively understand the decision logic and verify that the system prioritizes work order urgency and professional matching.

[0177] The macro-fairness indicators defined by the system include:

[0178] Average response time difference between regions Calculate the standard deviation of the average response time for work orders in the four regions, with the goal of keeping this value below 10 minutes.

[0179] Resource allocation balance : Use the Gini coefficient to calculate the balance of workload distribution among processing personnel in each region. The value range is 0-1, and the closer to 1, the more balanced the distribution.

[0180] Micro-level fairness indicators include:

[0181] Timely processing rate of high-priority work orders : Calculate the proportion of various emergency work orders that are completed within the specified time limit, with a target value of over 95%.

[0182] Special needs work order satisfaction : Obtained through follow-up visits or feedback, the target satisfaction score for work orders submitted for special groups such as the elderly and people with disabilities is 90 points or above (out of 100 points).

[0183] By analyzing the attention weights of the historical decisions of the agent in area A, the system found that when the agent processed work orders from different streets, there were significant differences in the attention weights of the "applicant's street" feature: the average weight of streets in the central city was 0.08, while the average weight of streets in the suburbs was 0.21, which indicates that there may be a regional bias towards work orders in suburban areas.

[0184] Further correlation analysis shows that the average response time for work orders in suburban streets is 28 minutes longer than that in the central urban area. Quantitative analysis shows that if the attention weight of the "applicant's street" feature is forcibly corrected to a consistent value of 0.10, the average response time for suburban streets can be shortened by approximately 18 minutes.

[0185] In response to the identified regional biases, the system took the following actions:

[0186] Feature processing optimization: The "applicant's street" feature is desensitized and the standardized distance from the street to the city center is used instead of the specific street name;

[0187] Reward function adjustment: Modify the reward function of the agent in area A and introduce a fairness component:

[0188] ;

[0189] in is the total reward value after introducing the fairness component, is the original efficiency bonus value, is the average response time difference index value, The timely processing rate of high-priority work orders.

[0190] Data augmentation: We oversampled work order data from remote suburban areas, increasing its proportion in the training data from 12% to 20%, matching its actual proportion.

[0191] The system deployed a meta-coordinator to continuously monitor the macro and micro fairness indicators of the four regions. When it was found that the number of high-priority work orders in region B increased sharply after a heavy rain, When the indicator drops to 78%, the meta-coordinator automatically sends a collaboration request to the adjacent areas A and C, suggesting the temporary deployment of a total of 5 processing personnel with relevant professional skills to support area B for 24 hours.

[0192] Region A agreed to support 3 personnel based on its current high reputation score (92 points), while Region C only sent 1 person due to its heavy workload. The system automatically recorded the negotiation process, including negotiation requests, responses from all parties, resource allocation results, and reputation score changes. After the support was completed, Region B The index recovered to 93%, and the overall The indicator rose only slightly by 2.8 minutes, remaining within an acceptable range.

[0193] Detailed records of the entire scheduling process are saved in the audit log. Subsequent management personnel discovered the problem of insufficient allocation of rainstorm emergency resources in Area B through log analysis and optimized the emergency resource allocation strategy accordingly.

[0194] The results of the decision-making understanding evaluation of government work order managers before and after the implementation of this application method are shown in Table 1:

[0195] Table 1: Decision-making understanding evaluation results of government work order managers

[0196]

[0197] Six months after the system was implemented, the changes in the multi-level fairness indicators are shown in Table 2:

[0198] Table 2: Changes in multi-level fairness indicators

[0199]

[0200] The above data verifies the significant effectiveness of this application method in improving decision-making transparency and enhancing multi-level fairness. It not only enables managers to better understand and trust scheduling decisions, but also achieves substantial improvements in processing fairness between regions and different work order types, while maintaining high overall scheduling efficiency.

[0201] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. A government work order scheduling system based on deep reinforcement learning, characterized by: include: The agent scheduling module configures deep reinforcement learning scheduling agents for multiple government service areas, outputs execution work order allocation and priority ranking, and integrates an explainable artificial intelligence module in each agent to provide decision explanations; The fairness monitoring module defines and monitors fairness indicators based on the results of executing work order allocation and prioritization, measuring the fairness of service between regions and the fairness of handling specific types of work orders; Fairness indicators include macro-fairness indicators and micro-fairness indicators. Macro-fairness indicators include: Average response time difference, used to measure the degree of difference in work order response time between different regions; Resource allocation balance, which is used to measure the degree of balance in processing staff workload; Micro-level fairness indicators include: The timely processing rate of high-priority work orders is used to measure the proportion of high-priority work orders that are completed within the specified time limit; Personalized satisfaction scores for special needs work orders, used to measure satisfaction with the resolution of special needs work orders; The bias analysis module analyzes attention patterns in scheduling decisions based on fairness metric monitoring results, identifies potential bias features associated with changes in fairness metrics, and quantifies the impact of potential bias features on scheduling fairness. The agent optimization module optimizes the scheduling agent based on the results of quantifying potential bias characteristics; The collaborative scheduling and auditing module builds a collaborative scheduling system based on the optimization results of the scheduling agent and records the scheduling process to form an auditable log; Among them, the intelligent agent uses the ActorCritic-based reinforcement learning algorithm to output scheduling actions and evaluate the value of the actions output by the Actor network; The state of the agent includes the characteristics of the current pending work order queue and processing resource status information. The agent learns by interacting with the environment to maximize the cumulative reward.

2. A government work order scheduling system based on deep reinforcement learning according to claim 1, characterized in that: The deep reinforcement learning scheduling agent adopts the ActorCritic architecture, where the Actor network outputs scheduling actions and the Critic network evaluates the value of actions.

3. The government work order scheduling system based on deep reinforcement learning according to claim 1 is characterized in that: The explainable artificial intelligence module is implemented based on the attention mechanism, and provides an explanation of the decision-making process by analyzing the attention weight distribution of the scheduling agent on the input features.

4. The government work order scheduling system based on deep reinforcement learning according to claim 1 is characterized in that: The optimization of the scheduling agent includes: Adjust feature weights or inputs to reduce bias through feature transformation, feature weighting, and feature masking; Optimize the reward function and introduce fairness-related components into the original reward function to form a comprehensive reward function that includes efficiency and fairness; Data augmentation or resampling balances the training data distribution through sampling techniques or synthetic sample generation techniques.

5. The government work order scheduling system based on deep reinforcement learning according to claim 1 is characterized in that: The collaborative scheduling system includes a meta-coordinator, a negotiation model, and a reputation scoring model. The meta-coordinator is an upper-level reinforcement learning agent. Its input includes the macro-fairness indicators and micro-fairness indicators of each regional agent and the resource status. Its output includes the reward function adjustment parameters and resource allocation suggestions for the lower-level agents.

6. A government work order scheduling system based on deep reinforcement learning according to claim 5, characterized in that: The negotiation model defines the collaborative request and response protocol between regional agents; The reputation scoring model dynamically updates the reputation score of the regional agent according to the collaborative behavior of the regional agent, and is used to adjust the weight of the regional agent in the negotiation.

7. The government work order scheduling system based on deep reinforcement learning according to claim 1 is characterized in that: The auditable log records scheduling decisions, decision explanations, fairness indicator changes, meta-coordinator guidance parameters, negotiation processes, and reputation score changes, and uses a multi-level record structure to ensure data integrity and retrievability.

8. A storage medium, characterized in that: It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a government work order scheduling system based on deep reinforcement learning as described in any one of claims 1-7.

Images