Reinforcement learning based process routing optimization method and system

Abstract

The application belongs to the technical field of process routing optimization, and discloses a process routing optimization method and system based on reinforcement learning; the method comprises the following steps: collecting real-time work order data and load data of a system, and cross-department data to obtain original data; pre-processing and standardizing the original data to obtain standard data; based on the configuration data of each department collected, a triadic graph is constructed, and a preset graph neural network algorithm is used to train the triadic graph to obtain a trained graph neural network model; the input of the graph neural network model is the triadic graph, and the output is a static feature vector; based on the standard data, a dynamic feature vector is constructed, and the dynamic feature vector and the static feature vector are spliced to obtain a state space vector; the cross-department collaboration efficiency is improved, the service compliance and stability are ensured, service interruption during a business peak period is avoided, compliance risks and business losses are reduced, and the collaboration efficiency and service quality of group management are improved.

Description

Technical Field

[0001] This invention relates to the field of process routing optimization technology, and more specifically, to a process routing optimization method and system based on reinforcement learning. Background Technology

[0002] In the service management scenarios of large enterprise groups, due to the complex organizational structure and diverse business needs, existing technologies generally adopt a group-wide multi-service desk management system to achieve cross-departmental collaboration. In actual business operations, most work orders have a default specific processing department, such as financial work orders belonging to the finance department and IT fault work orders belonging to the IT department. Therefore, while supporting each department to independently configure work order processing nodes, workflow rules, and other processes, as well as permission systems such as knowledge access scope and operation permission levels, and data standards such as information entry formats and key field definitions, the system also presets default routing paths based on business types and connects with OA, HR, and other systems through preset interfaces to achieve basic collaboration with priority processing by specific departments.

[0003] However, this department-centric model still reveals inherent limitations in practice. On the one hand, many work orders are not exclusively handled by a single department but require collaboration among multiple departments. For example, a payment system failure may involve the IT department, operations department, or network department. The process configurations, permission rules, and data standards of each department differ significantly. Cross-departmental data transmission requires multiple steps such as interface conversion, permission verification, and format alignment. The time consumed accumulates as more departments participate, leading to a decrease in collaboration efficiency. On the other hand, even exclusively handled work orders may fail to respond efficiently or even completely interrupt service due to excessive load on a specific department, such as a backlog of work orders several times higher than usual during peak business periods, sudden failures such as server crashes, or temporary adjustments to permissions such as compliance requirements changing processing permissions. Existing technologies lack flexible alternatives and can only passively wait for the specific department to recover, further exacerbating delays.

[0004] The consequences of this limitation are particularly pronounced: when specific departments can respond normally, delays in cross-departmental collaboration already cause bottlenecks in work order processing; when specific departments cannot respond, high-priority tasks such as financial transaction system failures will directly face processing delays, making it difficult to meet the response requirements of Service Level Agreements (SLAs), and even causing service interruptions during peak business periods. In fields such as finance and securities, where timeliness and compliance requirements are stringent, these problems may also violate regulatory provisions, leading to compliance risks and business losses, ultimately hindering the improvement of collaborative efficiency and service quality in group-wide management.

[0005] In view of this, the present invention proposes a process routing optimization method and system based on reinforcement learning to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned shortcomings of the prior art and achieve the above objectives, the present invention provides the following technical solution: a process routing optimization method based on reinforcement learning, comprising:

[0007] The system collects real-time work order data, load data, and cross-departmental data to obtain raw data; the raw data is then preprocessed and standardized to obtain standard data.

[0008] Based on the configuration data collected from various departments, a ternary graph is constructed, and a pre-defined graph neural network algorithm is used to train the ternary graph to obtain a trained graph neural network model. The input of the graph neural network model is the ternary graph, and the output is a static feature vector.

[0009] Based on standard data, a dynamic feature vector is constructed, and the dynamic feature vector and the static feature vector are concatenated to obtain a state space vector.

[0010] Based on historical cross-departmental data collected within a set time window, a departmental fit score is constructed.

[0011] Based on the current work order data, match the departments that have had interaction records with the department that initiated the current work order from all historical cross-department data, and use them as candidate departments; sort the candidate departments in descending order according to the department fit score, select the top n candidate departments, and construct the action space;

[0012] Input the state space vector into the pre-trained reinforcement learning model, and select the department corresponding to the action with the highest probability in the output action space as the target node.

[0013] Furthermore, the work order data includes work order priority, service type, actual transmission time, SLA compliance indicator, remaining SLA duration, standard processing time, number of permission verification failures, and number of data conversion failures;

[0014] The load data includes the number of new work orders per unit time, the average engineer load rate, the backlog of work orders, and the maximum work order capacity.

[0015] The cross-departmental data includes SLA compliance rate, cross-departmental work order routing path records, average response latency of inter-departmental interfaces, and cross-departmental work order processing success rate.

[0016] The configuration data for each department includes permission rules and data format mapping relationships.

[0017] Furthermore, based on the collected configuration data from various departments, the methods for constructing a ternary graph include:

[0018] Using departments as nodes, the permission rules and data format mapping relationships of each department as edges, and the cross-departmental interaction frequency as edge weights, a ternary graph is constructed; the cross-departmental interaction frequency is calculated based on the routing path records of cross-departmental work orders.

[0019] Based on the permission rules of each department in the ternary graph, a Boolean matrix of permission rule table is constructed;

[0020] Based on the data format mapping relationship of each department in the ternary diagram, a field-to-field vector of the data format mapping relationship is constructed;

[0021] Each node in the ternary graph corresponds to an initial feature vector, which is composed of two parts: one part is the corresponding element of each department and other departments extracted from the Boolean matrix of the permission rule table; the other part is the corresponding element of each department and other departments extracted from the field correspondence vector of the data format mapping relationship.

[0022] Furthermore, methods for constructing dynamic feature vectors based on standard data include:

[0023] The work order priority, work order backlog, average engineer load rate of each department, average response delay of inter-department interface and SLA remaining time data in the standard data are converted into equal-length vectors, concatenated to obtain dynamic feature vectors, and initial weights are assigned to each standard data item.

[0024] Based on historical and current standard data, determine whether the current business status is in a peak period. If the current business status is in a peak period, adjust the weights of various standard data in the dynamic feature vector.

[0025] Furthermore, methods for determining whether the current business status is during peak business hours, based on historical and current standard data, include:

[0026] Based on the load data and work order data of each department and time period in the historical standard data, set the threshold for the number of new work orders per unit time, the threshold for the work order growth rate, the threshold for the real-time engineer load rate, and the threshold for the SLA warning ratio.

[0027] If the number of new work orders per unit time exceeds the threshold for the number of new work orders per unit time within a consecutive set number period, or if the real-time work order growth rate exceeds the work order growth rate threshold, then the current business status will be marked as load increase.

[0028] If the real-time engineer load rate exceeds the real-time engineer load rate threshold during a business state with rising load, the current business state will be marked as a peak warning.

[0029] If the peak warning status is not lifted within the set time and the real-time SLA warning ratio exceeds the SLA warning ratio threshold, then the current business status is determined to be in the peak period.

[0030] Furthermore, if the current business is in a peak period, the methods for adjusting the weights of various standard data in the dynamic feature vector include:

[0031] Based on the load data and work order data of each department and time period in historical data, and with reference to the conventional values ​​of SLA warning ratio, average response delay of inter-department interface, work order backlog and average engineer load rate in the same period of history or under normal business conditions, the baseline values ​​of each standard data are obtained.

[0032] Calculate the deviation ratio of each standard data from the benchmark value, and map the deviation ratio exceeding the preset deviation threshold to the interval between 0 and 1 as a weighting coefficient;

[0033] During peak business periods, the initial weights of various standard data are adjusted according to weighting coefficients, increasing the weight of remaining SLA time and average response delay of inter-department interfaces, and decreasing the weight of work order backlog and average engineer load rate.

[0034] During off-peak periods, the weights of each standard data item are restored to their initial weights.

[0035] Furthermore, methods for constructing departmental fit scores based on historical cross-departmental data collected within a defined time window include:

[0036] Collect data on SLA compliance rate, average response delay of inter-department interfaces, and cross-department work order processing success rate for each department within a set time window, and map the SLA compliance rate to the range of 0 to 1 and the average response delay of inter-department interfaces to the range of 0 to 1.

[0037] Departmental Adaptability Score = Compliance Rate Weight × SLA Compliance Rate + Delay Weight × (1 - Average Response Delay of Inter-department Interfaces) + Success Rate Weight × Cross-department Work Order Processing Success Rate.

[0038] Furthermore, training methods for reinforcement learning models include:

[0039] A reward function for a reinforcement learning model is constructed based on real-time work order data and workload data.

[0040] The algorithm is trained using a pre-defined near-end policy optimization algorithm. The input is a state space vector, and the output is the probability value of each action in the action space. Each action in the action space corresponds to a candidate department.

[0041] During the training iteration, the reinforcement learning model calculates the reward value of each output result through the reward function. After processing a set number of work orders, the parameters of the reinforcement learning model are updated based on the accumulated reward value. During the update, the magnitude of the parameter update is limited by a preset clipping objective function. After training is completed, a trained reinforcement learning model is obtained.

[0042] Furthermore, methods for constructing reward functions for reinforcement learning models based on real-time work order data and workload data include:

[0043] An efficiency positive term is constructed based on the historical average transmission time and the actual transmission time. Efficiency positive term = efficiency positive coefficient × (historical average transmission time - actual transmission time) / historical average transmission time;

[0044] A compliance positive item is constructed based on the SLA compliance mark, remaining SLA time, and standard processing time. The compliance positive item = compliance positive item coefficient × SLA compliance mark × (remaining SLA time / standard processing time).

[0045] A load negative term is constructed based on the actual backlog of work orders and the maximum work order capacity. The load negative term = the negative value of the load negative term coefficient × max(0, actual backlog of work orders - maximum work order capacity) / maximum work order capacity.

[0046] A stable negative term is constructed based on the number of permission verification failures and the number of data conversion failures. The stable negative term = the negative value of the stable negative term coefficient × (number of permission verification failures + data conversion failure coefficient × number of data conversion failures).

[0047] A priority weighting coefficient is set for work orders with different priorities. Higher priority tasks have a higher priority weighting coefficient. The reward function = priority weighting coefficient × (efficiency positive term + compliance positive term + load negative term + stability negative term).

[0048] A reinforcement learning-based process routing optimization system includes:

[0049] The data acquisition module is used to collect real-time work order data, load data, and cross-departmental data from the system to obtain raw data; the raw data is then preprocessed and standardized to obtain standard data.

[0050] The static feature module constructs a ternary graph based on the collected configuration data of each department, and trains the ternary graph using a preset graph neural network algorithm to obtain a trained graph neural network model. The input of the graph neural network model is the ternary graph, and the output is a static feature vector.

[0051] The state space module constructs dynamic feature vectors based on standard data, and concatenates the dynamic feature vectors with the static feature vectors to obtain the state space vectors.

[0052] The department adaptation module constructs a department adaptation score based on historical cross-department data collected within a set time window.

[0053] The action space module, based on the current work order data, matches departments that have had interaction records with the department that initiated the current work order from all historical cross-department data, and uses them as candidate departments; the candidate departments are sorted in descending order according to the department fit score, and the top n candidate departments are selected to construct the action space;

[0054] The department selection module is used to input the state space vector into the pre-trained reinforcement learning model and select the department corresponding to the action with the highest probability in the output action space as the target node.

[0055] Compared with existing technologies, the technical effects and advantages of the reinforcement learning-based process routing optimization method and system provided by this invention are as follows:

[0056] This invention collects real-time work order data, load data, and cross-department data, and obtains standard data through preprocessing and standardization. Based on department configuration data, a ternary graph is constructed, and a graph neural network is used to train static feature vectors. These are combined with dynamic feature vectors constructed from real-time indicators to form a state space vector. A reward function containing dimensions of efficiency, compliance, load, and stability is constructed. Based on historical cross-department data, a departmental fit score is constructed to screen candidate departments to form an action space. A reinforcement learning model is used to select the target department, and the weights of features and reward functions can be dynamically adjusted according to peak business periods.

[0057] This invention addresses the problems of low efficiency in cross-departmental collaboration, lack of flexible alternatives when specific departments are overloaded, difficulty in meeting SLA requirements, and potential compliance risks and business losses. It improves cross-departmental collaboration efficiency, balances departmental load, ensures service compliance and stability, avoids service interruptions during peak business periods, reduces compliance risks and business losses, and enhances the collaborative efficiency and service quality of group-based management. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of a reinforcement learning-based process routing optimization system according to an embodiment of the present invention;

[0059] Figure 2 This is a flowchart of the reinforcement learning-based process routing optimization method according to an embodiment of the present invention;

[0060] Figure 3 This is a flowchart of a method for determining whether the current business status is during a peak business period, according to an embodiment of the present invention.

[0061] Figure 4 This is a flowchart of a method for adjusting the weights of various types of data in a dynamic feature vector according to an embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of a ternary graph structure according to an embodiment of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be described in detail, clearly, and completely below with reference to the accompanying drawings. It should be particularly noted that the specific embodiments described below are only for better illustrating and explaining the technical solutions of the present invention, and are intended to enable those skilled in the art to better understand and implement the present invention, and should not be construed as limiting the scope of protection of the present invention. Without departing from the spirit and substance of the present invention, those skilled in the art can modify, adjust, or make equivalent substitutions based on the content disclosed in the present invention, and these should all be considered within the scope of protection of the present invention.

[0064] Example 1

[0065] Please see Figure 1 As shown, this embodiment discloses a process routing optimization system based on reinforcement learning, including various modules that achieve data transmission through wired and / or wireless connections.

[0066] The data acquisition module is used to collect real-time work order data, load data, and cross-departmental data from the system to obtain raw data; the raw data is then preprocessed and standardized to obtain standard data.

[0067] The work order data includes work order priority, business type, actual transmission time, SLA compliance flag, remaining SLA duration, standard processing time, number of permission verification failures, and number of data conversion failures. Specifically, work order priority and business type are recorded when the work order is created. Actual transmission time is tracked in real-time during the work order workflow. When the work order is completed, an SLA compliance flag is set (1 for compliance, 0 for non-compliance), and the remaining SLA duration (i.e., the time remaining from the current time to the SLA deadline) and the standard processing time (i.e., the preset processing time limit for this type of work order) are recorded simultaneously. The number of permission verification failures and data conversion failures are extracted from the interface logs of cross-departmental data interaction. Each time a work order fails to pass permission verification or perform a data conversion operation, the log automatically records the failure event, and the corresponding data is obtained by statistically analyzing the number of failure events within a specific time period.

[0068] Work order priority and business type can clearly define the urgency and category of work order processing; actual transmission time can reflect real-time interaction efficiency; SLA compliance indicator, remaining SLA time and standard processing time can be directly associated with compliance; number of permission verification failures and number of data conversion failures can capture stability issues in the interaction. These data together provide a basis for judging the compliance and efficiency of work order processing, avoiding blind routing decisions due to missing information.

[0069] The load data includes the number of new work orders per unit time, average engineer load rate, work order backlog, and maximum work order capacity. The number of new work orders per unit time is calculated at fixed intervals, such as every 5 minutes, to count the total number of newly created work orders for each department. The average engineer load rate is the ratio of the number of work orders currently being processed in each department to the number of online engineers, with the number of online engineers being synchronously obtained from the human resources management system. The work order backlog is the total number of unprocessed work orders for each department, aggregated in real time from the work order management system. The maximum work order capacity is based on historical data, such as the maximum processing volume of each department over the past 3 months or preset configuration information.

[0070] The number of new work orders per unit time, the backlog of work orders, and the average engineer load rate can reflect the processing pressure of each department in real time. Combined with the maximum work order capacity, it can determine whether a department is overloaded, thereby avoiding routing work orders to departments with excessive load, reducing work order backlog, and solving the response delay problem caused by uneven load.

[0071] The cross-departmental data includes SLA compliance rate, cross-departmental work order routing path records, average response latency of inter-departmental interfaces, and cross-departmental work order processing success rate. The SLA compliance rate is calculated from SLA compliance records in cross-departmental work orders within a fixed period in the database, such as 6 months; that is, SLA compliance rate = number of compliant work orders / total number of work orders. Cross-departmental work order routing path records are extracted from the work order flow log, which details the flow of work orders from the initiating department to each processing department. The average response latency of inter-departmental interfaces is monitored and statistically analyzed in real time by an interface monitoring program. This program records the response time of each cross-departmental interface call and calculates the average value over a fixed period for department pairs, such as Department A and Department B. The cross-departmental work order processing success rate is the ratio of the number of work orders that successfully completed cross-departmental flow to the total number of cross-departmental work orders initiated; this data comes from the historical completion records of the work order management system.

[0072] SLA compliance rate, average response latency of inter-department interfaces, and success rate of cross-department work order processing can filter out efficient and stable collaborative departments. The routing path records of cross-department work orders can analyze historical interaction patterns, provide a reference for selecting high-quality collaboration links, and reduce the inefficiency and uncertainty of cross-department interactions.

[0073] The configuration data of each department is collected and input into a pre-trained graph neural network model, and the static feature vectors of each department are output.

[0074] The configuration data for each department includes permission rules and data format mapping relationships. For example, the permission rules for each department can be expressed as "Department X only allows Department Y to view basic customer information and prohibits access to transaction data" and "All financial work orders from all departments must be approved by the headquarters finance department," which indicates whether there is a permission sharing relationship between departments. The data format mapping relationship between departments can be expressed as "Department A's 'Customer ID' field is a string, while Department B's is an integer" and "Department C's 'Work Order Creation Time' format is 'Year / Month / Day', while Department D's is 'Month-Day-Year'," which indicates the degree of matching between the data formats of departments.

[0075] Construct an N×N Boolean matrix for the permission rule table, where N is the total number of departments, the row index is the department initiating the work order, and the column index is the target department of the work order. Perform a binary mapping for each permission rule: if the department initiating the work order has access / operation permissions for a certain type of data to the target department (e.g., viewing, modifying, and approving), the corresponding position in the Boolean matrix of the permission rule table is recorded as 1; otherwise, it is recorded as 0. For example, if department A can transmit customer data to department B (i.e., there is a permission exchange relationship), the element at position [A,B] in the Boolean matrix of the permission rule table is 1; if department A cannot transmit financial data to department C (i.e., there is no permission exchange relationship), the element at position [A,C] in the Boolean matrix of the permission rule table is 0.

[0076] For multi-dimensional permissions, such as "view permission" and "modify permission", they can be expanded into an N×N×K tensor, where K is the number of permission types. However, to simplify the calculation, they can be merged into a Boolean matrix of permission rule tables for "whether there is at least one valid permission", where 1 indicates that any permission exists and 0 indicates that there is no permission at all.

[0077] The differences in data formats between different departments are transformed into field-to-field vectors that map data formats, enabling graph neural networks to learn the degree of matching between data formats. The higher the element value of a field, the lower the difficulty of field mapping and the less time is spent on data conversion.

[0078] The scoring of field matching includes three dimensions: data type matching, format matching, and semantic matching. The calculation methods for each are as follows: Data type matching is calculated by assigning similarity weights between types. For example, strings of the same type are scored as 1.0, weakly matched types such as strings to integers are scored as 0.3, and no match is scored as 0. Format matching is achieved by parsing and comparing field format patterns, using pattern similarity scoring. For example, the date format "yyyy-mm-dd" and "yyyy / mm / dd" have the same structure but different separators, scoring 0.8; while "mm-dd-yyyy" has a significantly different structure, scoring 0.5. Semantic matching is based on natural language processing techniques, such as word vectors or pre-trained language models. Field names are converted into semantic vectors, and cosine similarity is calculated. For example, the vector similarity between "creation time" and "submission time" can reach 0.9. Finally, the three types of scores are weighted and averaged to obtain the similarity score of all matching fields. The similarity score of each matching field is used as the element of the field-corresponding vector of the data format mapping relationship. If there is no matching field, the corresponding element is 0.

[0079] The static feature module constructs a ternary graph based on the collected configuration data of each department, and trains the ternary graph using a preset graph neural network algorithm to obtain a trained graph neural network model. The input of the graph neural network model is the ternary graph, and the output is a static feature vector.

[0080] Please see Figure 5 As shown, a ternary graph is constructed with departments as nodes, the permission rules and data format mapping relationships of each department as edges, and the cross-departmental interaction frequency as edge weights.

[0081] The cross-departmental interaction frequency is calculated based on the routing path records of cross-departmental work orders. Specifically, data is extracted from the routing path records of historical cross-departmental work orders. First, all work orders involving two or more departments within a fixed period, such as three months, are selected. Then, these work orders are grouped according to the combination of the initiating department and the target department. The total number of work order interactions for each department pair, such as department A and department B, or department B and department C, within that period is counted. This total number is used as the interaction frequency between the corresponding departments and is directly used as the edge weight connecting the two department nodes in the ternary graph. For example, if department A initiated 150 cross-departmental work orders to department B in the past three months, then the weight of the edge between department A and department B is 150, thus reflecting the frequency of interaction between the two departments.

[0082] Although the initial feature vector of a node is formed by concatenating the Boolean matrix of the permission rule table and the field corresponding vector of the data format mapping relationship, containing the direct permission relationship and direct format matching relationship between departments and other departments, these are only direct associations between pairs of departments, which are isolated one-to-one information and cannot reflect the more complex association characteristics in the ternary graph.

[0083] The ternary graph is trained using a pre-defined graph neural network algorithm to obtain a trained graph neural network model, as follows:

[0084] The input to the graph neural network model is a Boolean matrix of the permission rule table and a field-corresponding vector of the data format mapping relationship. The output is a 128-dimensional feature vector of each department, that is, the static feature vector of each department. The static feature vector is used to characterize the matching degree feature of data interaction between departments.

[0085] First, the initialization of the ternary graph and the construction of the initial feature vectors for each node need to be completed. Each node in the ternary graph represents a department. The initial feature vector of a node is composed of two parts: one part is the corresponding element extracted from the Boolean matrix of the permission rule table, which represents the department's corresponding element with all other departments. For example, if department A has shared permissions with departments B and C, the corresponding position is 1, and the rest are 0, forming a vector with a length equal to the total number of departments. The other part is the corresponding element extracted from the field correspondence vector of the data format mapping relationship, which represents the department's corresponding element with other departments. For example, if the field correspondence vector between department A and department B is [0.6, 0.55, 0.825], then its average value of 0.658 is taken as the format matching value with B, and then combined with the format matching values ​​of other departments to form another vector. These two vectors are then concatenated to form the initial feature vector for each department node.

[0086] During the training of a graph neural network (Graph Neural Network) model, GraphSAGE can be used. GraphSAGE learns node embeddings through an iterative mechanism of neighbor sampling, feature aggregation, and vector update. First, for each department node, neighbor sampling is performed based on edge weights; neighbors with higher interaction frequencies are sampled more frequently, ensuring the Graph Neural Network model prioritizes more important related departments. For example, if department A interacts with department D 100 times per month, far more than with department E (10 times), then department D is a more important related department for department A. Next, aggregation functions such as mean aggregation or pooling aggregation are used to fuse the sampled neighbor node features with the current node's features. For instance, when a department A aggregates, it will take a weighted average of the permission and format features of its neighbor department B with its own corresponding features. The weights are dynamically adjusted by the edge weights, ensuring that the more frequently interacting neighbors have a greater impact on the current node's features. In this way, each department node can not only learn its own permissions and format attributes, but also integrate the relevant features of neighboring departments and capture the transitive features in indirect associations such as department A-department B-department C. For example, if department A matches department B and department B matches department C, then department A can indirectly obtain the matching degree information with department C through department B.

[0087] The loss function used in training the graph neural network model employs structural similarity constraints on the nodes in the graph. Specifically, if two departments frequently interact in actual business and have high matching degrees in permissions and formats (e.g., high edge weights and high mean values ​​of corresponding field vectors), the loss function will bring their 128-dimensional feature vectors closer in Euclidean distance; conversely, it will make the vectors farther apart. Through multiple iterations, the model parameters are continuously optimized, ultimately outputting a 128-dimensional feature vector for each department—the static feature vector.

[0088] The static feature vector is an abstract encoding of the interaction characteristics of departments in the entire ternary graph. Its content is not a specific, artificially defined indicator, but rather a comprehensive feature learned automatically through training. For example, some dimensions implicitly represent the overall permission matching level between the department and other departments, such as how many departments the department has valid data access permissions with; some dimensions reflect format matching ability, such as the mean of the field-corresponding vectors between the department and frequently interacting departments, and the ease of data format conversion; some dimensions encode higher-order association features, such as indirect matching degree features passed through 2-3 layers of neighbors, for example, permission and format matching between department A and department B, and permission and format matching between department B and department C, so department A can indirectly obtain matching degree information with department C through department B; some dimensions integrate the influence of interaction frequency, such as the association strength between the department and neighboring departments with high edge weights, i.e., departments with high cross-departmental interaction frequency. These dimensions together form a compact feature space, which allows the Euclidean distance between vectors to directly reflect the overall cost of data interaction between departments. That is, the closer the Euclidean distance, the more matched the permission rules of the two departments are, the more compatible the data format mapping relationship is, and the higher the frequency of cross-departmental interaction. The less time is spent on interface conversion and format alignment during data transmission, providing a quantitative basis for the inter-departmental interaction matching degree for the state space of subsequent reinforcement learning.

[0089] GraphSAGE integrates isolated direct relationships between nodes in various departments through neighbor sampling and feature aggregation. On the one hand, it can capture transitive features in indirect associations such as department A-department B-department C, where department A indirectly obtains matching degree information with department C through department B. On the other hand, it can reflect the impact of interaction frequency on matching degree, making the influence of frequently interacting neighbors on the current node's features more significant.

[0090] The state space module constructs dynamic feature vectors based on standard data, and concatenates the dynamic feature vectors with the static feature vectors to obtain the state space vector.

[0091] Based on standard data, a dynamic feature vector is constructed as follows:

[0092] The work order priority, work order backlog, average engineer load rate of each department, average response latency of inter-department interfaces, and remaining SLA time in the standard data are selected and converted into equal-length vectors. These vectors are then concatenated to obtain a dynamic feature vector, and an initial weight is assigned to each data item. For example, work order priority and remaining SLA time are directly related to compliance, so they each have an initial weight of 25%; work order backlog and average engineer load rate reflect real-time load, each accounting for 20%; and average response latency of inter-department interfaces reflects transmission efficiency, accounting for 10%.

[0093] The work order priority in the dynamic feature vector directly reflects the urgency of work order processing and forms the basis for priority ranking during routing. The backlog of each work order and the average engineer load rate of each department reflect the real-time processing pressure of each department, avoiding routing work orders to overloaded departments and causing delays. The average response latency of inter-department interfaces reflects the real-time efficiency of current cross-departmental data transmission, providing a basis for selecting efficient transmission paths. The remaining SLA time is associated with compliance, ensuring that decisions prioritize work orders that are about to reach their SLA deadline, avoiding breaches of contract. These dynamic indicators together constitute a real-time snapshot of the current business status, allowing the reinforcement learning model to perceive real-time load, efficiency bottlenecks, and compliance risks.

[0094] Static feature vectors, as abstract encodings of the interaction characteristics of departments in a ternary graph, implicitly contain the level of permission matching, format matching ability, higher-order association features, and the impact of interaction frequency between departments. They reflect the inherent cost of cross-departmental data interaction, which is a long-term stable characteristic. This can help reinforcement learning models identify which departments are inherently more compatible, such as those with high permission matching degree and low format conversion cost, fundamentally reducing the time spent on interface conversion and format alignment caused by poor compatibility.

[0095] By concatenating dynamic and static feature vectors into a state space vector, reinforcement learning models can simultaneously consider both dynamic and static features. In the final decision-making process, they can take into account real-time constraints such as current departmental load, interface efficiency, and compliance deadlines, as well as the long-term interaction compatibility between departments. This results in a routing strategy that meets real-time business needs while reducing inherent interaction costs, thus solving problems such as inefficiency, high compliance risks, and poor stability caused by incomplete information in cross-departmental collaboration.

[0096] Please see Figure 3 As shown, based on historical and current standard data, the system determines whether the current business status is during a peak period, as detailed below:

[0097] Based on historical data on load and work order data for each department and time period, thresholds for new work orders per unit time, work order growth rate, real-time engineer load rate, and SLA warning ratio are established.

[0098] The thresholds are determined based on historical data on the load and work order data of each department and time period: the threshold for new work orders per unit time is determined by statistically analyzing the normal maximum and outlier boundaries of new work orders per unit time for each department during historical time periods, such as peak weekday hours and holidays; the work order growth rate threshold is set based on the normal fluctuation range of historical work order growth, combined with the growth threshold that leads to process congestion; the real-time engineer load rate threshold refers to the matching relationship between the number of online engineers and work order processing volume in each department in historical data, based on the upper limit of load without significant processing delays; the SLA warning ratio threshold is based on the distribution of SLA compliance in historical cross-departmental work orders, using the proportion of service level risk triggering the critical point as the standard. The setting of the thresholds for new work orders per unit time, work order growth rate, real-time engineer load rate, and SLA warning ratio all rely on the characteristics of load and work order data of each department in different time periods in historical data, reflecting the boundary between normal and abnormal business states.

[0099] If the number of new work orders per unit time exceeds the threshold for new work orders per unit time within a consecutive set number period, or if the real-time work order growth rate exceeds the work order growth rate threshold, the current business status will be marked as load increase. Under the load increase business status, if the real-time engineer load rate exceeds the real-time engineer load rate threshold, the current business status will be marked as peak warning. If the peak warning business status is not lifted within the set time, and the real-time SLA warning ratio exceeds the SLA warning ratio threshold, the current business status will be judged to be in the business peak period.

[0100] Please see Figure 4 As shown, if the current business is in a peak period, the weights of various data types in the dynamic feature vector will be adjusted as follows:

[0101] Based on historical load and work order data for each department and time period, and referencing conventional values ​​for SLA warning rates, average inter-department interface response latency, work order backlog, and average engineer workload under normal business conditions, baseline values ​​for each data point are obtained. The deviation rate of each data point from the baseline value is calculated, and deviations exceeding preset thresholds are mapped to a 0-1 range as weighting coefficients. During peak business periods, the initial weights of each data point are adjusted according to these weighting coefficients, increasing the weight of remaining SLA time and average inter-department interface response latency while decreasing the weight of work order backlog and average engineer workload. During off-peak periods, the weights of each data point are restored to their initial weights. The adjustment effect is assessed by evaluating the improvement in SLA compliance rate, average inter-department interface response latency, and cross-department work order processing success rate within the adjusted observation period compared to the same period in history or the period before adjustment.

[0102] Based on historical standard data within a set time window, combined with load data and work order data for each department and time period, and referring to the conventional values ​​of SLA warning ratio, average response delay of inter-department interfaces, work order backlog, and average engineer load rate in the same period of history or under normal business conditions, deviation thresholds for each data item are determined according to the thresholds for new work orders per unit time, work order growth rate, real-time engineer load rate, and SLA warning ratio.

[0103] For example, the initial weights for each data point are: SLA remaining time 0.25, average inter-department interface response latency 0.25, work order backlog 0.25, and average engineer load rate 0.25. Based on historical and current standard data, the baseline values ​​for each data point are: SLA remaining time 60, average inter-department interface response latency 200, work order backlog 100, and average engineer load rate 0.60. The preset deviation thresholds for each data point are: SLA remaining time 0.10, average inter-department interface response latency 0.10, work order backlog 0.10, and average engineer load rate 0.10. When the current business is in peak period, the current data are: SLA remaining time 30, average inter-department interface response latency 350, and work order backlog 0.25. 180. Average engineer load rate: 0.85. Calculate the deviation ratio of each data point from the benchmark value, and map the deviation ratios exceeding the deviation threshold to the interval between 0 and 1 to obtain weighting coefficients: SLA remaining time: 0.50, average response delay of inter-departmental interface: 0.40, work order backlog: 0.30, average engineer load rate: 0.25. Adjust the initial weights according to these weighting coefficients, increasing the weight of SLA remaining time and average response delay of inter-departmental interface, and decreasing the weight of work order backlog and average engineer load rate, resulting in weights of SLA remaining time: 0.33, average response delay of inter-departmental interface: 0.31, work order backlog: 0.19, average engineer load rate: 0.17. During off-peak periods, the weights of each data point are restored to their initial weights.

[0104] The department adaptation module constructs a department adaptation score based on historical cross-departmental data collected within a set time window.

[0105] The system collects SLA compliance rate, average inter-department interface response latency, and cross-department work order processing success rate for each department within a set time window. The SLA compliance rate and average inter-department interface response latency are mapped to the 0-1 range. The departmental suitability score is calculated as: Compliance Rate Weight × SLA Compliance Rate + Latency Weight × (1 - Average Inter-department Interface Response Latency) + Success Rate Weight × Cross-department Work Order Processing Success Rate. For example, the system extracts the SLA compliance rate, average inter-department interface response latency, and cross-department work order processing success rate for each department over the past 6 months from the database and performs standardization: the SLA compliance rate is mapped to the [0,1] range (higher is better), and the average inter-department interface response latency is mapped to the [0,1] range (lower is better). The departmental suitability score is calculated as: 0.6 × SLA Compliance Rate + 0.3 × (1 - Average Inter-department Interface Response Latency) + 0.1 × Cross-department Work Order Processing Success Rate, comprehensively reflecting the department's suitability as a target node.

[0106] The weighting of compliance rate, latency, and success rate is selected based on the degree of influence each indicator has on the department's suitability as a target node. SLA compliance rate, directly related to compliance, has the greatest impact on suitability and is therefore assigned a high weight, such as 0.6 in the example. Average response latency of inter-department interfaces reflects transmission efficiency and has a secondary impact, so it is assigned a medium weight, such as 0.3 in the example. Cross-department work order processing success rate reflects overall collaboration effectiveness and has a relatively small impact, so it is assigned a low weight, such as 0.1 in the example. This weighting allocation allows the department suitability score to comprehensively reflect the importance of each indicator and more accurately reflect the department's suitability as a target node.

[0107] The action space module, based on the current work order data, matches departments that have had interaction records with the department that initiated the current work order from all historical cross-department data, and uses them as candidate departments; the candidate departments are sorted in descending order according to their department fit score, and the top n candidate departments are selected to construct the action space.

[0108] For example, if the initiating department of the current work order is department A, based on the current work order data, five departments that have had interaction records with department A are matched from the routing path records of historical cross-department work orders. These five departments are department B, department C, department D, department E, and department F. These five departments are the candidate departments.

[0109] For these five candidate departments, a departmental fit score was constructed based on historical cross-departmental data from the past six months: Department D's SLA compliance rate was 0.95, the average inter-departmental interface response latency after mapping was 0.1, and the cross-departmental work order processing success rate was 0.9. Departmental fit score = 0.6 × 0.95 + 0.3 × (1 - 0.1) + 0.1 × 0.9 = 0.93; Department B's SLA compliance rate was 0.9, the interface latency after mapping was 0.2, and the processing success rate was 0.85. Department F's SLA compliance rate was 0.88, the interface latency after mapping was 0.15, the processing success rate was 0.88, and the department's adaptability score was [not specified]. Department C's SLA compliance rate is 0.8, the interface latency after mapping is 0.3, the processing success rate is 0.8, and the department's adaptability score is... Department E's SLA compliance rate is 0.75, the interface latency after mapping is 0.4, the processing success rate is 0.75, and the department's adaptability score is... The candidate departments were sorted in descending order of fit score as follows: Department D (0.93), Department F (0.871), Department B (0.865), Department C (0.77), and Department E (0.705).

[0110] The selection of n is based on the balance between the decision efficiency of the reinforcement learning model and the coverage of candidate departments in the actual business scenario. For example, based on historical route optimization experience, when n=3, it can ensure that the reinforcement learning model has enough selection space, and the decision efficiency will not be reduced due to too many candidate departments. Therefore, the first 3 candidate departments are selected to construct the action space of reinforcement learning.

[0111] The department selection module is used to input the state space vector into the pre-trained reinforcement learning model and select the department corresponding to the action with the highest probability in the output action space as the target node.

[0112] A reward function for a reinforcement learning model is constructed based on real-time work order data and workload data.

[0113] An efficiency positive term is constructed based on historical average transmission time and actual transmission time, specifically: Efficiency positive term = Efficiency positive coefficient × (Historical average transmission time - Actual transmission time) / Historical average transmission time; A compliance positive term is constructed based on SLA compliance indicator, remaining SLA time, and standard processing time, specifically: Compliance positive term = Compliance positive term coefficient × SLA compliance indicator × (Remaining SLA time / Standard processing time); A load negative term is constructed based on actual work order backlog and maximum work order capacity, specifically: Load negative term = Negative value of load negative term coefficient × max(0, Actual work order backlog - Maximum work order capacity) / Maximum work order capacity; A stability negative term is constructed based on the number of permission verification failures and the number of data conversion failures, specifically: Stability negative term = Negative value of stability negative term coefficient × (Number of permission verification failures + Data conversion failure coefficient × Number of data conversion failures); The data conversion failure coefficient can be 2. In cross-departmental data interaction, the impact of data conversion failure on process stability is usually greater than that of permission verification failure. Data conversion involves mapping and aligning data formats from different departments. Failures can lead to data being unparseable or unusable, requiring additional time for format adjustments, which has a more significant impact on work order processing efficiency. In contrast, permission verification failures are mostly due to permission configuration issues, which are relatively easy to troubleshoot and fix. By assigning higher weight to the number of data conversion failures, the reward function can more accurately reflect the impact of different failure types on system stability, guiding the reinforcement learning model to prioritize avoiding data conversion failures and improving the stability of cross-departmental collaboration.

[0114] If the current business is in a peak period, the coefficients of the compliance positive term and load negative term in the reward function are increased, the coefficient of the efficiency positive term is decreased, and the coefficient of the stability negative term remains unchanged. The increased and decreased coefficient values ​​are determined according to the rule that the greater the deviation from the baseline, the larger the weight adjustment. For example, the initial allocation of coefficients in the reward function is: efficiency positive term coefficient 0.3, compliance positive term coefficient 0.4, load negative term coefficient 0.2, stability negative term coefficient 0.1, and the sum of the four coefficients is 1. When the system determines that the current business is in a peak period, real-time monitoring reveals that the SLA warning ratio is 60% higher than the baseline value, indicating a large deviation from the baseline; the departmental average load rate is 40% higher than the baseline value; and among the deviations, the actual transmission time is 20% higher than the baseline value, indicating a small deviation from the baseline. Following the rule that the greater the deviation of each data point from the baseline value, the larger the weight adjustment, the adjustment of the compliance positive term coefficient β is 60%, i.e. (after adjustment) The adjusted β is calculated to be 0.64; the adjustment range for the load negative term coefficient γ is 40%, i.e. (after adjustment) The adjusted γ is calculated to be 0.28; the adjustment range for the efficiency positive term coefficient α is -20%, i.e. (after adjustment) The adjusted α is calculated to be 0.24; the coefficient of the stable negative term remains unchanged at 0.1.

[0115] In addition, differentiated reward coefficients are set for work orders with different priorities to strengthen the processing priority of high-priority tasks. The work order priority is extracted from the work order attributes. For example, if the work order priority is P1 / P2 / P3 / P4, the corresponding priority weighting coefficients are set as follows: P1 corresponds to 1.5, P2 corresponds to 1.2, P3 corresponds to 1.0, and P4 corresponds to 0.8. The reward function is finally: Reward function = Priority weighting coefficient × (efficiency positive term + compliance positive term + load negative term + stability negative term).

[0116] Taking a cross-departmental work order with a priority level of P1 as an example, its historical average transmission time is 100ms, and the actual transmission time is 80ms; the SLA compliance indicator is 1, the remaining SLA time is 2 hours, and the standard processing time is 4 hours; the actual backlog of work orders is 120, and the maximum work order capacity is 100; the number of permission verification failures is 0, and the number of data conversion failures is 1. Therefore, the adjusted reward function is calculated as follows: Efficiency positive term = 0.24 × (100 - 80) / 100 = 0.048; Compliance positive term... Load negative term = -0.28 × max(0, 120 - 100) / 100 = -0.056; Stability negative term The total reward value is 1.5 × (0.048 + 0.32 - 0.056 - 0.2) = 0.168. It can be seen that the reward function during peak business periods focuses more on compliance and load control. Compared with off-peak periods, it can prioritize SLA compliance and avoid departmental work order overload.

[0117] The reinforcement learning model is trained using a proximal policy optimization algorithm. The input is a state space vector, and the output is the selection probability of each action in the action space. Each action corresponds to a candidate department, and the probability value reflects the suitability of routing the current work order to that department. During training iterations, after processing 100 work orders, the reinforcement learning model updates its parameters based on the accumulated reward value. The update is limited by a clipping objective function. The clipping objective function sets a fixed range of clipping coefficients, constraining the probability ratio of the new policy to the old policy within the interval determined by the clipping coefficients. When the probability ratio exceeds the upper limit of the interval, the upper limit is used as the replacement; when the probability ratio falls below the lower limit, the lower limit is used as the replacement. This limits the maximum magnitude of policy updates, preventing oscillations in the reinforcement learning model due to excessive parameter adjustments, ensuring the stability of the training process, and ultimately obtaining a well-trained reinforcement learning model.

[0118] The state space vector is input into the reinforcement learning model, and the department corresponding to the action with the highest probability in the action space is selected as the target node to complete the routing and scheduling of work orders.

[0119] Example 2

[0120] Please see Figure 2 As shown, this embodiment provides a process routing optimization method based on reinforcement learning, including:

[0121] The system collects real-time work order data, load data, and cross-departmental data to obtain raw data; the raw data is then preprocessed and standardized to obtain standard data.

[0122] Based on the configuration data collected from various departments, a ternary graph is constructed, and a pre-defined graph neural network algorithm is used to train the ternary graph to obtain a trained graph neural network model. The input of the graph neural network model is the ternary graph, and the output is a static feature vector.

[0123] Based on standard data, a dynamic feature vector is constructed, and the dynamic feature vector and the static feature vector are concatenated to obtain a state space vector.

[0124] Based on historical cross-departmental data collected within a set time window, a departmental fit score is constructed.

[0125] Based on the current work order data, match the departments that have had interaction records with the department that initiated the current work order from all historical cross-department data, and use them as candidate departments; sort the candidate departments in descending order according to the department fit score, select the top n candidate departments, and construct the action space;

[0126] Input the state space vector into the pre-trained reinforcement learning model, and select the department corresponding to the action with the highest probability in the output action space as the target node.

[0127] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0128] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A process routing optimization method based on reinforcement learning, characterized in that, include: The system collects real-time work order data, load data, and cross-departmental data to obtain raw data. The raw data is preprocessed and standardized to obtain standard data; Based on the collected configuration data of each department, a ternary graph is constructed, and a preset graph neural network algorithm is used to train the ternary graph to obtain a trained graph neural network model. The input of the graph neural network model is the ternary graph, and the output is a static feature vector. The configuration data of each department includes permission rules and data format mapping relationships. With departments as nodes, the permission rules and data format mapping relationships of each department as edges, and the cross-departmental interaction frequency as edge weights, a ternary graph is constructed. Based on standard data, a dynamic feature vector is constructed, and the dynamic feature vector and the static feature vector are concatenated to obtain a state space vector. Based on historical cross-departmental data collected within a set time window, a departmental fit score is constructed. Based on the current work order data, match the departments that have had interaction records with the department that initiated the current work order from all historical cross-department data, and use them as candidate departments; sort the candidate departments in descending order according to the department fit score, select the top n candidate departments, and construct the action space; Input the state space vector into the pre-trained reinforcement learning model, and select the department corresponding to the action with the highest probability in the output action space as the target node.

2. The process routing optimization method based on reinforcement learning according to claim 1, characterized in that: The work order data includes work order priority, service type, actual transmission time, SLA compliance indicator, remaining SLA duration, standard processing time, number of permission verification failures, and number of data conversion failures. The load data includes the number of new work orders per unit time, the average engineer load rate, the backlog of work orders, and the maximum work order capacity. The cross-departmental data includes SLA compliance rate, cross-departmental work order routing path records, average response latency of inter-departmental interfaces, and cross-departmental work order processing success rate.

3. The process routing optimization method based on reinforcement learning according to claim 2, characterized in that, Based on the collected configuration data from various departments, the methods for constructing a ternary graph include: The frequency of cross-departmental interaction is calculated based on the routing path records of cross-departmental work orders; Based on the permission rules of each department in the ternary graph, a Boolean matrix of permission rule table is constructed; Based on the data format mapping relationship of each department in the ternary diagram, a field-to-field vector of the data format mapping relationship is constructed; Each node in the ternary graph corresponds to an initial feature vector, which is composed of two parts: one part is the corresponding element of each department and other departments extracted from the Boolean matrix of the permission rule table; the other part is the corresponding element of each department and other departments extracted from the field correspondence vector of the data format mapping relationship.

4. The process routing optimization method based on reinforcement learning according to claim 3, characterized in that, Methods for constructing dynamic feature vectors based on standard data include: The work order priority, work order backlog, average engineer load rate of each department, average response delay of inter-department interface and SLA remaining time data in the standard data are converted into equal-length vectors, concatenated to obtain dynamic feature vectors, and initial weights are assigned to each standard data item. Based on historical and current standard data, determine whether the current business status is in a peak period. If the current business status is in a peak period, adjust the weights of various standard data in the dynamic feature vector.

5. The process routing optimization method based on reinforcement learning according to claim 4, characterized in that, Methods for determining whether the current business status is during peak hours, based on historical and current standard data, include: Based on the load data and work order data of each department and time period in the historical standard data, set the threshold for the number of new work orders per unit time, the threshold for the work order growth rate, the threshold for the real-time engineer load rate, and the threshold for the SLA warning ratio. If the number of new work orders per unit time exceeds the threshold for the number of new work orders per unit time within a consecutive set number period, or if the real-time work order growth rate exceeds the work order growth rate threshold, then the current business status will be marked as load increase. If the real-time engineer load rate exceeds the real-time engineer load rate threshold during a business state with rising load, the current business state will be marked as a peak warning. If the peak warning status is not lifted within the set time and the real-time SLA warning ratio exceeds the SLA warning ratio threshold, then the current business status is determined to be in the peak period.

6. The process routing optimization method based on reinforcement learning according to claim 5, characterized in that, If the current business is in a peak period, the methods for adjusting the weights of various standard data in the dynamic feature vector include: Based on the load data and work order data of each department and time period in historical data, and with reference to the conventional values ​​of SLA warning ratio, average response delay of inter-department interface, work order backlog and average engineer load rate in the same period of history or under normal business conditions, the baseline values ​​of each standard data are obtained. Calculate the deviation ratio of each standard data from the benchmark value, and map the deviation ratio exceeding the preset deviation threshold to the interval between 0 and 1 as a weighting coefficient; During peak business periods, the initial weights of various standard data are adjusted according to weighting coefficients, increasing the weight of remaining SLA time and average response delay of inter-department interfaces, and decreasing the weight of work order backlog and average engineer load rate. During off-peak periods, the weights of each standard data item are restored to their initial weights.

7. The process routing optimization method based on reinforcement learning according to claim 6, characterized in that, Methods for constructing department fit scores based on historical cross-departmental data collected within a set time window include: Collect data on SLA compliance rate, average response delay of inter-department interfaces, and cross-department work order processing success rate for each department within a set time window, and map the SLA compliance rate to the range of 0 to 1 and the average response delay of inter-department interfaces to the range of 0 to 1. Departmental Adaptability Score = Compliance Rate Weight × SLA Compliance Rate + Delay Weight × (1 - Average Response Delay of Inter-department Interfaces) + Success Rate Weight × Cross-department Work Order Processing Success Rate.

8. The process routing optimization method based on reinforcement learning according to claim 7, characterized in that, Training methods for reinforcement learning models include: A reward function for a reinforcement learning model is constructed based on real-time work order data and workload data. The algorithm is trained using a pre-defined near-end policy optimization algorithm. The input is a state space vector, and the output is the probability value of each action in the action space. Each action in the action space corresponds to a candidate department. During the training iteration, the reinforcement learning model calculates the reward value of each output result through the reward function. After processing a set number of work orders, the parameters of the reinforcement learning model are updated based on the accumulated reward value. During the update, the magnitude of the parameter update is limited by a preset clipping objective function. After training is completed, a trained reinforcement learning model is obtained.

9. The process routing optimization method based on reinforcement learning according to claim 8, characterized in that, Methods for constructing reward functions for reinforcement learning models based on real-time work order data and workload data include: An efficiency positive term is constructed based on the historical average transmission time and the actual transmission time. Efficiency positive term = efficiency positive coefficient × (historical average transmission time - actual transmission time) / historical average transmission time; A compliance positive item is constructed based on the SLA compliance mark, remaining SLA time, and standard processing time. The compliance positive item = compliance positive item coefficient × SLA compliance mark × (remaining SLA time / standard processing time). A load negative term is constructed based on the actual backlog of work orders and the maximum work order capacity. The load negative term = the negative value of the load negative term coefficient × max(0, actual backlog of work orders - maximum work order capacity) / maximum work order capacity. A stable negative term is constructed based on the number of permission verification failures and the number of data conversion failures. The stable negative term = the negative value of the stable negative term coefficient × (number of permission verification failures + data conversion failure coefficient × number of data conversion failures). For work orders with different priorities, set a priority weighting coefficient. The reward function = priority weighting coefficient × (efficiency positive term + compliance positive term + load negative term + stability negative term).

10. A reinforcement learning-based process routing optimization system, used to implement the reinforcement learning-based process routing optimization method according to any one of claims 1-9, characterized in that, include: The data acquisition module is used to collect real-time work order data, load data, and cross-departmental data from the system to obtain raw data. The raw data is preprocessed and standardized to obtain standard data; The static feature module constructs a ternary graph based on the collected configuration data of each department, and trains the ternary graph using a preset graph neural network algorithm to obtain a trained graph neural network model. The input of the graph neural network model is the ternary graph, and the output is a static feature vector. The configuration data of each department includes permission rules and data format mapping relationships. The ternary graph is constructed with departments as nodes, permission rules and data format mapping relationships of each department as edges, and cross-department interaction frequency as edge weights. The state space module constructs dynamic feature vectors based on standard data, and concatenates the dynamic feature vectors with the static feature vectors to obtain the state space vectors. The department adaptation module constructs a department adaptation score based on historical cross-department data collected within a set time window. The action space module, based on the current work order data, matches departments that have had interaction records with the department that initiated the current work order from all historical cross-department data, and uses them as candidate departments; the candidate departments are sorted in descending order according to the department fit score, and the top n candidate departments are selected to construct the action space; The department selection module is used to input the state space vector into the pre-trained reinforcement learning model and select the department corresponding to the action with the highest probability in the output action space as the target node.

Images