A method and system for business process decision optimization based on reinforcement learning

Abstract

The application relates to the technical field of business process intelligent decision, and discloses a business process decision optimization method and system based on reinforcement learning, which comprises the following steps: obtaining initial state information of a current work order in a business process, inputting the initial state information into a reinforcement learning intelligent agent to generate an initial dispatching decision; in response to an intermediate state generated after the initial dispatching decision is executed, generating an expected reward of a terminal service quality of the business process based on the intermediate state, and preliminarily optimizing a decision strategy of the reinforcement learning intelligent agent; after a terminal service result of the business process is fed back, calculating a real reward; generating a lag correction signal according to a reward deviation value between the real reward and the expected reward; backtracking to a node of generating the initial dispatching decision, and secondarily optimizing the decision strategy based on the lag correction signal, so that feedback attribution dislocation between the initial decision and the terminal service result can be corrected, and the terminal result obtained in lag can be accurately fed back to the initial decision node to complete strategy correction.

Description

Technical Field

[0001] This application relates to the field of intelligent decision-making technology for business processes, and more specifically, to a method and system for optimizing business process decisions based on reinforcement learning. Background Technology

[0002] In the field of business process decision optimization, especially in after-sales service work order processing scenarios involving multi-stage and cross-cycle execution, reinforcement learning models are widely used in the generation of dispatch and escalation strategies at the initial acceptance node. Existing technologies typically output decision actions based on the business status obtained at the current node, such as fault type, customer level, and regional load, and rely on the feedback of subsequent work order execution results to generate reward signals for continuous strategy optimization.

[0003] However, in real-world multi-stage workflow scenarios, work order processing involves multiple stages, including appointment scheduling, on-site visit, repair, re-inspection, customer confirmation, and follow-up evaluation. Core service quality results, such as repair completion rate, secondary repair rate, and final customer satisfaction, often only become fully apparent after several days or even weeks. To accelerate model convergence, existing technologies often generate interim rewards at intermediate stages, such as appointment completion or initial on-site arrival, to provide immediate feedback for strategy updates; or, when a final repair occurs, negative results are simply attributed to the most recent remedial action rather than the initial dispatch decision.

[0004] This approach leads to two key problems: First, there is a serious misalignment between the feedback attribution between the initial decision and the final service outcome. The model may mistakenly regard actions that result in smooth short-term nodes but high long-term repair rates as high-quality decisions and reinforce them. Second, the lack of accurate source tracing of cross-node lagging results makes it impossible for the decision-making strategies formed by reinforcement learning to truly optimize the quality of the final service.

[0005] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application provides a business process decision optimization method and system based on reinforcement learning. This method can correct the feedback attribution misalignment between the initial decision and the final service result, accurately transmit the delayed final result back to the initial decision node to complete the strategy correction, improve the convergence speed of the reinforcement learning model, and optimize the quality of the final service of the business process.

[0007] Firstly, this application provides a business process decision optimization method based on reinforcement learning, including: Obtain the initial status information of the current work order in the business process, and input the initial status information into the reinforcement learning agent to generate the initial dispatch decision; In response to the intermediate state generated after the initial dispatch decision, an expected reward for the final service quality of the business process is generated based on the intermediate state, and the expected reward is used to perform preliminary optimization of the decision-making strategy of the reinforcement learning agent. After the final service result of the business process is fed back, the actual reward is calculated based on the final service result; Calculate the reward deviation value between the actual reward and the expected reward, and generate a lag correction signal based on the reward deviation value; The decision-making strategy is optimized again based on the lag correction signal by tracing back to the node that generated the initial order dispatch decision.

[0008] Furthermore, this application also proposes that the steps for generating the expected reward for the end-service quality of the business process based on intermediate states include: Invoke the end-value prediction engine that has been pre-trained based on the complete historical business process trajectory; Input the initial state information, initial order dispatch decision and the time series characteristics of intermediate states into the final value prediction engine to estimate the probability value of the business process reaching the preset high-quality final result. Map the probability value to the expected reward.

[0009] Furthermore, this application proposes steps for initially optimizing the decision-making strategy of a reinforcement learning agent using anticipated rewards, including: Utilize expected rewards to compute temporal difference errors to optimize the value evaluation network in reinforcement learning agents; Based on the advantage function in the optimized value assessment network, the parameters of the decision policy are initially optimized according to the policy gradient algorithm.

[0010] Furthermore, this application also proposes that, after the step of initially optimizing the decision-making strategy of the reinforcement learning agent using anticipated rewards, it further includes: Establish a waiting queue that uniquely corresponds to the business process; The initial state information, initial order dispatch decision, expected reward, and the original strategy execution probability when executing the initial order dispatch decision are associated and stored in the waiting queue.

[0011] Furthermore, this application also proposes a step of tracing back to the node that generated the initial dispatch decision and performing a secondary optimization of the decision strategy based on the hysteresis correction signal, including: Extract the initial state information and original strategy execution probability corresponding to the business process from the waiting queue; Input the initial state information into the decision policy at the current moment to obtain the probability of executing the current policy; Calculate the ratio of the current strategy execution probability to the original strategy execution probability, calculate the product of the ratio and the reward deviation value, and correct the product to keep it within the preset confidence interval. The corrected product value is multiplied by the logarithmic probability gradient of the policy under the initial state information to generate the corrected gradient. The parameters of the decision policy are then residually optimized based on the corrected gradient.

[0012] Furthermore, this application proposes that, after the final service result of the business process is fed back, the steps for calculating the actual reward based on the final service result include: Obtain the end-point metrics of the business process, including the second-rework indicator, customer satisfaction score, and total processing time. The final score indicators are combined and calculated to generate the actual reward.

[0013] Furthermore, this application proposes that the steps for calculating the final outcome metrics to generate the real reward include: When some final indicators are missing from the final indicators, alternative indicators are generated based on the acquired final indicators and historical statistical data. Based on alternative metrics and the acquired endgame metrics, the real reward is calculated according to the preset real reward calculation function.

[0014] Furthermore, this application also proposes, after the step of calculating the reward deviation value between the actual reward and the expected reward, to include: The average absolute value of the reward deviation of all business processes within a preset time window; When the absolute mean exceeds the preset safety threshold, the learning rate weight of the expected reward in the initial optimization is reduced, and the final value prediction engine is retrained and its parameters are fine-tuned. Once the absolute mean value falls back to within the preset safety threshold, the learning rate weight for the expected reward is restored.

[0015] Furthermore, this application also proposes that the step of generating a hysteresis correction signal based on the reward deviation value includes: Obtain the unique identifier of the business process and the node information corresponding to the initial order dispatch decision; The reward deviation value is associated with and packaged with a unique identifier and node information to generate a hysteresis correction signal carrying a backtracking positioning tag.

[0016] Secondly, this application also proposes a business process decision optimization system based on reinforcement learning, used to run the aforementioned business process decision optimization method based on reinforcement learning. The system includes: The decision-making module is used to obtain the initial status information of the current work order in the business process and input the initial status information into the reinforcement learning agent to generate the initial dispatch decision. The preliminary optimization module is used to respond to the intermediate state generated after the initial order dispatch decision, generate the expected reward for the final service quality of the business process based on the intermediate state, and use the expected reward to perform preliminary optimization of the decision-making strategy of the reinforcement learning agent. The results tracking module is used to calculate the actual reward based on the final service result after the final service result of the business process is fed back. The deviation calculation module is used to calculate the reward deviation value between the actual reward and the expected reward, and generate a lag correction signal based on the reward deviation value; The secondary optimization module is used to backtrack to the node that generated the initial dispatch decision and perform secondary optimization of the decision strategy based on the hysteresis correction signal.

[0017] As can be seen from the above, the business process decision optimization method and system based on reinforcement learning provided in this application first optimizes the decision strategy by generating expected rewards based on intermediate states, and then generates a lag correction signal based on the deviation between the actual reward and the expected reward after the final result is fed back, and backtracks to the initial order dispatch decision node to complete the secondary optimization. This can solve the problems of misattribution between the initial decision and the feedback of the final service result in the prior art, and the inability to accurately transmit the lag result back to correct the initial strategy. It has the advantages of improving the convergence speed of the reinforcement learning model and effectively optimizing the quality of the final service of the business process. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a business process decision optimization method based on reinforcement learning, provided as an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the structure of a business process decision optimization system based on reinforcement learning, provided in an embodiment of this application.

[0020] Labeling Explanation: 210, Decision Module; 220, Preliminary Optimization Module; 230, Result Tracking Module; 240, Deviation Calculation Module; 250, Secondary Optimization Module. Detailed Implementation

[0021] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] In the field of business process decision optimization, especially in after-sales service work order processing scenarios involving multi-stage and cross-cycle execution, reinforcement learning models are widely used in the generation of dispatch and escalation strategies at the initial acceptance node. Existing technologies typically output decision actions based on the business status obtained at the current node and rely on the feedback of subsequent work order execution results to form reward signals to optimize strategies. However, work order processing involves multiple nodes such as appointment, on-site visit, repair, re-inspection, customer confirmation, and follow-up evaluation. Core service quality results, such as repair completion rate, secondary repair rate, and final customer satisfaction, often only fully manifest after several days. Existing technologies generate interim rewards in intermediate states or perform simple attribution of the final result, leading to a serious misalignment between the feedback attribution between the initial decision and the final service result. The model may consider actions that bring short-term smoothness but high long-term repair rates as high-quality decisions and reinforce them. Simultaneously, the lack of accurate tracing of lagging results across nodes prevents the decision strategies formed by reinforcement learning from truly optimizing the final service quality.

[0024] For example, in the process of processing work orders for home appliance after-sales service, when the work order is at the appointment stage, a dispatch decision is generated based on the fault type and regional load, assigning the work order to a specific repair team; at the initial on-site arrival stage, a phased reward signal is generated based on the timeliness of the on-site arrival to update the strategy. However, if the work order requires a second repair due to incompatible parts after the repair is completed, and the customer's final satisfaction is significantly reduced, existing technologies often simply attribute the negative result to the remedial operations at the repair stage, rather than tracing back to the deficiency in the skill matching of the repair team in the initial dispatch decision.

[0025] If the above problems are not addressed, the decision-making strategy will continue to optimize around intermediate state indicators, causing the model's convergence direction to deviate from the final service quality objective. The lag in the final service result will be amplified by the erroneous attribution mechanism, making it impossible for the reinforcement learning agent to establish an accurate mapping relationship between the initial decision and the final result. This will cause the overall service quality assessment system to fail, and the long-term operational efficiency of the business process will be affected.

[0026] Regarding this, firstly, see... Figure 1 This application proposes a business process decision optimization method based on reinforcement learning, the method including: S1. Obtain the initial status information of the current work order in the business process, and input the initial status information into the reinforcement learning agent to generate the initial dispatch decision; S2. In response to the intermediate state generated after the initial dispatch decision is executed, generate the expected reward for the final service quality of the business process based on the intermediate state, and use the expected reward to perform preliminary optimization of the decision-making strategy of the reinforcement learning agent. S3. After the final service result of the business process is returned, calculate the actual reward based on the final service result. S4. Calculate the reward deviation value between the actual reward and the expected reward, and generate a lag correction signal based on the reward deviation value; S5. Backtrack to the node that generated the initial dispatch decision, and perform secondary optimization of the decision strategy based on the hysteresis correction signal.

[0027] A reinforcement learning agent is a decision-making model built on deep neural networks, which may contain an Actor network and a Critic network. The Actor network is responsible for generating the probability distribution of decision actions based on the input state information, while the Critic network is responsible for evaluating the value of the current state. Through the synergistic effect of these two networks, the reinforcement learning agent can perceive the environment and make decisions.

[0028] The initial work order dispatch decision is a decision action generated by the reinforcement learning agent based on the initial state information of the current work order. It includes not only dispatching the work order to engineers or service outlets with specific skill levels, but may also include whether to set a high-priority expedited processing flag, and whether to trigger the remote support channel of senior technical experts.

[0029] The expected reward is the probability value estimated by the end-value prediction engine of the business process reaching a preset high-quality end result during the execution of the business process in response to the intermediate state generated after the initial order dispatch decision. This probability value is then mapped as a reward signal to perform preliminary optimization of the decision-making strategy of the reinforcement learning agent.

[0030] Real rewards are reward signals calculated based on actual final service results, such as secondary repair indicators, customer satisfaction scores, and total processing time, after the final service results of the business process are fed back. They reflect the actual impact of the initial order dispatch decision on the quality of final service.

[0031] The reward bias is the difference between the actual reward and the expected reward. It quantifies the cognitive error that occurs when using intermediate states for expectation assessment in the early stages, and is used to make secondary corrections to the decision-making strategy in the future.

[0032] The lag correction signal is a signal generated based on the reward deviation value. It carries a backtracking positioning tag and is used to backtrack to the node that generated the initial order dispatch decision to perform secondary optimization on the decision-making strategy of the reinforcement learning agent.

[0033] Specifically, the initial status information of the current work order in the business process is first obtained and input into the reinforcement learning agent to generate an initial dispatch decision. During the initial work order acceptance phase, a massive amount of input and status information is acquired and organized, serving as the basis for the reinforcement learning model to make decisions.

[0034] When a brand-new after-sales service work order is generated in the front-end acceptance system, the data acquisition engine will capture the multi-dimensional contextual information of the work order in real time. This information mainly comes from the customer relationship management system, including the customer's membership level, historical repair frequency, past service satisfaction rating, the customer's geographical area, and the customer's sensitivity tags to service response time; the equipment and product database, which covers the specific model of the faulty equipment, its service life and warranty status, the life cycle degradation of core components, and the initial fault phenomenon text recorded by the front-end customer service; a built-in pre-trained lightweight text feature extraction module, which maps the fault phenomenon into a fixed-dimensional fault semantic vector; and a real-time service resource scheduling pool, which includes the real-time load rate of engineers of different skill levels in various regional service outlets, the current scheduling status, the engineers' historical one-time repair rate index, and the real-time inventory quantity of relevant repair spare parts in the spare parts warehouse closest to the fault location.

[0035] After extracting these three pieces of information, they are time-aligned and feature-concatenated. Following normalization and one-hot encoding, they are organized into a high-dimensional continuous state representation vector. This vector characterizes the global business state faced by the current work order at the moment of dispatch, serving as the sole basis for the reinforcement learning model to perceive the environment and make decisions. After acquiring and organizing the state representation vector, it is input into a reinforcement learning agent built on a deep neural network. This agent maintains two cooperating deep neural networks: the Actor network responsible for outputting decision actions and the Critic network responsible for evaluating state value. The Actor network receives the high-dimensional state representation vector and generates a probability distribution of each possible action in the current state. Each action includes not only assigning the work order to a specific skill level engineer or service point, but also whether to set a high-priority expedited processing flag for the work order, and whether to directly trigger the remote support channel of a senior technical expert at the same time as dispatching the work order. Based on the probability distribution output by the Actor network, a specific decision action is sampled using a strategy. Once the action is determined, the work order and escalation instructions are sent to the backend engineer scheduling system for execution via an interface.

[0036] After the dispatch instruction is issued, the work order enters a multi-stage physical execution process. The first challenge in this process is the rapid feedback and semantic recognition of intermediate states. Typically, within minutes to hours, the backend scheduling system will provide initial execution results for the work order, such as whether the engineer successfully accepted the order, whether an appointment time with the customer was scheduled, and the time of the first on-site check-in. This information constitutes the intermediate states of the work order. When an intermediate state is detected, directly outputting an absolute positive reward is strictly prohibited. Instead, a final value prediction engine trained on complete trajectory data of massive historical closed-loop work orders is introduced. Knowing only the initial state, the executed actions, and the current feedback intermediate state, this engine estimates the probability that the work order will achieve a high-quality final result in the future.

[0037] When an intermediate status such as appointment completion or first visit is received, the final value prediction engine extracts the initial state vector, action features, and time consumption features of the intermediate status of the current work order and inputs them into the prediction model. The prediction model outputs a probability value between zero and one, representing the expected probability that the work order will not require a second repair in the future and that the customer will be satisfied.

[0038] Subsequently, this probability value is used to calculate the expected reward, which equals the expected probability value multiplied by a preset high-quality final state benchmark reward constant, plus a small penalty term for the time taken in intermediate states. This calculated expected reward represents the best value estimate based on the current limited information. The Critic network draws samples from the short-term experience replay pool, uses the expected reward to calculate the temporal difference error, and updates its own value assessment parameters. The Actor network, on the other hand, uses the advantage function provided by the Critic network to perform preliminary parameter updates according to the policy gradient algorithm.

[0039] If a work order does not require a second repair during the observation period and the customer's evaluation is extremely high, the actual reward will be given a high positive value; if a second repair occurs, no matter how quickly the order was accepted in the early stages, the actual reward will be given a significant negative penalty value; if the customer's evaluation is average and the repair takes a long time, a medium or low actual reward will be calculated according to the specific business rules.

[0040] After calculating the actual reward, the expected reward already used for the initial update is subtracted from the actual reward to obtain a reward deviation value. This value quantifies the cognitive error that occurred when using intermediate states for early expectation assessment. If the deviation value is positive, it indicates that the long-term true value of the dispatch action has been underestimated, and the resulting end-service quality exceeds early expectations. If the deviation value is negative, it indicates that although the action performed smoothly in the short term, it ultimately led to rework or customer dissatisfaction, and its long-term value was severely overestimated. This reward deviation value is used as a lag correction signal, traced back to the initial dispatch node, in preparation for a secondary correction of the reinforcement learning model.

[0041] Finally, this reward deviation value is used as a lag correction signal, traced back to the initial dispatch node, and prepared for a secondary correction of the reinforcement learning model. The long-term waiting queue not only saves the initial state and action, but also the original action execution probability output by the Actor network when the action was initially performed. The saved initial state is re-inputted into the latest version of the Actor network to calculate the new probability of executing the action under the current policy. Dividing the new probability by the original probability yields an importance sampling ratio. To prevent gradient explosion, this ratio is pruned to keep it within a reasonable confidence interval.

[0042] Using the pruned importance sampling ratio, reward bias, and the log probability gradient of the action pair in this state under the current policy, a rigorously calibrated correction gradient is calculated. This correction gradient directly affects the deep parameters of the Actor network, fine-tuning the direction and compensating for the magnitude of the initial updates based on expected rewards. If the bias is negative, the correction gradient strongly reduces the probability of selecting the dispatch action in this state; if the bias is positive, it further increases the probability. Simultaneously, the Critic network's value evaluation parameters are also updated using the reward bias, allowing the value network to gradually learn the true value distribution over long periods. Through this secondary optimization, the decision-making strategy of the reinforcement learning agent is further refined, making it more accurately reflect the actual impact of the initial decision on the final service quality.

[0043] In summary, this application effectively addresses the issues of misaligned feedback attribution and lack of accurate source tracing capabilities for lagging results in traditional multi-stage, cross-cycle business process decision-making by employing a dual mechanism: initial optimization through the introduction of expected rewards and secondary optimization based on the deviation between actual and expected rewards. This not only improves the convergence efficiency of reinforcement learning models in multi-stage, cross-cycle business processes but also enables precise source tracing of lagging results, thereby allowing decision-making strategies to continuously and effectively optimize the final service quality of the business process.

[0044] Furthermore, the steps for generating the expected reward for the end-service quality of the business process based on intermediate states include: Invoke the end-value prediction engine that has been pre-trained based on the complete historical business process trajectory; Input the initial state information, initial order dispatch decision and the time series characteristics of intermediate states into the final value prediction engine to estimate the probability value of the business process reaching the preset high-quality final result. Map the probability value to the expected reward.

[0045] Among them, the final value prediction engine is a trained prediction model whose core function is to make a forward-looking assessment of the final service quality result of the business process based on the currently available information. The training data comes from a massive amount of historical business process trajectories that have been completely closed-loop. These trajectories contain the entire chain of data from the initial state, all intermediate flow nodes, the decision-making actions executed, to the final service result.

[0046] By learning from this historical data, we can capture the complex nonlinear relationships between different initial conditions, decision-making behaviors, intermediate processes, and final service quality. For example, the end-value prediction engine can be a deep learning-based sequence model, such as a Long Short-Term Memory network or a Transformer network, capable of handling business process data with strong temporal dependencies; or it can be an ensemble learning model, such as a gradient boosting decision tree or a random forest, which uses feature engineering to transform temporal information into static features for prediction. Its role is to provide a reliable quantitative expectation of future end-service quality before the end result of the business process is apparent.

[0047] The initial dispatch decision refers to the specific dispatch and upgrade strategies made by the reinforcement learning agent at the beginning of the business process. The temporal features of intermediate states record the key events and their timing information experienced by the business process from the execution of the initial decision to the current intermediate node, such as appointment completion time and the engineer's first on-site visit time. This information can be integrated into a high-dimensional feature vector, for example, through feature concatenation, one-hot encoding, or embedding layers, and then used as input to the model. After receiving these comprehensive inputs, the final value prediction engine outputs a probability value between 0 and 1, quantifying the likelihood that the current business process will achieve a preset high-quality final outcome in the future.

[0048] Mapping probability values ​​to expected rewards transforms the estimated probability values ​​into expected rewards that the reinforcement learning agent can understand and utilize. One mapping method is a linear transformation, where the expected reward equals the probability value multiplied by a preset reward coefficient; for example, expected reward = probability value * 100. Another mapping method can be a non-linear function, such as transforming the probability value using a sigmoid or tanh function to provide more sensitive reward feedback within a specific probability range. Furthermore, other factors can be combined in the mapping; for example, the expected reward could equal the expected probability value multiplied by a preset high-quality final reward constant, plus a small penalty term for the time spent in intermediate states, thus more precisely reflecting the actual value of the business process.

[0049] Through the above technical solution, this application effectively solves the problems of inaccurate expected rewards and the inability to correlate initial decisions with final results caused by the lag in final outcome in traditional methods. By calling a final value prediction engine pre-trained based on the complete historical business process trajectory, and inputting initial state information, initial order dispatch decisions, and the temporal characteristics of intermediate states, the probability value of the business process reaching a preset high-quality final outcome can be accurately estimated. Mapping this probability value to expected rewards allows the reinforcement learning agent to obtain a reward signal that closely approximates the actual final service quality even before the final outcome of the business process is reflected back. This significantly improves the quality of the reward signal in the initial optimization stage, enabling the reinforcement learning agent's decision-making strategy to more effectively learn how to make initial order dispatch decisions that truly benefit the improvement of final service quality, thereby avoiding strategy bias caused by short-term reward misleading.

[0050] Furthermore, the steps for initially optimizing the decision-making strategy of the reinforcement learning agent using anticipated rewards include: Utilize expected rewards to compute temporal difference errors to optimize the value evaluation network in reinforcement learning agents; Based on the advantage function in the optimized value assessment network, the parameters of the decision policy are initially optimized according to the policy gradient algorithm.

[0051] The expected reward is the estimated reward for the final service quality of the business process when the intermediate state occurs. As an immediate feedback signal, it is used to perform preliminary optimization of the model before the final result is fed back. The calculation method can be to map the estimated probability value of the business process reaching the preset high-quality final result, or to estimate it by combining the initial state information, the initial order dispatch decision and the temporal characteristics of the intermediate state through a pre-trained final value prediction engine.

[0052] Temporal difference error is a core concept in reinforcement learning used to update the value function. It measures the difference between the value estimate of the current state and the value estimate based on the next state plus the immediate reward, and can be expressed as R_t + γ*V(S_{t+1}) - V(S_t), where R_t is the immediate reward (in this case, the expected reward), γ is the discount factor, and V(S) is the state value function. Another implementation is R_t + γ*Q(S_{t+1}, A_{t+1}) - Q(S_t, A_t), used to update the Q-value function. The value evaluation network is a component of the reinforcement learning agent, typically a deep neural network, called the Critic network in the Actor-Critic architecture. Its role is to estimate the value V(S) of a given state, learning to predict the cumulative reward obtainable in a specific state. Optimization refers to adjusting the internal parameters of the value evaluation network, such as the weights and biases of the neural network, to make its output value estimate more accurate and more consistent with the actual expected reward. This is usually achieved through gradient descent or its variants, with the goal of minimizing the temporal difference error.

[0053] The advantage function A(S,A) measures the additional benefit gained from performing a specific action A in a given state S, compared to the average value V(S) of that state. It is typically defined as Q(S,A) - V(S), where Q(S,A) is the state-action value function, and V(S) is the state value function. The advantage function more clearly indicates which actions are better in a given state, thus guiding the direction of policy updates. Policy gradient algorithms are a class of reinforcement learning algorithms that directly optimize the parameters of the policy function. They calculate the gradient of the policy function with respect to its parameters and update the parameters along the gradient direction to maximize the expected cumulative reward. In the Actor-Critic architecture, the Actor network uses the advantage function provided by the Critic network to update its policy parameters. The decision policy is another component of the reinforcement learning agent, usually also a deep neural network, referred to as the Actor network in the Actor-Critic architecture. Its role is to output the probability distribution of performing a certain action based on the current state. The parameters of the decision policy determine which action the agent takes in different states. Preliminary optimization refers to the initial parameter adjustment of the decision-making strategy using expected rewards before the final result of the business process is fed back.

[0054] Through the above technical solution, this application utilizes a hierarchical and step-by-step optimization logic, relying on expected rewards to sequentially update the value assessment network and decision-making strategy, effectively solving the problems of feedback attribution misalignment and strategy optimization direction deviation caused by the lag in final results in traditional methods. The value assessment network is optimized using the time-series difference error in expected reward calculation, enabling it to adjust its estimation of state value in advance with the help of real-time feedback. Based on this, the parameters of the decision-making strategy are initially optimized according to the policy gradient algorithm, based on the dominance function in the optimized value assessment network, ensuring that the adjustment direction of the policy parameters always moves towards improving the expected final service quality.

[0055] Furthermore, after the initial optimization of the decision-making strategy of the reinforcement learning agent using anticipated rewards, the process also includes: Establish a waiting queue that uniquely corresponds to the business process; The initial state information, initial order dispatch decision, expected reward, and the original strategy execution probability when executing the initial order dispatch decision are associated and stored in the waiting queue.

[0056] Specifically, establishing a waiting queue uniquely corresponding to each business process refers to assigning a unique identifier to each independent business process and using this identifier as an index to create a queue specifically for long-term storage of decision context information related to that business process. This can be a memory-based distributed caching system, for example, using key-value storage technology, storing the unique identifier of the business process as the key and its corresponding decision information as the value to support high-concurrency access and fast retrieval. Alternatively, this waiting queue can also be a persistent database table, where the unique identifier of the business process serves as the primary key, and corresponding indexes are created to optimize query performance, thereby providing longer-term storage guarantees and data reliability. According to some embodiments of this application, an initial experience record containing the current work order's unique identifier, current timestamp, input state representation vector, and selected execution action is generated in memory and stored in a specially designed multi-level experience replay pool's tracking area, awaiting subsequent execution feedback. After the work order generates the expected reward and completes the initial policy update, the complete context information of the work order is not discarded but transferred from the tracking area to a long-term waiting queue with long-term storage capabilities.

[0057] Storing initial state information, initial dispatch decision, expected reward, and the original strategy execution probability when executing the initial dispatch decision in a waiting queue refers to integrating and packaging multiple key data items related to a specific business process and storing them in a searchable manner in the waiting queue. Initial state information serves as the input basis for the reinforcement learning agent's initial dispatch decision; it is typically a high-dimensional vector containing multi-dimensional information such as customer, device, and resources. The initial dispatch decision is the specific action output by the reinforcement learning agent based on the initial state information, such as which engineer to dispatch to or whether to expedite the process. The original strategy execution probability when executing the initial dispatch decision is the probability value of the decision output by the reinforcement learning agent when generating the initial dispatch decision; it is a necessary parameter for calculating the correction gradient in subsequent secondary optimization. This information can be serialized into a structured data object, such as a JSON object or a ProtocolBuffer message, and then stored as a single record in the waiting queue, with the unique identifier of the business process serving as the index of this record. Alternatively, a record can be created in the database for each business process, with fields set for the initial state information, initial dispatch decision, expected reward, and original strategy execution probability for storage.

[0058] Through the above technical solution, this application effectively addresses the problem of missing initial decision-related information in subsequent secondary optimizations when the final service result is delayed in multi-stage business processes. By pre-linking and storing the corresponding information after the initial optimization is completed, a complete and accurate data foundation is provided for subsequent backtracking and correction of decision-making strategies. This ensures that the reinforcement learning agent can perform effective secondary optimization based on long-term real feedback, avoiding decision-making strategy deviations caused by information loss or misalignment, and significantly improving the accuracy of business process decisions and the quality of final services.

[0059] Furthermore, tracing back to the node that generated the initial dispatch decision, the steps for secondary optimization of the decision strategy based on the hysteresis correction signal include: Extract the initial state information and original strategy execution probability corresponding to the business process from the waiting queue; Input the initial state information into the decision policy at the current moment to obtain the probability of executing the current policy; Calculate the ratio of the current strategy execution probability to the original strategy execution probability, calculate the product of the ratio and the reward deviation value, and correct the product to keep it within the preset confidence interval. The corrected product value is multiplied by the logarithmic probability gradient of the policy under the initial state information to generate the corrected gradient. The parameters of the decision policy are then residually optimized based on the corrected gradient.

[0060] The original policy execution probability refers to the probability value of the actual selected action in the probability distribution of each optional action output by the reinforcement learning agent in response to the initial state information when the initial order dispatch decision is generated. Its function is to record the original tendency at the time of decision-making and provide a benchmark for subsequent policy correction. It can be stored together with the initial state information after the Actor network outputs the action probability distribution, or it can be obtained by recording the forward propagation results of the Actor network in the initial state.

[0061] A waiting queue is a storage structure used for temporary or long-term storage of business process-related data. It carries critical information that needs to be tracked and corrected from the initial decision to the final result feedback, enabling historical data traceability across time and nodes. This waiting queue can be built on a distributed caching system to support high-concurrency read / write and data persistence, or it can be combined with a message queue and a database, with the message queue used for real-time event delivery and the database used for long-term storage.

[0062] The current decision policy refers to the latest version of the reinforcement learning agent after initial optimization. Its function is to reflect the model's latest decision tendency regarding the same initial state at the current time. This can be achieved by loading and using the latest training weights and parameters of the reinforcement learning agent, or by managing model versions at different time points through a version control system and calling the latest version for inference. The current policy execution probability refers to the probability value output for the action actually executed after inputting the original initial state information into the latest version of the decision policy. Its function is to quantify the degree to which the current policy recognizes that historical action. This can be achieved by using the extracted initial state information as input, performing forward inference through the current Actor network to obtain the probability distribution of all actions, and extracting the probability corresponding to the original action.

[0063] The ratio refers to the quotient between the current policy execution probability and the original policy execution probability. It measures the degree of change of the current policy in a specific action relative to the original policy, and its purpose is to correct for data distribution mismatch caused by policy updates. This ratio can be directly calculated using floating-point division, or, to avoid instability caused by a division by zero or an excessively small number, the original policy execution probability can be smoothed.

[0064] Correction processing refers to the process of limiting or adjusting the product of the calculated ratio and the reward deviation value. Its purpose is to prevent excessive policy update magnitude from causing model instability or collapse, ensuring the stability of optimization. Pruning techniques can be used to limit the product to a preset upper and lower limit, such as using the pruning function in the PPO algorithm. The preset confidence interval is a predefined numerical range used to limit the product value after correction processing, ensuring the stability and reliability of policy updates.

[0065] The policy log probability gradient under initial state information refers to the gradient of the log probability of the action output by the current decision policy with respect to the network parameters under the initial state information. This gradient indicates how to adjust the network parameters to increase the log probability of a specific action, and its role is to provide the direction for policy parameter updates. This can be achieved by calculating the partial derivatives of the log probability of the Actor network with respect to the network parameters given the initial state and action using an automatic differentiation framework. The correction gradient is the final gradient obtained by multiplying the corrected product value with the policy log probability gradient. This gradient integrates reward bias, the degree of policy change, and the policy update direction, and is the core signal used for secondary optimization of policy parameters, its role being to precisely guide the adjustment of policy parameters. Residual optimization refers to incrementally adjusting the parameters of the decision policy using the correction gradient based on the existing preliminary optimization results. Its role is to avoid destroying the results of the preliminary optimization and achieve refined, cumulative policy improvement. This can be achieved by using the gradient descent algorithm to directly apply the correction gradient to the parameter updates of the Actor network.

[0066] Through the above technical solution, this application effectively solves the problem of policy optimization bias caused by feedback lag and attribution misalignment in multi-stage business processes of traditional reinforcement learning. By extracting initial state information and original policy execution probabilities from the waiting queue and comparing them with the current policy, the changes in the policy during the update process can be accurately quantified. Combined with the final reward bias value, a calibrated correction gradient is generated, ensuring that the secondary optimization of the decision policy is based on real and complete historical information, avoiding correction errors caused by missing or inaccurate information. In particular, by calculating the ratio of the current policy execution probability to the original policy execution probability and correcting the product to maintain it within a preset confidence interval, the magnitude of policy updates can be effectively controlled, preventing policy oscillations caused by over-correction, thereby ensuring the stability and convergence of the reinforcement learning model training. Furthermore, multiplying the corrected product value with the logarithmic probability gradient of the policy under the initial state information and performing residual optimization allows the reward bias to be accurately converted into incremental adjustments to the decision policy parameters, thereby achieving fine-grained correction of the initial decision without destroying the initial optimization results. This secondary optimization mechanism, combined with the scheme of establishing a waiting queue and storing the original strategy execution probability in the pre-processing information, can achieve accurate tracing and attribution of cross-node lagging results.

[0067] Furthermore, after the final service result of the business process is fed back, the steps for calculating the actual reward based on the final service result include: Obtain the end-point metrics of the business process, including the second-rework indicator, customer satisfaction score, and total processing time. The final score indicators are combined and calculated to generate the actual reward.

[0068] The acquisition of end-of-process metrics aims to collect key data points reflecting the final service quality from completed business processes. These data points serve as objective criteria for evaluating service effectiveness, customer experience, and efficiency. They can be queried in real-time from the business database; for example, data collection is triggered when a work order status changes to "completed" or "closed." Alternatively, relevant data can be extracted from multiple business systems, such as customer relationship management systems, work order management systems, and customer feedback systems, through periodic data synchronization tasks. Another approach is through an event-driven mechanism, where the event bus pushes relevant data to the reward calculation module when specific end-of-process events, such as customer confirmation or follow-up completion, occur. End-of-process metrics include secondary repair indicators, customer satisfaction scores, and total processing time, used to comprehensively measure the end-of-process service quality of the business process.

[0069] The second-repair indicator measures the effectiveness and durability of service results. It can be implemented as a Boolean value (e.g., 0 for no second repair, 1 for a second repair) or an enumeration value (e.g., no repair, one repair, multiple repairs). This is typically determined by the work order system during an observation period after a work order is closed, by monitoring for new related work orders. Customer satisfaction ratings directly reflect customer perception and experience of the service. They can be implemented as a numerical value (e.g., a 1-5 scale, 0-100 scale) or a category label (e.g., very satisfied, satisfied, neutral, dissatisfied). This is usually collected through customer follow-ups, SMS feedback links, and in-app reviews. Total processing time measures service efficiency. It can be implemented as a time difference (e.g., the total number of hours or days from work order creation to closure) or a time period (e.g., an integer in minutes). This is typically automatically recorded and calculated by the work order management system.

[0070] Calculating a combination of end-game metrics to generate a real reward involves integrating multiple dimensions of end-game metrics into a single, quantifiable reward value to enhance the learning agent's policy optimization. This combination can be achieved through a pre-defined weighted summation function. For example, customer satisfaction scores can be multiplied by one weight, second-return indicators can be converted into penalties and multiplied by another weight, and total processing time can be converted into either a penalty or reward and multiplied by a third weight. These weighted values ​​are then summed. Another approach is to use a rule-based decision tree or lookup table to directly map different combinations of metrics to a predefined reward value.

[0071] Through the above technical solution, this application can comprehensively and accurately evaluate the final service quality of the business process after the final service result is fed back. By acquiring and combining three multi-dimensional final indicators—secondary repair indicator, customer satisfaction score, and total processing time—it overcomes the limitation of traditional methods where single-dimensional indicators cannot accurately reflect the overall service quality. The secondary repair indicator ensures long-term consideration of the effectiveness of the service result, avoiding decision-making misguidance caused by short-term success but frequent long-term problems; the customer satisfaction score directly reflects the customer's true feelings, enabling decision optimization to take user experience into account; and the total processing time ensures the evaluation of service efficiency. This combination of multi-dimensional indicators allows the generated real reward to more comprehensively and objectively quantify the final value of the business process, providing an accurate and reliable benchmark for subsequently calculating the deviation between the real reward and the expected reward. This ensures that the optimization direction of the reinforcement learning agent's decision-making strategy can accurately align with the goal of improving the overall final service quality, effectively avoiding deviations in strategy optimization caused by inaccurate reward signals.

[0072] Furthermore, the steps for combining and calculating the final outcome metrics to generate the actual reward include: When some final indicators are missing from the final indicators, alternative indicators are generated based on the acquired final indicators and historical statistical data. Based on alternative metrics and the acquired endgame metrics, the real reward is calculated according to the preset real reward calculation function.

[0073] Specifically, when some final metrics are missing, it means that after the final service result of the business process is returned, when retrieving the final metrics used to calculate the actual reward, it is discovered that some preset key metric data failed to be collected or obtained successfully. For example, a data integrity verification module can be used to perform real-time checks on the returned final metric data, such as checking whether specific fields are empty, invalid in format, or outside a reasonable range. Once any key metric is found to be inconsistent with the preset integrity requirements, it is determined that some final metrics are missing. Alternatively, a list of final metrics can be preset, and the received final service result can be compared with this list. If any mandatory metric in the list cannot be found in the returned data, the metric missing judgment is triggered.

[0074] Generating alternative indicators based on acquired final indicators and historical statistical data refers to using successfully acquired, intact final indicator information, combined with patterns or models derived from historical business data, to infer and generate reasonable estimates of the missing indicators when some final indicators are detected as missing. One implementation method is to use a pre-trained machine learning model, using acquired final indicators as input features and complete final indicators from historical data as training targets, to predict the value of the missing indicator. For example, when customer satisfaction is missing, prediction can be made based on the second-repair indicator, total processing time, and satisfaction data from similar historical work orders. Another implementation method is to use rule-based interpolation or weighted averaging methods. For example, for missing customer satisfaction, a substitute value can be generated by weighting the acquired second-repair indicator (higher weight if there is no second-repair) and the total processing time (higher weight if the processing time is shorter), combined with the historical average satisfaction of similar businesses, regions, or customer groups.

[0075] Based on alternative indicators and the acquired final indicators, the true reward is calculated according to a pre-defined true reward calculation function. This means that after generating substitute values ​​for the missing indicators, these substitute indicators, along with the originally acquired complete final indicators, are used as input into a pre-defined mathematical model or algorithm for calculating the true reward, thus yielding a comprehensive true reward value. As one implementation method, the true reward calculation function can be a multivariate linear or non-linear function. For example, True Reward = w1 * (1 - Secondary Repair Flag) + w2 * Customer Satisfaction + w3 * (1 / Total Processing Time). Here, w1, w2, and w3 are pre-defined weighting coefficients, the secondary repair flag is a Boolean value (0 or 1), customer satisfaction is a rating value, and total processing time is a time value. When customer satisfaction is missing, its substitute indicator is used. As another implementation approach, the actual reward calculation function can also be a rule-based scoring system. For example, if there are no second repairs and customer satisfaction is above a certain threshold, the reward is a high score; if a second repair occurs, the reward is a low score; if customer satisfaction is average and the total processing time is moderate, the reward is a medium score. In the calculation, substitute indicators are treated as actual indicator values.

[0076] Through the above technical solution, this application effectively solves the problem in real-world business scenarios where missing end-game metrics in the business process prevent the calculation of actual rewards, thus affecting the continuous optimization of reinforcement learning agent strategies. By intelligently generating alternative metrics, the continuity of actual reward calculation is ensured, avoiding optimization interruptions due to incomplete data. Furthermore, the alternative metrics generated based on the acquired end-game metrics and historical statistical data, compared to simple default value filling, better reflect the actual situation of the current business and the overall statistical patterns of the business, thereby improving the accuracy of the final generated actual rewards. This not only ensures the continuity of reinforcement learning agent strategy iteration but also makes subsequent reward deviation calculations and secondary optimization of decision-making strategies more accurate and effective.

[0077] Furthermore, after the step of calculating the reward deviation value between the actual reward and the expected reward, the following is included: The average absolute value of the reward deviation of all business processes within a preset time window; When the absolute mean exceeds the preset safety threshold, the learning rate weight of the expected reward in the initial optimization is reduced, and the final value prediction engine is retrained and its parameters are fine-tuned. Once the absolute mean value falls back to within the preset safety threshold, the learning rate weight for the expected reward is restored.

[0078] Specifically, monitoring the average absolute value of reward deviations across all business processes within a preset time window involves continuously tracking and recording the difference between the actual reward and the expected reward for each completed business process. The absolute values ​​of these differences are accumulated and averaged over a pre-defined time period to obtain an indicator reflecting the overall level of prediction accuracy. This monitoring can be implemented through a separate monitoring service that periodically retrieves the reward deviations of all business processes within a specified time window from data storage and calculates their average absolute value. Alternatively, a streaming data processing platform can be used to maintain the average absolute value of reward deviations within a sliding time window in real time. When the average absolute value exceeds a preset safety threshold, it indicates a significant decrease in the prediction accuracy of the final value prediction engine, exceeding an acceptable range. This safety threshold is a configurable value used to define the upper limit of tolerance for prediction deviation. Reducing the learning rate weight of expected rewards in initial optimization refers to dynamically adjusting the influence of expected rewards in the initial optimization process of the enhancement learning agent's decision-making strategy when a decrease in prediction engine accuracy is detected. This can be achieved by multiplying the learning rate used to update the value evaluation network or policy network by a factor less than 1, or by directly setting the learning rate to a preset small value. Retraining and fine-tuning the end-value prediction engine involves reducing the learning rate weights while comprehensively retraining or finely adjusting the model using the latest business process data containing real end-value results to adapt it to the changing business environment. When the absolute mean falls back to within the preset safety threshold, it means that after retraining and parameter fine-tuning, the prediction accuracy of the end-value prediction engine has been restored, and the absolute mean of its reward bias has once again fallen below the safety threshold. Restoring the learning rate weights for expected rewards means restoring the previously reduced learning rate weights to their normal or initial levels, allowing the expected rewards to once again fully play their role in accelerating policy convergence.

[0079] Through the above technical solution, this application enables the entire decision optimization system to have the ability to adapt and self-correct by dynamically adjusting the learning rate weight of expected reward in the initial optimization and timely retraining and fine-tuning the parameters of the final value prediction engine.

[0080] Furthermore, the step of generating a hysteresis correction signal based on the reward deviation value includes: Obtain the unique identifier of the business process and the node information corresponding to the initial order dispatch decision; The reward deviation value is associated with and packaged with a unique identifier and node information to generate a hysteresis correction signal carrying a backtracking positioning tag.

[0081] The unique identifier for a business process refers to assigning a unique identifier to each independent business process throughout its entire lifecycle. This unique identifier binds the current reward deviation to the corresponding specific business process, effectively distinguishing different work orders and preventing the confusion of correction information when processing multiple business processes in parallel. This unique identifier can be a primary key ID automatically generated by the system, such as a globally unique identifier or a database auto-incrementing ID; or it can be a globally unique string generated using a hash algorithm by combining business attributes, such as work order number, customer ID, and timestamp.

[0082] Obtaining the node information corresponding to the initial dispatch decision refers to recording the specific context information of the decision within the business process trajectory when generating the initial dispatch decision. The purpose of this node information is to accurately pinpoint the specific decision location requiring correction, preventing misalignment of corrections between different decision nodes within the same business process. Node information may include the timestamp of the decision, the business stage identifier (e.g., the initial acceptance stage), the system module ID executing the decision, and the hash value of the work order status snapshot on which the decision is based; alternatively, node information may be a specific record ID in the decision log, containing all relevant context at the time of the decision, such as the version of the decision agent and a summary of the input state features. Associating and packaging the reward deviation value with the unique identifier and node information means logically or physically combining the calculated reward deviation value, the unique identifier of the business process, and the node information corresponding to the initial dispatch decision into a complete data unit. This associative packaging operation ensures that the information within the delayed correction signal maintains the correct correspondence throughout the entire storage and circulation process, preventing errors.

[0083] Specifically, these three pieces of information can be encapsulated into a structured data object, such as a JSON object or a Protobuf message, containing explicit fields to store their respective values; alternatively, this information can be serialized into a specific string format, distinguished by predefined delimiters or fixed-length fields, and parsed when needed. Generating a hysteresis correction signal carrying a backtracking location label means using the aforementioned associated packaged information as part of the hysteresis correction signal. The unique identifier of the business process and the node information corresponding to the initial dispatch decision together constitute the backtracking location label of this signal, enabling the secondary optimization module to quickly and accurately find the initial dispatch decision node that needs correction, achieving precise attribution. The hysteresis correction signal can be a data packet containing the aforementioned packaged data, which is sent to the secondary optimization module via a message queue, where the packaged data is the backtracking location label; or it can be a database record containing a reward deviation value, and its primary key or index field is the backtracking location label composed of a unique identifier and node information. This reward deviation value is used as the hysteresis correction signal to trace back to the initial dispatch node, preparing for secondary correction of the reinforcement learning model.

[0084] Secondly, see Figure 2 This application proposes a business process decision optimization system based on reinforcement learning, used to execute the aforementioned business process decision optimization method based on reinforcement learning. The system includes: The decision module 210 is used to obtain the initial status information of the current work order in the business process and input the initial status information into the reinforcement learning agent to generate the initial dispatch decision. The preliminary optimization module 220 is used to generate an expected reward for the final service quality of the business process based on the intermediate state generated after the execution of the initial order dispatch decision, and to use the expected reward to perform preliminary optimization of the decision-making strategy of the reinforcement learning agent. The results tracking module 230 is used to calculate the actual reward based on the final service result after the final service result of the business process is returned. The deviation calculation module 240 is used to calculate the reward deviation value between the actual reward and the expected reward, and generate a lag correction signal based on the reward deviation value; The secondary optimization module 250 is used to backtrack to the node that generated the initial dispatch decision and perform secondary optimization of the decision strategy based on the hysteresis correction signal.

[0085] The above technical solutions can effectively prevent actions that are smooth in the short term but have a high return rate in the long term from being reinforced as high-quality decisions. This ensures that the decision-making strategies formed by reinforcement learning are truly centered around the goal of optimizing the end-service quality, and significantly improve the service quality indicators of business processes.

[0086] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A business process decision optimization method based on reinforcement learning, characterized in that, The method includes: Obtain the initial status information of the current work order in the business process, and input the initial status information into the reinforcement learning agent to generate an initial dispatch decision; In response to the intermediate state generated after executing the initial dispatch decision, an expected reward for the final service quality of the business process is generated based on the intermediate state, and the expected reward is used to perform preliminary optimization of the decision-making strategy of the reinforcement learning agent. After the final service result of the business process is fed back, the actual reward is calculated based on the final service result; Calculate the reward deviation value between the actual reward and the expected reward, and generate a hysteresis correction signal based on the reward deviation value; The decision-making strategy is then optimized a second time based on the hysteresis correction signal, tracing back to the node that generated the initial dispatch decision.

2. The method according to claim 1, characterized in that, The step of generating the expected reward for the final service quality of the business process based on the intermediate state includes: Invoke the end-value prediction engine that has been pre-trained based on the complete historical business process trajectory; The initial state information, the initial order dispatch decision, and the temporal characteristics of the intermediate state are input into the final value prediction engine to estimate the probability value of the business process reaching a preset high-quality final result. The probability value is mapped to the expected reward.

3. The method according to claim 1, characterized in that, The step of initially optimizing the decision-making strategy of the reinforcement learning agent using the expected reward includes: The expected reward is used to calculate the temporal difference error in order to optimize the value evaluation network in the reinforcement learning agent; Based on the advantage function in the optimized value assessment network, the parameters of the decision-making strategy are initially optimized according to the policy gradient algorithm.

4. The method according to claim 1, characterized in that, After the step of initially optimizing the decision-making strategy of the reinforcement learning agent using the expected reward, the method further includes: Establish a waiting queue that uniquely corresponds to the aforementioned business process; The initial state information, the initial order dispatch decision, the expected reward, and the original strategy execution probability when executing the initial order dispatch decision are associated and stored in the waiting queue.

5. The method according to claim 4, characterized in that, The step of tracing back to the node that generated the initial dispatch decision and performing a secondary optimization of the decision strategy based on the hysteresis correction signal includes: Extract the initial state information and the original strategy execution probability corresponding to the business process from the waiting queue; The initial state information is input into the decision policy at the current moment to obtain the current policy execution probability; Calculate the ratio of the current strategy execution probability to the original strategy execution probability, calculate the product of the ratio and the reward deviation value, and correct the product to keep it within a preset confidence interval. The corrected product value is multiplied by the logarithmic probability gradient of the policy under the initial state information to generate a corrected gradient, and the parameters of the decision policy are residually optimized based on the corrected gradient.

6. The method according to claim 1, characterized in that, The step of calculating the actual reward based on the final service result after the final service result of the business process is fed back includes: Obtain the final indicators of the business process, including the second repair indicator, customer satisfaction score, and total processing time. The final outcome metrics are combined and calculated to generate the actual reward.

7. The method according to claim 6, characterized in that, The step of combining and calculating the final indicators to generate the real reward includes: When some of the final indicators are missing, alternative indicators are generated based on the acquired final indicators and historical statistical data. Based on the alternative indicators and the acquired endgame indicators, the real reward is calculated according to a preset real reward calculation function.

8. The method according to claim 2, characterized in that, After the step of calculating the reward deviation value between the actual reward and the expected reward, the following is included: The average absolute value of the reward deviation of all the business processes within the preset monitoring time window; When the mean absolute value exceeds a preset safety threshold, the learning rate weight of the expected reward in the initial optimization is reduced, and the final value prediction engine is retrained and its parameters are fine-tuned. Once the mean absolute value falls back to within the preset safety threshold, the learning rate weight of the expected reward is restored.

9. The method according to claim 1, characterized in that, The step of generating a hysteresis correction signal based on the reward deviation value includes: Obtain the unique identifier of the business process and the node information corresponding to the initial order dispatch decision; The reward deviation value is associated and packaged with the unique identifier and the node information to generate the hysteresis correction signal carrying the backtracking positioning tag.

10. A business process decision optimization system based on reinforcement learning, used to execute the business process decision optimization method based on reinforcement learning as described in any one of claims 1 to 9, characterized in that, The system includes: The decision module is used to obtain the initial status information of the current work order in the business process, and input the initial status information into the reinforcement learning agent to generate the initial dispatch decision. The preliminary optimization module is used to respond to the intermediate state generated after the execution of the initial order dispatch decision, generate an expected reward for the final service quality of the business process based on the intermediate state, and use the expected reward to perform preliminary optimization of the decision-making strategy of the reinforcement learning agent. The result tracking module is used to calculate the actual reward based on the final service result after the final service result of the business process is fed back. The deviation calculation module is used to calculate the reward deviation value between the actual reward and the expected reward, and generate a hysteresis correction signal based on the reward deviation value; The secondary optimization module is used to backtrack to the node that generated the initial dispatch decision and perform secondary optimization on the decision strategy based on the hysteresis correction signal.

Images