A workflow execution method and system based on RPA technology

Abstract

The present application relates to a workflow execution method and system based on RPA technology, and to the technical field of automatic workflow execution, wherein the method includes steps such as building a database, receiving requests, data analysis, selecting RPA robots, reinforcement learning, business execution and output. The system includes: a database building module, a request receiving module, a data analysis module, an RPA robot selection module, a reinforcement learning module, a business execution module and an output module. The present application can reasonably set the RPA workflow according to the type of user requirements, further reducing resource overhead and workflow execution time.

Description

Technical Field

[0001] The present application relates to the technical field of automatic execution of workflows, and in particular to a workflow execution method and system based on RPA technology. Background Art

[0002] Robotic process automation (RPA) is an automation technology that has developed rapidly in recent years. The development of artificial intelligence has forced people to examine what work should be automated and what work should be done by humans. RPA automation technology is a technology that uses robot software to simulate and execute manual business processes. It can simulate human user operations, such as mouse clicks, keyboard input, data extraction and processing, to complete various tedious and repetitive tasks. RPA technology usually does not require writing code, but creates automated tasks by simply recording operations or using predefined operations.

[0003] The Chinese invention patent with application publication number: CN116610082A and application announcement date: August 18, 2023, provides an RPA job workflow redundant scheduling method and system based on deep reinforcement learning. It obtains the execution data of each RPA job through the RPA job workflow, and converts the execution data into job features; obtains the resource fluctuations and hardware information of the corresponding machine through the RPA executor, and converts the resource fluctuations and hardware information into resource features of the cluster machine; constructs a deep reinforcement learning agent, constructs the input state of the agent through job features and resource features, trains and optimizes the agent according to the reward function, and outputs a redundant scheduling strategy.

[0004] Although the above patents reduce resource overhead and workflow execution time to a certain extent, the workflow execution process is different for different user needs. Therefore, it is necessary to reasonably set the RPA workflow according to user needs to further reduce resource overhead. Summary of the invention

[0005] In order to further reduce resource overhead, the present application provides a workflow execution method and system based on RPA technology.

[0006] In the first aspect, the present application provides a workflow execution method based on RPA technology, which adopts the following technical solution:

[0007] A workflow execution method based on RPA technology, comprising:

[0008] Database construction: Establish a knowledge base and store historical data of business types and processing strategies in the knowledge base;

[0009] Receive request: receive the pending user request, recorded as the first request;

[0010] Data analysis: including first modeling, feature extraction and classification;

[0011] First modeling: building a deep neural network traffic model;

[0012] Feature extraction: Extract demand features from the first request based on a deep neural network model;

[0013] Classification: taking the demand feature as the input of the decision tree to obtain the business type to which the first request belongs;

[0014] Select RPA robot: Select the RPA robot based on the business type;

[0015] Reinforcement learning: including defining reward functions and adjustments;

[0016] Define the reward function: Define the reward function based on the business execution goals;

[0017] Adjustment: Using deep reinforcement learning algorithms, the RPA robot continuously adjusts its processing strategy based on the feedback of the reward function until the value of the reward function reaches the maximum value.

[0018] Business execution: Based on the first request and business rules, the RPA robot after reinforcement learning is used to perform business operations and obtain operation results;

[0019] Output: Output the operation results.

[0020] By adopting the above technical solution, the present application extracts the demand features in the user request through a deep neural network model and classifies it using a decision tree. The method can automatically map the user request to different business types. According to the different business types, the most suitable RPA robot can be intelligently selected to perform the corresponding business operations, so that the RPA workflow can be dynamically adjusted according to different user requests. The present application defines a reward function and uses a deep reinforcement learning algorithm to adjust the processing strategy so that the RPA robot can continuously learn and optimize its own behavior when facing different user needs and business scenarios. This means that the RPA robot can automatically adjust its workflow and execution strategy according to changes in user needs and changes in the business environment to better meet user expectations and needs and reduce resource overhead. With the continuous accumulation and updating of the knowledge base, the present application can gradually learn and master more types of user needs and business scenarios, and then more flexibly adjust the RPA workflow to adapt to different user needs.

[0021] Optionally, after executing the step of defining the reward function and before executing the step of adjusting, the method further includes:

[0022] Task decomposition: decompose the business to be executed into multiple subtasks;

[0023] Task planning: Develop a task execution plan based on subtasks;

[0024] Task allocation: Each subtask is assigned to the corresponding RPA robot through the HRL algorithm.

[0025] By adopting the above technical solution, the present application decomposes the complex business process corresponding to the user's needs into multiple specific subtasks, which helps to reduce the complexity and execution difficulty of each task and improve the controllability and manageability of the overall workflow. Through task decomposition, the present application can more flexibly adjust the execution order and method of subtasks to adapt to different business scenarios and user needs. The decomposed subtasks are independent and can be executed in parallel without affecting each other, thereby shortening the execution time of the overall business process. Based on the dependencies and priorities between subtasks, the present application formulates a reasonable task execution plan, which can enable the business process to be executed efficiently and orderly. Through task planning, resource conflicts and execution order conflicts between different subtasks can be foreseen and avoided, and the stability and reliability of the system can be improved. Reasonable task planning can reduce unnecessary waiting time and repetitive work, thereby improving the execution efficiency of the overall business process. By assigning subtasks to corresponding RPA robots through the HRL (Hierarchical Reinforcement Learning) algorithm, each RPA robot can give full play to its expertise and make full use of system resources. Considering the load capacity and current status of the RPA robot when assigning tasks helps achieve load balancing and reduce the probability of individual RPA robots being overloaded while other RPA robots are idle. Assigning subtasks to the most suitable RPA robot can improve the accuracy and success rate of task execution, thereby improving the execution quality of the overall business process and reducing the time of task execution. The introduction of the three steps of task decomposition, task planning, and task assignment can further optimize the execution process of the RPA workflow. They not only improve the flexibility and controllability of the system, but also achieve efficient resource utilization and load balancing through reasonable planning and assignment of tasks, ultimately improving the execution efficiency and quality of business processes.

[0026] Optionally, after executing the task allocation step and before executing the adjustment step, the method further includes:

[0027] Initialization: Set up the initial processing strategy and rule set for each RPA robot.

[0028] By adopting the above technical solution and setting the initial processing strategy and rule set, the RPA robot has a certain execution capability before starting to execute tasks. These initial settings enable the RPA robot to respond quickly and accurately when facing simple or common business scenarios, thereby improving the overall reliability of the system. The processing strategy and rule set in the initialization step not only provide a foundation for the RPA robot, but also provide a starting point for the subsequent reinforcement learning and adjustment process. These initial settings can serve as a basis for learning, helping the RPA robot to continuously accumulate experience and optimize its own processing strategy during the actual execution process. Reasonable initial processing strategies and rule sets help the RPA robot to use system resources more efficiently when executing tasks. By reducing unnecessary exploration behaviors and trial and error costs, the RPA robot can find the optimal execution path faster and improve resource utilization. Although the initialization step sets the initial processing strategy and rule set for the RPA robot, this does not mean that they are fixed. On the contrary, these initial settings are to help the RPA robot adapt to different business scenarios and user needs more quickly during the subsequent learning and adjustment process.

[0029] Optionally, after the step of executing the business execution and before the step of executing the output, the step further includes:

[0030] Result judgment: judge whether the operation result meets expectations. If so, execute the output step; if not, execute the step of adding request;

[0031] Add request: Feedback the first request to the operation and maintenance personnel and add the current first request to the knowledge base.

[0032] By adopting the above technical solution, the first request that does not meet expectations is fed back to the operation and maintenance personnel, so that the operation and maintenance personnel can promptly obtain the user needs whose processing results do not meet expectations, and store the first request in the knowledge base. As time goes by, the continuous enrichment of the knowledge base will help improve the RPA robot's ability to handle complex and rare situations, making it more intelligent and efficient.

[0033] Optionally, after the step of determining the result and before the step of adding the request, the method further includes:

[0034] Calculate similarity: Calculate the similarity between the first request and all user requests in the knowledge base;

[0035] Obtain source task: obtain the user request with the greatest similarity to the first request in the knowledge base, record it as the second request, and record the second request and the processing strategy of the second request as the source task;

[0036] Get target task: take the first request as the target task;

[0037] Second modeling: establishing the first LSTM model and the second LSTM model;

[0038] Model training: The first LSTM model is trained using the source task and the processing strategy of the source task;

[0039] Transfer learning: transfer the weights of the trained first LSTM model to the corresponding layers of the second LSTM model;

[0040] Obtain processing strategy: Input the target task into the second LSTM model to obtain the processing strategy of the target task.

[0041] By adopting the above technical solution, the present application calculates the similarity between the first request and the user request in the knowledge base, and obtains the user request with the greatest similarity as the source task. This process realizes the automatic identification and classification of similar problems. Subsequently, the processing strategy of the source task is applied to the new target task through transfer learning, which greatly improves the automation level of the RPA process and reduces the need for manual intervention. The introduction of the LSTM (Long Short-Term Memory Network) model for modeling and training can capture the sequence dependencies in user requests, so as to understand user intentions more accurately. Through transfer learning, the effective features and processing strategies learned in the source task are migrated to the new target task, which helps to quickly and accurately generate processing strategies suitable for new tasks. The method based on similarity matching and transfer learning can more accurately predict and process new tasks, thereby reducing resource waste and process interruptions caused by incorrect processing.

[0042] Optionally, in the step of executing the adding request, the first request and the processing strategy obtained in the step of obtaining the processing strategy are stored in the knowledge base together.

[0043] By adopting the above technical solution, the content of the knowledge base is greatly enriched by adding the first request and its corresponding processing strategy to the knowledge base. This includes not only new user requests, but also effective processing strategies for these new user requests. This richness helps to improve the accuracy and efficiency of the RPA robot when processing similar requests in the future. As the knowledge base continues to expand, the RPA robot can adapt to new business scenarios and user needs more quickly. When encountering a new user request, the system can quickly find similar processing strategies by matching the similarity with the requests in the knowledge base, thereby realizing the automated processing of new user requests.

[0044] Optionally, after executing the step of transfer learning and before executing the step of acquiring the processing strategy, the step further includes:

[0045] Add labels: Add labels to some target tasks and record the labeled target tasks as training data;

[0046] Fine-tuning: Use the training data to train the second LSTM model. During the training process, monitor the changing trend of the output accuracy of the second LSTM model;

[0047] Accuracy judgment: judge whether the change trend of the output accuracy of the second LSTM model meets expectations. If so, execute the step of obtaining the processing strategy; if not, execute the step of adjusting the learning rate;

[0048] Adjust the learning rate: Reduce the learning rate of the second LSTM model. The calculation model is as follows:

[0049] ;

[0050] in, is the current learning rate of the second LSTM model, Adjusted learning rate for the second LSTM model.

[0051] By adopting the above technical solution, the present application further refines the training process of the second LSTM model by adding labels to some target tasks and using them as training data. This helps the second LSTM model to better understand the characteristics of the target task, thereby improving its generalization ability on new tasks. The fine-tuning step uses labeled training data to train the second LSTM model so that the model can adapt to the target task more accurately. By monitoring the changing trend of the output accuracy of the second LSTM model during the training process, potential problems can be discovered and solved in a timely manner to ensure the stable improvement of the performance of the second LSTM model. If the accuracy judgment result shows that the performance of the second LSTM model does not meet expectations, the training process of the second LSTM model can be further optimized by adjusting the learning rate. Reducing the learning rate helps the second LSTM model to adjust parameters more carefully in the later stage of training, thereby improving the convergence and stability of the model. This flexible learning rate adjustment strategy enables the second LSTM model to adapt to different training stages and data characteristics. By setting a reasonable initial learning rate, the training process of the second LSTM model can be accelerated. A higher learning rate can enable the second LSTM model to quickly approach the optimal solution area in the early stage of training, and further fine-tune it through steps such as fine-tuning. This helps save training time and improves overall efficiency.

[0052] Optionally, after performing the step of adding labels and before performing the step of fine-tuning, the following step is further included:

[0053] Set learning rate: Set the initial learning rate of the second LSTM model.

[0054] By adopting the above technical solution, the present application sets the initial learning rate of the second LSTM model. The appropriate learning rate setting helps the model to jump out of the local optimal solution and find the global optimal solution during the training process.

[0055] Optionally, after executing the step of adjusting the learning rate and before executing the step of obtaining the processing strategy, the step further includes:

[0056] Set threshold: Set the minimum threshold of learning rate;

[0057] Learning rate judgment: judge the learning rate Is it lower than the minimum learning rate threshold? If so, execute the step of obtaining the processing strategy; if not, execute the step of fine-tuning.

[0058] By adopting the above technical solution and setting the minimum threshold of the learning rate, it can be ensured that the learning rate will not be too low during the training process, thereby avoiding the model from falling into a deadlock of a local optimal solution or a too slow training process. The learning rate judgment step flexibly adjusts the subsequent training strategy based on the comparison result between the current learning rate and the minimum threshold of the learning rate. If the learning rate is lower than the threshold, the step of obtaining the processing strategy is executed. If the learning rate is still higher than the threshold, the step of fine-tuning is continued to further refine the training of the model. A learning rate that is too low may increase the risk of the model falling into overfitting in the later stages of training. By setting the minimum threshold of the learning rate and taking timely measures (such as obtaining a new processing strategy) when the learning rate is lower than the threshold, it is possible to avoid the model from continuing to overtrain when it is already close to the optimal solution, thereby maintaining the generalization ability of the model.

[0059] In a second aspect, the present application provides a workflow execution system based on RPA technology, including:

[0060] A database building module, used to build a knowledge base, wherein the knowledge base stores historical data of business types and processing strategies;

[0061] A request receiving module, used for receiving a user request to be processed, recorded as a first request;

[0062] Data analysis module, including modeling unit, feature extraction unit and classification unit;

[0063] Modeling unit, building a deep neural network traffic model;

[0064] A feature extraction unit, extracting demand features in the first request based on a deep neural network model;

[0065] A classification unit, taking the demand feature as an input of a decision tree, and obtaining a business type to which the first request belongs;

[0066] Select the RPA robot module and select the RPA robot according to the business type;

[0067] Reinforcement learning module, including reward function definition unit and adjustment unit;

[0068] Define the reward function unit and define the reward function according to the business execution goals;

[0069] The adjustment unit uses a deep reinforcement learning algorithm to enable the RPA robot to continuously adjust its processing strategy based on the feedback of the reward function until the value of the reward function reaches the maximum value;

[0070] The business execution module uses the RPA robot after reinforcement learning to execute business operations and obtain operation results according to the first request and business rules;

[0071] Output module, outputs the operation results.

[0072] By adopting the above technical solutions, the automated and intelligent processing of user requests is realized by integrating the database building module, request receiving module, data analysis module, RPA robot selection module, reinforcement learning module, business execution module and output module. The system not only improves the accuracy and efficiency of business processing and reduces the risk of human errors, but also continuously optimizes the processing strategy of RPA robots through deep learning and reinforcement learning algorithms to adapt to complex and changing business needs and reduce resource overhead. At the same time, its good scalability provides a solid foundation for the addition and optimization of future functions, demonstrating its wide application potential and important practical value in multiple fields.

[0073] In summary, the present application includes at least one of the following beneficial technical effects:

[0074] 1. This application extracts demand features in user requests through a deep neural network model and classifies them using a decision tree. This method can automatically map user requests to different business types. According to different business types, the most suitable RPA robot can be intelligently selected to perform corresponding business operations, so that the RPA workflow can be dynamically adjusted according to different user requests. This application defines a reward function and uses a deep reinforcement learning algorithm to adjust the processing strategy so that the RPA robot can continuously learn and optimize its own behavior when facing different user needs and business scenarios. This means that the RPA robot can automatically adjust its workflow and execution strategy according to changes in user needs and changes in the business environment to better meet user expectations and needs and reduce resource overhead. With the continuous accumulation and updating of the knowledge base, this application can gradually learn and master more types of user needs and business scenarios, and then more flexibly adjust the RPA workflow to adapt to different user needs.

[0075] 2. This application decomposes the complex business process corresponding to user needs into multiple specific subtasks, which helps to reduce the complexity and execution difficulty of each task and improve the controllability and manageability of the overall workflow. Through task decomposition, this application can more flexibly adjust the execution order and method of subtasks to adapt to different business scenarios and user needs. The decomposed subtasks are independent and can be executed in parallel without affecting each other, thereby shortening the execution time of the overall business process. This application formulates a reasonable task execution plan based on the dependencies and priorities between subtasks, which can make the business process efficient and orderly. Through task planning, resource conflicts and execution order conflicts between different subtasks can be foreseen and avoided, improving the stability and reliability of the system. Reasonable task planning can reduce unnecessary waiting time and repetitive work, thereby improving the execution efficiency of the overall business process. By assigning subtasks to corresponding RPA robots through the HRL (Hierarchical Reinforcement Learning) algorithm, each RPA robot can give full play to its expertise and make full use of system resources. Considering the load capacity and current status of the RPA robot when assigning tasks helps achieve load balancing and reduce the probability of individual RPA robots being overloaded while other RPA robots are idle. Assigning subtasks to the most suitable RPA robot can improve the accuracy and success rate of task execution, thereby improving the execution quality of the overall business process and reducing the time of task execution. The introduction of the three steps of task decomposition, task planning, and task assignment can further optimize the execution process of the RPA workflow. They not only improve the flexibility and controllability of the system, but also achieve efficient resource utilization and load balancing through reasonable planning and assignment of tasks, ultimately improving the execution efficiency and quality of business processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a flow chart of Example 1 of the present application;

[0077] Figure 2 This is a flowchart of task adjustment in Example 2 of the present application;

[0078] Figure 3 This is a flow chart of model adjustment in Example 2 of the present application. DETAILED DESCRIPTION

[0079] The following combination Figures 1 to 3 This application is described in further detail.

[0080] Embodiment 1: This embodiment discloses a workflow execution method based on RPA technology, referring to Figure 1The method includes the following steps: S1 building a database, S2 receiving a request, S3 data analysis, S4 selecting an RPA robot, S5 reinforcement learning, S6 business execution and S7 output. First, a knowledge base is established based on historical user requests and processing strategies, then a user request to be processed is received, then data analysis is performed on the first request, then a suitable RPA robot is selected based on the analysis result, then reinforcement learning is performed on the RPA robot, then the RPA robot after reinforcement learning is used to execute business operations according to the processing strategy, and then the processing result is output. The specific process is as follows:

[0081] S1 Database Building: Establish a structured knowledge base and store historical data of business types and historical data of business type processing strategies in the knowledge base. The knowledge base supports data retrieval and update to facilitate rapid matching and calling of business types and processing strategies in subsequent steps.

[0082] S2 receives a request: receives a user request to be processed and records the user request as the first request.

[0083] S3 data analysis: includes S31 first modeling, S32 feature extraction and S33 classification.

[0084] S31 First modeling: Based on deep learning algorithm, establish a deep neural network traffic model.

[0085] S32 Feature Extraction: Extract demand features in the first request based on a deep neural network model.

[0086] S33 Classification: taking the demand feature as an input of a decision tree, and determining the business type to which the first request belongs through a classification algorithm of the decision tree.

[0087] S4 selects an RPA robot: Based on the business type, an RPA robot is selected from the preset RPA robot library. During the selection process, the processing capability, historical performance, and matching degree of the RPA robot with the first request are taken into consideration. The comprehensive performance CP of the RPA robot is calculated, and the RPA robot with the largest comprehensive performance CP is selected as the most suitable RPA robot.

[0088] In the preset RPA robot library, each RPA robot predefines the business type that it can handle, helping operation and maintenance personnel to quickly narrow down the selection range.

[0089] In this step, the comprehensive performance of the RPA robot is quantified by the following methods: :

[0090] ;

[0091] in, The weight of the processing capability of the RPA robot; is the execution speed of the RPA robot, in seconds; V is the maximum execution speed of all RPA robots, in seconds; R is the value of the resource overhead of the RPA robot; S is the stability score of the RPA robot, and the value range of the stability score S is 0-100. The closer to 100, the higher the stability of the RPA robot, and the closer to 0, the worse the stability of the RPA robot; The weight of the historical performance of the RPA robot; Indicates the success rate of the RPA robot's task execution. Its value is obtained through statistical analysis and its initial value is 100. Indicates the failure rate of RPA robot execution tasks. Its value is obtained through statistical analysis and its initial value is 0. The weight of the matching degree between the RPA robot and the current task; The weight of RPA robot priority and availability; .

[0092] M is the score of the matching degree between the RPA robot and the current task, which is calculated based on the matching degree of requirements, data format and business rules. The calculation model is as follows:

[0093] ;

[0094] Among them, B is the matching score of the demand, C is the data format matching score, and D is the business rule matching score. The matching score B of the demand, the data format matching score C, and the business rule matching score D all have a value range of 0-10, and the closer to 10, the higher the matching degree, and the closer to 0, the lower the matching degree.

[0095] The matching score B of the requirement is the percentage of the current task steps that the RPA robot can execute. For example, if the RPA robot can execute 81% of the steps of the current task, the matching score B of the requirement is 8.1 points.

[0096] The data format matching score C is obtained by analyzing the proportion of all data sources directly processed by the RPA robot involved in the current task. For example, if the RPA robot can process 62% of the data of all data sources involved in the current task, the data format matching score C is 6.2 points.

[0097] The business rule matching score D is obtained by analyzing whether the RPA robot needs to add additional configurations to adapt to the business rules, and the proportion of the added additional configurations in the entire processing strategy. For example, due to the adjustment of industry standards, the RPA robot needs to add an additional process, and the process occupies 10% of the entire processing strategy. The score calculation method is 10-1=9. Similarly, the process occupies 12% of the business rules involved in the entire processing strategy, and the score calculation method is 10-1.2=8.8.

[0098] P represents the priority score of the RPA robot, which is set according to the urgency of business needs. The specific values ​​are shown in Table 1.

[0099] Table 1 Value table of the priority score P of the RPA robot

[0100]

[0101] As can be seen from Table 1, the higher the priority score, the more urgent it is.

[0102] Q represents the availability score of the RPA robot, which is calculated based on the current status and resource consumption of the RPA robot. The calculation model is as follows:

[0103] ;

[0104] Where E is the current state of the RPA robot, and the calculation model is as follows:

[0105] ;

[0106] F is the system resources occupied by the RPA robot, and G is the total system resources.

[0107] In other embodiments, the business may also be performed by manually selecting an RPA robot.

[0108] S5 reinforcement learning: includes S51 definition of reward function and S52 adjustment.

[0109] S51 defines a reward function: defines a reward function according to the goal of business execution.

[0110] S52 Adjustment: Using the deep reinforcement learning algorithm, the RPA robot continuously adjusts its processing strategy based on the feedback of the reward function until the value of the reward function reaches the maximum value.

[0111] S6 Business Execution: Based on the first request and business rules, the RPA robot after reinforcement learning is used to perform business operations and obtain operation results.

[0112] S7 output: output operation results.

[0113] This embodiment obtains the user request and the business type to which it belongs, and selects an RPA robot based on the business type. Based on the reinforcement learning algorithm, the selected RPA robot is strengthened. Then, based on the user request and the business type to which it belongs, the RPA robot after reinforcement learning is used to process it, perform business operations, and output operation results. The RPA robot of this embodiment can automatically adjust its workflow and execution strategy according to changes in user needs and changes in the business environment to better meet user expectations and needs, and further reduce resource overhead and workflow execution time.

[0114] Embodiment 2: The difference from Embodiment 1 is that, referring to Figure 2 After executing S51 to define the reward function and before executing S52 to adjust, the process further includes: a task adjustment step, which is as follows:

[0115] S511 Task decomposition: Decompose the business process to be executed into multiple subtasks. Each subtask is a specific link or operation in the business process. Such decomposition helps to better understand the composition of the business process and provides a basis for subsequent task planning and allocation. Through task decomposition, it can ensure that every part of the business process is fully considered and arranged.

[0116] S512 Task Planning: Based on the decomposed subtasks, a detailed task execution plan is developed. The plan clarifies the execution order, time nodes and required resources of each subtask, so that the entire business process can proceed smoothly according to the established goals and requirements.

[0117] S513 Task Allocation: The HRL algorithm intelligently allocates tasks to the most suitable RPA robot based on the characteristics of the subtasks and the capabilities of the RPA robot. This allocation method not only improves the efficiency of task execution, but also ensures the effective use of resources and the smooth progress of business processes.

[0118] S514 Initialization: Set the initial processing strategy and rule set for each RPA robot. The processing strategy defines the basic method and logic that the robot should follow when performing tasks, while the rule set contains a series of specific rules and guidelines to guide the robot's behavior in different situations. Through the initialization settings, the RPA robot has the correct basic configuration and guidance when it starts to perform tasks, thereby improving the accuracy and efficiency of task execution.

[0119] This embodiment decomposes the task and formulates a detailed task execution plan based on the decomposed subtasks. It also uses the HRL algorithm to intelligently assign the task to the most suitable RPA robot according to the characteristics of the subtasks and the capabilities of the RPA robot, thereby improving the effective use of resources and reducing resource overhead.

[0120] Reference Figure 3 In other embodiments, after executing S6 business execution and before executing S7 output, it also includes: S8 model adjustment step, which is as follows:

[0121] S801 result judgment: judge whether the operation result meets expectations. If so, execute S7 output to output the operation result. If not, execute S802 to calculate the similarity.

[0122] S802 Calculate similarity: Calculate the similarity between the first request and all user requests in the knowledge base, aiming to find the historical request most similar to the current request, so as to use the processing strategy of the historical request to assist in solving the current problem.

[0123] S803 Obtain source task: Based on the similarity calculation result, obtain the user request with the greatest similarity to the first request in the knowledge base, record it as the second request, and record the second request and the processing strategy associated with it as the source task as the basis for subsequent learning and migration.

[0124] S804 Obtaining a target task: taking the first request as a target task.

[0125] S805 Second Modeling: In order to perform transfer learning, a first LSTM model and a second LSTM model are established.

[0126] S806 Model training: Use the source task and its processing strategy to train the first LSTM model, so that the first LSTM model can learn the processing strategy of the source task so as to transfer it to the target task.

[0127] S807 Transfer Learning: The weights of the trained first LSTM model are transferred to the corresponding layer of the second LSTM model. Through transfer learning, the second LSTM model can use the knowledge of the source task to better process the target task.

[0128] In this step, other parameters of the first LSTM model can be migrated as needed.

[0129] S808 adding labels: adding labels to some target tasks, and recording the labeled target tasks as training data for the fine-tuning process of the second LSTM guidance model.

[0130] S809 sets the learning rate: sets the initial learning rate of the second LSTM model, which controls the learning speed of the second LSTM model during the training process.

[0131] S810 fine-tuning: Use the training data to train the second LSTM model. During the training process, monitor the changing trend of the output accuracy of the second LSTM model so as to adjust the training strategy in time.

[0132] S811 Accuracy judgment: judge whether the change trend of the output accuracy of the second LSTM model meets expectations. If so, execute S815 to obtain the processing strategy; if not, execute S812 to adjust the learning rate.

[0133] S812 Adjust learning rate: Reduce the learning rate of the second LSTM model. The calculation model is as follows:

[0134] ;

[0135] in, is the current learning rate of the second LSTM model, This is the adjusted learning rate for the second LSTM model. The purpose of reducing the learning rate is to make the second LSTM model more stable during the training process and to avoid the failure of the second LSTM model to converge due to an excessively large learning rate.

[0136] S813 Set threshold: Set the minimum threshold of learning rate.

[0137] S814 Learning rate judgment: judge the learning rate Is it lower than the minimum learning rate threshold? If so, execute S815 to obtain the processing strategy; if not, execute S810 to fine-tune.

[0138] S815 Obtaining a processing strategy: Inputting the target task into the second LSTM model after the learning rate judgment processing in S814 to obtain a processing strategy for the target task.

[0139] S816 Add request: Feedback the first request to the operation and maintenance personnel, and store the first request and the processing strategy obtained in S815 Obtain processing strategy in the knowledge base.

[0140] After the business is executed, if the result does not meet expectations, this embodiment will start a series of optimization processes. First, by calculating the similarity, the closest historical user request in the knowledge base is found as the source task, and the current request is set as the target task. Then, the LSTM model is used for transfer learning to transfer the knowledge of the source task to the target task, and the output accuracy of the second LSTM model is improved through fine-tuning. Finally, not only the processing strategy of the target task is output, but also the first request and its processing strategy are stored in the knowledge base to enrich the system's knowledge base resources and provide quick responses and solutions for similar requests in the future.

[0141] Embodiment 3: This embodiment discloses a workflow execution system based on RPA technology, including:

[0142] The database building module is used to build a knowledge base, which stores historical data of business types and processing strategies.

[0143] Request receiving module, when the user needs the system to process a certain business, a request will be sent, and this request will be captured and recorded by the request receiving module and recorded as the first request.

[0144] The data analysis module includes a modeling unit, a feature extraction unit and a classification unit.

[0145] Modeling unit, builds deep neural network traffic model.

[0146] The feature extraction unit extracts the demand features in the first request based on the deep neural network model.

[0147] The classification unit uses the demand feature as an input of a decision tree, and determines the business type to which the first request belongs through reasoning and judgment of the decision tree.

[0148] Select the RPA robot module, and according to the business type determined by the classification unit, select the most suitable robot from the pre-set RPA robot list to perform subsequent business operations.

[0149] Reinforcement learning module, including reward function definition unit and adjustment unit.

[0150] Define a reward function unit. Define a reward function unit. According to the business execution goals and requirements, design a reward function that can quantify the processing effect of the RPA robot. The reward function will give corresponding rewards or penalties based on the performance of the RPA robot to guide the robot to continuously optimize its processing strategy.

[0151] The adjustment unit uses a deep reinforcement learning algorithm to enable the RPA robot to continuously adjust its processing strategy according to the feedback of the reward function during the actual business execution process. Through continuous trial and error and learning, the RPA robot can gradually find the optimal processing strategy so that the value of the reward function reaches the maximum value.

[0152] The business execution module uses the RPA robot after reinforcement learning to perform business operations based on the first request and business rules. During this process, the RPA robot will process the data and information in the request according to the established processing strategy and finally generate the operation results.

[0153] Output module, outputs the operation results.

[0154] This embodiment provides a workflow execution system based on RPA technology, which integrates a database building module to build and maintain a knowledge base that stores historical data on business types and processing strategies. The request receiving module captures the user's first request, and the data analysis module determines the requested business type through deep neural network modeling, feature extraction, and decision tree classification, and then selects the RPA robot. The reinforcement learning module defines the reward function and uses the deep reinforcement learning algorithm to guide the RPA robot to optimize the processing strategy to the best. The business execution module performs business operations based on the reinforcement learning results, and the output module finally displays the operation results to the user, realizing the automation and intelligence of the entire process from request reception to processing completion.

[0155] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A workflow execution method based on RPA technology, Including, building a knowledge base: building a knowledge base, and storing historical data of business types and processing strategies in the knowledge base, which is characterized by including: Receive request: receive the pending user request, recorded as the first request; Data analysis: including first modeling, feature extraction and classification; First modeling: building a deep neural network traffic model; Feature extraction: Extract demand features from the first request based on a deep neural network model; Classification: taking the demand feature as the input of the decision tree to obtain the business type to which the first request belongs; Select RPA robot: Select the RPA robot based on the business type; Reinforcement learning: including defining reward functions and adjustments; Define the reward function: Define the reward function based on the business execution goals; Adjustment: Using deep reinforcement learning algorithms, the RPA robot continuously adjusts its processing strategy based on the feedback of the reward function until the value of the reward function reaches the maximum value. Business execution: Based on the first request and business rules, the RPA robot after reinforcement learning is used to perform business operations and obtain operation results; Result judgment: judge whether the operation result meets expectations. If so, execute the output step; if not, execute the similarity calculation step; Calculate similarity: Calculate the similarity between the first request and all user requests in the knowledge base; Obtain source task: obtain the user request with the greatest similarity to the first request in the knowledge base, record it as the second request, and record the second request and the processing strategy of the second request as the source task; Get target task: take the first request as the target task; Second modeling: establishing the first LSTM model and the second LSTM model; Model training: The first LSTM model is trained using the source task and the processing strategy of the source task; Transfer learning: transfer the weights of the trained first LSTM model to the corresponding layers of the second LSTM model; Obtaining a processing strategy: Input the target task into the second LSTM model to obtain the processing strategy of the target task; Add request: Feedback the first request to the operation and maintenance personnel, and add the current first request to the knowledge base; Output: Output the operation results.

2. The workflow execution method based on RPA technology according to claim 1, characterized in that: After the step of defining the reward function and before the step of adjusting, it also includes: Task decomposition: decompose the business to be executed into multiple subtasks; Task planning: Develop a task execution plan based on subtasks; Task allocation: Each subtask is assigned to the corresponding RPA robot through the HRL algorithm.

3. The workflow execution method based on RPA technology according to claim 2 is characterized in that: After the step of performing task allocation and before the step of performing adjustment, it also includes: Initialization: Set up the initial processing strategy and rule set for each RPA robot.

4. The workflow execution method based on RPA technology according to claim 1, characterized in that: In the step of executing the adding request, the first request and the processing policy obtained in the step of obtaining the processing policy are stored in the knowledge base together.

5. The workflow execution method based on RPA technology according to claim 1, characterized in that: After performing the transfer learning step and before performing the step of acquiring the processing strategy, the step further includes: Add labels: Add labels to some target tasks and record the labeled target tasks as training data; Fine-tuning: Use the training data to train the second LSTM model. During the training process, monitor the changing trend of the output accuracy of the second LSTM model; Accuracy judgment: judge whether the change trend of the output accuracy of the second LSTM model meets expectations. If so, execute the step of obtaining the processing strategy; if not, execute the step of adjusting the learning rate; Adjust the learning rate: Reduce the learning rate of the second LSTM model. The calculation model is as follows: ； in, is the current learning rate of the second LSTM model, Adjusted learning rate for the second LSTM model.

6. The workflow execution method based on RPA technology according to claim 5 is characterized in that: After the labeling step and before the fine-tuning step, the following steps are also included: Set learning rate: Set the initial learning rate of the second LSTM model.

7. The workflow execution method based on RPA technology according to claim 6, characterized in that: After executing the step of adjusting the learning rate and before executing the step of obtaining the processing strategy, it also includes: Set threshold: Set the minimum threshold of learning rate; Learning rate judgment: judge the learning rate Is it lower than the minimum learning rate threshold? If so, execute the step of obtaining the processing strategy; if not, execute the step of fine-tuning.

8. A workflow execution system based on RPA technology, comprising: a database building module for building a knowledge base, wherein the knowledge base stores historical data of business types and processing strategies; characterized in that: The system further comprises: A request receiving module, used for receiving a user request to be processed, recorded as a first request; Data analysis module, including modeling unit, feature extraction unit and classification unit; Modeling unit, building a deep neural network traffic model; A feature extraction unit, which extracts demand features in the first request based on a deep neural network model; A classification unit, taking the demand feature as an input of a decision tree, and obtaining a business type to which the first request belongs; Select the RPA robot module and select the RPA robot according to the business type; Reinforcement learning module, including reward function definition unit and adjustment unit; Define the reward function unit and define the reward function according to the business execution goals; The adjustment unit uses a deep reinforcement learning algorithm to enable the RPA robot to continuously adjust its processing strategy based on the feedback of the reward function until the value of the reward function reaches the maximum value; The business execution module uses the RPA robot after reinforcement learning to execute business operations and obtain operation results according to the first request and business rules; The result judgment module is used to judge whether the operation result meets expectations; A similarity calculation module is used to calculate the similarity between the first request and all user requests in the knowledge base; A source task acquisition module is provided to acquire a user request having the greatest similarity to the first request in the knowledge base, record the request as the second request, and record the second request and the processing strategy of the second request as the source task; Get the target task module and take the first request as the target task; The second modeling module establishes the first LSTM model and the second LSTM model; The model training module trains the first LSTM model using the source task and the processing strategy of the source task; The transfer learning module transfers the trained weights of the first LSTM model to the corresponding layer of the second LSTM model; Obtaining a processing strategy module, inputting the target task into the second LSTM model, and obtaining a processing strategy for the target task; Add a request module, feed back the first request to the operation and maintenance personnel, and add the current first request to the knowledge base; Output module, outputs the operation results.

Images