RPA code generation method, device, system and storage medium

Abstract

This application relates to an RPA code generation method, apparatus, system, and storage medium, and pertains to the field of RPA technology. It aims to at least address the problems of inflexibility, low accuracy, and difficulty in handling complex and ever-changing business scenarios in related technologies, particularly in achieving precise operations within complex user interfaces. The method includes: determining a target model from multiple preset models based on the target business type in business requirement information; different preset models are models trained based on UI metadata corresponding to different business types, used for generating RPA code; extracting target UI metadata from a metadata database according to the business requirement information; inputting the target UI metadata into the target model to obtain initial RPA code; and optimizing the initial RPA code based on a reinforcement learning model to obtain the target RPA code.

Description

Technical Field

[0001] This application relates to the field of RPA technology, and in particular to RPA code generation methods, apparatus, systems and storage media. Background Technology

[0002] Robotic Process Automation (RPA) is a technology that uses software robots or "digital labor" to automate repetitive, routine, rule-based business processes. RPA code is the program code written to implement these automated processes. This code can simulate human operations on a computer, such as mouse clicks, keyboard input, and data reading and writing, thereby helping companies improve work efficiency, reduce labor costs, and minimize human error. In related code generation methods, to ensure that RPA code is adapted to business needs, historical user operation data is collected and combined with business requirements to generate corresponding RPA code using a large language model. This code generation method mainly relies on analyzing user operation data to extract operational logic and patterns, and then using a pre-trained large language model to generate corresponding RPA code. However, because user operation data often depends on specific environments and interface layouts, the generated RPA code lacks flexibility and struggles to cope with complex and ever-changing business scenarios, making it difficult to achieve precise operations in complex user interfaces. Summary of the Invention

[0003] In view of this, the present invention provides an RPA code generation method, apparatus, system, and storage medium to at least solve the problems of inflexibility, low accuracy, and difficulty in coping with complex and ever-changing business scenarios in RPA code generated in related technologies, i.e., difficulty in achieving precise operations in complex user interfaces. The technical solution of the present invention is as follows.

[0004] According to a first aspect of the present invention, an RPA code generation method is provided, the method comprising: determining a target model from multiple preset models based on a target business type in business requirement information; different preset models being models trained based on UI metadata corresponding to different business types and used to generate RPA code; extracting target UI metadata from a metadata database according to the business requirement information; inputting the target UI metadata into the target model to obtain initial RPA code; and optimizing the initial RPA code based on a reinforcement learning model to obtain target RPA code.

[0005] According to a second aspect of the present invention, an RPA code generation apparatus is provided. The apparatus includes: a determining unit configured to determine a target model from a plurality of preset models based on a target business type in business requirement information; wherein the different preset models are models trained based on UI metadata corresponding to different business types and are used to generate RPA code; an extraction unit configured to extract target UI metadata from a metadata database according to the business requirement information; a generation unit configured to input the target UI metadata into the target model to obtain initial RPA code; and an optimization unit configured to optimize the initial RPA code based on a reinforcement learning model to obtain target RPA code.

[0006] According to a third aspect of the present invention, an RPA code generation system is provided, which is configured to perform an RPA code generation method as described in the first aspect and any possible implementation thereof.

[0007] According to a fourth aspect of the present invention, a code generation apparatus is provided, comprising: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the RPA code generation method as described in the first aspect and any possible implementation thereof.

[0008] According to a fifth aspect of the present invention, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of a code generation device, the code generation device is able to perform an RPA code generation method as described in the first aspect and any possible implementation thereof.

[0009] According to a sixth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions, which, when executed on a service terminal, cause the service terminal to execute the RPA code generation method of the first aspect and any possible implementation thereof.

[0010] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects: To avoid interference from different business scenarios and improve the accuracy of code generation by the code generation model, the initial code is generated using a target model corresponding to the target business type, thereby improving the accuracy of the initial code. Furthermore, given that UI metadata can accurately describe the essential attributes of interface elements, the initial RPA code generated based on UI metadata in this application can clearly represent the correspondence between code and interface elements. This allows for flexible adjustments to interface elements based on different business rules and requirements to generate initial RPA code that matches the business scenario. Simultaneously, to improve the environmental adaptability and task expansion flexibility of the generated RPA code, a reinforcement learning model is used to further optimize the initial RPA code. This enables the reinforcement learning-optimized RPA code to automatically adjust its execution strategy to adapt to new environments and rules, thereby giving the target RPA code better flexibility and scalability, making it easier to add new task steps or functions.

[0011] In the above code generation method, the RPA code generated and optimized according to UI metadata and reinforcement learning model can cope with complex and ever-changing business scenarios, so as to automatically execute precise operations in complex user interfaces.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0014] Figure 1 This is a flowchart illustrating an RPA code generation method according to an exemplary embodiment; Figure 2 This is a block diagram illustrating an RPA code generation apparatus according to an exemplary embodiment; Figure 3 This is a framework diagram of an RPA code generation system according to an exemplary embodiment; Figure 4 This is a schematic diagram of an RPA code generation device according to an exemplary embodiment. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0016] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0017] Before providing a detailed description of the RPA code generation method provided in the embodiments of this application, let's first briefly introduce the application scenarios involved in the embodiments of this application.

[0018] Robotic Process Automation (RPA) is a technology that uses software robots or "digital labor" to automate repetitive, routine, rule-based business processes. RPA code is the program code written to implement these automated processes. This code can simulate human operations on a computer, such as mouse clicks, keyboard input, and data reading and writing, thereby helping companies improve work efficiency, reduce labor costs, and minimize human error. In related code generation methods, to ensure that RPA code is adapted to business needs, historical user operation data is collected and combined with business requirements to generate corresponding RPA code using a large language model. This code generation method mainly relies on analyzing user operation data to extract operational logic and patterns, and then using a pre-trained large language model to generate corresponding RPA code. However, because user operation data often depends on specific environments and interface layouts, the generated RPA code lacks flexibility and struggles to cope with complex and ever-changing business scenarios, making it difficult to achieve precise operations in complex user interfaces.

[0019] Research has revealed that in relevant RPA development technologies, the analysis of business requirements and the writing of process code primarily rely on manual processes. Typically, business requirements are presented in natural language, and developers need to manually translate these requirements into specific operational steps and then write the corresponding RPA process code—a time-consuming and labor-intensive method. Automatic RPA code generation technology has developed rapidly in recent years, beginning to explore the use of large language models to automate certain processes.

[0020] In some implementations, template-driven RPA code generation is employed.

[0021] Template-driven RPA code generation technology relies on predefined operation templates. These templates are typically written based on common business processes and operational steps, enabling the rapid generation of standardized RPA code. For example, consider the following RPA process automation generation method based on a large language model and custom requirement templates. Users input business requirements in natural language, and the system generates corresponding code based on predefined templates. This method generates RPA code templates that meet user needs through specialized training of a large model. Users can input business requirements in natural language, and the system generates corresponding code based on predefined templates.

[0022] In some implementations, data-driven RPA code generation is employed.

[0023] Data-driven RPA code generation technology analyzes historical user operation data, extracts operation logic and patterns, and then uses a large language model to generate corresponding code. For example, the following is a data-driven RPA code generation method. This method collects historical user operation data, combines it with business requirements, and uses a large language model to generate corresponding RPA code. This technology mainly relies on analyzing user operation data to extract operation logic and patterns, and then using a pre-trained large language model to generate the corresponding code.

[0024] The above methods have three main problems: 1) Lack of adaptability to complex business needs: They typically rely on predefined templates or historical data, making it difficult to handle complex and ever-changing business scenarios. 2) Inability to achieve adaptive optimization: These RPA code generation methods lack adaptive optimization mechanisms. Once deployed, the generated code cannot be dynamically adjusted and optimized based on actual execution results. This statically generated code has limitations in execution efficiency and accuracy, especially when the business environment changes, making it difficult to respond quickly. 3) Insufficient precision of UI operations: In these RPA code generation methods, UI operations are often based on templates or simple operation patterns, making it difficult to achieve precise operations in complex user interfaces. This problem is particularly pronounced when dealing with complex interfaces, potentially leading to operational errors or mismatches in the generated code during execution.

[0025] Furthermore, the specific problems arising from this code generation method are as follows: First, poor stability: User operation data usually depends on a specific environment and interface layout. If the interface undergoes minor changes, such as button repositioning or window resizing, the code generated based on the user operation data may become invalid. For example, after a software update, the interface layout changes, and the previously recorded mouse click positions may no longer correspond to the correct buttons. Second, lack of flexibility: The code is mainly a mechanical copy of user operations, making it difficult to cope with complex and ever-changing business scenarios. When business rules change, it may be necessary to re-record user operations, making it difficult to adapt to new requirements by simply modifying the code. Third, inclusion of invalid operations: Users may make some mistakes or unnecessary steps during the operation process, and these operations will also be recorded in the code, increasing the complexity of the code and execution time, and potentially affecting the efficiency of automated processes.

[0026] RPA code consists of program code used to automate tasks, and different RPA tools use different code syntax and structures.

[0027] Therefore, to address the aforementioned issues, this application proposes an RPA code generation method. To avoid interference from different business scenarios and improve the accuracy of the code generation model, initial code is first generated using a target model corresponding to the target business type, thereby improving the accuracy of the initial code. Furthermore, given that UI metadata can accurately describe the essential attributes of interface elements, the initial RPA code generated based on UI metadata in this application clearly represents the correspondence between code and interface elements. This allows for flexible adjustments to interface elements based on different business rules and requirements, generating initial RPA code that matches the business scenario. Simultaneously, to improve the environmental adaptability and task expansion flexibility of the generated RPA code, a reinforcement learning model is used to further optimize the initial RPA code. This enables the reinforcement learning-optimized RPA code to automatically adjust its execution strategy to adapt to new environments and rules, resulting in better flexibility and scalability of the target RPA code, making it easier to add new task steps or functions.

[0028] In the above code generation method, the RPA code generated and optimized according to UI metadata and reinforcement learning model can cope with complex and ever-changing business scenarios, so as to automatically execute precise operations in complex user interfaces.

[0029] To facilitate understanding, the RPA code generation method provided in this application will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart illustrating an RPA code generation method according to an exemplary embodiment, such as... Figure 1As shown, the RPA code generation method includes the following steps.

[0031] S11. Based on the target business type in the business requirement information, determine the target model from multiple preset models.

[0032] The preset model can be a large language model.

[0033] The target business type can be a financial business type or a medical business type, etc. The financial business type corresponds to a preset model trained using UI metadata for the financial business. The medical business type corresponds to a preset model trained using UI metadata for the medical business.

[0034] Different preset models are trained based on UI metadata corresponding to different business types and are used to generate RPA code.

[0035] Different preset models are configured for different business scenarios. These preset models are code generation models. Different preset models are set for different business types within different business scenarios.

[0036] The RPA codebase builds an initial vocabulary, and then expands it by adding domain-specific terms and new vocabulary. It determines whether the business requirement information includes preset keywords that match the target business type, and then uses these matching preset keywords to infer the target business type in reverse.

[0037] S12: Extract the target UI metadata from the metadata database according to the business requirements information.

[0038] As one implementation method, the preset model corresponds one-to-one with the business type, and the target UI metadata includes target interface elements and target interface attributes. Step S11 above extracts the target UI metadata from the metadata database according to the business requirement information, including: extracting business process information from the business requirement information; the business process information includes business steps and business operations; mapping the business process information to fields in the metadata database according to a preset mapping relationship to construct target query conditions; and obtaining the target UI metadata from the metadata database based on the target query conditions.

[0039] In some embodiments, metadata of UI elements is collected from various software systems, web pages, etc., including element type (such as buttons, text boxes, dropdown lists, etc.), ID, name, location, attributes, etc. Automation tools (such as browser developer tools, UI automation frameworks) can be used to extract this information. RPA code corresponding to UI operations is collected; this code can come from existing RPA projects, open-source code repositories, or scripts in actual business processes. Ensure the code covers multiple RPA tools and programming languages, such as Python + Selenium, UiPath, Automation Anywhere, etc.

[0040] In some implementations, the business requirements clearly define the tasks that need to be automated, while the UI metadata provides specific objects and operational guidelines for achieving these tasks. For example, if the business requirement is to automatically complete and submit financial statements, the RPA robot will perform the corresponding automated operations based on the position and attributes of elements such as report input boxes and submit buttons defined in the UI metadata.

[0041] The specific advantages of UI metadata will be explained in detail from the following three aspects.

[0042] First, UI metadata describes the essential attributes of interface elements, such as position, size, and text. Even if the interface layout changes, as long as the key attributes of the elements remain the same, the code can still accurately locate and manipulate these elements. For example, when the font size and color of the software interface change, but the position and text content of the buttons remain the same, the code based on UI metadata can still run normally.

[0043] Secondly, the correspondence between code and UI elements is clear. When UI elements change, only the corresponding UI metadata needs to be updated, without requiring large-scale modifications to the entire code logic. This makes code maintenance and upgrades more convenient and reduces maintenance costs.

[0044] Third, the operation of interface elements can be flexibly adjusted according to different business rules and needs. For example, when dealing with complex form filling tasks, the corresponding input code can be dynamically generated according to different form field requirements, better adapting to diverse business scenarios.

[0045] S13, input the target UI metadata into the target model to obtain the initial RPA code.

[0046] S14, based on a reinforcement learning model, optimizes the initial RPA code to obtain the target RPA code.

[0047] Through the above implementation methods, to avoid interference from different business scenarios and improve the accuracy of code generation by the code generation model, initial code is first generated using a target model corresponding to the target business type to improve the accuracy of the initial code. Furthermore, given that UI metadata can accurately describe the essential attributes of interface elements, the initial RPA code generated based on UI metadata in this application can clearly represent the correspondence between code and interface elements. This allows for flexible adjustments to interface elements based on different business rules and requirements to generate initial RPA code that matches the business scenario. Simultaneously, to improve the environmental adaptability and task expansion flexibility of the generated RPA code, a reinforcement learning model is used to further optimize the initial RPA code. This enables the reinforcement learning-optimized RPA code to automatically adjust its execution strategy to adapt to new environments and rules, thereby giving the target RPA code better flexibility and scalability, making it easier to add new task steps or functions.

[0048] In the above code generation method, the RPA code generated and optimized according to UI metadata and reinforcement learning model can cope with complex and ever-changing business scenarios, so as to automatically execute precise operations in complex user interfaces.

[0049] As one implementation method, in order to ensure that both the RPA code and the operation information and UI metadata are compatible, the following steps are taken to further determine the initial RPA code in S13 above.

[0050] First, determine the process operation description information based on business requirements information.

[0051] Process operation description information is also known as user operation information.

[0052] Secondly, the process operation description information and target UI metadata are input into the target model to call the target model to generate initial RPA code based on the process operation description information, and the initial RPA code is modified based on the target UI metadata to obtain the modified initial RPA code, so as to implement the various implementation methods of this application using the modified initial RPA code.

[0053] Specifically, the initial RPA code is generated based on user operation information, which can quickly and accurately reproduce the user's operation behavior on the interface. At the same time, the generated initial RPA code is modified based on UI metadata to make the initial code compatible with the interface elements.

[0054] For example, when operating a complex financial trading system, users may have a unique set of operating sequences and rhythms. Code generated based on user operation data can fully preserve these details, ensuring consistency between automated processes and manual operations.

[0055] As one implementation method, the specific process of inputting the process operation description information and target UI metadata into the target model is as follows.

[0056] First, construct the initial model.

[0057] The first initial model includes process operation description information, UI metadata, and the first preset coding relationship between RPA code. The first preset coding relationship includes the first preset model parameters.

[0058] Secondly, the historical process operation description information and the first historical metadata are input into the initial model to obtain the first training output code.

[0059] Third, determine that the difference between the first historical RPA code and the first training output code corresponding to the historical process operation description information and the first historical metadata is greater than or equal to the first preset difference, adjust the first preset model parameters, and input the historical process operation description information and the first historical metadata into the first initial model after adjusting the first preset model parameters to obtain the adjusted first training output code.

[0060] Fourth, determine that the difference between the first historical RPA code corresponding to the historical process operation description information and the first historical metadata and the adjusted first training output code is less than the first preset difference, and determine the first initial model corresponding to the adjusted first preset model parameters as the target model.

[0061] In one implementation, the reinforcement learning model is generated using the Q-learning algorithm. In this embodiment, the reinforcement learning model is trained in the following manner.

[0062] First, the code generation context of historical RPA code is used as the sample state, and the historical operations corresponding to each historical code fragment in the historical RPA code are used as the sample actions.

[0063] Specifically, define the State. The State represents the context of code generation, including: determining the characteristics of the business requirements (text embedding vectors); determining the structure of the generated code (such as API call methods and UI operation types); and determining the history of code execution (success / failure).

[0064] Define the action. Action 'a' represents an optional RPA code snippet or operation step, such as: different UI element selection methods (XPath, CSS selectors); different API call methods (Selenium API, PyAutoGUI); whether to add exception handling mechanism (try-except block).

[0065] Secondly, based on the code execution metrics generated during the execution of each historical code, the evaluation results of the historical code generation paths of each historical RPA code under each sample action set for each sample state are determined.

[0066] Third, each sample state, each sample action set, each corresponding historical code generation path, and each corresponding evaluation result are used as the experience pool of the reinforcement learning model. Furthermore, a preset reward mechanism for the reinforcement learning model is constructed based on each evaluation result. The preset reward mechanism includes positive rewards and negative rewards.

[0067] Positive reward indicates the unit reward generated under a positive trend in code execution metrics for a sample state and a sample action set. Negative reward indicates the unit reward generated under a negative trend in historical code execution metrics for a sample state and a sample action set.

[0068] Furthermore, the specific process of optimizing the initial RPA code using the above reinforcement learning model is as follows.

[0069] First, determine the initial code generation path, initial generation environment, and initial operations for the initial RPA code.

[0070] Secondly, the experience pool in the reinforcement learning model is used to optimize the initial operation and initial code generation path in the initial generation environment one by one, so as to obtain the optimized code generation path and code operation after each optimization; and the initial evaluation result is used as the reward reference, and a preset reward mechanism is used to reward the optimized code generation path and code operation in the initial generation environment, so as to obtain the evaluation results of the optimized code generation path and code operation after each optimization.

[0071] The initial evaluation result is the result of the experience pool's evaluation of the initial operation and initial code generation path in the initial generation environment, or the preset initial result of the initial operation and initial code generation path in the initial generation environment.

[0072] Third, determine the target code generation path and target operation corresponding to the optimal evaluation result among the various evaluation results and the initial evaluation results.

[0073] Fourth, generate target code based on the target code generation path and target operation.

[0074] Through the above implementation method, the target code, target code generation path, target operation generation, and target UI metadata are input into the target model, and the target model is optimized so that the optimized target model can be used to perform code conversion on the subsequently input UI metadata.

[0075] As another implementation, the initial RPA code in S13 is further determined using the following steps.

[0076] First, construct a second initial model.

[0077] The second initial model includes a second preset encoding relationship between UI metadata and RPA code; the second preset encoding relationship includes second preset model parameters.

[0078] Secondly, the second historical metadata is input into the second initial model to obtain the second training output code.

[0079] Third, determine that the difference between the second historical RPA code corresponding to the second historical metadata and the second training output code is greater than or equal to the second preset difference, adjust the second preset model parameters, and input the second historical metadata into the second initial model with the adjusted second preset model parameters to obtain the adjusted second training output code.

[0080] Fourth, if the difference between the second historical RPA code corresponding to the second historical metadata and the adjusted second training output code is less than the second preset difference, then the second initial model corresponding to the adjusted second preset model parameters is determined as the target model.

[0081] Historical metadata is input into the initial model to obtain the training output code. During the training process using historical metadata, the model parameters of the initial model are adjusted based on the loss condition that the difference between the historical RPA code corresponding to the historical metadata and the training output code is less than a preset difference. The historical metadata is then input into the adjusted initial model until the loss condition is met, at which point the target model is obtained.

[0082] As a specific implementation method, an RPA code generation system is used to implement the above-mentioned code generation method based on a large language model and Q-learning reinforcement learning algorithm to automatically generate RPA code. This method achieves efficient and accurate RPA code generation by parsing business requirements, generating operation step descriptions, and combining real-time and offline reinforcement learning strategies for optimization.

[0083] like Figure 2 As shown, the RPA code generation system includes core components such as a requirements analysis module 21, an RPA code generation module 22, a UI metadata model library 23, an execution feedback module 24, an adaptive optimization module 25, and a reinforcement learning module 26. Through the collaborative work of these modules, an automated closed loop from requirements to execution is achieved.

[0084] Step 1: Requirements Analysis. Business requirements are input into the requirements analysis module 21. First, based on an open-source large language model (such as the LLama series models), the model is fine-tuned using domain-specific data to more accurately understand and process the business requirements of that domain. The fine-tuned large language model then parses the requirements, generating operation step descriptions. These operation step descriptions serve as input for RPA code generation.

[0085] The analysis of the above business requirements includes the following steps.

[0086] 1) Input business requirements: Users input business requirements documents or natural language descriptions into the system. The system then passes these requirements to the requirements analysis module 21.

[0087] 2) Requirements Analysis and Operation Step Generation: The requirements analysis module 21 uses a fine-tuned large language model to parse the input business requirements. The model generates corresponding operation step descriptions and outputs them to the RPA code generation module 22.

[0088] Step 2: RPA Code Generation. The RPA code generation module 22 receives the operation step description and generates preliminary RPA code using a large language model. This preliminary code is then adjusted using the UI metadata model library 23 to ensure the generated code matches the actual UI operations. The final generated RPA code is then passed to the execution feedback module 24.

[0089] The RPA code generation process described above includes the following steps.

[0090] 1) Preliminary RPA code generation: The RPA code generation module 22 receives the operation step description. Based on the operation step description, it calls the large language model to generate preliminary RPA code.

[0091] 2) Adjustments based on UI metadata model library 23: The system calls UI metadata model library 23, which stores and manages structured data models of interface elements and their attributes. Referring to UI elements and attributes, the generated initial RPA code is adjusted. The adjusted code ensures accurate mapping with UI operations, generating the final version of the RPA code.

[0092] Step 3: Code Execution and Feedback. The execution feedback module 24 executes the generated RPA code and monitors the execution process in real time. The system records execution logs, including successful and failed operations, and the generated feedback data will be used for strategy adjustments.

[0093] The code execution and feedback collection process includes the following steps.

[0094] 1) Executing RPA code: The execution feedback module 24 receives the final generated RPA code and executes it in the actual environment. The system monitors the code execution process in real time and records the execution log.

[0095] 2) Collect execution feedback: The system records success and failure information of code execution, including execution time, operation results, and error logs. This feedback data is stored and passed to the adaptive optimization module 25.

[0096] Step 4: Adaptive Optimization. The adaptive optimization module 25 updates the Q-value of the state-action pair based on real-time feedback data.

[0097] The aforementioned real-time strategy adjustment includes the following steps.

[0098] 1) Analyze Feedback Data: The adaptive optimization module 25 analyzes the feedback data and extracts useful features from it. These features may include: UI element attributes: such as the behavior of elements like buttons and input boxes during execution; and contextual information of the operation sequence: the order of execution steps and the correlation between input and output.

[0099] 2) Success / Failure Patterns in Historical Execution: The success or failure probability of certain combinations of operations. Based on the Q-learning algorithm, the Q-value of the state-action pair is updated. The state represents the system's execution status at a given moment, such as the current UI element configuration, the context of operation steps, and the sequence of partially executed operations. The action represents the specific operation the system can take in the current state, such as selecting a code generation strategy, adjusting the parameters of the generation model, or modifying specific operation steps.

[0100] First, the Q-value table is updated: the system adjusts the strategy for generating RPA code in real time based on the updated Q-value table. If a strategy can significantly improve the execution success rate of RPA code under certain conditions, the Q-value of that strategy will increase. Conversely, if a strategy leads to execution failure or exception, the Q-value of that strategy will decrease. Action paths with higher Q-values ​​are prioritized to avoid generating failed code sequences again. Q-value update formula (1) (Bellman equation): After each step of execution, the Q-value is adjusted according to the following update formula.

[0101] Formula (1).

[0102] in, Let Q be the Q value of action a in the current state s. The initial value can be zero or a random small number. For learning rate, Here, r is the discount factor, and r is the current reward. This represents the maximum Q-value for the next state. Using this formula (1), the system can continuously update the Q-value table and optimize the strategy during each generation process.

[0103] The specific steps for updating the Q-value table are as follows.

[0104] First, record the data of RPA code execution. Define the state, where state s represents the context of code generation, including: characteristics of business requirements (text embedding vectors); the structure of generated code (such as API call methods and UI operation types); and the history of code execution (success / failure).

[0105] Secondly, define the action. Action 'a' represents an optional RPA code snippet or operation step, such as: different UI element selection methods (XPath, CSS selectors), different API call methods (Selenium API, PyAutoGUI), and whether to add exception handling mechanisms (try-except block).

[0106] Secondly, execute RPA code and collect feedback.

[0107] 1) Execute the code. Run the generated RPA code and monitor the execution process.

[0108] 2) Calculate the reward. If the code executes successfully, a positive reward is given: r = +1 # Code execution successful; if the code fails to execute, a negative reward is given: r = -1 # Code execution failed.

[0109] Third, calculate the new Q value.

[0110] 1) Get the current Q value.

[0111] Query the Q-value table to get the Q-value of action a in the current state s: Q_current = Q_table.get((s,a), 0) # The default value is set to 0.

[0112] 2) Calculate the future optimal Q value.

[0113] To obtain the highest Q value of all possible actions a' for the next state s': max_Q_next = max([Q_table.get((s', a'), 0) for a' in actions]).

[0114] 3) Update the Q-value. Update the Q-value using the Q-learning formula: alpha = 0.1 # learning rate; gamma = 0.9 # discount factor.

[0115] .

[0116] Fourth, update the Q-value table.

[0117] 1) Store the newly calculated Q value into the Q value table. Q_table[(s, a)] = Q_new.

[0118] 2) Record the updated state. print(f"Updated Q({s}, {a}) = {Q_new}").

[0119] Step 5: Offline Reinforcement Learning. After collecting sufficient execution data, the system uses this data for offline reinforcement learning. The reinforcement learning module 26 further optimizes the overall strategy through experience replay and Q-learning algorithms. Based on the updated Q-value table, it prioritizes code generation paths with high Q-values ​​to avoid repeating operations that lead to failure. This enables the system to handle more complex and diverse scenarios.

[0120] Step 5: Offline Reinforcement Learning and Global Optimization 1) Offline Reinforcement Learning: After the system collects enough execution data, the reinforcement learning module 26 performs offline learning. Through an experience replay mechanism, the system uses the Q-learning algorithm to optimize the strategy in the offline environment. Based on the updated Q-value table, the reinforcement learning model adjusts the generation strategy of the RPA code generation model to improve the effectiveness of the overall generation strategy.

[0121] a. Select a better generation path: Based on the Q value, decide which generation path or strategy to use to maximize the code execution success rate. When the RPA code involves multiple steps, the system prioritizes the path with a higher historical success rate for generation.

[0122] For example: Operation steps description: Enter data in the webpage.

[0123] Historical data: XPath positioning has a high success rate → its Q value is higher. CSS selector positioning fails more often → its Q value is lower.

[0124] Adjustment strategy: When generating code, prioritize methods with high Q values.

[0125] For example, if Q(XPath) > Q(CSS Selector): use_xpath() else: use_css_selector().

[0126] b. Modify the generation logic: Automatically optimize RPA code and select more robust code generation logic, mainly including two types: (1) Exception handling: Add try-except capture mechanism to improve execution stability.

[0127] 2) Retry mechanism: If a step fails, try to execute it again.

[0128] For example: Initial code (prone to failure): driver.find_element_by_xpath(" / / button[text()='Submit']").click() Optimized code (enhanced robustness): try: button = driver.find_element_by_xpath(" / / button[text()='Submit']") button.click() except Exception as e: print("Retrying...") time.sleep(2) driver.find_element_by_xpath(" / / button[text()='Submit']").click().

[0129] 3) Global Strategy Update: After offline learning is complete, the system applies the optimized strategy to the next round of RPA code generation. The system then feeds back the further optimized generation strategy to the RPA code generation module 22, forming an optimization loop.

[0130] In the above implementation, Large Language Model (LLM) is integrated with Robotic Process Automation (RPA). Through a fine-tuned LLM combined with reinforcement learning algorithms, a technology is developed to automatically generate efficient RPA process code from business requirements described in natural language. This implementation not only automatically transforms business requirements but also introduces real-time strategy adjustment and offline reinforcement learning mechanisms to ensure that the generated RPA code has adaptive optimization capabilities, significantly improving the efficiency and accuracy of automated process development.

[0131] I. Improve RPA code generation efficiency. Automated generation: Domain-specific fine-tuning was performed on the large language model, enabling automatic generation from business requirements to operational steps, and finally to RPA code. This greatly simplifies the traditional RPA code development process, significantly reducing development time and manual input. Real-time strategy adjustment: The system can adjust the generation strategy in real time during code generation, avoiding the repetition of erroneous operations, thereby accelerating code iteration speed and delivery efficiency.

[0132] II. Enhancing Code Generation Accuracy. Precise UI Operation Mapping: Through a UI metadata model library, the system ensures that the generated RPA code highly matches actual UI operations, reducing code execution errors caused by UI changes. This improves the code's execution success rate and stability. Reinforcement Learning Optimization: Utilizing the Q-learning algorithm, the system can continuously optimize the generation strategy based on execution feedback, ensuring that the generated RPA code maintains high accuracy across various scenarios.

[0133] III. Enhancing System Adaptability and Robustness. Adaptive Optimization: The system adjusts its generation strategy in real time based on execution feedback and performs global optimization through offline reinforcement learning. This enables the system to adapt to diverse business needs and complex UI environments, improving its robustness. Continuous Learning Capability: This invention continuously accumulates and utilizes historical data for strategy optimization through offline reinforcement learning, allowing the system to quickly adapt to new requirements and maintain efficient code generation capabilities.

[0134] IV. Resource and Cost Savings. Reduced Manual Intervention: Since the system can automatically generate and optimize RPA code, manual code development and debugging are reduced, thus lowering labor costs. Improved Resource Utilization: Through real-time policy adjustment and offline reinforcement learning, the system can quickly converge to the optimal policy, reducing wasted computing resources and optimizing resource utilization efficiency.

[0135] V. Enhance System Scalability. Modular Design: The system structure of this invention adopts a modular design, facilitating expansion and maintenance. As business needs change, the system can adapt to new scenarios by adjusting the large language model and reinforcement learning strategies, maintaining the system's continuous evolution capability. Strong Compatibility: The system can interface with various UI environments through the UI metadata model library, possessing strong compatibility and suitable for different types of RPA application scenarios.

[0136] To achieve the above functions, the RPA code generation apparatus includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art will readily recognize that, based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0137] This disclosure also provides an embodiment such as Figure 3 The RPA code generation device shown includes: a determination unit 31, an extraction unit 32, a generation unit 33, and an optimization unit 34.

[0138] The determination unit 31 is configured to determine the target model from multiple preset models based on the target business type in the business requirement information; different preset models are models trained based on UI metadata corresponding to different business types and are used to generate RPA code; the extraction unit 32 is configured to extract the target UI metadata from the metadata database according to the business requirement information; the generation unit 33 is configured to input the target UI metadata into the target model to obtain the initial RPA code; the optimization unit 34 is configured to optimize the initial RPA code based on a reinforcement learning model to obtain the target RPA code.

[0139] In one implementation, the generation unit 33 is specifically configured to perform: determining process operation description information based on business requirement information; inputting the process operation description information and target UI metadata into the target model to call the target model to generate initial RPA code based on the process operation description information, and correcting the initial RPA code based on the target UI metadata.

[0140] In one implementation, the reinforcement learning model is generated using a Q-learning algorithm; the optimization unit 34 is further configured to perform: using the code generation context of historical RPA code as sample states, and using the historical operations corresponding to each historical code fragment in the historical RPA code as sample actions; determining the evaluation results of the historical code generation paths of each historical RPA code under each sample action set for each sample state according to the code execution metrics generated during the execution of each historical code; using each sample state, each sample action set, the corresponding historical code generation paths, and the corresponding evaluation results as the experience pool of the reinforcement learning model; and constructing a preset reward mechanism for the reinforcement learning model based on the evaluation results. The preset reward mechanism includes positive rewards and negative rewards. Positive rewards indicate the unit reward generated under a positive trend of code execution metrics under a sample state and a sample action set; negative rewards indicate the unit reward generated under a negative trend of historical code execution metrics under a sample state and a sample action set.

[0141] In one implementation, the optimization unit 34 is specifically configured to perform: determining the initial code generation path, initial generation environment, and initial operation of the initial RPA code; using the experience pool in the reinforcement learning model, successively optimizing the initial operation and initial code generation path in the initial generation environment to obtain the optimized code generation path and code operation for each iteration; using the initial evaluation result as a reward reference, using a preset reward mechanism to reward the optimized code generation path and code operation in the initial generation environment for each iteration to obtain various evaluation results for each optimized code generation path and code operation; the initial evaluation result is the result of the experience pool evaluating the initial operation and initial code generation path in the initial generation environment or the preset initial result for the initial operation and initial code generation path in the initial generation environment; determining the target code generation path and target operation corresponding to the optimal evaluation result among the various evaluation results and the initial evaluation result; and generating target code based on the target code generation path and target operation.

[0142] In one implementation, the generation unit 33 is specifically configured to perform: constructing a first initial model; the first initial model includes a first preset encoding relationship between process operation description information, UI metadata, and RPA code; the first preset encoding relationship includes first preset model parameters; inputting historical process operation description information and first historical metadata into the initial model to obtain a first training output code; determining that the difference between the first historical RPA code and the first training output code corresponding to the historical process operation description information and the first historical metadata is greater than or equal to a first preset difference, adjusting the first preset model parameters, and inputting the historical process operation description information and the first historical metadata into the first initial model after adjusting the first preset model parameters to obtain the adjusted first training output code; determining that the difference between the first historical RPA code and the adjusted first training output code corresponding to the historical process operation description information and the first historical metadata is less than a first preset difference, and determining the first initial model corresponding to the adjusted first preset model parameters as the target model.

[0143] In one implementation, the optimization unit 34 is specifically configured to perform: constructing a second initial model; the second initial model includes a second preset encoding relationship between UI metadata and RPA code; the second preset encoding relationship includes second preset model parameters; inputting second historical metadata into the second initial model to obtain second training output code; determining that the difference between the second historical RPA code corresponding to the second historical metadata and the second training output code is greater than or equal to a second preset difference, adjusting the second preset model parameters, and inputting the second historical metadata into the second initial model with adjusted second preset model parameters to obtain adjusted second training output code; determining that the difference between the second historical RPA code corresponding to the second historical metadata and the adjusted second training output code is less than a second preset difference, and determining the second initial model corresponding to the adjusted second preset model parameters as the target model.

[0144] In one implementation, a preset model corresponds one-to-one with a business type, and the target UI metadata includes target interface elements and target interface attributes; the extraction unit 32 is specifically configured to perform: extracting business process information from business requirement information; the business process information includes business steps and business operations; mapping the business process information to fields in the metadata database according to a preset mapping relationship to construct target query conditions; and obtaining target UI metadata from the metadata database based on the target query conditions.

[0145] Regarding the apparatus in the above embodiments, the specific manner in which each unit module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0146] Figure 4This is a schematic diagram of an RPA code generation device provided in this application. Figure 4 The RPA code generation device 50 may include at least one processor 501 and a memory 503 for storing processor-executable instructions. The processor 501 is configured to execute the instructions in the memory 503 to implement the RPA code generation method in the following embodiments.

[0147] This RPA code generation device is also called an RPA code generation device, such as a server.

[0148] In addition, the RPA code generation device 50 may also include a communication bus 502, at least one communication interface 504, an input device 506, and an output device 505.

[0149] The processor 501 may be a processor (central processing unit, CPU), a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application.

[0150] The communication bus 502 may include a path for transmitting information between the aforementioned components.

[0151] Communication interface 504 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0152] Input device 506 is used to receive input signals and output device 505 is used to output signals.

[0153] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0154] The memory 503 stores instructions for executing the scheme of this application, and the processor 501 controls the execution. The processor 501 executes the instructions stored in the memory 503 to implement the functions of the method of this application.

[0155] In a specific implementation, as one example, the processor 501 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 in the CPU.

[0156] In a specific implementation, as one example, the RPA code generation device 50 may include multiple processors, for example... Figure 4 Processors 501 and 507 are mentioned. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0157] The RPA code generation device, such as Figure 4 The diagram includes a processor 501 and a memory 503 for storing executable instructions of the processor 501; wherein the processor 501 is configured to execute executable instructions to implement the RPA code generation method as described in any of the possible embodiments above. And it can achieve the same technical effect, so to avoid repetition, it will not be described again here.

[0158] This application also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of an RPA code generation apparatus or an RPA code generation device, the blockchain digital bond data processing apparatus or the RPA code generation device can execute the RPA code generation method as described in any of the above possible implementations. And it can achieve the same technical effect; to avoid repetition, it will not be described again here.

[0159] This application also provides a computer program product, including a computer program or instructions, which are executed by a processor using the RPA code generation method as described in any of the possible implementations above. This achieves the same technical effects, and to avoid repetition, will not be repeated here.

[0160] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0161] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An RPA code generation method, characterized in that, The method includes: Based on the target business type in the business requirements information, a target model is determined from multiple preset models; the multiple preset models are models trained based on UI metadata corresponding to different business types and are used to generate RPA code. Based on the aforementioned business requirements, extract the target UI metadata from the metadata database; The target UI metadata is input into the target model to obtain the initial RPA code; Based on a reinforcement learning model, the initial code generation path and initial operations in the initial generation environment of the initial RPA code are optimized to obtain the target RPA code; the step of inputting the target UI metadata into the target model to obtain the initial RPA code includes: Based on the aforementioned business requirements information, determine the process operation description information; The process operation description information and the target UI metadata are input into the target model to call the target model to generate the initial RPA code based on the process operation description information, and to modify the initial RPA code based on the target UI metadata.

2. The RPA code generation method according to claim 1, characterized in that, The reinforcement learning model is generated using the Q-learning algorithm; before optimizing the initial code generation path and initial operations in the initial generation environment of the initial RPA code based on the reinforcement learning model to obtain the target RPA code, the method further includes: The code generation context of the historical RPA code is used as the sample state, and the historical operations corresponding to each historical code fragment in the historical RPA code are used as the sample actions. Based on the code execution metrics generated during the execution of each historical code, determine the evaluation results of the historical code generation path of each historical RPA code under each sample action set for each sample state; The sample states, sample action sets, corresponding historical code generation paths, and corresponding evaluation results are used as the experience pool of the reinforcement learning model. A preset reward mechanism for the reinforcement learning model is constructed based on the evaluation results. The preset reward mechanism includes positive rewards and negative rewards. The positive reward indicates the unit reward generated under a positive trend of code execution metrics under a sample state and a sample action set. The negative reward indicates the unit reward generated under a negative trend of historical code execution metrics under a sample state and a sample action set.

3. The RPA code generation method according to claim 2, characterized in that, The reinforcement learning model is used to optimize the initial code generation path and initial operations in the initial generation environment of the initial RPA code to obtain the target RPA code, including: Determine the initial code generation path, initial generation environment, and initial operations for the initial RPA code; Using the experience pool in the reinforcement learning model, the initial operation and the initial code generation path in the initial generation environment are successively optimized to obtain the optimized code generation path and code operation for each iteration. Using the initial evaluation result as a reward reference, a preset reward mechanism is used to reward the optimized code generation path and code operation in the initial generation environment for each iteration, resulting in various evaluation results for each optimized code generation path and code operation. The initial evaluation result is either the result of the experience pool evaluating the initial operation and the initial code generation path in the initial generation environment or a preset initial result for the initial operation and the initial code generation path in the initial generation environment. Determine the target code generation path and target operation corresponding to the optimal evaluation result among the various evaluation results and the initial evaluation results; The target RPA code is generated based on the target code generation path and the target operation.

4. The RPA code generation method according to claim 1, characterized in that, The step of inputting the process operation description information and the target UI metadata into the target model includes: Construct a first initial model; the first initial model includes process operation description information, UI metadata and a first preset coding relationship between RPA code; the first preset coding relationship includes a first preset model parameter; Input the historical process operation description information and the first historical metadata into the first initial model to obtain the first training output code; Determine that the difference between the first historical RPA code and the first training output code corresponding to the historical process operation description information and the first historical metadata is greater than or equal to the first preset difference, adjust the first preset model parameters, and input the historical process operation description information and the first historical metadata into the first initial model after adjusting the first preset model parameters to obtain the adjusted first training output code. The difference between the first historical RPA code corresponding to the historical process operation description information and the first historical metadata and the adjusted first training output code is determined to be less than the first preset difference, and the first initial model corresponding to the adjusted first preset model parameters is determined as the target model.

5. The RPA code generation method according to claim 1, characterized in that, The method further includes inputting the target UI metadata into the target model to obtain the initial RPA code. Construct a second initial model; the second initial model includes a second preset encoding relationship between UI metadata and RPA code; the second preset encoding relationship includes second preset model parameters; The second historical metadata is input into the second initial model to obtain the second training output code; Determine that the difference between the second historical RPA code corresponding to the second historical metadata and the second training output code is greater than or equal to the second preset difference, adjust the second preset model parameters, and input the second historical metadata into the second initial model that adjusts the second preset model parameters to obtain the adjusted second training output code; If the difference between the second historical RPA code corresponding to the second historical metadata and the adjusted second training output code is less than the second preset difference, the second initial model corresponding to the adjusted second preset model parameters is determined as the target model.

6. The RPA code generation method according to any one of claims 1 to 5, characterized in that, The preset model corresponds one-to-one with the business type, and the target UI metadata includes target interface elements and target interface attributes; The step of extracting target UI metadata from the metadata database according to the business requirement information includes: Business process information is extracted from the business requirement information; the business process information includes business steps and business operations. According to the preset mapping relationship, the business process information is mapped to the fields of the metadata database to construct the target query conditions; Based on the target query conditions, the target UI metadata is obtained from the metadata database.

7. An RPA code generation device, characterized in that, The device includes: The determining unit is configured to determine the target model from multiple preset models based on the target business type in the business requirement information; the multiple preset models are models trained based on UI metadata corresponding to different business types and are used to generate RPA code. The extraction unit is configured to extract target UI metadata from the metadata database according to the business requirement information; The generation unit is configured to input the target UI metadata into the target model to obtain initial RPA code; The optimization unit is configured to optimize the initial code generation path and initial operations in the initial generation environment of the initial RPA code based on a reinforcement learning model, so as to obtain the target RPA code. The generation unit is specifically configured to determine process operation description information based on the business requirement information; input the process operation description information and the target UI metadata into the target model to call the target model to generate the initial RPA code based on the process operation description information, and modify the initial RPA code based on the target UI metadata.

8. An RPA code generation system, characterized in that, It is configured to perform the RPA code generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions, characterized in that when the instructions in the computer-readable storage medium are executed by the processor of the RPA code generation device, the RPA code generation device is able to perform the RPA code generation method as described in any one of claims 1-6.

Images