Method and system for generating and processing. Usa file to realize RPA operation

Abstract

The invention discloses a method and a system for RPA operation based on a. Usa file, and relates to the technical field of robot process automation (RPA), in particular to a method and a system for RPA operation based on a. Usa file. According to the scheme, the method comprises the following steps: packaging one or more process steps into a single. Usa file; the file loading and analyzing module loads and analyzes the. Usa file, the script compiling module compiles the read operation file to generate a corresponding operation script, the interpretation module executes the corresponding operation script according to the description file, and the standard function module can be called in the operation script to achieve interface operation. According to the scheme, the interface operation is scripted through the usa file, so that a front-end developer or a script developer can easily participate in interface RPA operation development, restrictive functions or standard functions do not need to be repeatedly developed, variable development requirements can be met, and application transmission and sharing convenience is improved.

Description

Technical Field

[0001] This invention relates to the field of Robotic Process Automation (RPA) technology, specifically to an RPA system and method based on Chrome DevTools Protocol (CDP) for automating web browser and desktop application window operations and data interaction, and particularly to an RPA operation method and system based on .usa (UI Script Automation) files. Background Technology

[0002] Technologies such as WebDriver, WebDriver BiDi, and Chrome DevTools Protocol (CDP) have varying degrees of limitations in simulating mouse clicks, trajectory simulation, input method input, mouse scrolling, system shortcuts, copy and paste, meaning they cannot simulate these actions. Existing RPA platforms generally use pre-built operation steps, and their functionality is usually fixed at the time of software release. If users need to add certain functions, they often rely on the software vendor to release a complete new version, and the configured application flow can only be shared, not transmitted, resulting in poor flexibility. Meanwhile, browser plugins also have strict isolated execution environments and access controls, making it impossible to simulate user operations.

[0003] Therefore, the industry needs a user interface solution that simplifies functionality, allows for flexible development, facilitates transmission, and supports cross-window and browser interfaces. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes an RPA method and system based on .usa (UI Script Automation) files. This solution encapsulates restrictive or standard functionalities into standard functional modules, and then generates an independently distributable .usa file as an application by developing or configuring the operation flow steps.

[0005] A first aspect of the present invention provides an RPA method and system based on .usa files, comprising: S1: Provide at least one process step, each process step being encapsulated as an independent file unit (e.g., folder or filename distinction), a process step containing a description file and one or more operation files; S2: Package and compress the at least one process step to generate a single .usa format file, wherein the .usa file is the package file of this solution; S3: When this solution is running, the .usa file is loaded, its internal structure is parsed, and the description file and operation file of each process step are read; S4: Compile the read operation file using a compilation module to generate the corresponding operation script; if it is already a script, you can skip this step. S5: Through an interpretation module, based on the description file, at the defined timing points and corresponding interfaces (desktop window or browser page), the operation script for the corresponding process steps is executed. S6: The operation script can call the standard functions or extended functions encapsulated in the standard function module through the API to perform interface operations.

[0006] Furthermore, the description file is a text file based on JSON, XML, or YAML format, used to define metadata such as the name of the process step, interface name, script path, runtime, matching rules, initial path or URL, and account identifier. The name of the process step, interface name, initial path or URL, and account identifier are optional, and the script path can be a relative path or an absolute path. The operation file includes both a pre-compiled logic document and a compiled executable script. The interpretation module can adapt to execute both forms, balancing development convenience and operational efficiency.

[0007] In a second aspect, the present invention provides a system for implementing the above method, comprising: a file packaging module, a file loading and parsing module, a script compilation module, an interpretation module, a standard function module, and an extended function module.

[0008] A third aspect of the present invention provides a computer device and a computer-readable storage medium. Beneficial effects

[0009] Flexible Development: Supports writing logic using scripting languages ​​(such as JavaScript) and supplemented by structured description files, enabling front-end developers or script developers to easily participate in the development of automated interface operations without repeatedly developing restrictive functions, while still meeting diverse development needs. This solution gives control to the user; whether developers, enterprises, or ordinary users, they can achieve more efficient, flexible, and personalized needs through programming.

[0010] Easy to share: The application process steps are encapsulated in a single .usa file, making it easy to transfer, use, distribute and upgrade. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the architecture of an RPA method and system based on .usa files.

[0012] Figure 2This is a flowchart of an embodiment of the present invention in the .NET environment using the WebView2 control.

[0013] Figure 3 This is a flowchart of an embodiment of the present invention in a node.js environment using the chrome-remote-interface component.

[0014] Figure 4 This is a schematic diagram of the internal structure of a .usa file.

[0015] Figure 5 This is a YAML example of a description file for a process step. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0017] Please see Figure 1 The system architecture of the present invention includes: .usa files: These are the physical units that are distributed.

[0018] File packaging module: Used to package and compress one or more of the process steps into a .usa file; File loading and parsing module: Used to load and parse the structure of .usa files when this solution starts or runs; Script compilation module: used to compile operation files into operation scripts; Interpretation module: used to interpret and run the operation steps defined in the description file and operation file, and complete the interface operation at specific time points; Permissions module: Used to configure restrictive function access control for users; Standard functional modules: responsible for implementing restricted functions such as mouse click, mouse trajectory simulation, input method input, system shortcut keys, paste, and file, as well as other standardized functions as required.

[0019] Extended functionality modules: These can be installed or uninstalled according to individual needs without modifying the main program code, enabling functional customization.

[0020] Please see Figure 2 An implementation using the WebView2 control in a .NET environment is detailed below: 1. Initialization and Binding The interpreter module first initializes the WebView2 control (a modern browser kernel component) and sets listeners for two key events: NavigationStarting and NavigationCompleted. Next, the interpreter module creates a concrete function object instance (e.g., a standard function module named UsaMouse) and "binds" or "injects" this object into the WebView2 using the WebView2.CoreWebView2.AddHostObjectToScript method.

[0021] Key function: After this step, JavaScript code in subsequently loaded web pages will be able to directly access and call this UsaMouse object.

[0022] 2. Navigation and Page Loading The interpreter module commands the WebView2 to navigate to a specific website (e.g., example.com). When the page begins loading, the `NavigationStartin` event is triggered, and the interpreter module executes some initialization scripts (possibly to set up the page environment or prepare the page). After the page is fully loaded, the `NavigationCompleted` event is triggered, and the interpreter module executes a completion script (possibly to notify the page that the application is ready or to perform initial data transfer). At this point, the webpage has full interactive capabilities.

[0023] 3. Web page scripts calling host functions JavaScript logic within a webpage (such as responding to a button click) can call methods of the previously bound host object, UsaMouse, such as click(x, y) (simulating click coordinates). This call originates from the webpage script and is received by the WebView2 control.

[0024] 4. Call forwarding and execution WebView2 forwards the received click(x, y) method call request to the interpretation module. The interpretation module, acting as the central unit, identifies the request and calls the corresponding click method in the actual functional module (i.e., the UsaMouse object).

[0025] 5. Results returned After the UsaMouse.click method completes execution, it returns the result (if any) to the interpretation module. The interpretation module then passes the result back through WebView2, and finally delivers it to the JavaScript of the webpage that initially made the call, thus completing a full two-way communication.

[0026] Please see Figure 3 The implementation of a Chrome DevTools Protocol environment in conjunction with the Chrome browser is detailed below: 1. Establishing a Connection and Setup: The process begins with the interpreter module, with the goal of connecting to and controlling a Chrome browser instance.

[0027] Initialize controls: Explains whether the module starts or connects to a Chrome instance (possibly a headless browser) that supports the DevTools Protocol.

[0028] Listening for critical events: The interpreter module begins listening for lifecycle events of the browser page via a protocol, specifically: Page.frameStartedLoading: Triggered when the page starts loading.

[0029] Page.loadEventFired: Triggered when the main page frame finishes loading. These two events are key timing points for subsequent script injection.

[0030] Establishing the binding: This is the most crucial step. The interpreter module uses the `Runtime.addBinding` command to create a binding named `UsaMouse.click` in the browser's JavaScript execution environment. This is equivalent to registering a "stub function" on the page's `window` object. When the webpage's JavaScript calls it, the browser notifies the outside world via CDP.

[0031] Listening for bound calls: The interpreter module also listens for the Runtime.bindingCalled event. When a script in the webpage calls window.UsaMouse.click(...), the browser triggers this event and passes the call parameters to the interpreter module.

[0032] Navigation and Page Initialization Once the connection is established, the target page is loaded and its capabilities are injected.

[0033] Initiate page navigation: The interpreter module sends the Target.createTarget command, instructing the browser to create a new tab and navigate to the target URL (http: / / www.example.com).

[0034] Page load begins: The browser triggers the Page.frameStartedLoading event. Upon receiving this event, the interpreter module immediately executes the initialization script on the page using the Runtime.evaluate command. This script is typically used to prepare the page environment.

[0035] Page loading complete: Once the main content of the page has been loaded, the browser triggers the Page.loadEventFired event. The interpreter module receives this event and executes the loading completion script again via Runtime.evaluate. At this point, the page DOM is ready, making it an ideal time to inject main logic or perform initial operations.

[0036] Web page script calling and function execution The page is ready to receive instructions from users (or automation scripts).

[0037] Script execution call: JavaScript within the page (possibly a previously injected "operation script") executes window.UsaMouse.click(x, y). This call is initiated actively by the page's internal logic.

[0038] Triggering the listener: Since UsaMouse.click is a binding that has been registered in advance via Runtime.addBinding, the browser will not attempt to execute a non-existent JS function. Instead, it will immediately trigger the Runtime.bindingCalled event and send the function name "UsaMouse.click" and the parameters x and y (as payload) to the interpreter module.

[0039] Function Invocation: The interpreter module received a call request from the webpage in the callback function of the bindingCalled event. Based on the call name, it deduced that the required function was "click".

[0040] Response result: The interpreter forwards this request to the actual "standard / extended function module" running in the external environment (such as a mouse control library). This function module performs the actual click operation (possibly simulating a click at the operating system level) and generates the result.

[0041] 4. Results returned Returning results to the webpage: After the interpretation module obtains the execution result of the functional module, if it needs to return the result to the waiting webpage JavaScript, it will execute a piece of JS code in the page's execution environment again through the Runtime.evaluate command and pass the result back.

[0042] Please see Figure 4A typical .usa file has the following internal structure: the root directory index.yaml contains application metadata such as version; step1 and step2 are process step files; index.yaml is the description file within the process step; and start.js and end.js are the operation files within the process step.

[0043] Please see Figure 5 This is a YAML example of a process description file. `name` is the process name, `title` is the interface name, `url` is the browser's initial navigation URL, `account_id` is the account identifier, `content_scripts` are the scripts to be loaded, `matches` is an array of matching logic, `js` is an array of file paths to be loaded, and `run_at` is the loading time.

Claims

1. A method and system for generating and processing .usa files to implement RPA operations, characterized in that, Includes the following steps: S1: Provide at least one process step, each process step being encapsulated as an independent file unit (e.g., folder or filename distinction), a process step containing a description file and one or more operation files; S2: Package and compress the at least one process step to generate a single .usa format file, wherein the .usa file is the package file of this solution; S3: When this solution is running, the .usa file is loaded, its internal structure is parsed, and the description file and operation file of each process step are read; S4: Compile the read operation file using a compilation module to generate the corresponding operation script; if it is already a script, you can skip this step. S5: Through an interpretation module, based on the description file, at the defined timing points and corresponding interfaces (desktop window or browser page), the operation script for the corresponding process steps is executed. S6: The operation script can call the standard function module or extended function module encapsulated by the API to perform interface operations.

2. The solution according to claim 1, characterized in that, In step S1, the description file is a text file based on JSON, XML or YAML format, used to define metadata such as the name of the process step, interface name, script path, runtime, matching rules, initial path or URL, and account identifier. The name of the process step, interface name, initial path or URL, and account identifier are optional.

3. The solution according to claim 1, characterized in that, In step S1, the operation file includes two forms: The first type is a pre-compiled logical document, written in a scripting language or a domain-specific language; The second type is a compiled executable file, which is generated by compiling the first type of logical document, or it can be developed and written directly. The interpretation module can adapt to and interpret the operation files in the two presentation formats.

4. The method according to claim 3, characterized in that, The scripting language includes TypeScript, JavaScript, C#, Java, Lua, or Python. The domain-specific language is a dedicated syntax defined for interface operations.

5. The method according to claim 1, characterized in that, In steps S5 and S6, when the interpretation module executes the operation file, the operation file can call limited and extended functions such as mouse click, mouse trajectory simulation, input method input, system shortcut keys, paste, and file in the corresponding process steps of this solution.

6. The method according to claim 1, characterized in that, The .usa file uses a ZIP, TAR, or custom compression / archive format.

7. This solution is used to implement the method as described in any one of claims 1-6, characterized in that, include: The file packaging module is used to package and compress one or more of the process steps into a .usa file; The file loading and parsing module is used to load and parse the structure of .usa files when this solution starts or runs. The script compilation module is used to compile operation files into operation scripts; The interpretation module is used to interpret and run the operation steps defined in the description file and operation file, and complete the interface operation at specific time points; The permissions module is used to configure user access control for restrictive functions; Standard functional modules pre-encapsulate both restricted and commonly used functions; Extended functional modules can be installed or uninstalled according to individual needs without modifying the main program code, enabling functional customization.

8. The system according to claim 7, characterized in that, It also includes a development tools module, which provides a graphical interface or command-line tool to assist users in creating and editing the description file and the pre-compilation logic document, and to encapsulate and package the process steps into a .usa file.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-6.

Images