Robot process automation method and system, computer equipment and storage medium

Abstract

A robot process automation method comprises the following steps: running a robot process automation main program with a barrier-free operation authority in an operation system; importing an automation script into the robot process automation main program; starting the target application program in the operating system; the robot process automation main program obtains user interface view information of the target application program and analyzes operable items; the robot process automation main program executes the automation script to perform automatic operation on operable items of the target application program; obtaining dynamic response data of the target application program after the target application program is automatically operated from the operating system, and comparing the dynamic response data with expected response data of the target application program to obtain response difference data; and keeping or updating the automation script based on the response difference data. The robot process automation method and system, the computer equipment and the storage medium are high in expansibility and easy to implement.

Description

Technical Field

[0001] The present invention relates to the field of robotic process automation, and in particular to a robotic process automation method, system, computer equipment and storage medium. Background Art

[0002] RPA (Robotic Process Automation) is a type of process automation software tool that uses automation technology to simulate human back-end tasks, such as extracting data, filling out forms, and moving files. Robotic process automation technology combines APIs and user interfaces to integrate and execute repetitive tasks between enterprises and productivity applications. By deploying scripts that simulate manual processes, robotic process automation tools can automatically perform various activities and transactions in various unrelated software systems. There are two main ways to implement existing robotic process automation technology.

[0003] The first is to execute the automated script after obtaining the underlying permissions in the operating system. For example, in Android devices, it is necessary to use the adbshell tool to simulate user behavior after obtaining root permissions. The specific implementation principle is: because the underlying Android system is a Linux system, Linux commands can be sent through adbshell, and when a real user touches the phone screen, the system actually writes a semaphore to / dev / input / event, and the Android system reads the output in a loop and distributes it downward, thereby completing the entire touch user behavior feedback. After obtaining root permissions, Android devices can send custom Linux touch events through adbshell and continuously send them to / dev / input / event, thereby naturally simulating the click behavior of the Android system. The main problems with this method are: first, you need to obtain the underlying permissions before you can use it, and the method of injecting events into the underlying system driver is inefficient. When injecting events, you also need to obtain the coordinates of the UI controls. It is more troublesome to adapt to devices with different resolutions, and it is not suitable for mass production.

[0004] The second method is to use JavaScript based on Node.js as the scripting language to develop automation scripts, and use the auxiliary service functions provided by the operating system to run the automation scripts. For example, for security devices, with the capabilities provided by the Android auxiliary service function, it is possible to directly specify controls for operation, such as UI control retrieval, simulated clicks, slides, and long presses on Android devices. The main disadvantage of this method is that the overall functional scalability is poor. If you want to add functional modules, you can only develop incrementally based on existing functions. For example, if you need to use image color retrieval for game-related automation programs, you need to put the entire image color retrieval function module into the project and recompile and package it. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the present invention provides a robotic process automation method, system, computer device and storage medium, which are highly scalable and easy to implement.

[0006] In order to achieve the above object, the specific scheme adopted by the present invention is:

[0007] A robotic process automation method comprises the following steps:

[0008] Run the Robotic Process Automation main program with unimpeded operation permissions in the operating system;

[0009] Import automation scripts into the main RPA program;

[0010] Launch the target application in the operating system;

[0011] The main program of the RPA obtains the user interface view information of the target application and parses out the actionable items;

[0012] The main RPA program executes the automation script to automate the actionable items of the target application;

[0013] Acquire dynamic response data of the target application after the automated operation from the operating system, and compare it with the expected response data of the target application to obtain response difference data;

[0014] The automation script is maintained or updated based on the response difference data.

[0015] Preferably, the main program of the robotic process automation is driven by Python language, a Python runtime environment and multiple Python standard libraries are integrated in the main program of the robotic process automation, and the automation script is coded in Python language;

[0016] When you import an automation script into the main RPA program, the automation script is imported as a Python standard library.

[0017] Preferably, before starting the target application in the operating system, the running status of the RPA main program is checked, and the target application is started when the RPA main program runs normally, otherwise the operation is terminated and a warning is issued through the operating system.

[0018] Preferably, after the robotic process automation main program parses the operational items, it extracts the automation scripts that can automate the operational items based on the operational items, and generates a script execution plan for determining the execution order of the automation scripts. When the robotic process automation main program automates the operational items of the target application, it executes the automation script based on the script execution plan.

[0019] Preferably, the script execution plan includes a plurality of script tags for marking automation scripts, script serial numbers for sorting the script tags, and operable items associated with the automation scripts.

[0020] Preferably, when comparing the dynamic response data and the expected response data, if the dynamic response data completely covers the expected response data, the automation script is maintained; if the expected response data completely covers the dynamic response data, the automation script is increased; if there is a difference between the expected response data and the dynamic response data, the automation script is updated.

[0021] Preferably, the specific method for updating the automation script includes adding, deleting and / or modifying.

[0022] A robotic process automation system, used to implement the above-mentioned robotic process automation method, the system comprising:

[0023] Script creation module, used to create or obtain automation scripts;

[0024] An application activation module, used for running the robot process automation main program in the operating system and importing the automation script into the robot process automation main program;

[0025] A data acquisition module, used to obtain the user interface view information of the target application and obtain dynamic response data;

[0026] The data processing module is used to parse the operable items from the user interface view information and compare the dynamic response data with the expected response data of the target application.

[0027] Computer equipment, including:

[0028] a memory storing a computer program;

[0029] The processor is used to read the computer program from the memory and execute it to implement the above-mentioned robot process automation method.

[0030] The storage medium is used to store a computer program, which implements the above-mentioned robot process automation method when executed.

[0031] The present invention has strong scalability. By building a Python runtime environment into the RPA program, it supports a large number of Python standard libraries. It can also create and import Python libraries as needed to implement more automated scripts with high development efficiency. The present invention has low invasiveness and can be implemented using the operating system's accessibility services without obtaining the underlying permissions of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 is a flow chart of the robotic process automation method of the present invention;

[0034] Figure 2 It is a software framework diagram when the present invention is applied to an Android system. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] like Figure 1 As shown, the present invention first provides a robot process automation method, including S1 to S7.

[0037] S1. Run the robot process automation main program with barrier-free operation authority in the operating system. In order to facilitate development and improve development efficiency, in the present invention, the robot process automation main program is driven by Python language, and a Python operating environment and multiple Python standard libraries are integrated in the robot process automation main program. The robot process automation main program driven by Python language can utilize a huge number of Python standard libraries in the prior art, which can greatly reduce the development difficulty and cost, and shorten the development cycle.

[0038] S2. Importing the automation script into the main program of the robot process automation. The automation script is coded in Python language to match the main program of the robot process automation. When the automation script is imported into the main program of the robot process automation, the automation script is imported in the form of Python standard library.

[0039] S3. Start the target application in the operating system. In order to ensure that the robot process automation operation can be smoothly implemented, before starting the target application in the operating system, check the running status of the robot process automation main program. When the robot process automation main program runs normally, start the target application, otherwise terminate the operation and issue a warning through the operating system. For different operating systems, the robot process automation main program will have different background running modes, and you can determine whether the robot process automation main program is running by checking the process of the operating system. It should also be noted that for different target applications, each target application needs to determine whether the robot process automation main program is running before starting. If a target application is closed after running for a period of time, it is also necessary to check whether the robot process automation main program is running when it is restarted next time, or when it needs to be restarted due to program errors. When the robot process automation main program is accidentally closed, you can also try to start it yourself, that is, the robot process automation main program can have self-starting functions and permissions, so as to ensure that the robot process automation main program can run stably enough in the operating system, thereby ensuring that the robot process automation can be smoothly implemented.

[0040] S4. The main program of the robotic process automation obtains the user interface view information of the target application and parses out the operable items. Considering that the user interface view information of any target application may be different, when the main program of the robotic process automation obtains the user interface view information, it mainly obtains the operable items by checking the objects in the user interface view information. Common objects include buttons, forms, input boxes, and carousels. The operation methods of the same type of objects in different target applications are mostly the same. Therefore, the operable items and the operation methods corresponding to the operable items can be extracted by parsing the objects.

[0041] Furthermore, after the robot process automation main program parses the operable items, it extracts the automation scripts that can perform automated operations on the operable items based on the operable items, and generates a script execution plan for determining the execution order of the automation scripts. When the robot process automation main program performs automated operations on the operable items of the target application, it executes the automation scripts based on the script execution plan. For the same target application, there may be multiple automation scripts that jointly control the target application, and the operable items that need to be controlled by different automation scripts may be the same or different. Therefore, after obtaining the operable items, the robot process automation main program can first determine the automation scripts that can perform automated operations on the operable items based on the operable items, and create a script execution plan. When the robot process automation main program starts to perform automated operations on the operable items in the target application based on the automation scripts, it can quickly operate based on the script execution plan without repeatedly querying the available automation scripts, thereby greatly improving the automation operation efficiency of the robot process automation main program.

[0042] In actual implementation, the script execution plan may include multiple script tags for marking automation scripts, script serial numbers for sorting script tags, and operable items associated with automation scripts. Among them, the script tag is used to make each automation script have a unique tag symbol, which is convenient for the main program of the robotic process automation to quickly read and apply the automation script based on the tag symbol; when multiple automation scripts are needed to operate the same target application, the script serial number is used to determine the execution order of multiple automation scripts, for example, after the automation script with the script serial number 01 is executed, the automation script with the script serial number 02 is started to be executed. The script serial number may also include the number of loops, which is used to determine the number of times an automation script is repeatedly executed in a loop; the operable items are used to determine the corresponding relationship between the automation script and the operable items in the target application, for example, the automation script can only operate a certain button when it is executed once.

[0043] S5. The robot process automation main program executes the automation script to automate the operable items of the target application.

[0044] S6. Obtain the dynamic response data of the target application after the automated operation from the operating system, and compare it with the expected response data of the target application to obtain the response difference data. The purpose of robotic process automation is to replace manual operations on the target application, but it is difficult to determine whether the target application responds correctly to the automated operation as quickly as manual operations. When there are deficiencies in the writing of the automated script, it is easy to fail to achieve the required operation. Therefore, it is necessary to obtain the dynamic response data of the target application to determine whether the automated script has performed the correct operation on the target application. The expected response data is the response expected to appear when the target application is automated. For example, when the automated script is used to click a button, the expected response data is to jump to a new page. On this basis, when the dynamic response data and the expected response data are compared, if the dynamic response data completely covers the expected response data, the automated script is maintained, if the expected response data completely covers the dynamic response data, the automated script is increased, and if there is a difference between the expected response data and the dynamic response data, the automated script is updated. Specifically, if the dynamic response data completely covers the expected response data, it means that the target program is fully operated, and the automation script remains unchanged; if the expected response data completely covers the dynamic response data, it means that some of the expected response data may not be generated smoothly, that is, the automation script fails to perform certain operations smoothly, and the automation script needs to be updated to achieve adjustment. It should also be noted that if the dynamic response data and the expected response data are completely consistent, the automation script remains unchanged.

[0045] S7: Maintain or update the automation script based on the response difference data. The specific method of updating the automation script includes adding, deleting and / or modifying.

[0046] In one embodiment of the present invention, the robot process automation method of the present invention is applied in an Android system, and the software architecture at this time is as follows Figure 2 shown.

[0047] The present invention further provides a robotic process automation system for implementing the above-mentioned robotic process automation method, wherein the system comprises a script creation module, an application activation module, a data acquisition module and a data processing module.

[0048] The script creation module is used to create or obtain automation scripts.

[0049] The application activation module is used to run the robot process automation main program in the operating system and import the automation script into the robot process automation main program.

[0050] The data acquisition module is used to obtain the user interface view information of the target application and obtain dynamic response data.

[0051] The data processing module is used to parse the operable items from the user interface view information and compare the dynamic response data with the expected response data of the target application.

[0052] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0053] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0054] The present invention also provides a computer device, including a memory and a processor.

[0055] A memory storing a computer program.

[0056] The processor is used to read the computer program from the memory and execute it to implement the above-mentioned robot process automation method.

[0057] The present invention finally provides a storage medium for storing a computer program, which implements the above-mentioned robot process automation method when executed.

[0058] Based on the storage medium, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0059] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0060] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robotic process automation method, characterized in that: The steps include: Run the Robotic Process Automation main program with unimpeded operation permissions in the operating system; Import automation scripts into the main RPA program; Launch the target application in the operating system; The main program of the RPA obtains the user interface view information of the target application and parses out the actionable items; The main RPA program executes the automation script to automate the actionable items of the target application; Acquire dynamic response data of the target application after the automated operation from the operating system, and compare it with the expected response data of the target application to obtain response difference data; The automation script is maintained or updated based on the response difference data.

2. A robotic process automation method as claimed in claim 1, characterized in that: The main program of the robotic process automation is driven by the Python language. The Python runtime environment and multiple Python standard libraries are integrated in the main program of the robotic process automation. The automation script is coded in the Python language. When you import an automation script into the main RPA program, the automation script is imported as a Python standard library.

3. A robotic process automation method as claimed in claim 1, characterized in that: Before starting the target application in the operating system, the running status of the robot process automation main program is checked, and the target application is started when the robot process automation main program runs normally, otherwise the operation is terminated and a warning is issued through the operating system.

4. A robotic process automation method as claimed in claim 1, characterized in that: After the Robotic Process Automation main program parses the operational items, it extracts the automation scripts that can automate the operational items based on the operational items, and generates a script execution plan for determining the execution order of the automation scripts. When the Robotic Process Automation main program automates the operational items of the target application, it executes the automation scripts based on the script execution plan.

5. A robotic process automation method as claimed in claim 4, characterized in that: The script execution plan includes a plurality of script tags for marking automation scripts, script sequence numbers for sorting the script tags, and actionable items associated with the automation scripts.

6. A robotic process automation method as claimed in claim 1, characterized in that: When comparing the dynamic response data and the expected response data, if the dynamic response data completely covers the expected response data, the automation script is maintained; if the expected response data completely covers the dynamic response data, the automation script is added; if there is a difference between the expected response data and the dynamic response data, the automation script is updated.

7. A robotic process automation method as claimed in claim 6, characterized in that: Specific methods for updating the automation script include adding, deleting and / or modifying.

8. A robotic process automation system, characterized in that: Used to implement a robotic process automation method as described in any one of claims 1 to 7, the system comprising: Script creation module, used to create or obtain automation scripts; An application activation module, used for running the robot process automation main program in the operating system and importing the automation script into the robot process automation main program; A data acquisition module, used to obtain the user interface view information of the target application and obtain dynamic response data; The data processing module is used to parse the operable items from the user interface view information and compare the dynamic response data with the expected response data of the target application.

9. Computer device, characterized in that include: a memory storing a computer program; A processor is used to read a computer program from a memory and execute it to implement a robotic process automation method as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that Used to store a computer program, which, when executed, implements a robotic process automation method as described in any one of claims 1 to 7.

Images