Dynamic scaling scheduling method and system based on distributed RPA Worker

Abstract

The invention discloses a dynamic scaling scheduling method and system based on distributed RPA Worker, and relates to the technical field of robot process automation. According to the dynamic telescopic scheduling method and system implemented based on the distributed RPA Worker, step 1, group information is configured, the group information is added through a Worker group management page, and after the group information is added, a corresponding third-party RPA console can check the added group information; 2, worker information is added, an ECS cloud server needs to be purchased in advance during configuration, a server instance id is obtained, and configuration of the instance id, a worker group, a worker type and the like is completed; 3, building a capacity expansion and contraction core system, wherein the capacity expansion and contraction core system is built according to the technical solution described in the patent; and step 4, creating an RPA task, automatically establishing a set of management function of the RPA task, and recording basic information, execution information, log information and the like of the task.

Description

Technical Field

[0001] The present invention relates to the field of robotic process automation technology, and in particular to a dynamic scaling scheduling method and system based on distributed RPA Worker. Background Art

[0002] RPA (Robotic Process Automation) is a technology that uses software robots or "robot workers" to automate enterprise processes. These software robots can simulate human operation of computer applications to perform repetitive, regular and structured tasks, thereby improving efficiency, reducing errors and saving time. It is worth noting that each RPA robot must be deployed on a computer. Generally speaking, only one robot is deployed on a machine, so we call the computer where the robot is deployed a worker.

[0003] UiPath: As the world's leading RPA provider, UiPath provides a powerful RPA platform and tools with rich functions and excellent scalability. It supports a variety of applications, systems and technologies, while also providing powerful analysis and management capabilities.

[0004] Automation Anywhere: Automation Anywhere is another leading RPA vendor that offers ease of use and flexibility in its platform. The platform supports multiple technology environments and provides specialized tools and features to enhance security and reliability.

[0005] ENCOO: ENCOO is an innovative leader in the RPA field, committed to building an industry-leading hyper-automation platform to help enterprises improve intelligent productivity and accelerate digital transformation. With its self-developed ENCOO RPA as its core, ENCOO is committed to providing intelligent RPA robot products and solutions to customers in various industries, continuously creating value for customers, and helping enterprises promote digital transformation. Specific products include editors, robots, and consoles. With ENCOO, you can achieve full process control from process editing to deployment and permission.

[0006] WorkFusion: WorkFusion provides a complete set of enterprise-level automation solutions, including RPA, intelligent OCR, and machine learning. Its platform has powerful automation capabilities and supports multiple applications and system integration.

[0007] Currently, RPA suppliers are used to build enterprise-specific worker clusters, and the basic architecture is shown in the figure above. The supplier's self-developed Worker cluster console system is used to manage multiple RPA Workers established within the enterprise. Users can submit RPA tasks to be executed to the console through various client forms and the internal network. The console will check, distribute, collect execution results, and perform execution record statistics on the received tasks.

[0008] As of 2023, the types of support provided by common RPA vendors on the market for robots can be roughly divided into two categories: (the names vary slightly, but the essential functions are the same)

[0009] 1. Attended operation: This type of robot is usually installed directly on the business personnel's computer. When in use, the robot can be started by directly triggering it manually or through the console.

[0010] 2. Unattended operation: This type of robot is generally installed on a non-business personnel computer (such as a cloud computer), and the start and stop of the robot are completely controlled by commands sent by the console.

[0011] Regardless of the method, the server must be kept turned on during execution and unless it is manually turned on or off by human control, existing RPA vendors do not dynamically increase or decrease the number of workers in the cluster based on the amount of work.

[0012] In the current downturn of the Internet industry, enterprises need to actively find ways to reduce costs. Cloud servers are a major expense, and cloud servers that run 24 hours a day will lead to unnecessary expenses. Today, companies face a problem in the use of cloud servers, that is, all current cloud servers run 24 hours a day, but in fact, when the server is turned off, except for the disk fee, other resources such as CPU, memory and network do not incur any fees. With the maturity and development of cloud computing technology, the intelligent cloud server power on and off system has become an effective solution to reduce costs. Summary of the invention

[0013] The purpose of the present invention is to solve at least one of the technical problems existing in the prior art, and to provide a dynamic scaling scheduling method and system based on distributed RPA Worker, which can solve the problem of high cost of traditional technology.

[0014] To achieve the above object, the present invention provides the following technical solutions: a dynamic scaling scheduling method and system based on distributed RPA Worker, step 1: configure group information, add group information through the worker group management page, and after the addition is completed, the corresponding third-party RPA console can view the added group information;

[0015] Step 2: Add worker information. You need to purchase an ECS cloud server in advance and obtain the server instance ID. Then, configure the instance ID, worker group, worker type, etc.

[0016] Step 3: Build the core system for expansion and contraction, and complete the construction of the expansion and contraction system according to the "technical solution" described in this patent;

[0017] Step 4: Create an RPA task. You can create a set of RPA task management functions to record the basic information, execution information, log information, etc. of the task.

[0018] After a task is created, the platform needs to send the task to a third-party RPA console, and the third-party console receives the task.

[0019] Step 5: The worker expands / reduces capacity. When the expansion / reduction system detects that there are tasks to be executed, it will start or shut down the machine according to the calculation results of the expansion / reduction strategy. The machine status can be observed in real time. At the same time, we can design a database table to fully record the machine's power on / off status information.

[0020] Step 6: View the task execution results and implement the task execution result log function by yourself. You can use this function to view the task execution process, the machine to which the execution belongs, and other information to verify whether the expansion and contraction is real and effective.

[0021] Preferably, the construction of the expansion and contraction system in step 3 specifically includes:

[0022] Determine the solution path based on technology selection, using the "pre-purchase mode" or "real-time purchase mode";

[0023] Build the overall system according to the technical architecture;

[0024] Calculate the number of workers for scaling up and down based on the scaling strategy;

[0025] Implement the power on / off connection of ECS servers according to the ECS interface specification;

[0026] Implement the cluster pull-in or pull-out function according to the API interface specification of the third-party RPA console.

[0027] Preferably, the database table in step 5 includes the startup time, shutdown time, cluster-exit time, and cluster-in time of a specific IP machine.

[0028] Preferably, during the process of step 2, the newly added server is synchronously added to the third-party RPA console, and the subsequent assignment of tasks to be executed is implemented by the console.

[0029] Preferably, the initial state: the newly added machine through the worker management page has not been powered on or joined to the cluster. After being powered on and joined to the cluster, the state is normal and can receive and execute scheduling tasks;

[0030] Normal state: The ECS machine is powered on and the cluster is in the pulled-in state. The machine can receive and execute tasks normally.

[0031] Shutdown state: The machine is shut down through the scaling logic or the manual shutdown operation on the worker page. When the machine needs to be shut down, the ECS shutdown interface will be called to stop the service;

[0032] Offline state: In this state, the ECS machine is not shut down and is still powered on, but the cluster state is in the state of being pulled out of the cluster and cannot receive or execute tasks. This can be achieved through the pull-out cluster function on the worker page. Generally, this state is required when you need to manually log in to the server to view logs and other information.

[0033] Preferably, the scaling system "status check" operation is performed under different matching relationships between the worker status and the actual ECS status. For example, when the worker is online but the ECS machine is actually shut down, the system will automatically turn on the ECS machine.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) The dynamic scaling scheduling method and system based on distributed RPA Worker has a normal monthly fee of RMB 409 for a 4C8G machine with the same configuration. After dynamic capacity expansion, the actual monthly fee for a 4C8G machine is about RMB 133. Therefore, it is expected that the monthly cost savings will be about 70%, and the cost will be reduced: by dynamically adjusting the number of server startups and shutdowns according to the real-time task volume, enterprises can maximize the use of resources and avoid unnecessary expenses on cloud servers.

[0036] (2) The dynamic scaling scheduling method and system based on distributed RPA Worker,Resource optimization: The system can intelligently schedule resources according to task requirements to ensure that the utilization rate of each server is maximized, thereby improving the overall resource utilization efficiency,Environmental friendliness: Reduce unnecessary startup time of cloud servers, thereby reducing energy consumption, reducing the impact on the environment, and reflecting the company's environmental responsibility,Automated operation and maintenance: The system can automatically monitor and adjust the server status, reducing the workload of operation and maintenance personnel, allowing them to focus on higher value tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention is further described below in conjunction with the accompanying drawings and embodiments:

[0038] Figure 1 It is a schematic diagram of the device connection of the present invention;

[0039] Figure 2 It is a schematic diagram of the pre-purchase inventory mode and real-time purchase mode system of the present invention;

[0040] Figure 3 It is a schematic diagram of the technical framework of the present invention;

[0041] Figure 4 This is a schematic diagram of the worker life cycle of the present invention;

[0042] Figure 5 The figure is a schematic diagram of the capacity expansion and contraction system of the present invention under different matching relationships between the worker status and the actual status of the ECS. DETAILED DESCRIPTION

[0043] This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0044] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0045] In the description of the present invention, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0046] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0047] The present invention provides a technical solution: please refer to the figure, a dynamic scaling scheduling method and system based on distributed RPA Worker; step 1: configure group information, add group information through the worker group management page, and after the addition is completed, the corresponding third-party RPA console can view the added group information;

[0048] Step 2: Add worker information. You need to purchase the ECS cloud server in advance and obtain the server instance ID. Then configure the instance ID, worker group, worker type, etc. At the same time, add the newly added server to the third-party RPA console, and the subsequent tasks to be executed will be assigned by the console.

[0049] Step 3: Build the core system for expansion and contraction. Complete the construction of the expansion and contraction system according to the "technical solution" described in this patent, including:

[0050] Determine the solution path based on technology selection, using the "pre-purchase mode" or "real-time purchase mode";

[0051] Build the overall system according to the technical architecture;

[0052] Calculate the number of workers for scaling up and down based on the scaling strategy;

[0053] Implement the power on / off connection of ECS servers according to the ECS interface specification;

[0054] Implement the cluster pull-in or pull-out function according to the API interface specification of the third-party RPA console;

[0055] Step 4: Create an RPA task. You can create a set of RPA task management functions to record the basic information, execution information, log information, etc. of the task.

[0056] After a task is created, the platform needs to send the task to a third-party RPA console, and the third-party console receives the task.

[0057] Step 5: The worker expands / reduces capacity. When the expansion / reduction system detects that there are tasks to be executed, it will start or shut down the machine according to the calculation results of the expansion / reduction strategy. The machine status can be observed in real time. At the same time, we can design a database table to fully record the machine's power on / off status information.

[0058] It includes the startup time, shutdown time, cluster pull-out time, cluster pull-in time, etc. for specific IP machines;

[0059] Step 6: View the task execution results and implement the task execution result log function by yourself. You can use this function to view the task execution process, the machine to which the execution belongs, and other information to verify whether the expansion and contraction is real and effective.

[0060] For further information, see Figure 4 Initial state: The machine newly added through the worker management page has not been powered on or added to the cluster. After being powered on and added to the cluster, the state is normal and can receive and execute scheduling tasks;

[0061] Normal state: The ECS machine is powered on and the cluster is in the pulled-in state. The machine can receive and execute tasks normally.

[0062] Shutdown state: The machine is shut down through the scaling logic or the manual shutdown operation on the worker page. When the machine needs to be shut down, the ECS shutdown interface will be called to stop the service;

[0063] Offline state: In this state, the ECS machine is not shut down and is still powered on, but the cluster state is in the state of being pulled out of the cluster and cannot receive or execute tasks. This can be achieved through the pull-out cluster function on the worker page. Generally, this state is required when you need to manually log in to the server to view logs and other information.

[0064] For further information, see Figure 5 , the "status check" operation of the scaling system under different matching relationships between the worker status and the actual ECS status. For example, when the worker is online but the ECS machine is actually shut down, the system will automatically start the ECS machine.

[0065] For further information, see Figure 2 After the RPA task is created, the task creation action will be monitored to trigger the calculation of the scaling rules. When scaling is required, there are two underlying technology options, namely "pre-purchase inventory mode" and "real-time procurement mode";

[0066] Pre-purchase inventory mode: In this mode, you need to purchase a batch of ECS cloud servers in advance and set the CPU, memory, disk and other specifications. When you need to scale up or down, you only need to call the ECS interface to turn the machine on and off. The advantages are that the scale-up and scale-down range is free and the startup time is fast. The disadvantage is that you need to continue to pay for the disk of the machine that has been shut down.

[0067] Real-time purchasing mode: In this mode, there is no need to purchase ECS cloud servers in advance. When you need to scale up or down, you only need to call the ECS interface to purchase and release cloud server resources. That is, you can purchase when you need them. The advantage is that you do not need to pay for disks continuously, and the cost is relatively low. The disadvantage is that the startup time is calculated in minutes, which has a greater impact on the effectiveness of task scheduling.

[0068] It should be noted that the startup and shutdown of ECS need to be coordinated with the RPA vendor console. When the ECS server is started, it is necessary to add the ECS server to the existing RPA Worker cluster. When the ECS server is shut down, it is necessary to pull the ECS server out of the existing RPA Worker cluster. After the RPA task enters the console, the task is assigned to a robot server based on the cluster. Therefore, pulling in and out of the cluster is a necessary operation.

[0069] At present, this patent is designed based on the "pre-purchase inventory model" according to actual business needs, and subsequent technical introductions are all based on this premise.

[0070] See also Figure 1 Currently, RPA suppliers are used to build enterprise-specific worker clusters. The basic architecture is shown in the figure above. The supplier's self-developed Worker cluster console system is used to manage multiple RPA Workers established in the enterprise. Users can submit RPA tasks to be executed to the console through various client forms and the internal network. The console will check, distribute, collect execution results, and perform execution record statistics on the received tasks.

[0071] For further information, see Figure 3 , link task module: link refers to the RPA task platform. This module is responsible for receiving external RPA request tasks, such as requiring RPA robots to enter forms or download files. When the task arrives at the link platform, the platform is responsible for task information verification, persistence, task scheduling, log collection, execution result push, etc. Other related functions include worker information maintenance and worker machine group maintenance;

[0072] Worker scaling module: The worker scaling module performs calculations based on the current task backlog and the current status of machines in the RPA cluster. The calculation result is to determine whether a cluster needs to expand or reduce a fixed number of workers.

[0073] There are two triggering times for worker scaling: first, when a new task is created, scaling calculations are triggered; second, by setting a fixed five-minute job schedule to regularly trigger calculations.

[0074] The scaling module is also responsible for checking the worker machine status. It checks whether the worker status (online, offline, shutdown) currently maintained by the link system is consistent with the corresponding ECS ​​machine status through a scheduled job with a fixed time of 20 minutes. If there is any inconsistency, corrections and reminders will be made.

[0075] Furthermore, the service is released. This system is a package of the API interface of external ECS server providers such as Alibaba Cloud. Specific functions include startup, shutdown, status query, etc.

[0076] Alibaba Cloud Platform: A third-party ECS server provider that provides basic usage, configuration, and supporting control API functions for ECS servers;

[0077] Yunkuo Console: A third-party RPA supplier that provides RPA robot-related machine management, task execution, task scheduling, log monitoring and other related services. This patent uses the "Yunkuo" supplier as an example, and other suppliers have similar functions and services;

[0078] Furthermore, the core of the worker scaling module is the implementation of the scaling rule strategy. The complete rule strategy is as follows:

[0079]

[0080] Furthermore, worker classification

[0081] Permanent type: This type of machine remains permanently powered on and is not affected by the expansion and contraction strategy;

[0082] Time period type: This type of machine is powered on during fixed working hours and powered off during non-working hours;

[0083] Temporary type: When the number of machines that have been powered on for permanent and time-slot types cannot meet the number of machines required for the current number of tasks, the scaling module will dynamically power on or off the temporary type machines.

[0084] Furthermore, channel: We agree that each task that needs to be performed by a worker does not exist in isolation but belongs to a "channel", which can be understood here as an operation process and a business specification. Tasks under the same channel are completely consistent in the robot operation process.

[0085] Task: must belong to a certain channel and carry the data required for the RPA robot to execute, such as the account password for logging into the web page.

[0086] Number of tasks backed up in a channel: the number of tasks waiting to be executed by the robot in a certain channel;

[0087] Channel average task time: the average task execution time recorded by a channel itself.

[0088] Furthermore, for the workday strategy, we agree that the working hours are from 8 am to 8 pm from Monday to Friday. During the working hours, permanent machines and time-slot machines will be started. During this period, these two types of machines will not be expanded or reduced as the number of tasks changes.

[0089] Furthermore, for the weekend strategy, only permanent machines will be kept powered on during the weekend period, and other types of machines will be in the shutdown stage. If the workload increases and the permanent machines are not enough to complete the operation, temporary machines will be turned on according to the calculation rules.

[0090] Furthermore, task peak strategy: During the peak phase, the task volume is high, and it is necessary to calculate the capacity expansion strategy to implement the startup operation. The specific case is as follows:

[0091] Known:

[0092] 1. Task expected waiting time (seconds): 60;

[0093] 2. Number of temporary workers: 2;

[0094] 3. Number of resident (permanent) workers: 5

[0095] set up:

[0096] Task expected waiting time: Y

[0097] Number of channels: N

[0098] Number of tasks accumulated in the channel (minus the number of tasks being executed, i.e. the current number of workers): T

[0099] Average channel task time (seconds): P

[0100] The number of workers required is: X

[0101] Then: X = [∑N((T*P) / Y) and P<=Y] + [∑N(T) and P>Y], calculated according to the channel, accumulated and rounded up;

[0102] Expansion example: The final number of temporary workers is: 17-5-2=12

[0103] Furthermore, the task valley strategy, when the task volume is low during the valley phase, needs to calculate the scaling-down strategy to implement the shutdown operation. The specific case is as follows:

[0104] Known:

[0105] 1. Task expected waiting time (seconds): 60;

[0106] 2. Number of temporary workers: 2;

[0107] 3. Number of resident workers: 5

[0108] set up:

[0109] Task expected waiting time: Y

[0110] Number of channels: N

[0111] Number of tasks accumulated in the channel (minus the number of tasks being executed, i.e. the current number of workers): T

[0112] Average channel task time (seconds): P

[0113] The number of workers required is: X

[0114] but:

[0115] 1. If X = X < the number of permanent workers, then take the number of all temporary workers

[0116] 2. If X=X>number of permanent workers, then take the absolute value of |X-5-2|

[0117]

[0118] Furthermore, in the scaling-down example, the number of temporary workers that are finally scaled down is 2.

[0119] Furthermore, the normal monthly subscription fee for a 4C8G machine with the same configuration is 409 yuan. After dynamic expansion, the actual monthly payment for a 4C8G machine is about 133 yuan. Therefore, the monthly cost savings are expected to be around 70%. Cost reduction: By dynamically adjusting the number of server startups and shutdowns based on the real-time task volume, enterprises can maximize the use of resources and avoid unnecessary expenses on cloud servers.

[0120] Resource optimization: The system can intelligently schedule resources according to task requirements to ensure that the utilization of each server is maximized, thereby improving the overall resource utilization efficiency.

[0121] Environmentally friendly: Reduce unnecessary startup time of cloud servers, thereby reducing energy consumption, reducing the impact on the environment, and reflecting the company's environmental responsibility.

[0122] Automated operation and maintenance: The system can automatically monitor and adjust server status, reducing the workload of operation and maintenance personnel and allowing them to focus on higher-value tasks.

[0123] The embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in the technical field without departing from the purpose of the present invention.

Claims

1. A dynamic scaling scheduling method and system based on distributed RPA Worker, characterized by: Step 1: Configure group information. Add group information on the worker group management page. After adding, you can view the added group information on the corresponding third-party RPA console. Step 2: Add worker information. You need to purchase an ECS cloud server in advance and obtain the server instance ID. Then, configure the instance ID, worker group, worker type, etc. Step 3: Build the core system for expansion and contraction, and complete the construction of the expansion and contraction system according to the "technical solution" described in this patent; Step 4: Create an RPA task. You can create a set of RPA task management functions to record the basic information, execution information, log information, etc. of the task. After a task is created, the platform needs to send the task to a third-party RPA console, and the third-party console receives the task. Step 5: The worker expands / reduces capacity. When the expansion / reduction system detects that there are tasks to be executed, it will start or shut down the machine according to the calculation results of the expansion / reduction strategy. The machine status can be observed in real time. At the same time, we can design a database table to fully record the machine's power on / off status information. Step 6: View the task execution results and implement the task execution result log function by yourself. You can use this function to view the task execution process, the machine to which the execution belongs, and other information to verify whether the expansion and contraction is real and effective.

2. According to claim 1, a dynamic scaling scheduling method and system based on distributed RPA Worker is characterized in that: The construction of the expansion and contraction system in step 3 specifically includes: Determine the solution path based on technology selection, using "pre-purchase mode" or "real-time purchase mode"; Build the overall system according to the technical architecture; Calculate the number of workers for scaling up and down based on the scaling strategy; Implement the power on / off connection of ECS servers according to the ECS interface specification; Implement the cluster pull-in or pull-out function according to the API interface specification of the third-party RPA console.

3. According to claim 1, a dynamic scaling scheduling method and system based on distributed RPA Worker is characterized in that: The database table in step 5 includes the startup time, shutdown time, cluster pull-out time, and cluster pull-in time for a specific IP machine.

4. According to claim 1, a dynamic scaling scheduling method and system based on distributed RPA Worker is characterized in that: During step 2, the newly added server is simultaneously added to the third-party RPA console, and the subsequent assignment of tasks to be executed is implemented by the console.

5. According to claim 1, a dynamic scaling scheduling method and system based on distributed RPA Worker is characterized in that: Initial state: The machine newly added through the worker management page has not been powered on or added to the cluster. After being powered on and added to the cluster, the state is normal and can receive and execute scheduling tasks; Normal state: The ECS machine is powered on and the cluster is in the pulled-in state. The machine can receive and execute tasks normally. Shutdown state: The machine is shut down through the scaling logic or the manual shutdown operation on the worker page. When the machine needs to be shut down, the ECS shutdown interface will be called to stop the service; Offline state: In this state, the ECS machine is not shut down and is still powered on, but the cluster state is in the state of being pulled out of the cluster and cannot receive or execute tasks. This can be achieved through the pull-out cluster function on the worker page. Generally, this state is required when you need to manually log in to the server to view logs and other information.

6. According to claim 1, a dynamic scaling scheduling method and system based on distributed RPA Worker is characterized in that: The "status check" operation of the scaling system under different matching relationships between the worker status and the actual ECS status. For example, when the worker is online but the ECS machine is actually shut down, the system will automatically start the ECS machine.

Images