Robot process automation method based on image recognition

Abstract

The invention discloses a robot process automation method based on image recognition, relates to the technical field of process automation, and solves the technical problems that existing resources are difficult to utilize to the maximum extent, task decomposition and distribution strategies are inconvenient to quickly adjust, and meanwhile, dynamic adjustment of load balance is difficult to realize. By decomposing the tasks and reasonably distributing the tasks to different RPA robots, existing resources can be utilized to the maximum degree, the multiple RPA robots can process different tasks at the same time, and the overall processing speed is increased. And if the user demand changes, the task decomposition and distribution strategy can be quickly adjusted, and the same sub-task does not need to be repeatedly executed. By monitoring and analyzing the load condition of the robot executing the task in each task group in real time, the overload or idle robot can be identified in time, the task allocation is dynamically adjusted, and the optimal utilization of resources is ensured. Redistribution of stacking tasks can effectively avoid overload of some robots, thereby realizing load balancing.

Description

Technical Field

[0001] The present invention belongs to the technical field of process automation, and in particular is a robot process automation method based on image recognition. Background Art

[0002] With the acceleration of digital transformation, enterprises are facing increasingly complex tasks and data processing requirements in their operations. Traditional manual operations are not only inefficient, but also prone to human errors, leading to rising costs and declining customer satisfaction. In order to improve work efficiency, reduce operating costs and improve service quality, more and more companies are beginning to adopt robotic process automation (RPA) technology. At the same time, the rapid development of image recognition technology has provided new possibilities for RPA. Image recognition can extract valuable information from images, enabling automated systems to understand and process visual data. This technology has been widely used in manufacturing, logistics, medical care, security and other fields, providing companies with more intelligent solutions.

[0003] Existing robotic process automation methods lack task decomposition, making it difficult to establish standard sub-task processes, maximizing the use of existing resources, and quickly adjusting task decomposition and allocation strategies. At the same time, there is also a lack of reasonable redistribution of tasks to be executed based on the execution status of tasks in different groups, making it difficult to achieve dynamic load balancing. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a robotic process automation method based on image recognition, which is used to solve the technical problems that it is difficult to maximize the utilization of existing resources, it is not convenient to quickly adjust task decomposition and allocation strategies, and it is also difficult to achieve dynamic adjustment of load balancing.

[0005] To solve the above problems, a first aspect of the present invention provides a robotic process automation method based on image recognition, comprising the following steps:

[0006] Obtain the user's authorization application information, set permissions for the user, and confirm the user's identity through username and password, SSO single sign-on or other authentication mechanisms; based on the user role and permission settings, check whether the user has the right to access specific functions or data.

[0007] Provide a web form or application interface for users to fill in authorization application information, including required permissions, reasons for application, etc., and support users to upload relevant documents or supporting materials.

[0008] Submit the user's authorization application to the management staff or automated system for review; once the review is completed, the system will notify the user of the approval result.

[0009] Based on the audit results, the corresponding permission settings are updated in the database to ensure that users can access the resources and functions they need.

[0010] The user enters the required task flow and target image data within the scope of authority;

[0011] In response to the required task flow input by the user, the required task flow is decomposed, and the decomposed tasks are assigned to different edited standard task flows; the RPA robots in the area are grouped into tasks, and the RPA robots in each task group are assigned, and according to the image data input by the user, the running instructions of the edited standard task flow are assigned to the RPA robots in each group;

[0012] The image acquisition module of the RPA robot is used to collect images of the target area, and the collected image data of the target area is traversed. When the target object appears, the standard task process preset by the process automation robot is started;

[0013] Monitor the operation data of each robot in real time, monitor the task execution process of each task group, and the task accumulation situation, and analyze the load status of the robot executing tasks in each task group;

[0014] The accumulated tasks are redistributed according to the current task accumulation situation and the load condition of the robots executing tasks in each task group.

[0015] Preferably, in response to the required task flow input by the user, the required task flow is decomposed, and the decomposed tasks are assigned to different edited standard task flows, including the following steps:

[0016] Subdivide all task operations within the RPA robot's functional scope into standard subtasks and generate a standard subtask process library;

[0017] Through NLP natural language processing technology, keywords and phrases are extracted from the demand task flow input by the user, the operation goals are identified, and the operation goals are structured into a preset data format;

[0018] Split the operation objectives into subtasks based on business logic;

[0019] Based on the content recommendation system or machine learning model, the decomposed subtasks are compared with the processes in the standard subtask process library to select matching standard subtasks;

[0020] Sort the selected standard subtasks according to the order of subtasks divided by the operation target, and check whether the operation target can be completed;

[0021] If the operation goal can be achieved, the robot is assigned to perform the standard subtasks according to the standard subtask sorting, the results are assigned, and the automation script of the corresponding robot is called to start executing each subtask;

[0022] If the operation target cannot be completed, the required task process entered by the user is reported as an unauthorized task.

[0023] Preferably, all task operations within the functional scope of the RPA robot are subdivided into standard subtasks, and a standard subtask process library is generated, including the following steps:

[0024] Collect a list of all tasks related to RPA robots and clarify the application scenarios and functional scope of RPA robots in the organization;

[0025] Step by step, decompose the task operations in all task operation lists within the application scenarios and functional scope of the RPA robot, and form standardized subtasks with the decomposed operation steps;

[0026] Plan standard subtasks into a unified task template, which includes: subtask name, task description, input requirements, execution steps and output results;

[0027] The task templates of standard subtasks are classified and stored according to different categories or business lines to generate a standard subtask process library.

[0028] Among them, task description: briefly describe the purpose and function of the subtask; input requirements: required input data or conditions; execution steps: list in detail the specific steps required to execute the subtask; output results: the results that should be produced after the subtask is completed.

[0029] Preferably, the content-based recommendation system or machine learning model compares the decomposed subtasks with the processes of the standard subtask process library and selects matching standard subtasks, including the following steps:

[0030] Use the Ji eba word segmentation algorithm to split the decomposed subtask description text into phrases;

[0031] Through the TF-IDF algorithm, the frequency of occurrence of each phrase is counted, and the TF value of each phrase in the text is calculated;

[0032] By counting the number of documents containing each phrase in the documents of the demand task flow input by users in the historical data, and the total number of documents of the demand task flow input by users in the historical data, the IDF value of each phrase is calculated by the formula: IDF = log total number of documents / number of documents containing the phrase;

[0033] Multiply the TF value and the IDF value to get the weight of each phrase in the document;

[0034] Using the TF-IDF algorithm, extract the keywords in the subtask name and task description of each subtask in the standard subtask process library, and calculate the similarity between the subtask description text and the subtask name and task description of the extracted subtask;

[0035] According to the similarity calculation result, the similarity between the selected subtasks and the subtasks in the standard subtask process library exceeds a preset threshold, and the subtask in the standard subtask process library with the highest similarity is used as the selected matching standard subtask.

[0036] Preferably, calculating the similarity between the subtask description text and the subtask name and task description of the extracted subtask comprises the following steps:

[0037] Convert phrases into vector representations using the Word2Vec model;

[0038] Calculate the cosine similarity between the word vectors of the phrases in the subtask description text and the phrases in the subtask name and task description in the standard subtask process library;

[0039] The similarity between the subtask description text and the subtasks in the standard subtask process library is calculated using the following formula:

[0040]

[0041] Where W is the similarity between the subtask description text and the subtasks in the standard subtask process library, q i The weight of the i-th phrase in the document in the subtask description text, COSX i is the maximum value of the cosine similarity between the ith phrase in the subtask description text and the keywords in the subtask in the standard subtask process library, i∈(1,2,…,n), and n is the total number of phrases in the subtask description text.

[0042] Preferably, the screened standard subtasks are sorted according to the order of the subtasks divided into the operation target, and whether the operation target can be completed is detected, including the following steps:

[0043] Sort the selected standard subtasks according to the subtask sequence of the operation target, simulate and run the sorted standard subtask process in the UI Path Studio tool of the RPA robot, and check whether the standard subtasks are executed as expected;

[0044] Check whether each subtask is successfully completed and whether the operation goal is achieved. If each subtask is successfully completed and the actual execution result is consistent with the operation goal, it means that the screened standard subtask can achieve the operation goal; otherwise, the operation goal cannot be achieved.

[0045] Preferably, real-time monitoring of the operating data of each robot, monitoring of the task execution process of each task group, and task accumulation, and analysis of the load status of the robot executing the task in each task group include the following steps:

[0046] Real-time monitoring of each robot's operating data, including the name of the subtask being executed, the average time of the subtask executed, and the success rate of the subtask execution;

[0047] Monitor the execution progress of each task group, including: subtask execution status and CPU resources occupied during execution, as well as task accumulation, including: waiting time for subtask execution within a preset time period and the number of subtasks currently to be processed;

[0048] The following formula is used to analyze the load status of the robots performing tasks in each task group:

[0049]

[0050] Among them, Lo is the load condition evaluation value of the robot executing the task in the task group, f is the current total CPU occupancy, p is the success rate of executing subtasks, N is the number of subtasks being executed, N0 is the number of executed subtasks, C is the maximum CPU resource occupied when executing the executed subtasks, C0 is the maximum CPU load of the robot executing the task, t is the average waiting time for executing subtasks within the preset time period, M is the number of subtasks currently to be processed, t0 is the average time of executed subtasks, and M0 is the total number of executed and unexecuted subtasks within the preset time period.

[0051] Preferably, according to the current task accumulation situation and the load condition of the robot executing the task in each task group, the accumulated tasks are reallocated, including the following steps:

[0052] According to the load status of the robot executing the task in each task group, if the load status evaluation value of the robot executing the task in the task group is greater than the preset threshold, the subtask to be executed is reallocated to the robot with the smallest load status evaluation value in the task group;

[0053] If the smallest robot load condition evaluation value in the task group is still greater than the preset threshold, the tasks to be executed in the task group are assigned to the task group with the least number of subtasks to be processed for execution.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention facilitates the maximum utilization of existing resources by decomposing tasks and reasonably allocating them to different RPA robots. Multiple RPA robots can handle different tasks at the same time, thereby improving the overall processing speed. If user needs change, it is convenient to quickly adjust the task decomposition and allocation strategy, and the same subtask does not need to be repeated, and there is no need to redevelop the entire process. By using standard task processes, it is easier to update or replace a module without affecting the entire system. By standardizing the edited task process, manual intervention and operational errors are reduced, and the consistency and accuracy of execution are improved. For the image data input by the user, the RPA robot can apply specific standard processes for automated processing, reducing dependence on manual review.

[0056] By real-time monitoring of the robot load status, the present invention can promptly identify overloaded or idle robots, dynamically adjust task allocation, and ensure optimal use of resources. Reallocating accumulated tasks can effectively avoid overloading of certain robots, thereby achieving load balancing. Through real-time monitoring, the system can quickly respond to task accumulation and make timely adjustments, thereby reducing task delays and improving overall work efficiency. Through reasonable task allocation, it is easy to reduce the waiting time for each task and speed up the overall processing speed. Fast and efficient task processing capabilities can significantly improve customer satisfaction and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1 The first aspect of the present invention provides a method for robotic process automation based on image recognition, comprising the following steps:

[0061] Obtain the user's authorization application information, set permissions for the user, and the user enters the required task flow and target image data within the permissions;

[0062] In response to the required task flow input by the user, the required task flow is decomposed, and the decomposed tasks are assigned to different edited standard task flows; the RPA robots in the area are grouped into tasks, and the RPA robots in each task group are assigned, and according to the image data input by the user, the running instructions of the edited standard task flow are assigned to the RPA robots in each group;

[0063] The image acquisition module of the RPA robot is used to collect images of the target area, and the collected image data of the target area is traversed. When the target object appears, the standard task process preset by the process automation robot is started;

[0064] Monitor the operation data of each robot in real time, monitor the task execution process of each task group, and the task accumulation situation, and analyze the load status of the robot executing tasks in each task group;

[0065] The accumulated tasks are redistributed according to the current task accumulation situation and the load condition of the robots executing tasks in each task group.

[0066] Specifically, in this embodiment, the authorization application information of the user is obtained, and the authority is set for the user. The user enters the required task flow and target image data within the authority, including:

[0067] Confirm user identity through username and password, SSO single sign-on or other authentication mechanisms; check whether the user has permission to access specific functions or data based on user role and permission settings.

[0068] Provide a web form or application interface for users to fill in authorization application information, including required permissions, reasons for application, etc., and support users to upload relevant documents or supporting materials.

[0069] Submit the user's authorization application to the management staff or automated system for review, allowing the user to enter the required task flow and target image data within the authority. After the review is completed, the system will notify the user of the approval result.

[0070] Based on the audit results, the corresponding permission settings are updated in the database to ensure that users can access the resources and functions they need.

[0071] In this embodiment, by responding to the required task flow input by the user, the required task flow is decomposed, and the decomposed tasks are assigned to different edited standard task flows; the RPA robots in the area are grouped into tasks, and the RPA robots in each task group are assigned, and according to the image data input by the user, the running instructions of the edited standard task flow are assigned to the RPA robots in each group;

[0072] By decomposing tasks and assigning them to different RPA robots, it is easy to maximize the use of existing resources. Multiple RPA robots can handle different tasks at the same time, improving the overall processing speed. If user needs change, it is easy to quickly adjust the task decomposition and allocation strategy. The same subtask does not need to be repeated, and there is no need to redevelop the entire process. By using standard task processes, it is easier to update or replace a module without affecting the entire system. By standardizing the edited task process, manual intervention and operational errors are reduced, and the consistency and accuracy of execution are improved. For the image data entered by the user, the RPA robot can apply specific standard processes for automated processing, reducing the reliance on manual review.

[0073] The image acquisition module of the RPA robot is used to collect images of the target area, and the collected image data of the target area is traversed. When the target object appears, the standard task process preset by the process automation robot is started;

[0074] Automated image acquisition and target object recognition processes reduce manual involvement, labor costs and human errors. Once the target object is identified, the system can immediately start the corresponding standard task process, greatly improving the processing speed. Through image processing algorithms and machine learning models, high-precision target object recognition is achieved to ensure the accuracy of task execution.

[0075] This allows the system to be adjusted according to different needs to adapt to a variety of business scenarios and application areas, such as manufacturing, logistics, security, etc.

[0076] Monitor the operation data of each robot in real time, monitor the task execution process of each task group, and the task accumulation situation, and analyze the load status of the robots that are executing tasks in each task group; reallocate the accumulated tasks according to the current task accumulation situation and the load status of the robots that are executing tasks in each task group.

[0077] By monitoring the robot load in real time, overloaded or idle robots can be identified in time, and task allocation can be adjusted dynamically to ensure optimal use of resources. Redistributing accumulated tasks can effectively avoid overloading of certain robots, thereby achieving load balancing. Real-time monitoring enables the system to respond quickly to task accumulation and make timely adjustments, thereby reducing task delays and improving overall work efficiency. Through reasonable task allocation, it is easy to reduce the waiting time for each task and speed up the overall processing speed. Fast and efficient task processing capabilities can significantly improve customer satisfaction and user experience.

[0078] In one embodiment of the present invention, in response to a required task flow input by a user, the required task flow is decomposed, and the decomposed tasks are assigned to different edited standard task flows, including the following steps:

[0079] Subdivide all task operations within the RPA robot's functional scope into standard subtasks and generate a standard subtask process library;

[0080] Through NLP natural language processing technology, keywords and phrases are extracted from the demand task flow input by the user, the operation goals are identified, and the operation goals are structured into a preset data format;

[0081] Split the operation objectives into subtasks based on business logic;

[0082] Based on the content recommendation system or machine learning model, the decomposed subtasks are compared with the processes in the standard subtask process library to select matching standard subtasks;

[0083] Sort the selected standard subtasks according to the order of subtasks divided by the operation target, and check whether the operation target can be completed;

[0084] If the operation goal can be achieved, the robot is assigned to perform the standard subtasks according to the standard subtask sorting, the results are assigned, and the automation script of the corresponding robot is called to start executing each subtask;

[0085] If the operation target cannot be completed, the required task process entered by the user is reported as an unauthorized task.

[0086] In this embodiment, according to the business logic, the operation target is divided into subtasks, including the following steps:

[0087] Extract the primary goal from the user's needs. This is usually the core of the entire task process. For example, if the user wants to generate a report, the primary action goal may be "Complete the report."

[0088] List the key steps, including:

[0089] Based on the operational objective, list the key steps required to achieve the objective. These steps should be necessary to achieve the objective. For example: data collection, data cleaning, data analysis, and report writing;

[0090] Break down key steps into subtasks, including:

[0091] Break each key step down into smaller subtasks. Each subtask should be an independently executable and specific operation.

[0092] For example: Based on business logic, the operational objectives are generated into sales performance reports, which are divided into subtasks, including: Collecting sales data: determining the required data fields (such as date, product, sales, etc.), extracting relevant data from the database, and importing it into Excel or other analysis tools; Cleaning and preparing data: checking and processing missing values, formatting date fields, and removing unnecessary data columns; Analyzing sales trends: drawing sales trend charts and comparing the sales performance of different product lines; Writing report content: writing an introduction to explain background information and inserting charts and explaining trends.

[0093] In one embodiment of the present invention, all task operations within the functional scope of the RPA robot are subdivided into standard subtasks, and a standard subtask process library is generated, including the following steps:

[0094] Collect a list of all tasks related to RPA robots and clarify the application scenarios and functional scope of RPA robots in the organization;

[0095] Step by step, decompose the task operations in all task operation lists within the application scenarios and functional scope of the RPA robot, and form standardized subtasks with the decomposed operation steps;

[0096] Plan standard subtasks into a unified task template, which includes: subtask name, task description, input requirements, execution steps and output results;

[0097] The task templates of standard subtasks are classified and stored according to different categories or business lines to generate a standard subtask process library.

[0098] Among them, task description: briefly describe the purpose and function of the subtask; input requirements: required input data or conditions; execution steps: list in detail the specific steps required to execute the subtask; output results: the results that should be produced after the subtask is completed.

[0099] In one embodiment of the present invention, a content-based recommendation system or a machine learning model compares the decomposed subtasks with the processes of the standard subtask process library and selects matching standard subtasks, including the following steps:

[0100] Use the Ji eba word segmentation algorithm to split the decomposed subtask description text into phrases;

[0101] Through the TF-IDF algorithm, the frequency of occurrence of each phrase is counted, and the TF value of each phrase in the text is calculated;

[0102] By counting the number of documents containing each phrase in the documents of the demand task flow input by users in the historical data, and the total number of documents of the demand task flow input by users in the historical data, the IDF value of each phrase is calculated by the formula: IDF = log total number of documents / number of documents containing the phrase;

[0103] Multiply the TF value and the IDF value to get the weight of each phrase in the document;

[0104] Using the TF-IDF algorithm, extract the keywords in the subtask name and task description of each subtask in the standard subtask process library, and calculate the similarity between the subtask description text and the subtask name and task description of the extracted subtask;

[0105] According to the similarity calculation result, the similarity between the selected subtasks and the subtasks in the standard subtask process library exceeds a preset threshold, and the subtask in the standard subtask process library with the highest similarity is used as the selected matching standard subtask.

[0106] In one embodiment of the present invention, calculating the similarity between the subtask description text and the subtask name and task description of the extracted subtask includes the following steps:

[0107] Convert phrases into vector representations using the Word2Vec model;

[0108] Calculate the cosine similarity between the word vectors of the phrases in the subtask description text and the phrases in the subtask name and task description in the standard subtask process library;

[0109] The similarity between the subtask description text and the subtasks in the standard subtask process library is calculated using the following formula:

[0110]

[0111] Where W is the similarity between the subtask description text and the subtasks in the standard subtask process library, q i The weight of the i-th phrase in the document in the subtask description text, COSX i is the maximum value of the cosine similarity between the ith phrase in the subtask description text and the keywords in the subtask in the standard subtask process library, i∈(1,2,…,n), and n is the total number of phrases in the subtask description text.

[0112] In one embodiment of the present invention, the screened standard subtasks are sorted according to the order of the subtasks divided into the operation target, and whether the operation target can be completed is detected, including the following steps:

[0113] Sort the selected standard subtasks according to the subtask sequence of the operation target, simulate and run the sorted standard subtask process in the UI Path Studio tool of the RPA robot, and check whether the standard subtasks are executed as expected;

[0114] Check whether each subtask is successfully completed and whether the operation goal is achieved. If each subtask is successfully completed and the actual execution result is consistent with the operation goal, it means that the screened standard subtasks can achieve the operation goal; otherwise, the operation goal cannot be achieved.

[0115] In one embodiment of the present invention, real-time monitoring of the operation data of each robot, monitoring of the task execution process of each task group, and task accumulation, and analysis of the load status of the robot executing the task in each task group include the following steps:

[0116] Real-time monitoring of each robot's operating data, including the name of the subtask being executed, the average time of the subtask executed, and the success rate of the subtask execution;

[0117] Monitor the execution progress of each task group, including: subtask execution status and CPU resources occupied during execution, as well as task accumulation, including: waiting time for subtask execution within a preset time period and the number of subtasks currently to be processed;

[0118] The following formula is used to analyze the load status of the robots performing tasks in each task group:

[0119]

[0120] Among them, Lo is the load condition evaluation value of the robot executing the task in the task group, f is the current total CPU occupancy, p is the success rate of executing subtasks, N is the number of subtasks being executed, N0 is the number of executed subtasks, C is the maximum CPU resource occupied when executing the executed subtasks, C0 is the maximum CPU load of the robot executing the task, t is the average waiting time for executing subtasks within the preset time period, M is the number of subtasks currently to be processed, t0 is the average time of executed subtasks, and M0 is the total number of executed and unexecuted subtasks within the preset time period.

[0121] In one embodiment of the present invention, according to the current task accumulation situation and the load condition of the robot executing the task in each task group, the accumulated tasks are reallocated, including the following steps:

[0122] According to the load status of the robot executing the task in each task group, if the load status evaluation value of the robot executing the task in the task group is greater than the preset threshold, the subtask to be executed is reallocated to the robot with the smallest load status evaluation value in the task group;

[0123] In this embodiment, a preset threshold value for the load condition evaluation value of the robot that is performing the task in the task group is determined. The load condition evaluation values ​​of all robots in poor operating conditions are counted, and the minimum value is taken and set as the preset threshold value for the load condition evaluation value, which is specifically set to 0.82.

[0124] If the smallest robot load condition evaluation value in the task group is still greater than the preset threshold, the tasks to be executed in the task group are assigned to the task group with the least number of subtasks to be processed for execution.

[0125] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A robotic process automation method based on image recognition, characterized in that: The following steps are involved: Obtain the user's authorization application information, set permissions for the user, and the user enters the required task flow and target image data within the permissions; In response to the required task flow input by the user, the required task flow is decomposed, and the decomposed tasks are assigned to different edited standard task flows; The RPA robots in the area are divided into task groups, and the RPA robots in each task group are assigned running instructions for the edited standard task flow according to the image data input by the user; The image acquisition module of the RPA robot is used to collect images of the target area, and the collected image data of the target area is traversed. When the target object appears, the standard task process preset by the process automation robot is started; Monitor the operation data of each robot in real time, monitor the task execution process of each task group, and the task accumulation situation, and analyze the load status of the robot executing tasks in each task group; The accumulated tasks are redistributed according to the current task accumulation situation and the load condition of the robots executing tasks in each task group.

2. The method of robotic process automation based on image recognition according to claim 1, characterized in that: In response to the required task flow input by the user, the required task flow is decomposed, and the decomposed tasks are assigned to different edited standard task flows, including the following steps: Subdivide all task operations within the RPA robot's functional scope into standard subtasks and generate a standard subtask process library; Through NLP natural language processing technology, keywords and phrases are extracted from the demand task flow input by the user, the operation goals are identified, and the operation goals are structured into a preset data format; Split the operation objectives into subtasks based on business logic; Based on the content recommendation system or machine learning model, the decomposed subtasks are compared with the processes in the standard subtask process library to select matching standard subtasks; Sort the selected standard subtasks according to the order of subtasks divided by the operation target, and check whether the operation target can be completed; If the operation goal can be achieved, the robot is assigned to perform the standard subtasks according to the standard subtask sorting, the results are assigned, and the automation script of the corresponding robot is called to start executing each subtask; If the operation target cannot be completed, the required task process entered by the user is reported as an unauthorized task.

3. The image recognition-based robotic process automation method according to claim 2, characterized in that: Subdivide all task operations within the RPA robot's functional scope into standard subtasks and generate a standard subtask process library, including the following steps: Collect a list of all tasks related to RPA robots and clarify the application scenarios and functional scope of RPA robots in the organization; Step by step, decompose the task operations in all task operation lists within the application scenarios and functional scope of the RPA robot, and form standardized subtasks with the decomposed operation steps; Plan standard subtasks into a unified task template, which includes: subtask name, task description, input requirements, execution steps and output results; The task templates of standard subtasks are classified and stored according to different categories or business lines to generate a standard subtask process library.

4. The method for robotic process automation based on image recognition according to claim 2, characterized in that: Based on the content recommendation system or machine learning model, the decomposed subtasks are compared with the processes of the standard subtask process library, and matching standard subtasks are screened, including the following steps: The decomposed subtask description text is split into phrases using the Jieba word segmentation algorithm; Through the TF-IDF algorithm, the frequency of occurrence of each phrase is counted, and the TF value of each phrase in the text is calculated; By counting the number of documents containing each phrase in the documents of the demand task flow input by users in the historical data, and the total number of documents of the demand task flow input by users in the historical data, the IDF value of each phrase is calculated using the formula: IDF = log total number of documents / number of documents containing the phrase; Multiply the TF value and the IDF value to get the weight of each phrase in the document; Using the TF-IDF algorithm, extract the keywords in the subtask name and task description of each subtask in the standard subtask process library, and calculate the similarity between the subtask description text and the subtask name and task description of the extracted subtask; According to the similarity calculation result, the similarity between the selected subtasks and the subtasks in the standard subtask process library exceeds a preset threshold, and the subtask in the standard subtask process library with the highest similarity is used as the selected matching standard subtask.

5. The method for robotic process automation based on image recognition according to claim 4, characterized in that: Calculating the similarity between the subtask description text and the subtask name and task description of the extracted subtask includes the following steps: Convert phrases into vector representations using the Word2Vec model; Calculate the cosine similarity between the word vectors of the phrases in the subtask description text and the phrases in the subtask name and task description in the standard subtask process library; The similarity between the subtask description text and the subtasks in the standard subtask process library is calculated using the following formula: Where W is the similarity between the subtask description text and the subtasks in the standard subtask process library, q i The weight of the i-th phrase in the document in the subtask description text, COSX i is the maximum value of the cosine similarity between the ith phrase in the subtask description text and the keywords in the subtask in the standard subtask process library, i∈(1,2,…,n), and n is the total number of phrases in the subtask description text.

6. The image recognition-based robotic process automation method according to claim 2, characterized in that: According to the order of subtasks divided by the operation target, sort the selected standard subtasks and check whether the operation target can be completed, including the following steps: Sort the selected standard subtasks according to the order of subtasks divided by the operation objectives, simulate and run the sorted standard subtask process in the UiPath Studio tool of the RPA robot, and check whether the standard subtasks are executed as expected; Check whether each subtask is successfully completed and whether the operation goal is achieved. If each subtask is successfully completed and the actual execution result is consistent with the operation goal, it means that the screened standard subtask can achieve the operation goal; otherwise, the operation goal cannot be achieved.

7. The image recognition-based robotic process automation method according to claim 1, characterized in that: Monitor the operation data of each robot in real time, monitor the task execution process of each task group, and the task accumulation situation, and analyze the load status of the robot executing the task in each task group, including the following steps: Real-time monitoring of each robot's operating data, including the name of the subtask being executed, the average time of the subtask executed, and the success rate of the subtask execution; Monitor the execution process of each task group, including: subtask execution status and CPU resources occupied during execution, as well as task accumulation, including: waiting time for subtask execution within a preset time period and the number of subtasks currently to be processed; The following formula is used to analyze the load status of the robots performing tasks in each task group: Among them, Lo is the load condition evaluation value of the robot executing the task in the task group, f is the current total CPU occupancy, p is the success rate of executing subtasks, N is the number of subtasks being executed, N0 is the number of executed subtasks, C is the maximum CPU resource occupied when executing the executed subtasks, C0 is the maximum CPU load of the robot executing the task, t is the average waiting time for executing subtasks within the preset time period, M is the number of subtasks currently to be processed, t0 is the average time of executed subtasks, and M0 is the total number of executed and unexecuted subtasks within the preset time period.

8. The method of robotic process automation based on image recognition according to claim 7, characterized in that: According to the current task accumulation situation and the load status of the robots executing tasks in each task group, the accumulated tasks are redistributed, including the following steps: According to the load status of the robot executing the task in each task group, if the load status evaluation value of the robot executing the task in the task group is greater than the preset threshold, the subtask to be executed is reallocated to the robot with the smallest load status evaluation value in the task group; If the smallest robot load condition evaluation value in the task group is still greater than the preset threshold, the tasks to be executed in the task group are assigned to the task group with the least number of subtasks to be processed for execution.

Images