RPA process adaptive optimization methods, systems, devices, and media

Abstract

This invention discloses an adaptive optimization method, system, device, and medium for Robotic Process Automation (RPA), relating to the field of robotic process automation technology. The method includes the following steps: acquiring multi-dimensional environmental data during process execution and generating a state feature sequence based on the multi-dimensional environmental data; performing anomaly prediction based on the state feature sequence to obtain anomaly probability prediction values; when the anomaly probability prediction value is greater than a preset probability threshold, acquiring multi-modal features of the current target user interface and fusing the multi-modal features into comprehensive scene features; matching the comprehensive scene features with historical strategy features in the knowledge base to determine a target alternative strategy; when the system response corresponding to the target alternative strategy is in a delayed state, determining the retry waiting parameters corresponding to the target alternative strategy; and updating the execution path in the knowledge base according to the execution result of the target alternative strategy. This invention achieves adaptive process repair and dynamic optimization of execution paths.

Description

Technical Field

[0001] This invention relates to the field of robotic process automation (RPA) related technologies, specifically to RPA process adaptive optimization methods, systems, devices, and media. Background Technology

[0002] With the deepening of enterprise digital transformation, Robotic Process Automation (RPA) technology is now widely used to handle a large number of repetitive and well-defined business processes in enterprises. By simulating human operation, RPA technology interacts with the user interface of the business system according to pre-set static rules and fixed scripts to complete repetitive tasks such as data capture, entry, and comparison.

[0003] Existing RPA systems rely on static scripts and fixed execution strategies. When faced with changes in the business system interface, system performance fluctuations, network latency, and other environmental changes, they are prone to problems such as element location failure and process execution interruption. After an anomaly occurs, the system can only passively stop and rely on manual troubleshooting and repair, lacking self-optimization capabilities. If high-computing recognition models are used throughout to cope with complex scenarios, it will result in excessive robot resource consumption and make it difficult to meet real-time requirements. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an RPA process adaptive optimization method, system, device, and medium.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] This invention provides an adaptive optimization method for RPA processes, comprising the following steps:

[0007] Acquire multidimensional environmental data during the process execution, and generate a state feature sequence based on the multidimensional environmental data;

[0008] Anomaly prediction is performed based on the state feature sequence to obtain an anomaly probability prediction value;

[0009] When the predicted anomaly probability value is greater than a preset probability threshold, the multimodal features of the current target user interface are obtained, and the multimodal features are fused into comprehensive scene features;

[0010] Based on the comprehensive scenario features and the historical strategy features in the knowledge base, a target replacement strategy is determined; when the system response corresponding to the target replacement strategy is in a delayed state, a retry waiting parameter corresponding to the target replacement strategy is determined, and the execution sequence of the target replacement strategy is controlled according to the retry waiting parameter;

[0011] The execution path in the knowledge base is updated based on the execution result of the target substitution strategy.

[0012] As a preferred embodiment of the present invention, the step of generating a state feature sequence based on the multidimensional environmental data includes:

[0013] The multidimensional environmental data is time-aligned according to a preset time window and a global timestamp to obtain multiple initial features;

[0014] The multiple initial features are standardized, and the standardized initial features are concatenated in chronological order to form the state feature sequence.

[0015] The multidimensional environmental data includes image features, node structure features, system performance indicators, and process execution logs.

[0016] As a preferred embodiment of the present invention, the multimodal features include visual features and textual semantic features; the fusion of the multimodal features into comprehensive scene features includes:

[0017] The text semantic features are projected onto a feature space of the same dimension as the visual features through a fully connected layer to obtain mapped text features;

[0018] Based on the first weighting coefficient and the second weighting coefficient, the mapped text features and the visual features are weighted and fused to generate the comprehensive scene features.

[0019] As a preferred embodiment of the present invention, the step of matching the comprehensive scene features with historical strategy features in the knowledge base to determine the target replacement strategy includes:

[0020] Calculate the cosine similarity between the comprehensive scene features and multiple historical strategy features in the knowledge base, and extract the maximum value among the cosine similarities;

[0021] When the maximum value is greater than a preset similarity threshold, the strategy to which the historical strategy feature corresponding to the maximum value belongs is determined as the target replacement strategy.

[0022] As a preferred embodiment of the present invention, determining the retry waiting parameter corresponding to the target substitution strategy includes:

[0023] Obtain the state space and action space. The state space is composed of a combination of discretized system load and process steps, and the action space contains multiple discretized waiting times.

[0024] The feedback reward value after the action is executed is calculated based on a preset reward function, and the action value corresponding to each action in the action space is iteratively updated based on the feedback reward value. The preset reward function is composed of a weighted sum of a positive reward value for successful execution, a negative penalty value for waiting time, and a negative penalty value for retry times.

[0025] The target waiting time is determined from the action space based on the iteratively updated action value and used as the retry waiting parameter.

[0026] As a preferred embodiment of the present invention, updating the execution path in the knowledge base based on the execution result of the target substitution strategy includes:

[0027] A state transition graph is constructed in the knowledge base, wherein user interface states or process checkpoints are defined as state nodes and execution strategies are defined as connection edges.

[0028] Obtain the execution success rate of the target substitution strategy;

[0029] Based on the preset benchmark value, the execution success rate, and the preset compensation value to prevent zero weight, the difference between the preset benchmark value and the execution success rate is added to the preset compensation value to calculate the target weight, and the target weight is updated to the connection edge corresponding to the target substitution strategy in the state transition diagram.

[0030] Based on the shortest path search algorithm, the path with the smallest cumulative weight is determined as the execution path in the updated state transition graph.

[0031] As a preferred embodiment of the present invention, the method further includes a degradation processing mechanism for dealing with location anomalies:

[0032] When the maximum value is not greater than the preset similarity threshold and no target replacement strategy is matched, global text search or matching logic based on node edit distance is invoked to locate the target.

[0033] If the positioning fails for a preset number of consecutive times, the current process will be rolled back to the previous stable checkpoint, the current process node will be marked as abnormal, and subsequent execution will be suspended.

[0034] When the cumulative failure rate of the current execution route exceeds the tolerance threshold, abnormal snapshot data is generated and the current process is blocked.

[0035] The present invention also provides an RPA process adaptive optimization device, comprising:

[0036] The data acquisition unit is used to acquire multi-dimensional environmental data during the process execution and generate a state feature sequence based on the multi-dimensional environmental data.

[0037] An anomaly warning unit is used to predict anomalies based on the state feature sequence and obtain an anomaly probability prediction value.

[0038] The feature recognition unit is used to acquire the multimodal features of the current target user interface when the anomaly probability prediction value is greater than a preset probability threshold, and to fuse the multimodal features into a comprehensive scene feature;

[0039] The strategy decision unit is used to match the comprehensive scenario features with the historical strategy features in the knowledge base to determine the target replacement strategy; when the system response corresponding to the target replacement strategy is in a delayed state, it determines the retry waiting parameter corresponding to the target replacement strategy and controls the execution sequence of the target replacement strategy according to the retry waiting parameter.

[0040] The path optimization unit is used to update the execution path in the knowledge base based on the execution result of the target substitution strategy.

[0041] The present invention also provides an RPA process adaptive optimization system, comprising: a processor, a memory, a communication interface, and a database;

[0042] The memory stores a computer program, and the processor executes the computer program to implement the above method.

[0043] The database is used to store the state transition diagrams and historical strategy feature sets in the knowledge base;

[0044] The communication interface is used to receive multi-dimensional environmental data reported by the external execution terminal and to send control commands to the external execution terminal to support the external execution terminal in path optimization and strategy matching.

[0045] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0046] The beneficial effects of this invention are:

[0047] 1. In this invention, by setting the acquisition of multimodal features of the current target user interface only when the predicted abnormal probability value is greater than a preset probability threshold, and fusing the multimodal features into comprehensive scene features, the invention achieves the function of waking up high-computing-power multimodal recognition on demand. This solves the problem that existing technologies, which require the full use of high-computing-power recognition models to cope with complex scenes, result in excessive robot resource consumption and difficulty in meeting real-time requirements.

[0048] 2. In this invention, a target replacement strategy is determined by matching the comprehensive scenario features with the historical strategy features in the knowledge base. When the system response corresponding to the target replacement strategy is in a delayed state, a retry waiting parameter is determined and the execution sequence is controlled. The execution path in the knowledge base is updated according to the execution result. This plays a role in adaptive process repair and dynamic optimization of the execution path, solving the problem that existing RPA systems suffer from abnormal process interruption due to user interface changes or network latency and lack self-repair capabilities. Attached Figure Description

[0049] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0050] Figure 1 This is a schematic diagram of the workflow of the RPA process adaptive optimization method of the present invention. Detailed Implementation

[0051] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0052] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0053] like Figure 1 As shown, this embodiment addresses the problem in existing Robotic Process Automation (RPA) systems where static scripts are highly dependent on the stability of the target business system's interface structure, are prone to failure during system upgrades or network fluctuations, and cannot autonomously repair themselves. It provides an adaptive optimization method for RPA processes. This method presents a closed-loop approach: taking the environment as input, activating as needed, adjusting according to strategies, and solidifying experience, correspondingly dividing the process into several processing steps.

[0054] Regarding the entire process of the robot used in RPA to complete the business process, the system mentioned in the article uses a bypass listening method to obtain various environmental data in real time. Considering that the stability of the script is affected by various factors (such as incorrect rendering of the interface, changes in DOM node values, and excessive CPU or memory usage of the host machine), the system aligns and normalizes the underlying data with a unified time standard, and then forms a state feature sequence from the information of each dimension. The method used ensures that different data are not distorted due to the units of measurement, and the obtained sequence can coherently express the current running status of RPA.

[0055] Traditional RPA typically requires high-performance, full-time vision and language models to handle complex real-world situations, consuming excessive resources and hindering practical application. To address this challenge, this method proposes a threshold-controlled cascading triggering scheme. This involves anomaly prediction based on relevant state characteristics, yielding a predicted value with corresponding probabilistic properties. The predicted value serves as a lightweight indicator to determine whether to invoke the high-performance components (i.e., subsequent modules).

[0056] The anomaly probability prediction mentioned in the text is compared with a preset probability threshold. Only when the predicted anomaly probability is greater than the threshold (i.e., the original static script may fail more frequently at this time) will the system suspend the established execution flow, perform multimodal analysis on the current interface, and concentrate the information from each modality into a comprehensive scene feature. Then, a judgment is made based on this feature. If the prediction does not exceed the threshold, the regular script is processed according to the low-overhead solution. This arrangement is a design that balances detection accuracy with computing resources.

[0057] When the obtained comprehensive scene features can fully express the current abnormal situation, the system begins to make strategy decisions. It compares these comprehensive features with the historical strategy features in the knowledge base, measures similarity using vectors, and finds the alternative strategy that is closest to the current interface among known successful examples.

[0058] For practical applications encountering issues such as network congestion and slow server response, the method proposed in this paper improves upon the generally fixed retry waiting time. When there is a delay in the system response related to the target substitution strategy, the retry waiting parameters for the corresponding strategy are determined based on an analysis of the overall load situation. Then, the timing of the target substitution strategy is scheduled according to these parameters (i.e., when to apply it). This precise timing avoids unnecessary polling and reduces the possibility of timeout failures.

[0059] Once an anomaly is fixed, the actual application of the target replacement strategy should be analyzed, and the execution paths involved should be updated accordingly. Whether the fix was successful should be converted into a quantitative weight and incorporated into the description of the graph topology. This transforms a one-off solution into a long-term rule that can be followed, allowing RPA to have the potential for self-improvement and self-development.

[0060] Taking the automated expense reimbursement process used in a corporate financial shared service center as an example, when the front-end of the expense reimbursement system undergoes a framework upgrade and the absolute path of the original submit button becomes invalid, the system can use a bypass acquisition method to summarize the state characteristics, discover the existing changes in the interface structure, and estimate the abnormal probability, i.e., the overprobability threshold standard. At this time, a multimodal feature fusion method is activated as needed, treating the screenshot of the error page along with the existing HTML text as scene elements. Then, the knowledge base provides alternative solutions for the relative offset of the image anchor points. Considering that this is the peak of expense reimbursement at the end of the month, and observing the delay in backend response, the waiting time for retrying is dynamically adjusted. After a slight delay, the click command is issued. The temporal visual matching encountered after the process is completed is recorded as a knowledge base update item for processing similar tasks in the future.

[0061] Furthermore, during the operation of the RPA system, the various data obtained are inconsistent in terms of sampling number, numerical value and practical meaning. If the unprocessed data is used for calculation immediately, there will be distortion. This embodiment has proposed a countermeasure to this difficulty, namely, forming a state feature sequence in the manner described.

[0062] During the RPA process, the system has arranged relevant bypass agents at the lower level to uniformly obtain multi-dimensional information about the current environment of the target terminal and business system using a given, fixed sampling time (such as once every 200 milliseconds). The above data is a vector combination of four sub-feature categories, namely:

[0063] Image features: Based on a local or overall operation screen of the RPA robot, relevant images are captured. The processor analyzes the seen content using a lightweight convolution method and converts the obtained visual information into a fixed-length (e.g., 128-dimensional) floating-point vector, which represents the state of the interface and controls.

[0064] Node structure characteristics: The characteristics discussed are determined according to the Document Object Model (DOM) tree of the current page of the target business system. That is, the depth of the target element's corresponding position in the DOM tree is used as a numerical value, and a hash is calculated in the form of a combination of basic attributes (id or class, etc.) to serve as a logical representation of the non-visual structure of the interface.

[0065] System performance metrics: Monitoring information about the resources used by the underlying operating system is data at the sampling point, namely the CPU usage ratio (in percentage), memory usage ratio (in percentage), and latency (in milliseconds) for making network calls to the business server at that time.

[0066] Process execution log: The record of the current running stack of the execution engine used by RPA is the source of the most recent automatic command status. For the constructed vector, the statuses of "success", "failure" and "running" are numerically processed as 1.0, 0.0 and 0.5 respectively.

[0067] The system manages a fixed-length sliding time window (e.g., 15 sampling steps) in memory. During each sampling, it arranges and binds four types of asynchronous data from different sources according to the global timestamp. That is, the image information, node structure, performance value and process record corresponding to the same time are used as complete one-dimensional features and stored in the buffer area occupied by the sliding window in chronological order. If a certain type of data cannot be obtained at a specific time point, linear interpolation is performed using the sampling results of the same dimension from the previous sampling to fill in the missing part without missing the corresponding time-related items.

[0068] The four initial features mentioned appear on different orders of magnitude (for example, network latency can be hundreds of milliseconds, while log identifiers are only represented by 0 and 1). Directly concatenating them would cause some dimensions to become overly prominent in subsequent calculations. Therefore, the processor needs to standardize each group of initial features using a sliding window method.

[0069] If calculated according to actual needs, each feature dimension should be processed one by one: first, calculate the average and standard deviation of each dimension in the current sliding window, then shift (i.e., subtract) each original number according to the average of that dimension, and then scale (i.e., divide) according to the standard deviation. In this way, the environmental data related to each dimension are uniformly transformed (mean is close to 0, standard deviation is close to 1), so there will be no difficulty in comparison due to the units.

[0070] After appropriately scaling the values, the four types of standardized features corresponding to each global timestamp are placed in memory in a certain order (i.e., image features, node structure, system performance, and process log) and processed as a unified vector of single-step states. Then, using the timeline as a guide (from the earliest time point in the window to the currently seen time), all such single-step states contained in the sliding window are sequentially connected and assembled into a whole sequence.

[0071] The resulting long-dimensional vector should be understood as a feature sequence of the relevant state. It eliminates the difficulty of processing various heterogeneous information and provides a complete dynamic record of the RPA operating environment over a certain period of time, which is the basic material that can be used in the subsequent anomaly prediction part.

[0072] Furthermore, when the RPA workflow causes cascading responses due to predicted anomalies, the interface conditions must be clearly described.

[0073] It should be noted that before performing the feature fusion step, the system should vectorize the two heterogeneous features in the current environment.

[0074] The visual features referred to are single-column floating-point vectors obtained by convolution processing of screenshots of the corresponding interactive areas in the current target user interface. In practical applications, a lighter convolution structure (such as MobileNet) should be used to specify the output dimension (usually 128 dimensions). This setting does not lose the basic visual elements of the UI—namely, shape, color, or small-scale texture—while also providing stricter control over the memory used for the subsequent similarity analysis.

[0075] The text semantic features discussed are one-dimensional floating-point vectors obtained by natural language processing for the text of the target element (such as button descriptions, DOM tree node hints). To extract the semantic information completely, a trained language model is generally required, so the output dimension used is generally high-dimensional (such as 768-dimensional).

[0076] The visual and textual semantic features used come from their respective independent physical semantic spaces, and there is a significant dimensional difference between them (e.g., 128-dimensional vs. 768-dimensional). They cannot be simply merged mathematically. Therefore, the system uses a fully connected layer to transform the textual semantic features into a feature space of the same dimension as the visual features, resulting in mapped textual features. From a computational perspective, this projection is a linear matrix transformation, and its basic formula is:

[0077] ;

[0078] Among them, V m The obtained mapped text features have vector dimensions that are completely consistent with the visual features; W p This represents the projection weight matrix built into the fully connected layer, where the number of rows equals the dimension of the visual features, and the number of columns equals the initial dimension of the text semantic features; V t B represents the semantic features of the input text. p This represents the bias term vector, used to correct the zero points of the data distribution.

[0079] After unifying the feature dimensions, the system performs weighted feature fusion on the mapped text features and the visual features based on the first and second weight coefficients to generate the comprehensive scene features. This fusion process uses a weighted summation algorithm, with the specific formula as follows:

[0080] ;

[0081] Among them, S cThis refers to the comprehensive scene features related to the output, which will be used as a unique identifier input into the strategy; α refers to the first type of weight coefficient, used to specify the contribution ratio of text semantics to the overall judgment. From a practical point of view, the optimal value of α is 0.2~0.7; V m V represents the known mapped text features obtained using the preceding method; β is the second-type weighting coefficient, corresponding to the proportion of visual texture in the overall judgment, and empirically, β should be around 0.3 to 0.8; v That is, the visual features being discussed.

[0082] To ensure the stability of the magnitude after feature fusion and to prevent overflow in subsequent cosine similarity calculations, the system is configured with the following constraints:

[0083] ;

[0084] And α>0, β>0.

[0085] If the proposed weighted summation method is used to process visual and textual features, it is equivalent to performing linear interpolation in the same mathematical space according to a certain proportion. This does not change the dimension of each feature, and various business needs can be met by adjusting α and β. For example, if the business system frequently changes its UI skin while the terminology itself remains unchanged, then the α used in the initial configuration can be adjusted significantly (e.g., set to 0.6); when encountering a large number of systems that rely solely on icons without text, β should be increased (e.g., to 0.7). In addition, if both the visual and textual features used are all-zero vectors, it is considered that feature extraction has failed, and a degradation method should be initiated to prevent invalid fusion and matching deadlock.

[0086] Furthermore, the next step should be to consider the comprehensive characteristics of the current interface, analyze known historical success cases, and identify the most suitable solutions. In terms of strategy retrieval requirements, the proposed knowledge base uses relational and vector databases as its primary means to achieve long-term data storage, containing records of resolved processes for specific anomalies at previous stages of the system. In terms of content, the data tables in the knowledge base have two basic mapping relationships:

[0087] The first type of historical strategy is a one-dimensional floating-point array of the same dimension (usually stored as a binary large object BLOB in the database) organized according to the various features of the comprehensive scenario.

[0088] The second key point is a strategic description of the feature bindings used, namely, a summary of the location methods and parameters that have actually worked in the past (such as image template hashes saved in JSON type, or target retrieval rules designed based on OCR).

[0089] If the system needs to use matching logic, the index used is the identifier of the current business or the node of the relevant process. At this time, the database should provide several historical strategy features as candidates, and the obtained data should be used for subsequent calculations.

[0090] In practical RPA, the user interface of the business system used may change due to changes in resolution or page layout, which often leads to fluctuations in the absolute value of the feature vector. To avoid matching errors caused by numerical magnitude, the processor does not use distance as a direct indicator, but instead compares the current scenario (i.e., the query) features with the features of each historical strategy in the knowledge base using the cosine method.

[0091] In terms of actual computation, the processor used examines each of the selected historical strategy features, performs dot product processing on the current overall scene feature vector and the vectors corresponding to each historical strategy, and then divides the result by the product of the two moduli.

[0092] The calculated results are expressed as floating-point numbers in the range of 0 to 1. When the value approaches 1, the angle between the two multimodal features being analyzed on the vector is relatively small, and the historical situation is closer to the present situation (in terms of features).

[0093] After all candidate features have been calculated, the processor stores the obtained similarity values ​​in memory and selects the highest cosine similarity. The corresponding existing record should be the closest known repair instance or experience in the knowledge base, given the current situation.

[0094] Once the maximum similarity value is obtained, the system needs to make a judgment to avoid misusing or incorrectly applying other irrelevant strategies. At this point, the processor needs to consult the preset similarity threshold given by the underlying configuration (in actual engineering, a constant value with a fixed value in the range of 0.80 to 0.90 is often used). The threshold used is the boundary of the degree of similarity of the relevant features. The processor compares the found maximum value with this threshold, and only when the maximum value exceeds the set threshold is the current situation considered to have a large similarity to the past, that is, a conclusion of high matching is made.

[0095] When processing in this way, the historical strategy characteristics corresponding to the found maximum value should be understood as the alternative strategy to be sought. Then, all specific instructions related to the strategy are sent to the work queue used by the underlying execution engine of RPA, automatically skipping steps that cannot be completed temporarily in the process without rewriting the static script.

[0096] When the maximum value being compared does not reach the predetermined threshold, it should be considered that the current environment has changed significantly and existing knowledge cannot be used as a reference. At this time, the matching branch should be interrupted and the relevant transactions should be handed over to the degradation processing method described later.

[0097] Furthermore, for the selected target alternative, if the backend processing of the corresponding business is difficult or the network is unstable, the RPA instructions generally will not immediately obtain the execution result.

[0098] To address the issues raised and achieve dynamic control over the execution timing, the processor must initially store both the state and action spaces into memory in an appropriate manner.

[0099] The state space mentioned in the text is a two-dimensional mapping hash table, where the process steps and the discretized description of the system load are used as the composite primary key. The corresponding results are the various waiting parameters during candidate retries and the value of the actions at this time (Q value). To make the actual environment an analyzable load level object, the system has provided a corresponding linearly weighted evaluation method, specifically the following calculation formula:

[0100] ;

[0101] Among them, L c R represents the currently calculated overall system load rate (normalized to the [0,1] interval); c R m R n W1, W2, and W3 represent the normalized values ​​of the currently collected CPU utilization, memory usage, and network latency, respectively; W1, W2, and W3 represent the corresponding preset environmental weight coefficients, and satisfy W1+W2+W3=1.

[0102] The processor, based on the derived L c As a result, based on the preset numerical range (such as L) c ≤0.3 indicates low load, 0.3 <L c ≤0.7 indicates a medium load, L c >0.7 indicates high load), which is mapped to a specific discretization level identifier and combined with the current step identifier to uniquely determine the current state of the system.

[0103] Preferably, the preferred value range for each weight coefficient is: W1 preferably 0.4~0.5, W2 preferably 0.3~0.4, and W3 preferably 0.1~0.2. In this embodiment, W1=0.4, W2=0.4, and W3=0.2 are preferred. The response speed of RPA process execution is mainly affected by the CPU and memory resources of the device where the business system is located, and network latency is a secondary influencing factor. Therefore, allocating higher weights to CPU and memory can ensure a higher degree of matching between the calculation result of the overall load rate and the actual system response capability.

[0104] The action space is represented as a preset one-dimensional array containing multiple floating-point values, such as the set {0.5, 1.0, 2.0, 5.0, 10.0}, with the unit being seconds.

[0105] In a given state, after the system extracts a certain waiting time from the action space and executes it, the processor calculates the feedback reward value after the action is executed based on a preset reward function. This function converts the execution result into a scalar value, and the specific calculation formula is as follows:

[0106] ;

[0107] Among them, R v The calculated feedback reward value is represented by I1, which represents the execution success status indicator. If the strategy is successfully executed after this wait, the value is 1, otherwise it is 0. α1 is the weight of the corresponding positive reward value for successful execution. T1 represents the actual waiting time consumed this time, and α2 is the weight of the corresponding negative penalty value for waiting time. I2 represents the cumulative number of retries for the current node, and α3 is the weight of the corresponding negative penalty value for the number of retries.

[0108] By giving a positive reward for a successful state that is in the lead, the process can be ensured to proceed smoothly; at the same time, negative penalties for time and number of retries are introduced to force the system to minimize the invalid thread blocking time while ensuring the success rate.

[0109] Preferably, the preferred value range of α1 is 8~12, the preferred value range of α2 is 0.5~1.5, and the preferred value range of α3 is 2~4. In this embodiment, α1=10, α2=1, and α3=3 are preferred. The core objective of this invention is to improve the success rate of process execution. Therefore, the highest weight coefficient is set for successful execution to ensure that successful execution can obtain the highest positive reward and guide the system to prioritize the waiting time that can guarantee successful execution.

[0110] The penalty coefficient for waiting time is set to a small value. Under the premise of ensuring successful execution, the system is guided to choose a shorter waiting time, thereby improving execution efficiency. At the same time, it avoids the system sacrificing the success rate in order to shorten the waiting time due to excessively high penalty weight.

[0111] The penalty coefficient for the number of retries should be set to a moderate value, thereby encouraging the system to perform fewer retries and not to treat resource waste and slow processes as acceptable situations.

[0112] When the reward received is R v At that time, based on the temporal difference method, an iterative value analysis should be performed on each action in the action space, and update rules should be given accordingly, namely:

[0113] ;

[0114] Where Q1 is the latest definition of action value; Q0 is the record of past values ​​of the action in the current state (i.e., cache); η is a constant that adjusts the learning rate to appropriately balance new and old information; γ is the time discount factor; Q max This refers to the maximum action value that each possible action can achieve in the next state. The time discount factor γ is best taken in the range of 0.8 to 0.95. In this example, γ=0.9 is appropriate. It helps to properly coordinate the immediate reward for the action performed now with the long-term benefits of the future state. When γ is too small (i.e. less than 0.8), the system only pays attention to the one-time result at the current point. The decision made may not avoid the overall adverse consequences such as congestion in subsequent business links due to too many retries or short-term aggressiveness.

[0115] When γ is too large (like 0.95 or larger), the corresponding system focuses too much on long-term benefits that may change in the future. In this case, the estimation of the value of actions is not easy to stabilize in a dynamic RPA environment. If γ is set to 0.9, it can maintain high requirements for the quality of current actions (efficiency, success rate) and appropriate consideration for each link of the overall process. This perfectly meets the integrated need for multi-faceted and whole-process optimization in long process automation under complex conditions.

[0116] It should be pointed out that, from the perspective of the above closed-loop operation, the value table of relevant actions in system maintenance has taken into account the benefits corresponding to various load conditions. In actual application, when a certain type of business is encountered, the processor will locate the state according to the current load, perform Q1 analysis on all corresponding actions, and use the general maximum method (Argmax) to find the maximum item (e.g., 2.0 seconds) as the gear.

[0117] The proposed value is treated as a retry waiting parameter by the system. At this point, an instruction is sent to the underlying operating system thread scheduler to cause the RPA executor to sleep for exactly 2.0 seconds before starting the alternative solution. Using a value judgment method for dynamic scheduling is a systematic solution that completely eliminates the need for manually specifying the waiting time.

[0118] Furthermore, the system uses a knowledge base to construct and store graphs related to state transitions, i.e., logical models of underlying strategies and global paths. Following this line of thought, the state transition graph is a type of directed weighted graph in the database.

[0119] According to the definition in the diagram, the so-called state node is a checkpoint in the user interface state or process. In actual engineering, the unified resource identifier (URL) and window handle involved in the current business interface, as well as the hash value of important DOM elements, should be used for overall analysis. The result is used as the unique identifier of the state node, so that the same business interface can be identified no matter when it is run and will not change over time. The connecting edge is the execution strategy itself, set in the direction from the starting point to the ending point. Each edge is composed of the two end nodes together as the primary key. The relevant attributes should provide a description of the command sequence used, the total number of executions, the number of successful executions, and the target weight.

[0120] To express the reliability of the execution strategy in a manner permitted by graph algorithms, the system should dynamically adjust the target weights of relevant edges using internal methods based on the known success rates of alternative strategies. The corresponding calculation formula is:

[0121] ;

[0122] Among them, W e The target weights are calculated and should be understood as the attribute values ​​of the edges used in the state transition diagram; B0 is a known baseline value, generally taken as a fixed number of 1; S r This refers to the actual execution success rate, calculated by dividing the number of successful executions in the past by the total number of executions. The value falls within the range [0, 1]. ϵ is a preset compensation amount used to avoid zero weight, and it is advisable to use a very small positive floating-point number (such as 0.01) as its value.

[0123] When the weight W involved in a single edge e After completing the calculations and inputting the results into the state transition diagram, the relevant global path optimization can be used for the subsequent end-to-end process. For each reachable path from the starting point to the ending point given in the state transition diagram, the processor needs to adjust the relevant cumulative weight W. p Add them up:

[0124] ;

[0125] Where n is the total number of state nodes contained in the reachable path, W e (s i ,s i+1 ) indicates from state node s i To state node s i+1 The target weights corresponding to the connecting edges.

[0126] If the shortest path search method is used, the processor needs to examine the total weight W corresponding to each reachable path encountered. p The path with the least weight is identified by minimizing the value, and then it is decomposed into ordered instructions and sent to the RPA execution engine.

[0127] Regarding the long process of "candidate resume entry" in the human resources automation system, there are two simultaneously valid logics in the state transition diagram from "list page" to "details page": connection edge A is located by XPath, and connection edge B uses the image template method.

[0128] If the front-end of the business system is slightly modified, the success rate S of connecting edge A will increase. r It then begins to decrease to 0.5, and the target weight calculated according to the formula is 0.51 at this point; for connecting edge B, the success rate has always been 0.95, and the target weight becomes 0.06.

[0129] If a subsequent task related to data entry occurs, the total weight of edge A encountered in the shortest path search will be significantly greater than that of edge B. The system, based on security logic, avoids this high-risk point and uses edge B instead for the workflow. In this case, for RPA, the operations dashboard should display a structured trace log.

[0130] The global optimization test has already pointed out that some edges of the original route are too poor (degraded).

[0131] The shortest path algorithm for dynamic rerouting has now been recalculated, using a method that avoids high-weight edges, thus transforming the global path into [visually matched route 02];

[0132] The current topology and path have been stored in the knowledge base. The so-called closed-loop mechanism allows the system to operate according to certain logic without interruption, guiding the RPA workflow in a direction that consistently maintains the highest success rate.

[0133] Furthermore, regarding the strategy matching used in the knowledge base, if it is to be implemented and the target business undergoes a significant redesign, then the resulting scenario features may differ greatly from the previous knowledge base features.

[0134] If the processor has already calculated the maximum cosine similarity (i.e., the current maximum similarity is 0.5, while the preset similarity threshold is set at 0.85) and the conditions are still not met, the system will treat this situation as an unknown state and will not use the corresponding target substitution strategy. In this case, vector matching will be excluded, and the system will instead use full text search or the idea of ​​using node edit distance to determine whether a suitable object has been found and whether the desired matching result has been achieved.

[0135] Depending on the specific situation, the so-called global text search involves the processor directly examining the underlying DOM tree of the current interface, using regular expressions to match readable text, and processing based on node edit distance when the interface has replaced the text with a wordless icon: the system uses the absolute path of the control carried by the original static script (i.e., in the form of / html / body / div / form / button) as a benchmark to find the possible paths of each control of the same type, and then calculates the edit distance of each of them from the original path (the value is determined by the Levenshtein method).

[0136] When a candidate path (such as an additional layer of structure created by changes in the front-end framework, i.e., / html / body / div / section / form / button) has a close editing relationship with the original path (the distance is 2, which is lower than the set underlying threshold of 3), the system considers it a target element and provides a temporary positioning method, thereby providing actual protection or continuation for the process that may be interrupted in an undesirable state.

[0137] If the broad matching logic listed cannot locate the business node, the system records the number of failures in memory and determines whether to attempt to locate it based on the set number of consecutive attempts (usually 3 in actual engineering). To avoid business errors caused by meaningless retries, the processor will forcibly terminate the original execution path and rewind the process to the most recently determined checkpoint.

[0138] A stable checkpoint is a snapshot of the complete context (i.e., page session tags, existing form content, and related caches) and the old state of the DOM tree that the RPA process relies on before starting high-risk operations. If the current data in memory is cleared and restored according to this snapshot, the actual business situation will return to the previously determined normal point. After the processor registers the exception for the corresponding process node in the background database, it will actively stop running. This keeps the error within a certain local range and prevents it from affecting subsequent nodes. The state at this point will be considered for release or correction when the state transition diagram or human-machine interface provides new instructions.

[0139] If the cumulative failure rate of the execution route used has reached or exceeded the allowed limit (i.e., the total number of failures in the recent period is more than 30% of the total business volume), then the relevant external business interface is likely to be out of service or have obvious faults.

[0140] The situation encountered was a highly serious anomaly in the entire process. In this case, the processor must activate the highest level of security defense measures: taking a snapshot and suspending the current process. The relevant anomaly snapshot is the result of organizing the error scene in a structured manner, providing the screen image at this time, network latency values, and stack traces related to the error. The system stores the snapshot as a log entry in the database and simultaneously transmits the information to the operation and maintenance console interface via an interrupt.

[0141] The present invention also provides an RPA process adaptive optimization device, comprising the following units:

[0142] The data acquisition unit is used to acquire multi-dimensional environmental data during the process execution and generate a state feature sequence based on the multi-dimensional environmental data. The unit is deployed in a bypass proxy mode and collects images of the current user interface, DOM node structure, system performance indicators and process execution logs at a fixed sampling period. After global timestamp alignment and Z-Score standardization, the data is spliced ​​and output as a state feature sequence.

[0143] An anomaly warning unit is used to predict anomalies based on the state feature sequence and obtain an anomaly probability prediction value. This unit encapsulates a time-series prediction model to calculate the anomaly probability prediction value of the current state sequence. When the prediction value is greater than a preset probability threshold, an activation signal is sent to the subsequent unit.

[0144] The feature recognition unit is used to acquire the multimodal features of the current target user interface when the predicted abnormal probability value is greater than the preset probability threshold, and to fuse the multimodal features into comprehensive scene features. After activation, the unit calls the visual feature extractor and the text semantic feature extractor, maps the text semantic features to the same dimension as the visual features through a fully connected projection layer, and then performs weighted summation according to the set weight coefficients to output comprehensive scene features.

[0145] The strategy decision unit is used to match the comprehensive scenario features with the historical strategy features in the knowledge base to determine the target replacement strategy; when the system response corresponding to the target replacement strategy is in a delayed state, it determines the retry waiting parameter corresponding to the target replacement strategy, and controls the execution sequence of the target replacement strategy according to the retry waiting parameter. This unit determines the target replacement strategy through cosine similarity retrieval, and determines the retry waiting parameter according to the current load and process steps through a reinforcement learning parameter tuner, and encapsulates the strategy and parameters into control commands for issuance.

[0146] The path optimization unit is used to update the execution path in the knowledge base according to the execution result of the target substitution strategy. The unit obtains the execution result, calculates the execution success rate, updates the weight of the connecting edge in the state transition graph according to the preset benchmark value, the execution success rate and the preset compensation value, and triggers the shortest path search algorithm to redetermine the execution path.

[0147] The present invention also provides an RPA process adaptive optimization system, comprising: a processor, a memory, a communication interface, and a database.

[0148] The memory stores a computer program, and the processor executes the computer program to implement the above method. The database is used to store the state transition diagram and historical policy feature set in the knowledge base. The communication interface is used to receive multi-dimensional environmental data reported by the external execution terminal and send control commands to the external execution terminal to support the external execution terminal in path optimization and policy matching.

[0149] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method. The computer-readable storage medium is a non-transitory data storage carrier, including but not limited to solid-state drives, hard disk drives, random access memory, read-only memory, USB flash drives, or optical discs.

[0150] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An adaptive optimization method for RPA processes, characterized in that, Includes the following steps: Acquire multidimensional environmental data during the process execution, and generate a state feature sequence based on the multidimensional environmental data; Anomaly prediction is performed based on the state feature sequence to obtain an anomaly probability prediction value; When the predicted anomaly probability value is greater than a preset probability threshold, the multimodal features of the current target user interface are obtained, and the multimodal features are fused into comprehensive scene features; Based on the comprehensive scenario features and the historical strategy features in the knowledge base, a target replacement strategy is determined; when the system response corresponding to the target replacement strategy is in a delayed state, a retry waiting parameter corresponding to the target replacement strategy is determined, and the execution sequence of the target replacement strategy is controlled according to the retry waiting parameter; The execution path in the knowledge base is updated based on the execution result of the target substitution strategy.

2. The RPA process adaptive optimization method according to claim 1, characterized in that, The generation of the state feature sequence based on the multidimensional environmental data includes: The multidimensional environmental data is time-aligned according to a preset time window and a global timestamp to obtain multiple initial features; The multiple initial features are standardized, and the standardized initial features are concatenated in chronological order to form the state feature sequence. The multidimensional environmental data includes image features, node structure features, system performance indicators, and process execution logs.

3. The RPA process adaptive optimization method according to claim 1, characterized in that, The multimodal features include visual features and textual semantic features; the fusion of the multimodal features into comprehensive scene features includes: The text semantic features are projected onto a feature space of the same dimension as the visual features through a fully connected layer to obtain mapped text features; Based on the first weighting coefficient and the second weighting coefficient, the mapped text features and the visual features are weighted and fused to generate the comprehensive scene features.

4. The RPA process adaptive optimization method according to claim 1, characterized in that, The process of matching the comprehensive scenario features with historical strategy features in the knowledge base to determine the target alternative strategy includes: Calculate the cosine similarity between the comprehensive scene features and multiple historical strategy features in the knowledge base, and extract the maximum value among the cosine similarities; When the maximum value is greater than a preset similarity threshold, the strategy to which the historical strategy feature corresponding to the maximum value belongs is determined as the target replacement strategy.

5. The RPA process adaptive optimization method according to claim 1, characterized in that, The determination of the retry waiting parameters corresponding to the target substitution strategy includes: Obtain the state space and action space. The state space is composed of a combination of discretized system load and process steps, and the action space contains multiple discretized waiting times. The feedback reward value after the action is executed is calculated based on a preset reward function, and the action value corresponding to each action in the action space is iteratively updated based on the feedback reward value. The preset reward function is composed of a weighted sum of a positive reward value for successful execution, a negative penalty value for waiting time, and a negative penalty value for retry times. The target waiting time is determined from the action space based on the iteratively updated action value and used as the retry waiting parameter.

6. The RPA process adaptive optimization method according to claim 1, characterized in that, The step of updating the execution path in the knowledge base based on the execution result of the target substitution strategy includes: A state transition graph is constructed in the knowledge base, wherein user interface states or process checkpoints are defined as state nodes and execution strategies are defined as connection edges. Obtain the execution success rate of the target substitution strategy; Based on the preset benchmark value, the execution success rate, and the preset compensation value to prevent zero weight, the difference between the preset benchmark value and the execution success rate is added to the preset compensation value to calculate the target weight, and the target weight is updated to the connection edge corresponding to the target substitution strategy in the state transition diagram. Based on the shortest path search algorithm, the path with the smallest cumulative weight is determined as the execution path in the updated state transition graph.

7. The RPA process adaptive optimization method according to claim 4, characterized in that, The method also includes a degradation handling mechanism for dealing with location anomalies: When the maximum value is not greater than the preset similarity threshold and no target replacement strategy is matched, global text search or matching logic based on node edit distance is invoked to locate the target. If the positioning fails for a preset number of consecutive times, the current process will be rolled back to the previous stable checkpoint, the current process node will be marked as abnormal, and subsequent execution will be suspended. When the cumulative failure rate of the current execution route exceeds the tolerance threshold, abnormal snapshot data is generated and the current process is blocked.

8. An RPA process adaptive optimization device, characterized in that, include: The data acquisition unit is used to acquire multi-dimensional environmental data during the process execution and generate a state feature sequence based on the multi-dimensional environmental data. An anomaly warning unit is used to predict anomalies based on the state feature sequence and obtain an anomaly probability prediction value. The feature recognition unit is used to acquire the multimodal features of the current target user interface when the anomaly probability prediction value is greater than a preset probability threshold, and to fuse the multimodal features into a comprehensive scene feature; The strategy decision unit is used to match the comprehensive scenario features with the historical strategy features in the knowledge base to determine the target replacement strategy; when the system response corresponding to the target replacement strategy is in a delayed state, it determines the retry waiting parameter corresponding to the target replacement strategy and controls the execution sequence of the target replacement strategy according to the retry waiting parameter. The path optimization unit is used to update the execution path in the knowledge base based on the execution result of the target substitution strategy.

9. An RPA process adaptive optimization system, characterized in that, include: Processor, memory, communication interface, and database; The memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7; The database is used to store the state transition diagrams and historical strategy feature sets in the knowledge base; The communication interface is used to receive multi-dimensional environmental data reported by the external execution terminal and to send control commands to the external execution terminal to support the external execution terminal in path optimization and strategy matching.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images