Intelligent process automation robot system based on business rule engine

Abstract

The application relates to a service rule engine-based intelligent process automation robot system and relates to the technical field of process automation.The application comprises a service rule management module, a sensing semantic analysis module, a rule engine core module, a process automation robot executor and a dynamic path planning module; the service rule management module supports hot deployment of rules; the sensing semantic analysis module generates a context snapshot through computer vision and natural language processing; the rule engine core module performs rule matching based on a Rete algorithm and outputs real-time decision instructions; the executor is embedded with a pre-hook and a post-hook to form a closed-loop feedback; the dynamic path planning module generates an alternative path based on heuristic search when a path is interrupted; the application realizes dynamic hot loading of service rules, intelligent sensing of interface elements, closed-loop feedback of an execution process and path self-recovery in an abnormal scenario, and significantly improves the robustness and adaptability of an automated process.

Description

Technical Field

[0001] This invention relates to the field of computer process automation technology, and more specifically, to an intelligent process automation robot system based on a business rule engine. Background Technology

[0002] Robotic Process Automation (RPA) technology automates repetitive business processes by simulating human operations on a computer interface, and is widely used in finance, government affairs, manufacturing, healthcare, and other fields. Existing RPA systems typically use recording and playback or scripting to automate processes; developers pre-record user actions or write automation scripts, which the robot then executes repeatedly according to preset logic. However, existing RPA systems have the following technical shortcomings in practical applications: The high coupling between rules and execution leads to high maintenance costs. Traditional RPA robots typically hard-code business rules, such as approval thresholds, data validation logic, and exception handling methods, into automated scripts. When business rules change, developers need to modify the scripts, retest, and redeploy, resulting in high system maintenance costs and long response cycles. This is especially problematic in scenarios where business rules change frequently, making it impossible to achieve zero-interruption iteration of business processes. Furthermore, existing RPA systems lack dynamic perception and adaptive capabilities. If they encounter changes in user interface elements, abnormal pop-up windows, or abnormal data formats during execution, they often directly report errors and terminate the process, lacking real-time decision-making capabilities based on runtime context semantics and unable to dynamically adjust execution paths to adapt to environmental changes. The disconnect between decision-making and execution results in low levels of intelligence. While some existing improvement solutions introduce artificial intelligence models for interface recognition, there is a lack of deep integration between the rule engine and the automated executor. The rule engine typically only serves as a pre-judgment tool and cannot provide real-time intervention, path correction, or exception self-healing during robot operation, leading to insufficient robustness in the face of complex and ever-changing business scenarios. To address the aforementioned problems, an intelligent process automation robot system and its control method based on a business rule engine need to be designed to solve the technical problems of rule changes requiring downtime for maintenance, inability to cope with dynamic interface changes, and lack of anomaly self-healing capabilities in existing technologies. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of this invention is to provide an intelligent process automation robot system based on a business rule engine to solve the problems existing in the background technology.

[0004] The above-mentioned technical objective of this invention is achieved through the following technical solution: an intelligent process automation robot system based on a business rule engine, comprising: The business rules management module is used to define, store, and manage business rule sets in a visual manner; The Perception Semantic Parsing Module is used to capture runtime interface images in real time, use computer vision algorithms to identify the state, position and attributes of interface elements, and use natural language processing algorithms to perform semantic understanding of interface text, generating a context snapshot containing structured interface element information and semantic tags. The core module of the rule engine is communicatively connected to the business rule management module and the perception semantic parsing module. The core module of the rule engine has a built-in rule matching network based on the Rete algorithm, which is used to receive the context snapshot, perform pattern matching between the context snapshot and the condition rules, decision rules and exception handling rules in the business rule set, and output a real-time decision instruction including operation type, operation parameters, target element positioning strategy and exception fallback strategy. The process automation robot actuator is communicatively connected to the rule engine core module to parse and execute the real-time decision instructions, simulating human-computer interaction. The actuator is embedded with a pre-hook and a post-hook, which are used to send the current execution status to the rule engine core module for secondary confirmation before the operation is executed, and to feed back the execution result to the rule engine core module after the operation is executed. The dynamic path planning module is communicatively connected to the rule engine core module and the process automation robot actuator, respectively. It is used to maintain an operation path topology graph with interface elements as nodes and operation reachability relationships as edges. When the original operation path is detected to be interrupted, it calculates the shortest path from the current interface reachable node to the target node in the operation path topology graph based on a heuristic search algorithm and generates an alternative operation path.

[0005] Optionally, the business rule management module includes: The rule editor unit provides a graphical user interface, enables users to define rules in their own format, and compiles the defined rules into rule files in a preset format. A rule storage unit is used to store the rule files in a relational database or a distributed configuration center; The rule hot-loading unit is used to listen for change events in the rule storage unit, obtain rule change notifications through a message queue or publish-subscribe mechanism, and dynamically load the updated rules into the rule matching network of the rule engine core module, while keeping the currently executing automated process instance unaffected.

[0006] Optionally, the perceptual semantic parsing module includes: The computer vision unit is used to perform real-time reasoning on interface screenshots using a convolutional neural network-based object detection model, identify control types, control bounding boxes, control visibility, and control enable states, and output standardized control objects. An optical character recognition unit, coupled to the computer vision unit, is used to extract text regions in the interface and associate the recognized text content with the corresponding control object; The natural language processing unit is used to perform intent classification and keyword extraction on the text content extracted by the optical character recognition unit and the system pop-up text using a pre-trained language model based on the Transformer architecture, and to generate the semantic tags. The context aggregation unit is used to assemble the control object, the text content, and the semantic tags into a tree-like or graph-like context snapshot according to the interface hierarchy.

[0007] Optionally, the core module of the rules engine includes: The rule compilation unit is used to parse the rule file provided by the business rule management module into an abstract syntax tree and generate Alpha and Beta nodes of the Rete network. The fact-judgment unit is used to receive the context snapshot and send each data item in the context snapshot as a fact judgment to the Rete network; The conflict resolution unit is used to select one rule to trigger from the multiple matched rules when multiple rules are matched simultaneously in the Rete network, based on a preset conflict resolution strategy. The conflict resolution strategy includes a sorting strategy based on rule priority, a strategy based on the most recent update time, a strategy based on rule complexity, or a strategy based on rule usage frequency. The instruction generation unit is used to parse the action part of the triggered rule into an executable instruction object. The instruction object includes an operation type enumeration, a target control locator chain, a timeout parameter, a retry count parameter, and an exception compensation action list.

[0008] Optionally, the target control locator chain includes multiple locating strategies ordered by priority. Specifically, the multiple locating strategies are: a locating strategy based on control identifier, a locating strategy based on XPath, a locating strategy based on CSS selectors, a locating strategy based on control text content, a locating strategy based on the relative position of the control, and a locating strategy based on image feature matching. When generating the instruction object, the core module of the rule engine dynamically adjusts the priority order of each locating strategy in the locator chain according to the attribute integrity of the control in the context snapshot.

[0009] Optionally, the dynamic path planning module includes: The topology graph construction unit is used to scan the user interface structure of the target application during the system initialization phase or the process orchestration phase, construct the operation path topology graph with interface elements as nodes and navigation relationships between elements as directed edges, and record the unique identifier, locator set and node type metadata of each node. The path caching unit is used to cache historically successfully executed operation paths and record the execution success rate and average execution time for each cached path; The path search unit, upon receiving a path interruption signal, retrieves the current real-time node set as the starting node set and the unique identifier of the target node as the target node, using A... The algorithm or Dijkstra's algorithm performs path search in the operation path topology graph, and the execution success rate of the cached path in the path cache unit is used as a heuristic weight during the search process. The path verification unit is used to simulate the preliminary steps of the candidate alternative path in a sandbox environment after the path search unit generates the candidate alternative path, verify the validity of the path, and submit the verified path to the process automation robot executor.

[0010] Optionally, the process automation robot actuator includes: The instruction parsing unit is used to receive the real-time decision instruction and convert the real-time decision instruction into an operation sequence that can be recognized by the underlying automation framework. The operation execution unit is used to call the application programming interface of the underlying automation framework to execute the operation sequence, and record the start time, end time and execution result of each operation during the execution process; The status monitoring unit is used to monitor system logs, interface changes and abnormal pop-ups generated during the execution process. When a preset abnormal pattern is detected, the post-hook is actively triggered to feed back the abnormal information to the rule engine core module. The execution log unit is used to record the operation type, target control, execution result, time consumption information and exception stack of each operation as a structured log, and output the structured log to the audit system.

[0011] An intelligent process automation control method based on the above-mentioned intelligent process automation robot system based on a business rule engine includes the following steps: Step S1: Load the business rule set through the business rule management module, compile the rules into a Rete network and load it into memory to establish a rule memory image; Step S2: Obtain the visual and text features of the target interface through the perceptual semantic parsing module, and construct a runtime context containing an interface element tree and semantic tags; Step S3: The runtime context is used as a fact judgment in the Rete network of the rule engine core module to trigger rule matching and generate a real-time decision instruction containing operation type, location strategy chain and exception fallback strategy; Step S4: The process automation robot executor parses the real-time decision instruction, triggers a pre-hook to send the current status to the rule engine core module for secondary confirmation before execution, and executes the automated operation after confirmation. Step S5: After the process automation robot actuator performs the operation, it triggers a post-hook to feed back the execution result to the core module of the rule engine. If the execution result does not match the expectation or an interface abnormality is detected, the abnormal handling rule in the business rule set is triggered to generate a compensation operation or call the dynamic path planning module to switch to an alternative path. Step S6: Record the rule matching trajectory, decision instruction sequence and operation execution log throughout the entire execution process, and generate a visual audit report.

[0012] Optionally, the trigger rule matching in step S3 specifically includes: Step S3.1: Insert each interface element in the runtime context as an independent fact object into the Alpha node of the Rete network and perform single-condition matching; Step S3.2: Pass the fact object matched by the Alpha node to the Beta node for multi-condition cross-matching and variable binding; Step S3.3: When a fact object satisfies all the conditions of any rule, put the rule and its bound variable values ​​into the agenda queue; Step S3.4: The conflict resolution unit selects a rule from the agenda queue according to the preset conflict resolution strategy, triggers it, and executes the action part of the rule.

[0013] Optionally, step S5, which involves invoking the dynamic path planning module to switch to an alternative path, specifically includes: Step S5.1: Receive the path interruption signal sent by the process automation robot actuator and the real-time control tree of the current interface through the dynamic path planning module; Step S5.2: Using the dynamic path planning module, the operation path topology is traversed from the root node of the real-time control tree to the target operation node. Step S53: Use A The algorithm calculates the path cost, and the formula for calculating the path cost is: F(n)=G(n)+H(n), where G(n) is the actual cost from the starting point to the current node, calculated based on the number of operation steps and the historical execution success rate, and H(n) is the heuristic cost from the current node to the target node, calculated based on the spatial distance and semantic similarity between interface elements. Step S5.4: Select the path with the lowest calculated cost as the candidate alternative path, and verify the reachability of the first N nodes in the candidate alternative path in the sandbox environment through the path verification unit, where N is a preset positive integer; Step S5.5: Update the verified alternative path to the execution queue of the process automation robot actuator, and mark the original path as a failed path.

[0014] In summary, the present invention has the following beneficial effects: 1. Decoupling of rules and execution enables hot deployment. By setting up a business rule management module and a rule hot loading unit, dynamic loading of business rules is achieved. Rule updates can be completed without stopping system operation or interrupting the current automated process, reducing maintenance costs and achieving zero-interruption iteration of business processes.

[0015] 2. By acquiring runtime context in real time through the semantic parsing module, making real-time decisions through the rule engine core module, and forming a closed-loop feedback mechanism through the pre-hooks and post-hooks of the executor, the system achieves real-time monitoring and dynamic adjustment of execution results, thereby improving the system's adaptability to environmental changes.

[0016] 3. A locator chain containing multiple positioning strategies is adopted, and the priority order is dynamically adjusted according to the property integrity of the control in the context snapshot, which effectively solves the positioning failure problem caused by changes in interface elements and improves the stability of automated operations.

[0017] 4. The operation path topology is maintained through the dynamic path planning module. When the original operation path is interrupted, an alternative path is automatically generated based on the heuristic search algorithm, which realizes the self-healing capability in abnormal scenarios, avoids manual intervention, and ensures the continuity and reliability of the automated process. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the system flow architecture of the present invention; Figure 2 This is a schematic diagram of the system implementation method of the present invention; Figure 3 This is a schematic diagram of the workflow of the perceptual semantic parsing module of the present invention; Figure 4 This is a schematic diagram of the workflow of the core module of the rule engine of this invention; Figure 5 This is a schematic diagram of the workflow of the dynamic path planning module of the present invention. Detailed Implementation

[0019] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

[0020] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0021] In this invention, unless otherwise expressly specified and limited, "above" or "below" a second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of a second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" of a second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature. The terms "vertical," "horizontal," "left," "right," "above," "below," and similar expressions are for illustrative purposes only and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0022] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] This invention provides an intelligent process automation robot system based on a business rule engine, such as... Figure 1 As shown, it includes: a business rule management module, a semantic perception and parsing module, a rule engine core module, a process automation robot executor, and a dynamic path planning module.

[0024] In a specific embodiment, the business rule management module defines and manages the business rule set in a visual manner, while also enabling hot deployment of rules; the perception semantic parsing module captures the interface image of the target system in real time and generates a structured context snapshot through computer vision and natural language processing technologies; the rule engine core module receives this context snapshot, performs pattern matching with the business rule set, and outputs real-time decision instructions; the process automation robot executor parses and executes the decision instructions, and implements secondary confirmation before execution and result feedback after execution through pre-hooks and post-hooks; the dynamic path planning module maintains the operation path topology map and automatically generates an alternative path when the original path is interrupted; the above modules together constitute a closed-loop control loop of perception, decision-making, execution, and feedback.

[0025] The business rule management module is used to define, store, and manage business rule sets in a visual manner, and to dynamically load updated business rule sets into memory without stopping system operation. In a preferred embodiment, the business rule management module includes a rule editor unit, a rule storage unit, and a rule hot-loading unit. These three units work together to achieve full lifecycle management of business rules: the rule editor unit is responsible for defining and editing rules; the rule storage unit is responsible for persistent storage and version management of rules; and the rule hot-loading unit is responsible for dynamic updates and uninterrupted deployment of rules. The rule editor unit provides a web-based graphical user interface, enabling the definition of rules in the form of condition-action pairs, and compiling the defined rules into rule files in a preset format. In the specific rule definition method, the rule editor unit adopts a visual rule editing interface, which includes, but is not limited to, the following components: Rule list area: Displays defined rules in a table format, with each rule showing its name, status (enabled / disabled), priority, last modification time, and last modifier information; Rule editing area: Provides a rule name input box, a condition configuration area, and an action configuration area; Condition configuration area: Configures conditions using a field-operator-value triple format, enabling compound conditions (AND / OR logical combinations) and allowing nested condition groups; Action configuration area: Configures single or multiple actions, including interface operations (click, input, scroll), flow control (jump, terminate, wait), and data processing (extraction, transformation, validation), etc. Rule editor unit 110 implements the following operators: equal to (=), not equal to (≠), greater than (>), less than (<), greater than or equal to (≥), less than or equal to (≤), contains, not contains, regular expression matching, is null, is not null; The rule editor unit allows users to define custom rule priorities, with priority values ​​ranging from 1 to 1000, where higher values ​​indicate higher priority. When multiple rules meet the same condition, the rule with the higher priority is triggered first. In the next step, the rule editor unit compiles the user-defined rules into a rule file in a preset format. In this preferred embodiment, the preset format adopts the DRL format, which has good readability and execution efficiency. The rule compilation process includes the following steps: Step A1: Parse the conditional expression configured by the user in the graphical interface and convert it into the conditional part of the DRL syntax; Step A2: Parse the action expression configured by the user in the graphical interface and convert it into the action part in the DRL syntax; Step A3: Encapsulate the rule metadata (rule name, priority, rule group, etc.) into the header of the DRL rule to generate a complete rule file; The rule editor unit compiles multiple rules into a single rule file, forming a rule set. Rule set files have the .drl extension and are stored according to a preset directory structure.

[0026] The rule storage unit is used to persistently store the rule files generated by the rule editor unit to a relational database or distributed configuration center, and to implement rule version management; The rule storage unit adopts a dual-storage architecture, simultaneously implementing a relational database and a distributed configuration center. The choice can be made according to the deployment environment: Relational database storage mode: using MySQL or PostgreSQL database to create rule storage tables; Distributed configuration center mode: using ZooKeeper, Etcd, or Apollo as the configuration center, storing rule files as configuration nodes. The rule storage unit adopts a version management mechanism. Each time a rule is changed, a new version number is automatically generated. The version number is an incrementing integer with the format {major}.{minor}.{patch}, where: major: major version number, indicating major incompatible changes; minor: minor version number, indicating backward compatible functional additions; patch: revision number, indicating backward compatible bug fixes. Version management follows these rules: each time the rule content changes, the patch version number is automatically incremented; when a new rule is added or the rule structure changes, the minor version number is incremented and the patch version number is reset to 0; when the rule system undergoes a major restructuring, the major version number is incremented and the minor and patch version numbers are reset to 0. The rule storage unit retains the history of the most recent N versions (N is configurable, defaults to 10) to implement version rollback operations; when performing a version rollback, the rule storage unit will overwrite the rule content of the specified historical version with the current version and generate a new version record; Before persisting the rule file, the rule storage unit performs rule validation to ensure the syntax correctness and logical consistency of the rules. Rule validation includes: syntax validation: calling the DRL compiler to parse the rule file and check for syntax errors; dependency validation: checking whether the data model class referenced by the rule exists and whether the fields are accessible; circular dependency validation: detecting whether there is a circular dependency between rules (such as rule A triggering rule B, and rule B triggering rule A); conflict detection: detecting whether there are logically mutually exclusive rules enabled at the same time. When a conflict is detected, the rule storage unit 120 issues a warning, prompting the user to adjust the priority or rule conditions.

[0027] The rule hot-loading unit is used to listen for changes in the rule storage unit, obtain rule change notifications through message queues or publish-subscribe mechanisms, and dynamically load the updated rules into the rule matching network of the rule engine core module, while keeping the currently executing automated process instance unaffected. The rule hot-loading unit adopts a dual-channel listening mechanism, simultaneously implementing database-based polling and configuration center-based listening to ensure the reliability of change notifications. Database monitoring mode: The rule hot-loading unit starts a background thread that polls the rule storage table every T seconds (T is configurable, default T=5) to query rule records older than the last polling time. When a change is detected, the rule hot-loading unit obtains the list of changed rules and triggers the rule update process.

[0028] Configuration center listening mode: When using a distributed configuration center to store rules, the rule hot-loading unit utilizes the configuration center's native listening mechanism; The first optimal implementation of the rule hot reloading unit combines two listening modes to form a primary and backup mechanism: the configuration center listening mode is used first (high real-time performance), and when the configuration center is unavailable, it automatically degrades to the database polling mode to ensure the high availability of the system. The rule hot-loading unit implements publish-subscribe for rule changes through a message queue. When a rule changes, the rule storage unit sends a rule change message to a specified topic in the message queue; the rule hot-loading unit, as a consumer of the message queue, subscribes to the rule change topic; when it receives a change message, the rule hot-loading unit retrieves the complete rule content from the rule storage unit based on the rule ID in the message. The message queue mechanism enables multi-instance deployment scenarios: When the system is deployed in a cluster, the rule hot-loading unit of each instance can receive change notifications to achieve synchronous rule updates; to avoid duplicate loading, the rule hot-loading unit adopts a distributed lock mechanism, and only the instance that acquires the lock executes the actual rule loading, while other instances synchronously obtain the updated rules through the cache; The rule hot-loading unit employs a double-buffering mechanism to achieve uninterrupted rule updates. This mechanism establishes two rule set buffers in memory: Active buffer: stores the rule matching network currently being used by the rule engine core module; Reserve buffer: stores the rule matching network to be updated; The specific buffer hot-loading process is as follows: Step B1: After receiving the rule change notification, the rule hot-loading unit retrieves the updated rule file from the rule storage unit; Step B2: The rule hot-loading unit calls the rule compilation unit of the rule engine core module to compile the updated rule file into a new rule matching network and load it into the preparation buffer; Step B3: The rule hot-loading unit verifies the rule matching network in the preparatory buffer, including rule syntax verification, rule conflict detection, and fast simulation testing; Step B4: After verification, the rule hot-loading unit performs an atomic switching operation, pointing the pointer of the active buffer to the rule matching network of the preparation buffer, and marking the original active buffer as to be reclaimed. Step B5: After the rule hot-loading unit completes the switch, it delays for a certain period of time (configurable, default 30 seconds) to reclaim the memory resources of the original active buffer, ensuring that all currently executing process instances have left the old rule matching network; the atomic switch operation is implemented through memory barriers and atomic pointer assignment to ensure the atomicity and visibility of the switch operation; Furthermore, when a single rule is updated, it may affect other rules that depend on that rule or process instances that are currently being executed. The rule hot-loading unit uses a dependency analysis mechanism to identify the scope of the impact of rule changes. Set rule set The dependencies between rules can be represented as a directed graph. There are directed edges in it. Representation rules The execution may trigger rules ,Right now Conditions depend on The execution result; When the rules When a change occurs, the rule-based hot-loading unit calculates its impact set. That is, all direct or indirect dependencies The rules.

[0029] When loading updated rules, the rule hot-loading unit simultaneously recompiles all rules in the affected set to ensure the integrity of the rule network. When the affected set is too large, the rule hot-loading unit can trigger a full rule reload.

[0030] Furthermore, when an exception occurs during rule hot loading, such as a compilation error or verification failure, the rule hot loading unit automatically performs a rollback operation to ensure that the system is restored to its stable state before loading. The rollback mechanism includes the following steps: Step C1: Capture any exceptions thrown during rule loading and record the exception information to the error log; Step C2: Determine the current loading stage: If the exception occurs during the rule compilation stage, the buffer is not switched, and the original active buffer continues to be used; if the exception occurs during the verification stage, the preparation buffer is discarded, and the original active buffer remains unchanged. Step C3: Send an alert notification to the administrator, including information such as the exception type, rule ID, and error details; Step C4: Rule storage unit 120 marks the rule version that has an error as unstable and prevents the automatic loading of the version until it is manually confirmed and repaired.

[0031] The rule hot-reloading unit enables manually triggered rule rollback. Administrators can select a historical version for rollback through the management interface. The rule hot-reloading unit treats this historical version as a rule change and executes the standard hot-reloading process.

[0032] In a specific embodiment, the perceptual semantic parsing module is used to capture runtime interface images in real time, use computer vision algorithms to identify the state, position and attributes of interface elements, and use natural language processing algorithms to perform semantic understanding of interface text, generating a context snapshot containing structured interface element information and semantic tags. In this preferred embodiment, the perceptual semantic parsing module includes a computer vision unit, an optical character recognition unit, a natural language processing unit, and a context aggregation unit. The above four units work together in a pipeline manner: the computer vision unit is responsible for the visual detection and localization of interface elements; the optical character recognition unit is responsible for extracting the text content in the interface and associating it with the corresponding controls; the natural language processing unit is responsible for performing deep semantic understanding on the text content and generating semantic tags; and the context aggregation unit is responsible for assembling the outputs of the aforementioned units into a structured context snapshot according to the interface hierarchy.

[0033] The computer vision unit uses a target detection model based on convolutional neural networks to perform real-time reasoning on the interface screenshot, identify control types, control bounding boxes, control visibility and control enable states, and output standardized control objects. In this preferred embodiment, the computer vision unit 210 uses YOLOv8 as the object detection model. YOLOv8 is a single-stage object detection algorithm with high detection speed and high accuracy, suitable for real-time interface element recognition scenarios. The network structure of this model includes: Backbone Network: Employs the CSPDarknet structure, extracting multi-scale features of the image through cross-stage local connections; the Backbone network outputs three feature maps at different scales, corresponding to the original image downsampled by 8, 16, and 32 times, respectively, for detecting controls of different sizes; Neck Network: Adopting the PANet structure, it enhances the ability of feature maps to represent controls at different scales through bidirectional feature fusion from top to bottom and bottom to top. Head network: It adopts a decoupled head structure to separate the classification task and the regression task, and outputs the category probability, bounding box coordinates and confidence score of the control respectively; During the training process, the object detection model is pre-trained on a dataset of interface images labeled with control types. The dataset construction process is as follows: Data Acquisition: Collect screenshots of interfaces from different operating systems (Windows, macOS, Linux) and different application types (Web applications, desktop applications, mobile applications), covering common control types such as buttons, input boxes, drop-down menus, checkboxes, radio buttons, labels, and dialog boxes; Data annotation: Use annotation tools (such as LabelImg, CVAT) to annotate each control in the interface image. The annotation content includes control type (predefined category), control bounding box (coordinates of the top left corner and the bottom right corner), control visibility (visible / hidden), and control enabled state (enabled / disabled). Data augmentation: Data augmentation techniques are used to expand the training samples, including random rotation (-15° to 15°), random scaling (0.8x to 1.2x), random cropping, color jitter (brightness, contrast, and saturation adjustment), and Gaussian noise addition, to enhance the model's generalization ability. Training configuration: SGD optimizer is used, initial learning rate is set to 0.01, cosine annealing learning rate scheduling strategy is adopted, batch size is set to 32, training epochs are set to 300, and multi-scale training strategy is used (input image size is randomly adjusted every 10 epochs).

[0034] The optical character recognition unit is coupled with the computer vision unit to extract text areas in the interface and associate the recognized text content with the corresponding control object. The optical character recognition unit first receives the set of control objects output by the computer vision unit, and expands the bounding box bᵢ of each control object to generate a text recognition region. The expansion strategy is to expand the bounding box outward by δ pixels. In this embodiment, δ=5 is used to ensure that the text content is completely contained within the recognition region, as shown below: ; For plain text areas without corresponding control objects (such as prompt text in system pop-ups), the optical character recognition unit uses an edge detection-based text region detection algorithm for independent recognition: first, the interface image is grayscaled, then the edge information is extracted using the Canny edge detection operator, then adjacent edges are connected through morphological closing operations, and finally, candidate text regions are obtained through contour detection and filtering. The optical character recognition unit uses an OCR engine to recognize the extracted text region; in this preferred embodiment, PaddleOCR is used as the OCR engine, which can recognize Chinese, English, and multilingual mixed languages, and has high recognition accuracy; the PaddleOCR recognition process includes: Text detection: The DB algorithm is used to detect text regions and output the coordinates of the text boxes; Text orientation classification: A classification network is used to determine the orientation of the text (0°, 90°, 180°, 270°) and correct tilted text. Text recognition: The CRNN algorithm is used to recognize text content and output the recognition results and confidence scores; For each identified text region, the optical character recognition unit outputs the text content T and the text confidence score. The formula for calculating the text confidence score is: Where L is the text length. Let be the confidence score for recognizing the t-th character; The optical character recognition unit associates the recognized text content with the corresponding control object. The association strategy is based on spatial location relationships and adopts the following rules: Rule R1 (Including Association): If the bounding box of a text area is completely inside the bounding box of a control, then the text is associated with the embedded text of the control. Rule R2 (Adjacent Association): If the Euclidean distance between the bounding box of the text area and the bounding box of the control is less than the threshold τ (τ = 10 pixels in this embodiment), and the text area is located to the left, right, top, or bottom of the control, then the text is associated with the label text of the control. Rule R3 (Nearest Neighbor Association): If a text area does not satisfy R1 or R2, but is closest to the control, then the nearest neighbor principle is used for association; After association, the control object adds a `text` field and a `text_confidence` field. For button controls, the `text` field stores the text on the button; for input box controls, the `text` field stores the content inside the input box or placeholder text; for label controls, the `text` field stores the label text.

[0035] The natural language processing unit uses a pre-trained language model based on the Transformer architecture to perform intent classification and keyword extraction on the text content extracted by the optical character recognition unit and the system pop-up text, and generate semantic tags. In this preferred embodiment, the natural language processing unit uses BERT as a pre-trained language model. BERT is a Transformer-based bidirectional encoder model that learns deep semantic representations of text through two pre-training tasks: masked language model and next sentence prediction. The input to the BERT model consists of three parts: word embeddings, segment embeddings, and position embeddings. For the input text sequence... The BERT model outputs the hidden state vector at each location. ,in d is the dimension of the hidden layer, and in this embodiment, d=768; The natural language processing unit can be fine-tuned on domain-specific corpora to adapt to business scenarios. The fine-tuning process is as follows: Data preparation: Collect various text data in the business process, including system pop-up text (such as confirmation of deletion, network connection failure, operation success), interface label text (such as approver, amount, date), error message text (such as required fields cannot be empty, incorrect format), and label the corresponding intent category and keywords; Task Design: Add a classification head and a sequence labeling head to the BERT model. The classification head is used for intent classification, taking the [CLS] vector output by BERT as input, and outputting the intent category probability through a fully connected layer and a Softmax function. The sequence labeling head is used for keyword extraction, employing the BIO labeling scheme, and outputting a label for each character through a fully connected layer and a CRF layer. Training parameters: AdamW optimizer was used, and the learning rate was set to 2×10⁻ 5 The batch size is set to 16, the training rounds are set to 10, and an early stopping strategy is adopted (training is stopped if the validation set loss does not decrease for 3 consecutive rounds). In the next step, intent classification is performed, and the calculation process is as follows: Input the input text X into the BERT model, and take the output vector at the [CLS] position. As a global semantic representation of the text, this vector is input into the intent classification layer to calculate the score for each intent category: ,in, The weight matrix for the intent classification layer. As a bias term, the scores are converted into a probability distribution using the Softmax function: The system selects the intent category with the highest probability as the output. Intent categories include, but are not limited to: confirm, cancel, error, warning, info, success, input, and select. The keyword extraction calculation process is as follows: Input the input text X into the BERT model and extract the output vector at each position. The input sequence labeling layer uses a linear chain conditional random field to predict the optimal label sequence based on the dependency relationship of the label sequence. The score for the sequence labeling layer is: ,in, Assigning weights to the labels As a bias, T represents the label transition matrix. This represents the score for transitioning from label a to label b; then, the Viterbi algorithm is used to find the label sequence that maximizes the score: The tags use the BIO tagging scheme, where B-KEYWORD indicates the beginning of a keyword, I-KEYWORD indicates the content within a keyword, and O indicates a non-keyword. All characters tagged with B-KEYWORD and I-KEYWORD are extracted and merged into a keyword string. The natural language processing unit integrates the results of intent classification and keyword extraction to generate structured semantic tags.

[0036] The context aggregation unit is used to assemble the control objects output by the computer vision unit, the text content output by the optical character recognition unit, and the semantic tags output by the natural language processing unit into a tree-like or graph-like context snapshot according to the interface hierarchy. The context aggregation unit first constructs a hierarchical tree structure of UI elements. The hierarchical tree construction is based on the following rules: The window is the root node: each application window is the root node of a hierarchical tree, and window information is obtained through the operating system API (such as window title, window class name, and window handle). Parent-child relationship determined: The containment relationship between controls is determined by the containment property of their bounding boxes. Let the bounding box of control A be... The bounding box of control B is ,like Completely included And there is no control C that makes Include and Include If A is the parent node of B, then B is the child node of A. Sibling nodes are sorted by the top-left corner coordinate of the bounding box. , Sort the controls to form a stable control traversal order; After the hierarchy tree is built, each control node contains the following attributes: node identifier, control type, bounding box, visibility, enable state, text content, text confidence, semantic label, and list of child nodes; The context aggregation unit serializes the hierarchical tree into a structured context snapshot. The context snapshot uses JSON format and supports both tree and graph structures. The tree structure is suitable for most scenarios, while the graph structure is suitable for scenarios requiring complex navigation relationships between controls. In the graph structure, controls are nodes, and the interaction relationships between controls (e.g., displaying control B after clicking button A) are directed edges. Constructing a graph structure requires combining historical operation records; by analyzing the temporal sequence and spatial relationships of control appearances, the dependencies between controls can be inferred. The context aggregation unit performs a quality assessment on the generated context snapshots, calculating a snapshot integrity score. This integrity score comprehensively considers control recognition rate, text recognition rate, and semantic tag coverage. The specific calculation formula is as follows: ,in, The control recognition rate is equal to the ratio of the number of controls actually detected to the total number of controls in the interface (obtained by traversing the control tree). The text recognition rate is the ratio of the number of controls that successfully recognize text to the total number of controls containing text. Semantic tag coverage is the ratio of the number of controls that successfully generated semantic tags to the total number of controls. , , The preset weighting coefficients satisfy... In this embodiment, =0.4, =0.3, =0.3; Furthermore, when Q is lower than a preset threshold (0.7 in this embodiment), the context aggregation unit triggers a recapture and recognition process, or sends a low-quality alarm to the rule engine core module.

[0037] In a specific embodiment, the core module of the rule engine is communicatively connected to the business rule management module and the perception semantic parsing module, respectively. It has a built-in rule matching network based on the Rete algorithm, which is used to receive the context snapshot, perform pattern matching between the context snapshot and the condition rules, decision rules and exception handling rules in the business rule set, and output a real-time decision instruction containing operation type, operation parameters, target element positioning strategy and exception fallback strategy. In this preferred embodiment, the core module of the rule engine includes a rule compilation unit, a fact judgment unit, a conflict resolution unit, and an instruction generation unit. These four units work collaboratively in a pipeline manner: the rule compilation unit is responsible for compiling the rule files provided by the business rule management module into a Rete rule matching network; the fact judgment unit is responsible for asserting the context snapshot generated by the perceptual semantic parsing module as a fact object into the rule matching network; the conflict resolution unit is responsible for selecting the optimal rule to trigger when multiple rules match simultaneously; and the instruction generation unit is responsible for parsing the action portion of the triggered rule into an executable instruction object. The rule compilation unit is used to parse the rule files provided by the business rule management module into an abstract syntax tree and generate Alpha and Beta nodes of the Rete network. The rule compilation unit first receives the rule file (DRL format) provided by the business rule management module, performs lexical and syntactic analysis on the rule file, and constructs an abstract syntax tree; The lexical analysis phase decomposes the rule file into a sequence of tokens. Token types include: keywords (such as rule, when, then, end), identifiers (rule name, variable name), operators (>, <, =, !=), constants (number, string), punctuation marks (parentheses, semicolons), etc.

[0038] During the syntax analysis phase, the token sequence is organized into an AST structure according to the DRL syntax rules. The root node of the AST is the rule set node, and each rule node contains the following child nodes; Rule header node: contains metadata such as rule name, priority, rule group, and whether it is enabled; Condition node: Contains conditional expressions, implementing logical operators (AND, OR) and relational operators (>, <, =, !=, contains, matches, etc.); Action node: Contains an action expression, which can be a Java code snippet or a predefined action function; The rule compilation unit compiles the AST into a rule matching network based on the Rete algorithm. The Rete algorithm reduces the rule matching process from O(N·M) to O(N+M) by using a space-for-time strategy, where N is the number of fact objects and M is the number of rule conditions. The Rete network consists of the following node types. The root node is the entry point to the Rete network. All fact objects first enter the root node, and the root node broadcasts the fact objects to all Alpha nodes. Alpha nodes are used to perform single-condition matching. Each Alpha node contains a condition expression and maintains a memory cache of a fact object. When a fact object passes through an Alpha node, the node evaluates the condition expression. If the condition is met, the fact object is stored in Alpha Memory and passed to subsequent nodes. The construction rule for Alpha nodes is as follows: For each independent condition in the rule, a corresponding Alpha node is generated. The same condition expression shares the same Alpha node to avoid duplicate storage. Beta nodes are used to perform multi-condition cross-matching and variable binding. Each Beta node contains two inputs: a left input (usually from the Alpha Memory or the Beta Memory of the previous Beta node) and a right input (usually from the Alpha node). The Beta node performs a join operation, combining the fact objects of the left and right inputs and checking whether the combined object satisfies the cross-condition constraints. Beta nodes are of two types: Join nodes: perform a join operation on two input fact objects and output fact pairs that satisfy the join condition; Not nodes: perform a negative match, which is triggered when the left input exists and the right input does not exist; Beta nodes maintain Beta Memory to store intermediate results that have been matched. The last Beta node of each rule is connected to the terminal node. The terminal node indicates that all conditions of the rule have been met and adds the rule and its bound variable values ​​to the agenda queue to wait for triggering. The rule compilation unit optimizes the generated Rete network to improve rule matching efficiency. Specifically: Node sharing: Rules with the same condition expression share Alpha nodes, reducing memory usage and redundant calculations; Condition reordering: Conditions are reordered based on their selectivity, placing conditions with higher selectivity at the beginning to filter out fact objects that do not meet the conditions in advance, reducing the number of intermediate results. Selectivity is defined as the ratio of the number of fact objects that meet the condition to the total number of fact objects; the smaller the ratio, the higher the selectivity; Index optimization: Hash indexes are built on Alpha Memory and Beta Memory to accelerate the lookup and join operations of fact objects; hash indexes are used for equality conditions; ordered indexes are used for range conditions.

[0039] The fact-judgment unit is used to receive the context snapshot generated by the perceptual semantic parsing module, and to send each data item in the context snapshot as a fact judgment to the Rete network; The fact-determining unit iterates through each UI element in the context snapshot and encapsulates each element into a fact object; for non-control data such as text content and semantic tags in the context snapshot, the fact-determining unit encapsulates them into TextFact and SemanticFact objects respectively. The fact-judging process of the fact-judging unit is as follows: The fact-judging unit inserts the encapsulated fact object into the Rete network, and the process is as follows: Step E1: Insert the fact object into the root node; Step E2: The root node broadcasts the fact object to all Alpha nodes; Step E3: Each Alpha node evaluates whether the fact object satisfies its own conditional expression. If it does, the fact object is stored in Alpha Memory and passed to subsequent nodes of that Alpha node; Step E4: The Beta node receives the left input (from the Beta Memory) and the right input (from the Alpha node) and performs a connection operation. If the connection is successful, the result is stored in the Beta Memory and passed to subsequent Beta nodes. Step E5: Repeat step E4 until the fact object reaches the terminal node, and add the matching rule and its bound variable value to the agenda queue.

[0040] Furthermore, during actual operation, the interface state changes dynamically, requiring the fact objects to be updated. The fact judgment unit implements incremental updates of facts: New Fact: When a new UI element is detected, a new fact object is created and inserted into the Rete network; Deleting Facts: When a UI element disappears, the corresponding fact object is removed from the Rete network. This removal operation requires traversing the Alpha Memory and Beta Memory to delete all occurrences of the fact object. Modify facts: When the properties of UI elements change, delete the old facts first, then insert the new facts, triggering a network re-evaluation; To optimize the efficiency of modification operations, the fact judgment unit uses a two-pointer comparison before and after modification, and only triggers deletion and insertion operations when the attribute change affects rule matching.

[0041] The conflict resolution unit is used to select one rule from the multiple matched rules to trigger when multiple rules are matched simultaneously in the Rete network, based on a preset conflict resolution strategy. When multiple rules match simultaneously, the conflict resolution unit 330 places these rules into the agenda queue; each entry in the agenda queue contains: a rule object, a bound variable value (fact object), a rule priority, and a trigger timestamp.

[0042] The agenda queue performs the following operations: Add: When a rule matches successfully, it is added to the agenda queue; Remove: When a rule is triggered or a fact change causes the rule to no longer match, it is removed from the agenda queue; Sort: The rules in the agenda queue are sorted according to the conflict resolution strategy. The conflict resolution unit selects a rule from the agenda queue to trigger using a preset conflict resolution strategy. This embodiment implements the following strategies and their combinations. A priority-based sorting strategy is used to pre-define a priority weight value for each rule. (Range 1-1000), select the rule with the highest weight value to trigger: When multiple rules have the same priority, the secondary strategy is invoked. Based on the most recent update time strategy, the rule with the most recent update time is selected for triggering. This strategy is suitable for hot deployment scenarios and ensures that the most recently deployed rule takes effect first. ,in This is represented by the last update timestamp of rule i; Based on rule complexity, the strategy selects the rule with the most conditions to trigger, assuming that complex rules contain more business logic and should be processed first. ,in Let be the number of conditions for rule i; Based on a rule usage frequency strategy, the rule with the lowest historical trigger frequency is selected for execution to achieve rule load balancing and prevent certain rules from being left unexecuted for extended periods. ,in The historical trigger frequency (number of times / hour) of rule i is calculated using an exponentially weighted moving average (EWMA): , where α is the smoothing factor, and α = 0.3; The instantaneous frequency of the current trigger (1 / Δt); The comprehensive scoring strategy involves the conflict resolution unit weighted and summed from the above factors to calculate a comprehensive score. ,in Then, the rule with the highest overall score is selected for triggering; Furthermore, the conflict resolution unit is also responsible for detecting logical conflicts between rules. An alarm is triggered when the following situations are detected: Mutually exclusive conflict: The conditions of two rules are simultaneously met, but their actions are mutually exclusive (e.g., one rule requires clicking "confirm," and the other requires clicking "cancel"); Priority inversion: The conditions of a lower-priority rule are a subset of the conditions of a higher-risk rule, causing the higher-risk rule to never be triggered; Circular dependency: The triggering of rule A causes a change in facts, which in turn triggers rule B, and the triggering of rule B causes a change in facts, triggering rule A again, forming an infinite loop. The conflict resolution unit detects circular dependencies by analyzing the rule dependency graph. When a cycle is detected, it avoids infinite loops by setting an upper limit on the number of triggers (10 times in this embodiment) and a cooldown time (1 second).

[0043] The instruction generation unit is used to parse the action part of the triggered rule into an executable instruction object; the instruction object includes an operation type enumeration, a target control locator chain, a timeout parameter, a retry count parameter, and an exception compensation action list; The complete instruction generation process of the instruction generation unit is as follows: Step F1: Receive the rule object and its bound variable values ​​output by the conflict resolution unit; Step F2: Parse the expression of the rule action part to identify the action type and action parameters; Step F3: If the action involves control operation, obtain the fact object of the target control from the context snapshot, calculate the availability score of each positioning strategy, and generate the target control locator chain; Step F4: Extract the timeout parameter from the rule metadata (if the rule does not specify, use the default value of 3000 milliseconds). Step F5: Extract the retry count parameter from the rule metadata (if the rule does not specify, use the default value of 1); Step F6: If the rule contains exception handling logic, parse and generate a list of exception compensation actions; Step F7: Encapsulate the above information into an instruction object and output it to the process automation robot actuator.

[0044] The dynamic path planning module communicates with both the rule engine core module and the process automation robot actuator. It maintains an operation path topology graph with interface elements as nodes and operation reachability relationships as edges. When an interruption of the original operation path is detected, it calculates the shortest path from the current interface reachable node to the target node in the operation path topology graph based on a heuristic search algorithm and generates an alternative operation path. In this preferred embodiment, the dynamic path planning module includes a topology graph construction unit, a path caching unit, a path search unit, and a path verification unit. These four units work together to realize the planning, caching, searching, and verification of automated operation paths: the topology graph construction unit is responsible for constructing the operation path topology graph during the system initialization phase; the path caching unit is responsible for caching historically successfully executed operation paths and recording execution statistics; the path search unit is responsible for calculating alternative paths based on heuristic search algorithms when a path is interrupted; and the path verification unit is responsible for verifying the validity of candidate alternative paths.

[0045] Specifically, the topology graph construction unit is used to scan the user interface structure of the target application during the system initialization phase or process orchestration phase, construct an operation path topology graph with interface elements as nodes and navigation relationships between elements as directed edges, and record the unique identifier, locator set, and node type metadata of each node. The topology graph building unit scans the user interface structure of the target application in the following ways: Static scanning mode: During the system initialization phase, the topology graph construction unit performs static scanning by analyzing the interface definition file of the target application. The specific process is as follows: Start the target application and navigate to the initial interface of the application; call the perceptual semantic parsing module to obtain a complete context snapshot of the current interface and extract all visible controls; for each interactive control (button, link, menu item, input box, etc.), simulate user operation (such as clicking) and record the target interface navigated to after the operation; recursively execute the above steps until all reachable interfaces have been scanned or a preset depth limit has been reached (in this embodiment, the depth limit is 10 layers); summarize the scanning results and generate an operation path topology graph. Dynamic Learning Mode: During the process orchestration phase, the topology graph construction unit dynamically learns by analyzing historically successfully executed operation logs, supplementing paths not covered by static scanning. This dynamic learning mode is achieved through: monitoring the execution logs of the process automation robot's actuators and extracting successfully executed operation sequences; analyzing these sequences to identify new interface elements or new navigation relationships; and dynamically adding newly discovered nodes and edges to the operation path topology graph. Furthermore, the operation path topology graph consists of a set of nodes and a set of directed edges. Each node represents an interface element or interface state, and its data structure is as follows: Node Unique Identifier: generated using a timestamp plus sequence number or a universally unique identifier to ensure global uniqueness; Node Type: including predefined types such as window, dialog box, frame, tab, and menu; Interface Title: window title or page title, used for interface identification; Locator Set: containing multiple positioning strategies, each consisting of a positioning method (such as control identifier, XPath expression, CSS selector, text content, relative position, image template, etc.) and a corresponding positioning value, used to uniquely identify the node at runtime; Metadata: containing auxiliary information such as the number of controls, Uniform Resource Locator (suitable for web applications), and process name; Semantic Tags: a list of tags generated by the natural language processing unit, used for subsequent semantic similarity matching; First Discovery Timestamp: the time when the node was first scanned; Last Appearance Timestamp: the time when the node was most recently observed; Access Count: the number of times the node has been used as an operation target in history. A directed edge represents the reachability relationship from a source node to a target node. That is, by performing an operation on the interface of the source node, one can reach the interface of the target node. The specific data structure of each directed edge includes: a unique edge identifier; a source node identifier; a target node identifier; an operation type (including predefined types such as click, input, selection, navigation, and scrolling); a target control locator chain (containing multiple positioning strategies sorted by priority, used to locate the target control for the operation); a historical execution success rate (the historical execution success rate of this edge, ranging from 0 to 1, dynamically updated); an average execution time (the average execution time of this edge, in milliseconds, dynamically updated); the number of executions (the number of times this edge has been successfully executed); a last execution timestamp (the time when this edge was last executed); and a set of alternative operations (alternative operations that can be tried when the main operation fails, each alternative operation including its type and parameters). The topology graph construction unit stores the constructed operation path topology graph in an in-memory database or a graph database, thereby enabling efficient node query and path search. The graph database uses an adjacency list for storage, and each node maintains a list pointing to all its outgoing edges and a list pointing to all its incoming edges.

[0046] The path caching unit is used to cache historically successfully executed operation paths and records the execution success rate and average execution time for each cached path; The path caching unit establishes a path caching table. The specific data structure of each cached path is as follows: unique path identifier; starting node identifier; target node identifier; node sequence: a list of node identifiers traversed by the path arranged in order; edge sequence: a list of edge identifiers traversed by the path arranged in order; historical execution success rate: the historical execution success rate of this path, with a value ranging from 0 to 1; average execution time: the average execution time of this path, in milliseconds; execution count: the number of times this path has been successfully executed; last execution timestamp: the time when this path was last executed; path fitness score: a score calculated by combining the success rate and execution time, used as a heuristic weight in subsequent path searches. Path caching unit 520 calculates a fitness score for each cached path, which comprehensively considers execution success rate and execution time. Let the historical execution success rate of path p be... The average execution time is First, the execution time of all cached paths is normalized. Convert to normalized time values The calculation method is as follows: take the minimum average execution time among all cache paths and denote it as... The longest average execution time is denoted as ,but equal Divide by The formula for calculating the path fitness score F(p) is: α and β are preset weight coefficients that satisfy α+β=1. In this embodiment, α=0.7 and β=0.3, that is, the success rate weight is higher than the execution time weight. The path caching unit uses the following strategy to dynamically update the cache: Add cache: When a path is successfully executed and is not in the cache, it is added to the cache, the initial historical execution success rate is 1.0, the number of executions is 1, and the average execution time is the execution time of this time; Update cache: When a cached path is executed again, the success rate and average execution time are updated using the exponentially weighted moving average method. The new success rate is the smoothing factor multiplied by the current execution result (1 for success, 0 for failure) plus (1 minus the smoothing factor) multiplied by the original success rate. The new average execution time is the smoothing factor multiplied by the current execution time plus (1 minus the smoothing factor) multiplied by the original average execution time. The smoothing factor is 0.3. Elimination strategy: When the number of cached paths exceeds the preset limit (1000 in this embodiment), they are eliminated according to the elimination score. The elimination score is equal to (1 minus fitness score) multiplied by (1 minus time factor). The time factor is calculated based on the last execution time. The closer the last execution time, the larger the time factor and the lower the elimination score of the path, thus retaining the high fitness paths that have been used frequently recently.

[0047] The path search unit, upon receiving a path interruption signal, retrieves the current real-time node set as the starting node set and the unique identifier of the target node as the target node, using A... The algorithm or Dijkstra's algorithm performs path search in the operation path topology graph, and the execution success rate of the cached path in the path cache unit is used as a heuristic weight during the search process. Specifically, when the process automation robot actuator encounters the following situations during operation, it sends a path interruption signal to the dynamic path planning module: element positioning failure: the target control does not appear on the interface within the timeout period; navigation failure: the expected interface is not reached after the operation is executed; abnormal pop-up interference: an unexpected system pop-up appears and blocks the operation process. Interface structure changes: Changes in the identifiers of controls, XPath, and other positioning information lead to positioning failure. The path interruption signal includes the real-time control tree of the current interface (provided by the perceptual semantic parsing module), the identifier of the expected target node, the failed operation information, and the timestamp of the interruption. Furthermore, the path search unit determines the starting node set based on the real-time control tree of the current interface. Since the current interface may correspond to multiple nodes (such as different windows or different tabs), the starting node set is determined in the following way: extracting key features of the current interface, including window title, Uniform Resource Locator (URL) (applicable to web applications), process name, list of visible controls, etc.; searching all nodes in the operation path topology graph and calculating the similarity score between each node and the current interface. The similarity score is obtained by weighted summation of title similarity, URL similarity, control list similarity, and semantic tag similarity. Among them, title similarity is calculated based on string edit distance, URL similarity is calculated based on path matching, control list similarity is calculated based on Jaccard similarity to compare the similarity between the control identifier set of the current interface and the historical control identifier set of the node, and semantic tag similarity is calculated based on cosine similarity of word vectors; selecting nodes with similarity scores exceeding a preset threshold (0.7 in this embodiment) as the starting node set; if no starting node that meets the conditions is found, an interface unrecognized anomaly is triggered, the current interface information is recorded as a new node, and manual assistance in recognition is requested; The path search unit uses A The algorithm searches for the shortest path from the starting node set to the target node in the operation path topology graph. The algorithm evaluates the priority of each node through a cost function and prioritizes expanding the node with the lowest cost. A The cost function of the algorithm is defined as: F(n) = G(n) + H(n), where G(n) is the actual cost from the starting node set to the current node n, and H(n) is the heuristic cost from the current node n to the target node; The actual cost G(n) comprehensively considers the number of operation steps, historical execution success rate, and execution time. Let the edge sequence traversed by the path from the starting node to the current node n be denoted as . If the historical execution success rate of each edge e is R(e), and the normalized execution time is N(e) (its calculation method is similar to that of path normalization time), then the formula for calculating the actual cost G(n) is: ,in, This is expressed as a step weight, which is set to 1.0 in this embodiment; For time weighting, this embodiment uses 0.5. For edges not recorded in the cache (first exploration), the default values ​​are used: R(e) = 0.5, N(e) = 0.5; The heuristic cost H(n) estimates the optimal cost from the current node n to the target node v_target, using a weighted combination of the following three methods: Heuristics based on semantic similarity: ,in The semantic label similarity between node n and the target node is calculated based on the cosine similarity of word vectors. Heuristics based on spatial distance: ,in This represents the spatial distance between node n and the target node in the interface (calculated based on the coordinates of the control's center). The preset maximum distance (such as the screen diagonal length); Heuristic based on historical paths: When a historical path from node n to the target node exists in the path cache unit, Normalize by dividing the minimum average execution time of the historical path by the preset maximum time value; if there is no historical path, use the default value of 0.5.

[0048] The comprehensive heuristic cost is the weighted sum of the above three factors, specifically: in , , The preset weighting coefficients satisfy... In this embodiment, =0.3, =0.2, =0.5; To go further, A The specific execution flow of the algorithm is as follows: Initialize the open list and the closed list. Add all nodes in the starting node set to the open list, initialize the actual cost G(n) of each node to 0, and the cost function F(n) equals the heuristic cost H(n); Repeat the following steps until the open list is empty or the target node is found: Select the node with the smallest cost function F(n) from the open list as the current node; If the current node is the target node, the path search is successful, backtrack the path through the recorded parent node pointer and return; Move the current node from the open list to the closed list; Traverse all outgoing edges of the current node, for each outgoing edge pointing to a neighbor node, if the neighbor node is in the closed list, skip it; otherwise, calculate the new actual cost to reach the neighbor node through the current node. Its value is the actual cost G(current) of the current node plus the cost cost(edge) of the edge (cost(edge) is calculated in the same way as before); if the neighbor node is not in the open list, it is added to the open list, and its actual cost G(neighbor) is set to G_new, and the cost function is... And record the parent node as the current node; if the neighbor node is already in the open list and If the cost is less than the current actual cost, then update the actual cost, cost function, and parent node; if the open list is empty and the target node is still not found, then return a path search failure. Optionally, when the topology of the operational path does not satisfy the admissibility condition of the heuristic function (i.e., the heuristic cost may overestimate the actual cost), the path search unit can switch to Dijkstra's algorithm for path search. The difference between Dijkstra's algorithm and Algorithm A is that Dijkstra's algorithm does not use a heuristic function, but only expands based on the actual cost G(n), guaranteeing the finding of the shortest path but with lower search efficiency; the cost function of Dijkstra's algorithm is F(n)=G(n), and the other processes are the same as Algorithm A.

[0049] The process automation robot actuator is communicatively connected to the rule engine core module to parse and execute the real-time decision instructions, simulating human-machine interaction. The actuator has embedded pre-hook and post-hook, which are used to send the current execution status to the rule engine core module for secondary confirmation before the operation is executed, and to feed back the execution result to the rule engine core module after the operation is executed. In this preferred embodiment, the process automation robot actuator includes an instruction parsing unit 410, an operation execution unit, a status monitoring unit, and an execution log unit. These four units work together to achieve reliable execution of automated operations: the instruction parsing unit is responsible for converting the real-time decision instructions output by the rule engine core module into operation sequences that can be recognized by the underlying automation framework; the operation execution unit is responsible for calling the application programming interface of the underlying automation framework to execute the operation sequence and recording the operation time and results during the execution process; the status monitoring unit is responsible for monitoring the system logs, interface changes, and abnormal pop-ups generated during the execution process, and actively triggering a post-hook to feed back the abnormal information to the rule engine core module when a preset abnormal pattern is detected; the execution log unit is responsible for recording the operation type, target control, execution result, time consumption information, and abnormal stack of each step as a structured log and outputting the structured log to the audit system. Specifically, the instruction parsing unit receives real-time decision instructions output by the rule engine core module and converts them into an operation sequence recognizable by the underlying automation framework. The instruction parsing unit receives real-time decision instructions from the rule engine core module. These instructions use a structured data format and include the following fields: Operation Type: Identifies the type of operation to be performed, including click, double-click, right-click, text input, clear input box, dropdown selection, scroll page, wait, page navigation, data extraction, page refresh, window closing, and notification sending; Target Control Locator Chain: Contains multiple positioning strategies ordered by priority. Each positioning strategy includes a positioning method (such as control identifier, XPath expression, CSS selector, text content, relative position, image template, etc.) and a corresponding positioning value; Operation Parameters: Varies depending on the operation type. For example, text input requires the text content to be entered, scrolling requires the scroll distance, and waiting requires the waiting duration; Timeout Parameter: The maximum waiting time in milliseconds, used for timeout control of control positioning and operation execution; Retry Count Parameter: The maximum number of retries when the operation fails; Exception Compensation Action List: A sequence of alternative operations to execute when the operation fails. The instruction parsing unit selects the appropriate underlying automation framework based on the type of the target application and the deployment environment. These underlying automation frameworks include, but are not limited to: the Windows UI Automation framework for Windows desktop applications, which accesses user interface elements of desktop applications through a control tree and supports control location, attribute acquisition, and operation simulation; the Selenium framework for web applications, which interacts with web pages through browser drivers and supports automated control of multiple browsers; the Appium framework for mobile applications, which supports automated testing on iOS and Android platforms and simulates user operations through mobile terminal drivers; and a custom framework based on operating system message hooks for legacy systems, which achieves operation simulation by intercepting and sending window messages. The instruction parsing unit converts the standardized instruction format into a sequence of application programming interface (API) calls specific to the framework. For example, for a click operation, it converts it into a click method call for the corresponding framework; for an input operation, it first calls the positioning method to find the target input box, and then calls the input method to write the text; for a scroll operation, it calls the scroll or script execution method for the corresponding framework. The instruction parsing unit supports expanding a single instruction into a multi-step operation sequence to handle complex interaction scenarios. For example, for a control that requires hovering before clicking, the instruction parsing unit generates a two-step sequence that executes the mouse hover operation first, followed by the click operation; for an input box that requires focusing before input, the instruction parsing unit generates a two-step sequence that executes the focus operation first, followed by the input operation. Each step in the operation sequence contains complete information such as the operation type, the target control locator (selected from the locator chain), operation parameters, timeout parameters, and the number of retries. The operation sequences are arranged in execution order to form a queue to be executed.

[0050] Specifically, the operation execution unit is used to call the application programming interface of the underlying automation framework to execute the operation sequence, and records the start time, end time and execution result of each operation during the execution process; Before executing an operation, the operation execution unit first locates the target control according to the target control locator chain. The locator chain contains multiple positioning strategies ordered by priority. The operation execution unit tries each positioning strategy in descending order of priority until the control is successfully located or all strategies fail.

[0051] For each positioning strategy, the operation execution unit performs the following positioning attempts: Location based on control identifier: The control is searched using its unique identifier (such as AutomationId for Windows controls or the id property for Web controls). If found, the control object is returned. XPath-based positioning: It performs path matching in the UI control tree using XPath expressions, and returns if a unique matching control is found; CSS selector-based positioning: Matching elements in the document object model tree of a web page using CSS selectors, and returning the element if a unique match is found; Location based on control text content: Traverse all controls in the interface, find controls whose text content exactly matches or contains the target text, and return the control with the highest matching degree; Positioning based on relative control position: The target control is positioned according to its relative position (such as top, bottom, left, right) relative to its parent container or other known controls. The target control is determined by calculating the spatial distance and directional relationship between the candidate control and the reference control. Image feature matching-based localization: The template image of the target control is matched with the current screenshot of the interface for feature points, and the similarity is calculated. If the similarity exceeds a preset threshold, the control corresponding to the matching position is returned. The positioning process employs an intelligent waiting strategy, which involves polling and checking the existence of the control within a timeout period. Let the timeout period be... (milliseconds), the polling interval is Δt (200 milliseconds in this embodiment), then the maximum number of polling times is Divide by Δt and round down. In each polling cycle, the operation execution unit attempts to locate the control. If successful, it returns immediately; if it times out and still fails, the location fails. After successfully locating the target control, the operation execution unit performs the corresponding operation based on the operation type, such as click, double-click, or input. When the operation fails, the operation execution unit retryes according to the retry count parameter in the instruction. The retry mechanism uses an exponential backoff strategy, meaning the waiting time for each retry increases exponentially to avoid additional load caused by frequent retries when the system is busy. The operation execution unit records the start and end timestamps before and after each operation step and calculates the operation time. The operation time equals the end timestamp minus the start timestamp, in milliseconds. This time information is used for subsequent logging and performance analysis.

[0052] Specifically, the status monitoring unit is used to monitor system logs, interface changes and abnormal pop-ups generated during the execution process. When a preset abnormal pattern is detected, the post-hook is actively triggered to feed back the abnormal information to the core module of the rule engine. The status monitoring unit adopts a multi-threaded parallel monitoring architecture to ensure that it does not interfere with the normal operation of the main execution thread. The main execution thread is responsible for executing the operation sequence, while the monitoring thread continuously captures system messages and interface status. The monitoring thread and the main execution thread communicate through a shared memory queue. When the monitoring thread detects an anomaly, it writes the anomaly information to the queue, and the main execution thread handles the anomaly at an appropriate time. The status monitoring unit establishes an exception mode library, predefining the following exception modes: Network connection interruption mode: The system log contains keywords such as network connection failure, inability to connect to the server, and connection timeout; Target element not found mode: The exception information returned after the control positioning timeout; Timeout no response mode: The operation execution time exceeds the preset threshold (30 seconds in this embodiment) and is still not completed; System pop-up mode: Unexpected dialog boxes, warning boxes, or error message boxes are detected, and the pop-up text contains keywords such as error, warning, confirmation, and failure; Memory overflow warning mode: The system log contains keywords such as insufficient memory and OutOfMemory; Application crash mode: The process of the target application terminates unexpectedly; A post-hook is a callback function embedded in the executor, which is automatically triggered after the operation is completed. The status monitoring unit uses post-hooks to feed back the execution results to the core module of the rule engine. The triggering timing of post-hooks is as follows: After successful operation: the post-hook is triggered, carrying the success status and execution result; After operation failure: the post-hook is triggered, carrying the failure status and exception information; After the monitoring thread detects an exception: the post-hook is actively triggered, carrying the exception status and exception details. The feedback message sent by the post-hook uses a structured format and includes the following fields: Operation Identifier: a unique identifier for the corresponding executed instruction; Execution Status: success, failure, or exception; Execution Result: the data returned by the operation (such as the extracted data content); Exception Information: if the execution fails, it includes the exception type, exception description, and exception stack; Execution Time: the total time taken to execute the operation, in milliseconds; Timestamp: the time the operation was completed. After receiving the feedback message, the core module of the rules engine makes a decision based on the preset exception handling rules, which may generate compensation operations, adjust the execution strategy, or trigger path switching. To elaborate further, in the process of pre-hooking and post-hooking, taking the scenario where the rule engine core module generates a decision instruction to click the submit approval button, and the process automation robot executor prepares to execute the instruction as an example: In the pre-hook, the instruction parsing unit parses the decision instruction, extracts the operation type as click, and the target control locator chain includes a positioning strategy based on text content and a positioning strategy based on control identifier. Before the operation execution unit prepares to execute the operation, it triggers the pre-hook. The pre-hook captures the operation instruction to be executed and extracts the target control locator chain. The pre-hook calls the perceptual semantic parsing module to re-identify the current state of the target control in real time and obtain the real-time attributes of the target control, including visibility, enablement, and position coordinates. The pre-hook encapsulates the real-time attributes into a confirmation request and sends it to the rule engine core module. The rule engine core module matches the confirmation request with the pre-validation rules in the business rule set. For example, there is a pre-validation rule: if the target control is not visible, wait 2 seconds and retry; another rule: if the target control is unavailable, perform an alternative operation: click the enable button first, and then retry. If the rule matches successfully, a confirmation instruction is returned, allowing continued execution. After receiving the confirmation instruction, the operation execution unit executes the actual click operation. In the post-hook, after the operation execution unit completes the click operation, the post-hook is triggered. The post-hook obtains the operation execution result (success or failure) and records the execution time. If the operation is successful, the post-hook encapsulates the success status and execution result into a feedback message and sends it to the rule engine core module. If the operation fails (e.g., the expected interface does not appear after clicking), the post-hook encapsulates the failure status and exception information into a feedback message and sends it to the rule engine core module. After receiving the feedback message, the rule engine core module makes a decision based on the exception handling rules. If the operation fails, it may trigger retry rules, path switching rules, or compensation operation rules. The execution log unit records a complete operation log, including operation type, target control, execution result, and time consumption information.

[0053] The intelligent process automation control method based on the above system includes the following steps: Step S1, Rule Loading: The business rule set is loaded through the business rule management module, the rules are compiled into a Rete network and loaded into memory to establish a rule memory image; specifically, the rule compilation unit parses the rule file into an abstract syntax tree, generates Alpha nodes and Beta nodes, and constructs the Rete network; the rule hot loading unit adopts a double buffering mechanism to achieve uninterrupted rule updates; Step S2: Context Awareness: The visual and textual features of the target interface are obtained through the semantic parsing module, and a runtime context containing an interface element tree and semantic tags is constructed. Specifically, the computer vision unit uses an object detection model to identify interface controls; the optical character recognition unit extracts text content; the natural language processing unit uses a pre-trained language model to generate semantic tags; and the context aggregation unit assembles the above information into a structured context snapshot. Step S3: Rule Matching and Decision Making: The runtime context is used as a fact judgment in the Rete network of the rule engine core module to trigger rule matching and generate a real-time decision instruction containing the operation type, location strategy chain, and exception fallback strategy. This step further includes the following sub-steps: Step S3.1: The fact judgment unit inserts each UI element in the runtime context as an independent fact object into the Alpha node of the Rete network and performs single-condition matching; Step S3.2: Pass the fact object matched by the Alpha node to the Beta node for multi-condition cross-matching and variable binding; Step S3.3: When a fact object satisfies all the conditions of any rule, put the rule and its bound variable values ​​into the agenda queue; Step S3.4: The conflict resolution unit selects a rule from the agenda queue to trigger according to the preset conflict resolution strategy, and the instruction generation unit executes the action part of the rule to generate a real-time decision instruction.

[0054] Step S4: Pre-execution confirmation: The process automation robot executor parses the real-time decision instructions and triggers a pre-hook to send the current status to the rule engine core module for secondary confirmation before execution. After confirmation, the automated operation is executed. This step further includes the following sub-steps: Step S4.1: The pre-hook captures the operation command to be executed and extracts the target control locator from the operation command; Step S4.2: Call the perceptual semantic parsing module to re-identify the current state of the target control in real time and obtain the real-time attributes of the target control, including visibility, enablement and position coordinates; Step S4.3: Encapsulate the real-time attributes into a confirmation request and send it to the core module of the rule engine; Step S4.4: The core module of the rules engine matches the confirmation request with the pre-validation rules in the business rule set. If the match is successful, a confirmation instruction is returned; if the match is unsuccessful, a rejection instruction is returned along with alternative operation suggestions.

[0055] Step S5: Execution and Feedback: After the process automation robot executor performs the operation, it triggers a post-hook to feed the execution result back to the core module of the rules engine. If the execution result does not match the expectation or an interface anomaly is detected, the exception handling rules in the business rule set are triggered to generate a compensation operation or call the dynamic path planning module to switch to an alternative path. The specific steps of calling the dynamic path planning module to switch to an alternative path include: Step S5.1: The dynamic path planning module receives the path interruption signal sent by the process automation robot actuator and the real-time control tree of the current interface; Step S5.2: The dynamic path planning module traverses the operation path topology graph, starting from the root node of the real-time control tree and ending at the target operation node. Step S5.3: The path search unit uses A The algorithm calculates the path cost; Step S5.4: The path with the lowest calculated cost is selected as the candidate alternative path. The path verification unit 540 simulates the reachability of the first N nodes in the candidate alternative path in a sandbox environment, where N is a preset positive integer (e.g., N=3). Step S5.5: Update the verified alternative path to the execution queue of the process automation robot executor and mark the original path as a failed path.

[0056] Step S6: Auditing and Optimization: The execution log unit records the rule matching trajectory, decision instruction sequence, and operation execution log throughout the entire execution process, generating a visual audit report. The audit report includes the following: Rule Hit Statistics: the number of times each rule is triggered, the trigger time, and the trigger context; Execution Efficiency Analysis: the average time, maximum time, and minimum time for each operation; Anomaly Distribution Analysis: anomaly type statistics, anomaly frequency, and anomaly handling results; Path Usage Statistics: the number of times each operation path is used, the success rate, and the average time.

[0057] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An intelligent process automation robot system based on a business rule engine, characterized in that, include: The business rules management module is used to define, store, and manage business rule sets in a visual manner; The Perception Semantic Parsing Module is used to capture runtime interface images in real time, use computer vision algorithms to identify the state, position and attributes of interface elements, and use natural language processing algorithms to perform semantic understanding of interface text, generating a context snapshot containing structured interface element information and semantic tags. The core module of the rule engine is communicatively connected to the business rule management module and the perception semantic parsing module. The core module of the rule engine has a built-in rule matching network based on the Rete algorithm, which is used to receive the context snapshot, perform pattern matching between the context snapshot and the condition rules, decision rules and exception handling rules in the business rule set, and output a real-time decision instruction including operation type, operation parameters, target element positioning strategy and exception fallback strategy. The process automation robot actuator is communicatively connected to the rule engine core module to parse and execute the real-time decision instructions, simulating human-computer interaction. The actuator is embedded with a pre-hook and a post-hook, which are used to send the current execution status to the rule engine core module for secondary confirmation before the operation is executed, and to feed back the execution result to the rule engine core module after the operation is executed. The dynamic path planning module is communicatively connected to the rule engine core module and the process automation robot actuator, respectively. It is used to maintain an operation path topology graph with interface elements as nodes and operation reachability relationships as edges. When the original operation path is detected to be interrupted, it calculates the shortest path from the current interface reachable node to the target node in the operation path topology graph based on a heuristic search algorithm and generates an alternative operation path.

2. The intelligent process automation robot system based on a business rule engine according to claim 1, characterized in that, The business rules management module includes: The rule editor unit provides a graphical user interface, enables users to define rules in their own format, and compiles the defined rules into rule files in a preset format. A rule storage unit is used to store the rule files in a relational database or a distributed configuration center; The rule hot-loading unit is used to listen for change events in the rule storage unit, obtain rule change notifications through a message queue or publish-subscribe mechanism, and dynamically load the updated rules into the rule matching network of the rule engine core module, while keeping the currently executing automated process instance unaffected.

3. The intelligent process automation robot system based on a business rule engine according to claim 1, characterized in that, The perceptual semantic parsing module includes: The computer vision unit is used to perform real-time reasoning on interface screenshots using a convolutional neural network-based object detection model, identify control types, control bounding boxes, control visibility, and control enable states, and output standardized control objects. An optical character recognition unit, coupled to the computer vision unit, is used to extract text regions in the interface and associate the recognized text content with the corresponding control object; The natural language processing unit is used to perform intent classification and keyword extraction on the text content extracted by the optical character recognition unit and the system pop-up text using a pre-trained language model based on the Transformer architecture, and to generate the semantic tags. The context aggregation unit is used to assemble the control object, the text content, and the semantic tags into a tree-like or graph-like context snapshot according to the interface hierarchy.

4. The intelligent process automation robot system based on a business rule engine according to claim 1, characterized in that, The core module of the rule engine includes: The rule compilation unit is used to parse the rule file provided by the business rule management module into an abstract syntax tree and generate Alpha and Beta nodes of the Rete network. The fact-judgment unit is used to receive the context snapshot and send each data item in the context snapshot as a fact judgment to the Rete network; The conflict resolution unit is used to select one rule to trigger from the multiple matched rules when multiple rules are matched simultaneously in the Rete network, based on a preset conflict resolution strategy. The conflict resolution strategy includes a sorting strategy based on rule priority, a strategy based on the most recent update time, a strategy based on rule complexity, or a strategy based on rule usage frequency. The instruction generation unit is used to parse the action part of the triggered rule into an executable instruction object. The instruction object includes an operation type enumeration, a target control locator chain, a timeout parameter, a retry count parameter, and an exception compensation action list.

5. The intelligent process automation robot system based on a business rule engine according to claim 4, characterized in that, The target control locator chain includes multiple locating strategies ordered by priority. Specifically, the multiple locating strategies are: a locating strategy based on control identifier, a locating strategy based on XPath, a locating strategy based on CSS selectors, a locating strategy based on control text content, a locating strategy based on the relative position of the control, and a locating strategy based on image feature matching. When generating the instruction object, the core module of the rule engine dynamically adjusts the priority order of each locating strategy in the locator chain according to the attribute integrity of the control in the context snapshot.

6. The intelligent process automation robot system based on a business rule engine according to claim 1, characterized in that, The dynamic path planning module includes: The topology graph construction unit is used to scan the user interface structure of the target application during the system initialization phase or the process orchestration phase, construct the operation path topology graph with interface elements as nodes and navigation relationships between elements as directed edges, and record the unique identifier, locator set and node type metadata of each node. The path caching unit is used to cache historically successfully executed operation paths and record the execution success rate and average execution time for each cached path; The path search unit, upon receiving a path interruption signal, retrieves the current real-time node set as the starting node set and the unique identifier of the target node as the target node, using A... The algorithm or Dijkstra's algorithm performs path search in the operation path topology graph, and the execution success rate of the cached path in the path cache unit is used as a heuristic weight during the search process. The path verification unit is used to simulate the preliminary steps of the candidate alternative path in a sandbox environment after the path search unit generates the candidate alternative path, verify the validity of the path, and submit the verified path to the process automation robot executor.

7. The intelligent process automation robot system based on a business rule engine according to claim 1, characterized in that, The process automation robot actuator includes: The instruction parsing unit is used to receive the real-time decision instruction and convert the real-time decision instruction into an operation sequence that can be recognized by the underlying automation framework. The operation execution unit is used to call the application programming interface of the underlying automation framework to execute the operation sequence, and record the start time, end time and execution result of each operation during the execution process; The status monitoring unit is used to monitor system logs, interface changes and abnormal pop-ups generated during the execution process. When a preset abnormal pattern is detected, the post-hook is actively triggered to feed back the abnormal information to the rule engine core module. The execution log unit is used to record the operation type, target control, execution result, time consumption information and exception stack of each operation as a structured log, and output the structured log to the audit system.

8. An intelligent process automation control method for an intelligent process automation robot system based on a business rule engine as described in any one of claims 1-7, characterized in that, Includes the following steps: Step S1: Load the business rule set through the business rule management module, compile the rules into a Rete network and load it into memory to establish a rule memory image; Step S2: Obtain the visual and text features of the target interface through the perceptual semantic parsing module, and construct a runtime context containing an interface element tree and semantic tags; Step S3: The runtime context is used as a fact judgment in the Rete network of the rule engine core module to trigger rule matching and generate a real-time decision instruction containing operation type, location strategy chain and exception fallback strategy; Step S4: The process automation robot executor parses the real-time decision instruction, triggers a pre-hook to send the current status to the rule engine core module for secondary confirmation before execution, and executes the automated operation after confirmation. Step S5: After the process automation robot actuator performs the operation, it triggers a post-hook to feed back the execution result to the core module of the rule engine. If the execution result does not match the expectation or an interface abnormality is detected, the abnormal handling rule in the business rule set is triggered to generate a compensation operation or call the dynamic path planning module to switch to an alternative path. Step S6: Record the rule matching trajectory, decision instruction sequence and operation execution log throughout the entire execution process, and generate a visual audit report.

9. The intelligent process automation control method according to claim 8, characterized in that, The trigger rule matching in step S3 specifically includes: Step S3.1: Insert each interface element in the runtime context as an independent fact object into the Alpha node of the Rete network and perform single-condition matching; Step S3.2: Pass the fact object matched by the Alpha node to the Beta node for multi-condition cross-matching and variable binding; Step S3.3: When a fact object satisfies all the conditions of any rule, put the rule and its bound variable values ​​into the agenda queue; Step S3.4: The conflict resolution unit selects a rule from the agenda queue according to the preset conflict resolution strategy, triggers it, and executes the action part of the rule.

10. The intelligent process automation control method according to claim 8, characterized in that, The step S5, which involves calling the dynamic path planning module to switch to the alternative path, specifically includes: Step S5.1: Receive the path interruption signal sent by the process automation robot actuator and the real-time control tree of the current interface through the dynamic path planning module; Step S5.2: Using the dynamic path planning module, the operation path topology is traversed from the root node of the real-time control tree to the target operation node. Step S53: Use A The algorithm calculates the path cost, and the formula for calculating the path cost is: F(n)=G(n)+H(n), where G(n) is the actual cost from the starting point to the current node, calculated based on the number of operation steps and the historical execution success rate, and H(n) is the heuristic cost from the current node to the target node, calculated based on the spatial distance and semantic similarity between interface elements. Step S5.4: Select the path with the lowest calculated cost as the candidate alternative path, and verify the reachability of the first N nodes in the candidate alternative path in the sandbox environment through the path verification unit, where N is a preset positive integer; Step S5.5: Update the verified alternative path to the execution queue of the process automation robot actuator, and mark the original path as a failed path.

Images