Automatic process processing system

Abstract

The invention discloses an automatic process processing system, which is characterized in that an acquisition module acquires a target operation instruction and modeling data for target software, so that modeling of time sequence information of an operation behavior is realized; then, the analysis module can determine corresponding action nodes in target software when the target operation instruction is completed based on the target operation instruction, and action sequences corresponding to multiple continuous operations used for executing the target operation instruction can be rapidly determined by combining the dependency relationship among the action nodes in the modeling data; therefore, the action sequence for automatic process processing can be quickly determined through the dependency relationship between the action nodes, and compared with the related technology, a user does not need to separately encode each step of operation, so that the operation efficiency is improved, and the user experience is improved. The automatic processing of the process can be realized through the dependency relationship among the action nodes, and the efficiency of the automatic processing of the process is effectively improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a process automation system. Background Technology

[0002] Current desktop software suffers from a high degree of reliance on manual operation and insufficient automation in complex workflows. Tasks such as data import, spatial analysis, map output, and data management often require users to manually repeat a large number of operations (such as clicking, inputting, and setting parameters), resulting in low productivity and a high risk of errors.

[0003] The relevant technologies mainly use Python scripts or command-line tools to automate the operation process.

[0004] However, the automation of the process described above is inefficient. Summary of the Invention

[0005] In view of this, this application provides a process automation system that can quickly determine the sequence of actions for process automation by means of the dependencies between action nodes. Compared with related technologies, it eliminates the need for users to encode each step separately, thus effectively improving the efficiency of process automation.

[0006] To solve the above problems, the technical solution provided in this application is as follows:

[0007] On the one hand, this application provides a process automation system, which includes an acquisition module, a parsing module, and an execution module:

[0008] The acquisition module is used to acquire target operation instructions and modeling data for the target software. The modeling data is used to identify the action nodes corresponding to the user's operation behavior, and there are dependencies between multiple action nodes.

[0009] The parsing module is used to determine an action sequence based on the target operation instruction and the dependency relationship between the multiple action nodes in the modeling data. The action sequence includes action nodes corresponding to the continuous operations that complete the target operation instruction.

[0010] The execution module is used to process the target operation instruction according to the action sequence.

[0011] In one possible implementation, the parsing module includes an intent recognition module, a semantic matching module, and a determination module:

[0012] The intent recognition module is used to recognize the target operation command and obtain the target intent;

[0013] The semantic matching module is used to determine the corresponding action node based on the target intent;

[0014] The determining module is used to determine the action sequence based on the action node, the current action node, and the dependencies between the multiple action nodes in the modeling data.

[0015] In one possible implementation, the system further includes a repair module for:

[0016] During the process of processing the target operation instruction according to the action sequence, if the first node in the action sequence is abnormal, a second node matching the first node is determined based on the modeling data.

[0017] Based on the second node, the action sequence is adjusted.

[0018] In one possible implementation, the system further includes an optimization module for:

[0019] Optimization suggestions are generated based on historical execution data;

[0020] Based on the optimization suggestions, the action sequence is optimized.

[0021] In one possible implementation, the system further includes an assertion module for:

[0022] Set an assertion for the first target node in the action sequence using one or more of the following assertion forms: file assertion, value assertion, or state assertion;

[0023] or,

[0024] Assertions are automatically generated based on the semantic information of the first target node.

[0025] In one possible implementation, the system includes an error handling module for:

[0026] During the processing of the target operation instruction according to the action sequence, if an abnormality occurs in the third node of the action sequence, the execution is stopped, the process jumps to the error handling mechanism, or the third node is automatically re-executed after a preset waiting time is reached. If the third node becomes abnormal again, it is processed in one of the ways of the error handling mechanism.

[0027] In one possible implementation, the system further includes a debugging module for:

[0028] The action sequence can be debugged by setting pre-defined breakpoints, or the action sequence can be debugged step by step.

[0029] In one possible implementation, the parsing module is further configured to:

[0030] Obtain a modification instruction, wherein the modification instruction is used to identify an instruction to modify the action sequence;

[0031] The action sequence is adjusted based on the modification instructions.

[0032] In one possible implementation, the acquisition module includes an operation acquisition module and an operation modeling module:

[0033] The operation acquisition module is used to collect interaction data, which is used to identify multiple user operations on the target software.

[0034] The operation modeling module is used to construct modeling data based on the interaction data.

[0035] In one possible implementation, the system further includes a storage module for:

[0036] The execution result of the target operation instruction is saved.

[0037] As can be seen from the above technical solution, this solution acquires the target operation instructions and modeling data for the target software through the acquisition module. The modeling data is used to identify the action nodes corresponding to the user's operation behavior. Multiple action nodes have dependencies, thereby realizing the modeling of the temporal information of the operation behavior. Then, the parsing module can determine the corresponding action node in the target software when the target operation instruction is completed based on the target operation instruction. Combining the dependencies between multiple action nodes in the modeling data, the action sequence corresponding to multiple consecutive operations used to execute the target operation instruction can be quickly determined. Then, the execution module can realize the automated processing flow for the target operation instruction based on the action sequence. Thus, by using the dependencies between action nodes, the action sequence used for automated process processing can be quickly determined. Compared with related technologies, users do not need to encode each operation separately. The automated process processing can be realized through the dependencies between action nodes, which effectively improves the efficiency of automated process processing. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1One of the schematic diagrams of a process automation system provided in this application embodiment;

[0040] Figure 2 This is a flowchart illustrating an automated processing method provided in an embodiment of this application. Detailed Implementation

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0042] As described in the background section, although the complex operation processes of current GIS desktop software have fixed rules and clear logic, they are difficult to automate through traditional scripting methods. This is because the relevant automated testing or batch processing mechanisms (such as Python scripts or command-line tools) require users to have certain programming skills. Ordinary business users cannot get started quickly and need to code each step in the operation process, resulting in low processing efficiency.

[0043] To address the low efficiency issue in automated process processing mentioned above, this application provides an automated process processing system. An acquisition module obtains target operation instructions and modeling data for the target software. The modeling data identifies action nodes corresponding to user actions, with dependencies between multiple action nodes, thus modeling the temporal information of the operation. A parsing module, based on the target operation instructions, determines the corresponding action nodes in the target software when the target operation instruction is completed. Combining the dependencies between multiple action nodes in the modeling data, a sequence of actions corresponding to multiple consecutive operations for executing the target operation instructions can be quickly determined. The execution module then uses this action sequence to automate the process for the target operation instructions. Therefore, by leveraging the dependencies between action nodes, the action sequence for automated process processing can be quickly determined. Compared to related technologies, this eliminates the need for users to individually code each step; automated process processing is achieved through the dependencies between action nodes, effectively improving efficiency. The solution provided in this application relates to the field of data processing technology, and will be specifically described through the following embodiments.

[0044] See Figure 1 The diagram shown is one of the schematic diagrams of a process automation system provided in this application embodiment, including an acquisition module 101, a parsing module 102, and an execution module 103.

[0045] The acquisition module 101 is used to acquire the target operation instructions and modeling data for the target software.

[0046] The modeling data is used to identify the action nodes corresponding to the user's operation behavior. There are dependencies between multiple action nodes. For example, there are logical relationships between multiple action nodes (such as determining the next action node when modifying the variable of the current action node) or temporal relationships (such as the imported data can only be processed after the imported data is imported). This application embodiment does not impose specific limitations on this.

[0047] This application does not impose specific restrictions on the method of obtaining modeling data. For example, the user's operation behavior in the target software can be converted into a system-parseable event sequence to obtain the action nodes corresponding to multiple consecutive operations, or the modeling data can be obtained by constructing the dependency relationship between multiple action nodes.

[0048] Based on the dependencies between action nodes, the action sequence that can reach the target action node can be determined according to the current action node.

[0049] As one possible implementation, the acquisition module includes an operation acquisition module and an operation modeling module:

[0050] The operation acquisition module is used to collect interaction data, which is used to identify multiple user actions on the target software.

[0051] The operation node modeling module is used to construct modeling data based on interactive data.

[0052] Specifically, the operation acquisition module uses low-intrusion desktop event capture technology to convert user operations within the software into a sequence of events that the system can parse. For example, this operation acquisition module may include the following:

[0053] (1) System event listening submodule: Listen to mouse, keyboard, window and control events through the event interface, and record control position, control properties, control state before and after operation, key value and timestamp, etc. All the recorded information is called behavior object, which constitutes the interaction data with the target software.

[0054] (2) Event Analysis Submodule: Behavior objects need to have a hierarchical relationship, that is, if a behavior depends on the previous behavior, then the behavior belongs to the child level of the previous behavior. Therefore, control tree traversal is used. Based on the control object bound to the event, the hierarchical structure of the control tree is located, and then the hierarchical relationship of the behavior object corresponding to the recorded event is determined. For example: the user first clicks the right mouse button in a tree control to pop up a menu, then clicks the left mouse button on a menu item in the menu to pop up a form interface, and then clicks the left mouse button on a button in the form interface. Then there is such a progressive hierarchical relationship: right mouse button pop-up menu → left mouse button click menu item → pop-up form → left mouse button click button. This example can determine that the hierarchical relationship of the control bound to the left mouse button click event is: the child level of the pop-up form. Therefore, the parent behavior object of the left mouse button click event is the pop-up form behavior object. Analyze the semantics of the events, such as clicking a button, selecting a tree node, double-clicking a button, panning the map, zooming the map, selecting objects on the map, etc. If it is a control area with an absolute coordinate system, such as a map or layout, the absolute coordinates also need to be recorded so that the pixel coordinates can be determined based on the absolute coordinates during automatic restoration, thus achieving consistent restoration.

[0055] (3) The operation acquisition module also includes an event sequence cache: the operation records are saved based on the time order and the behavior object level, that is, the same behavior object level is saved from the oldest to the newest in time order; the record format is uniformly JSON format, and is finally saved as a JSON file, which is convenient for parsing and modification.

[0056] (4) The operation acquisition module may also include a recording control interface: providing functions for starting recording, pausing recording, ending recording and playback; supporting users to modify the recording results, such as copying, pasting, adjusting the order of actions, deleting and commenting.

[0057] The sequence of behavioral objects is then transformed into reusable action nodes through the operation modeling module. For example, the core structure of the operation modeling module includes:

[0058] (1) Action parser: Users can combine multiple strongly related semantic behaviors of operation identified by the event analysis submodule into a single action, such as: select dataset → right-click the mouse → click to open to new map, which is abstracted as the "open dataset to new map" action node. The dataset can be arbitrarily specified as the action input parameter.

[0059] Building upon the action parser, an intelligent action node recognition mechanism can be introduced. After capturing the user's event sequence, the system uses machine learning algorithms (such as sequence pattern mining and decision trees) to perform high-level semantic abstraction of user behavior. For example, when the system detects that a user continuously performs a series of operations such as "import data → open to map → apply map template → export map as PDF", it can automatically identify it as a "batch map matching" mode and automatically generate a parameterized workflow template.

[0060] (2) Action Library Manager: Establish a unified action library. Each action includes: execution logic (e.g., clicking a button first, then entering text; this behavior is required), ID (unique identifier of the action), type (action type, such as map type action, layout type action, etc.; not required), input parameters (e.g., a dataset as mentioned above; not required and can be left blank), output parameters (the output result after the action is executed; for example, if the execution logic is to copy text, then the type of the output parameter can be defined as text type; not required and can be left blank), assertion settings (to determine whether the action is executed correctly). Both input and output parameters include parameter names and parameter types. For example, the input parameter name of the dataset is inputDataset, and the type is dataset. Similarly, the output parameter of the dataset is outputDataset, and the type is also dataset. This parameter definition can ensure that action nodes are correctly connected. Actions are also stored in JSON format for easy versioning and loading.

[0061] (3) Variable Mechanism: The variable mechanism is one of the key design features for achieving flexibility and reusability in software automation processes. During software operation automation, users often need to process various types of data, such as data paths, buffer radii, projected coordinate systems, and output directories. If these parameters are fixed within action nodes, the reusability and adaptability of the process will be greatly limited. Introducing a variable mechanism provides a unified semantic basis for data transfer, dynamic parameter replacement, and execution status maintenance between nodes in the process, allowing users to freely define, reference, and manage data during execution without modifying the underlying script. Variables are defined using %actionID.parameterName%, and if it is a global variable or the process has no action name, it is in the form %parameterName%. Variables can be referenced in the process. The execution engine will parse the %...% expression at runtime and replace the variable with the actual value in the current context.

[0062] (4) Parameter mapper: It is responsible for extracting possible input and output parameters from the combined single action, and marking the parameter type to facilitate the user to build action nodes; it also supports converting parameters into variable reference form, such as %openDatasetToNewMap.dataset%, for easy reuse.

[0063] (5) Process Editor: Supports combining and rearranging the action nodes in the action library manager into a reusable automated process that meets automation requirements in the visual process editor.

[0064] Therefore, by using the operation acquisition module and the operation modeling module, real interaction data with the target software can be obtained, and then continuous operation behaviors can be converted into multiple action nodes with dependencies to obtain modeling data, providing a solid data foundation for the generation of automated processing flow.

[0065] The target operation instruction refers to the operation task input by the user for the target software. This application embodiment does not impose specific limitations on this; for example, it can be an instruction to directly create or modify an automated process. In this application embodiment, a series of procedural operations need to be performed within the target software interface to complete the target operation instruction. The target operation instruction can be expressed in voice or text form, etc., and this application embodiment does not impose specific limitations on this.

[0066] After obtaining the target operation instructions and modeling data, the action sequence used to implement the target operation instructions can be determined from the modeling data.

[0067] The parsing module 102 is used to determine the action sequence based on the target operation instructions and the dependencies between multiple action nodes in the modeling data.

[0068] The action sequence includes action nodes corresponding to the continuous operations that complete the target operation instruction. Based on the action sequence, the operation process of the target operation instruction can be automatically executed.

[0069] This parsing module supports parsing instructions that directly create or modify automated processes, determining the corresponding action sequence, and obtaining the automated process file. For example, when a user inputs a target operation command such as "Create a process that automatically imports vector data and performs buffer analysis" or "Add a condition: if the coordinate system is not WGS84, perform a coordinate system transformation," the system can obtain the action sequence based on the action nodes corresponding to the target operation command and the action nodes corresponding to the target software in its current state. Exemplarily, the core features of the parsing module include the following.

[0070] (1) Sequence control: The default execution mode is to execute the action nodes step by step from top to bottom.

[0071] (2) Condition control: refers to the fact that the action node contains condition parameters. The condition parameters require input of variable values ​​and other logical expressions, and the corresponding action is adjusted to the sub-action node of the action node, so as to realize the decision to run based on the specific state during automatic execution.

[0072] (3) Loop control: It supports repeating certain action nodes multiple times according to the set number of loops; it also supports traversing a data list multiple times, taking out new data from the data list each time the loop is repeated, and repeating certain action nodes.

[0073] (4) Variable scope system: To prevent variable pollution, as shown in Table 1, a three-level variable scope model was designed to control the life cycle and visibility of variables.

[0074] Table 1

[0075]

[0076] This hierarchical management mechanism ensures that variables with the same name do not conflict in different processes or nodes, while allowing necessary context data to be shared across nodes.

[0077] (5) Sub-processes and reuse: It supports adding existing processes to new processes for combined use. The added processes are called sub-processes and are displayed as individual nodes, which facilitates task reuse.

[0078] As one possible implementation, the intelligent automation engine module includes an intent recognition module, a semantic matching module, and a determination module:

[0079] The intent recognition module is used to identify the target operation command and obtain the target intent.

[0080] The semantic matching module is used to determine the corresponding action node based on the target intent.

[0081] The determination module is used to determine the action sequence based on the action node, the current action node, and the dependencies between multiple action nodes in the modeling data.

[0082] The current action node refers to the action node corresponding to the target software in its current state.

[0083] The parsing module has a built-in semantic parsing and action template mapping mechanism: First, it analyzes the target intent, entity and parameters of the target operation instruction through a large language model; then, it finds the action nodes that match the target intent in the modeling data, fills in the parameters, and combines multiple action nodes according to the dependency relationship between the action nodes to generate the action sequence corresponding to the complete process. The generated complete process can be directly visualized and modified by the user in the process editor.

[0084] Therefore, by identifying the target intent corresponding to the target operation instruction, determining the corresponding action node, and generating the action sequence corresponding to the operation process of the target operation instruction, the accuracy of generating automated processes based on natural language is effectively improved.

[0085] The execution module 103 is used to process the target operation instructions according to the action sequence.

[0086] By parsing the action nodes in the action sequence, the corresponding operation behavior can be determined, thereby achieving automated execution of the target operation command. For example, this execution module includes the following five aspects.

[0087] (1) Process interpreter: parses the action sequence node by node, and calls the corresponding action processor according to the action node type (for behavior action nodes, restores the behavior action, enters the operation control proxy mechanism, and executes the logical expression of the logic module to determine the execution direction of the process).

[0088] (2) Action scheduler: Controls the execution order of action nodes according to the dependencies between action nodes; provides execution status (ready / in execution / paused / completed / failed) management, and can intuitively see the status of each action.

[0089] (3) Control operation delegate: Different methods are used to perform control operations for different controls. For example, for the action object of clicking a button, first search for the button control object in the UI based on the control name. If the button name does not exist, then search for the object based on the button's display text. If the button's display text still does not exist, then search for the object based on the button's tooltip text. If the button's tooltip text still does not exist, then search for the object based on the button type. If the object is still not found, then perform the specific operation of the action based on the recorded absolute pixel coordinates of the mouse, such as whether it is a click or a double click, etc.

[0090] (4) Execution Context Manager: A complete automated task flow often contains multiple nested operation modules: main flow (such as map making flow), sub-flow (such as data import, buffer analysis, coordinate transformation), and flow control (such as if the imported data is point data, then buffer is not supported, and only line data is processed by the buffer, etc.). There are variable passing, variable parsing, and identification of available variables for the current node between these modules. Therefore, this invention introduces an execution context manager to uniformly manage runtime information, enabling the system to have variable lifecycle management: responsible for parsing and assigning local, process, and global variables, dynamically maintaining a three-level variable table (global, process, local) during execution, and responsible for: parsing variable references %varName%; checking scope conflicts and occlusion relationships; updating variables when action nodes are created / destroyed according to lifecycle rules; supporting parameter passing, passing the output parameters after the action is executed to the next action, as input parameters or variable assignment, etc.

[0091] (5) Process pause and resume mechanism: Supports "pause execution" and "resume from interruption point", automatically records execution snapshots, including action node ID, variable status and time point, and supports resumption from interruption point.

[0092] Therefore, the acquisition module obtains the target operation instructions and modeling data for the target software. The modeling data is used to identify the action nodes corresponding to the user's operation behavior. Multiple action nodes have dependencies, thus realizing the modeling of the operation behavior. Then, the parsing module can determine the corresponding action nodes in the modeling data based on the target operation instructions. Combining the dependencies between multiple action nodes in the modeling data, the action sequence corresponding to multiple consecutive operations used to execute the target operation instructions can be quickly determined. Then, the execution module can realize the automated processing of the target operation instructions based on the action sequence. Thus, by using the dependencies between action nodes, the action sequence for process automation can be quickly determined. Compared with related technologies, users do not need to code each operation separately, which effectively improves the efficiency of process automation.

[0093] As one possible implementation, the system also includes a repair module for:

[0094] During the process of processing the target operation instructions according to the action sequence, if the first node in the action sequence is abnormal, a second node matching the first node is determined based on the modeling data and execution data.

[0095] The action sequence is adjusted based on the second node.

[0096] This application provides an intelligent repair mechanism. During automated execution, if the first node fails, the failure reason (such as incorrect file path, unloaded data, or unavailable control) is determined by combining the generated execution data, such as execution logs, and modeling data, such as the context of the first node, the data information of the previous node and the next node. Then, through a semantic matching mechanism, a second node with a similarity to the first node that reaches a preset threshold is determined, and the action sequence is adjusted, such as replacing the first node in the action sequence with the second node, to ensure the normal execution of the automated process.

[0097] In addition, when an anomaly occurs at the first node, the action sequence can be adjusted in the following ways:

[0098] Automatically adjust control matching strategies (e.g., switch from control name matching to text matching); search for similar filenames in several adjacent folders to find the best matching file; adjust the waiting time or perform a retrieval; rebind the best matching action node through a semantic matching mechanism and retry, etc.

[0099] For example, although the operations "export map" and "output map results" correspond to different operations on the target software interface, they can be automatically matched to the same operation semantics through semantic similarity calculation, avoiding process execution failure. Moreover, by adjusting the action sequence, a new process file can be obtained. The new process file name format is: old process file name + version number, realizing a traceable and recoverable "self-healing automated process".

[0100] Therefore, the repair module can repair abnormal action nodes during automated execution, ensuring the normal execution of the automated process and effectively improving the reliability of the system.

[0101] As one possible implementation, the system also includes an optimization module for:

[0102] Optimization suggestions are generated based on historical execution data.

[0103] Based on the optimization suggestions, the action sequence was optimized.

[0104] Historical execution data refers to historical data on operations performed on the target software. This application does not impose specific limitations on this. For example, it can be historical execution data corresponding to multiple action nodes, including input information, execution time, performance indicators, etc. of each action node.

[0105] The optimization suggestion refers to a suggestion to optimize the action sequence. This application does not impose specific limitations on this. For example, it can be an optimization suggestion for the parameters of an action node in the action sequence, or it can be a process optimization suggestion for the corresponding action sequence.

[0106] Historical execution data can be used to generate optimization suggestions for action sequences. Based on these suggestions, the action sequences can be optimized directly, or corresponding operation controls can be provided to users. When the user triggers the corresponding operation control, the action sequences can be optimized.

[0107] For example, if action node A has an extremely high execution frequency and fixed parameters, the system can provide suggestions to the user, encapsulating it as a "common sub-process." If there are redundant waiting or repetitive actions in the process, the system automatically identifies and prompts for optimization to reduce waiting time or remove repetitive actions. If assertions fail frequently, the system automatically suggests adding delay control or conditional judgment nodes. The system also supports an AI-trained "process evaluation mechanism," which comprehensively evaluates based on dimensions such as execution success rate, average time consumption, and reuse count, and generates optimization suggestion reports. Users can apply optimization suggestions with one click, allowing the system to automatically adjust the action sequence.

[0108] Therefore, by using historical execution data, it is possible to optimize the sequence of actions, which in turn helps to improve the efficiency and accuracy of automated process handling.

[0109] As one possible implementation, the system also includes an assertion module for:

[0110] Set an assertion for the first target node in the action sequence using one or more of the following assertion forms: file assertion, value assertion, or state assertion;

[0111] or,

[0112] Assertions are automatically generated based on the semantic information of the first target node.

[0113] The first target node refers to one or more action nodes in the action sequence, and this application embodiment does not impose specific limitations on this.

[0114] Assertions are quality control points for process execution. The system supports various assertion forms, including file assertions (determining whether a file exists, the number of files, etc.), value assertions (whether a is equal to b, whether a is less than b, etc.), status assertions (whether a tree node is selected, whether a text box control is available, etc.), and so on.

[0115] By asserting the first target node in an action sequence using one or more of the following assertion forms: file assertion, value assertion, or state assertion, the execution status of the first target node can be comprehensively monitored.

[0116] Semantic reasoning capabilities can also be introduced to identify the semantic information of the first target node and automatically generate assertions. For example, when the system detects that the user performs the "export results to folder" operation during recording, it automatically adds an assertion that "the file exists and its size is greater than 0"; when it detects a map drawing object operation, it automatically suggests an assertion that "the result object has been added successfully".

[0117] Therefore, the assertion module can evaluate the intermediate steps of the automated process and enable timely handling of abnormal information, thereby further improving the reliability of the system.

[0118] As one possible implementation, the system includes an error handling module for:

[0119] During the processing of target operation instructions according to the action sequence, if an exception occurs in the third node of the action sequence, execution is stopped, the process jumps to the error handling mechanism, or the third node is automatically re-executed after a preset waiting time is reached. If the third node fails again, it is handled in one of the ways of the error handling mechanism.

[0120] This application does not impose specific restrictions on the error handling mechanism; for example, it may allow debugging of the third node or adjustment of the action sequence.

[0121] If the third node encounters an error, the number of retries can be configured. If multiple retries still fail, a screenshot of the current screen is taken and saved to help users find the cause, and then the execution is terminated.

[0122] Therefore, the error handling module can handle abnormally executed action nodes, thereby ensuring the reliability of the system.

[0123] As one possible implementation, the system also includes a debugging module for:

[0124] You can debug the action sequence by setting pre-defined breakpoints or perform step-by-step debugging of the action sequence.

[0125] This debugging module supports step-by-step debugging of automated processes, similar to code debugging, making it easier to pinpoint the cause of automated process failures. It provides the following capabilities: breakpoint setting (adding breakpoints to action nodes; when the process reaches that action node, the breakpoint is triggered, entering debugging mode); single-step execution (running actions step by step); process rollback (rolling back to previous actions); and variable injection (dynamically modifying variable values).

[0126] The debugging module has a corresponding debugging interface, which mainly includes three panels: the process panel, which displays the execution order of each action node in the process, from top to bottom and from newest to oldest, that is, the action node executed later is displayed at the top and the action node executed earlier is displayed at the bottom; the variable and parameter panel, which displays real-time variables and input and output parameters; and the injection panel, which allows you to dynamically modify variable values.

[0127] Therefore, the debugging module can be used to debug action nodes that may have execution errors, further improving the reliability and accuracy of automated process processing.

[0128] As one possible implementation, the parsing module is also used for:

[0129] Get the modification instructions.

[0130] The action sequence is adjusted based on the modification instructions.

[0131] The modification instruction is used to identify instructions that modify the action sequence. The method of obtaining the modification instruction in this embodiment is not specifically limited; for example, it can be a natural language instruction input by the user to modify the action sequence, or an instruction generated by the system through historical execution data to optimize the action sequence.

[0132] After obtaining the modification instruction, the action sequence is adjusted by parsing the modification instruction. For example, if the modification instruction is to add a sub-process, the sub-process is added to the appropriate position based on the dependency relationship between multiple actions in the action sequence. If the modification instruction is to modify the parameters in action node A, the subsequent action nodes are re-determined based on the modified action node A.

[0133] Therefore, by modifying the instructions, the action nodes can be further adjusted, thereby improving the flexibility of automated process handling.

[0134] As one possible implementation, the system also includes a storage module for:

[0135] Save the execution result of the target operation instruction.

[0136] All action execution results (including timestamps, time elapsed, error messages, execution information, etc.) are recorded in a log file, which can be used to locate the cause of execution failure and perform performance analysis.

[0137] Furthermore, this application designs a pluggable adapter layer, providing a unified external interface. Custom-developed software, and custom controls that fail to adapt correctly to software automation, all need to implement this interface. The execution engine loads the corresponding adapter at runtime based on the configuration file, enabling cross-software operations and meeting the needs of business automation scenarios.

[0138] To more clearly describe this type of automated process system, a specific implementation scenario will be used as an example. The following example illustrates an automated process task for a Geographic Information System (GIS).

[0139] While existing general-purpose software automation tools can achieve interface-level automated operation, they lack the ability to identify spatial objects, adapt map windows, and encapsulate analysis tasks for GIS scenarios, making it difficult to meet the specific needs of GIS.

[0140] Therefore, this invention proposes a process automation system for GIS software, aiming to solve the problems of traditional GIS software operation relying on manual labor, high repetition, high script development threshold, and the inability of general automation platforms to understand the unique mechanisms of GIS.

[0141] This system achieves semantic recognition, process extraction, intelligent generation, visual editing, and automatic execution of GIS software operation processes through the collaboration of event capture, operation modeling, process control, and execution engine. Simultaneously, the system introduces an intelligent automation engine that, through natural language parsing, contextual semantic matching, and automatic repair mechanisms, enables users to define, modify, and optimize automated processes in a "human language."

[0142] Through the technical solution of this invention, ordinary users do not need to write code; they only need to perform an operation once or input a natural language command (such as "import all vector data in the C drive data folder and batch export maps as PDFs using default map images") to automatically generate an editable GIS automated workflow. The system can not only recognize GIS software interface elements and geospatial operation semantics, but also perform workflow repair, parameter optimization, and adaptive learning based on the running results and user feedback, realizing the evolution from "passive execution" to "intelligent understanding and optimization".

[0143] The system includes an operation acquisition module, an operation modeling module, a process control module, an execution engine module, an assertion and error handling module, a debugging and monitoring module, an intelligent automation engine module, and an extension module.

[0144] First, the operation acquisition module is responsible for capturing all user operation events within the GIS software. The system listens for mouse, keyboard, window, and control events, and extracts interface elements from event analysis to generate a structured operation log script file. By recording information such as timestamps, control attributes, and coordinates, the system can completely reconstruct user interaction behavior, providing a data foundation for subsequent automated modeling.

[0145] Secondly, the operation modeling module abstracts these low-level events into high-level "action nodes." Users can abstract multiple consecutive operations (such as "open layer," "perform buffer analysis," "export map as image," etc.) into standard action nodes with parameters. Action nodes are stored in a unified action library and can be reused in different workflows. This module also supports parameter extraction and variableization, transforming fixed user input into variable references, thereby enabling configurable workflows.

[0146] During the process construction phase, the process control module is responsible for combining action nodes into complete execution logic. The system defines a process structure that supports sequential execution, conditional branching, loop structures, and variable management. Users can then orchestrate these processes through a visual process editor, creating automated processes with programming language capabilities.

[0147] The execution engine module is the core of the system, responsible for parsing script files and executing corresponding operations. The engine controls action execution and manages action states through an action scheduler. Simultaneously, the control operation proxy mechanism automatically selects the optimal execution method based on the action type and the current software state, including control property calls and keyboard / mouse event execution. During execution, the system maintains variables and context information to achieve dynamic data transfer and output binding within the process.

[0148] To ensure the accuracy and reliability of execution, the system includes assertion and error handling modules. Users can insert assertion nodes into the workflow to verify conditions such as file existence, parameter value matching, or consistency of GIS spatial data. If an assertion fails, the system can automatically execute an exception branch or retry logic.

[0149] In addition, the system provides a debugging and monitoring module that supports visual debugging functions such as breakpoints, single-step execution, variable monitoring, and log tracing. Users can view the process status in real time and dynamically modify variables, thereby quickly locating and fixing process logic problems.

[0150] To make the automation system more intelligent and capable of self-learning and evolution, an intelligent automation engine module was introduced. This engine, based on rule learning, semantic understanding, and context modeling technologies, provides the following three types of advanced functions:

[0151] (1) Natural language process generation: When the user inputs "import data in batches and output as PDF after plotting", the system automatically generates the corresponding process logic and parameter binding;

[0152] (2) Self-repair: When the control changes at the control tree level or the window becomes unresponsive, the system automatically adjusts the process nodes according to historical semantics and similarity matching rules;

[0153] (3) Intelligent optimization suggestions: By analyzing execution logs and exception records, performance bottlenecks, repetitive operations and potential error points are automatically identified, and improvement solutions are recommended.

[0154] Finally, the extension module defines a unified interface, allowing software customization to be adapted to the current business scenario. The execution engine dynamically loads the corresponding adapter at runtime based on the configuration, enabling automated processes to run across software versions without modifying the process itself.

[0155] Through the collaborative work of the aforementioned modules, the system achieves a complete closed loop from "operation capture—semantic modeling—logic control—execution engine—intelligent repair—process optimization." Ultimately, it realizes an intelligent automation system that is understandable, editable, learnable, and evolvable.

[0156] refer to Figure 2The diagram shown is a flowchart of an automated process processing method provided in an embodiment of this application. First, operation records for GIS software are acquired. Then, an event sequence is constructed based on the operation records. The event sequence includes mouse events, keyboard events, and window events. Next, process modeling is performed based on the event sequence, action nodes, and an intelligent automation engine. Then, the execution engine combines the process modeling results and the parsing results of the intelligent automation engine, and performs automated execution of the process through control operation agents. During the execution process, intelligent assertions, intelligent repairs, and process optimizations are performed through the intelligent automation engine, thereby realizing assertion and error handling, debugging, and monitoring.

[0157] Thus, this application promotes the development of GIS software from "passive operation" to an intelligent stage of "self-understanding, self-repair, and self-evolution," and has the following beneficial effects:

[0158] (1) Lowering the threshold for automation: Through operation recording and natural language understanding, non-technical users can also generate automated processes;

[0159] (2) Enhance controllability and robustness: Ensure process reusability and executableness through variable mechanisms, assertions and automatic repair mechanisms;

[0160] (3) Intelligent optimization and self-learning: The built-in intelligent automation engine analyzes process execution data, automatically detects anomalies, and generates optimization suggestions;

[0161] (4) Scalability: Through the interface adaptation layer, it supports the adaptation of GIS software for business expansion development and can also perform software automation.

[0162] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0163] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A process automation system, characterized in that, The system includes an acquisition module, a parsing module, and an execution module: The acquisition module is used to acquire target operation instructions and modeling data for the target software. The modeling data is used to identify the action nodes corresponding to the user's operation behavior, and there are dependencies between multiple action nodes. The parsing module is used to determine an action sequence based on the target operation instruction and the dependency relationship between the multiple action nodes in the modeling data. The action sequence includes action nodes corresponding to the continuous operations that complete the target operation instruction. The execution module is used to process the target operation instruction according to the action sequence.

2. The system according to claim 1, characterized in that, The parsing module includes an intent recognition module, a semantic matching module, and a determination module: The intent recognition module is used to recognize the target operation command and obtain the target intent; The semantic matching module is used to determine the corresponding action node based on the target intent; The determining module is used to determine the action sequence based on the action node, the current action node, and the dependencies between the multiple action nodes in the modeling data.

3. The system according to claim 1, characterized in that, The system also includes a repair module for: During the process of processing the target operation instruction according to the action sequence, if the first node in the action sequence is abnormal, a second node matching the first node is determined based on the modeling data and execution data. Based on the second node, the action sequence is adjusted.

4. The system according to claim 1, characterized in that, The system also includes an optimization module for: Optimization suggestions are generated based on historical execution data; Based on the optimization suggestions, the action sequence is optimized.

5. The system according to claim 1, characterized in that, The system also includes an assertion module for: Set an assertion for the first target node in the action sequence using one or more of the following assertion forms: file assertion, value assertion, or state assertion; or, Assertions are automatically generated based on the semantic information of the first target node.

6. The system according to claim 1, characterized in that, The system includes an error handling module for: During the processing of the target operation instruction according to the action sequence, if an abnormality occurs in the third node of the action sequence, the execution is stopped, the process jumps to the error handling mechanism, or the third node is automatically re-executed after a preset waiting time is reached. If the third node becomes abnormal again, it is processed in one of the ways of the error handling mechanism.

7. The system according to claim 1, characterized in that, The system also includes a debugging module for: The action sequence can be debugged by setting pre-defined breakpoints, or the action sequence can be debugged step by step.

8. The system according to claim 1, characterized in that, The parsing module is also used for: Obtain a modification instruction, wherein the modification instruction is used to identify an instruction to modify the action sequence; The action sequence is adjusted based on the modification instructions.

9. The system according to claim 1, characterized in that, The acquisition module includes an operation acquisition module and an operation modeling module: The operation acquisition module is used to collect interaction data, which is used to identify multiple user operations on the target software. The operation modeling module is used to construct modeling data based on the interaction data.

10. The system according to claim 1, characterized in that, The system also includes a storage module for: The execution result of the target operation instruction is saved.

Images