Automatic process generation method and system based on user operation capture

Abstract

The invention discloses an automatic process generation method and system based on user operation capture, and belongs to the technical field of process coding processing, the automatic process generation method based on real-time operation capture comprises the following steps: step S100: monitoring and capturing operation of a user in a browser or a client page, and obtaining user operation behaviors allowed to be recorded; s200, analyzing the user operation behaviors allowed to be recorded, and generating a user operation process; and S300, converting the user operation process into an automatic process script, and generating an automatic process. According to the method, the user operation can be converted into the executable script, the manual code development and test time is shortened, the method is suitable for a rapid iteration development environment, and non-technical users can easily capture the user operation through recording operation and generate an automatic process.

Description

Technical Field

[0001] The present invention belongs to the technical field of process coding processing, and specifically relates to an automated process generation method and system based on user operation capture. Background Art

[0002] In existing automated process development, users often need to manually write scripts or navigate complex user interfaces to define automated tasks. This is not only inefficient but also requires a high level of technical expertise. Furthermore, existing user operation recording technologies often only record user actions, making it difficult to convert them into reproducible automated processes. Therefore, the development of new process encoding technologies is urgently needed. Summary of the Invention

[0003] This invention provides a method and system for automated process generation based on user operation capture, aiming to partially or completely resolve the technical problems of low efficiency and complex operation of automated process generation in the prior art. To achieve the above objectives, this invention adopts the following technical solutions:

[0004] The first aspect is an automated process generation method based on user operation capture, including:

[0005] Step S100: monitoring and capturing the user's operations in the browser or client page, and obtaining the user's operation behaviors that can be recorded;

[0006] Step S200: Analyze the user operation behaviors that can be recorded and generate a user operation process;

[0007] Step S300: The user operation process is converted into an automated process script to generate an automated process.

[0008] Optionally, step S100 includes: a script or plug-in embedded in the browser or client page, which monitors and captures user operation behaviors that can be recorded in real time. User operation behaviors that can be recorded include user clicks, inputs, movements and other operations. The captured user operation behaviors that can be recorded will be marked with a timestamp and stored.

[0009] Optionally, in step S100, the user operation behaviors that may be recorded include browser operations, which include DOM events, keyboard input, mouse operations, and page navigation; and / or, the user operation behaviors that may be recorded include client interaction operations, which include button clicks, keyboard input, and mouse operations.

[0010] Optionally, the recorded user operation rows may be stored as structured format data, where the structured format data includes an operation type, a target element, an operation content, and a timestamp.

[0011] Optionally, step S200 includes: pre-processing the user operation behavior data that can be recorded, identifying the pattern and logical relationship of the user operation, extracting the operation node steps of the user operation behavior, and converting them into a structured user operation process.

[0012] Optionally, step S200 includes:

[0013] Step S201: pre-processing the user operation behavior data that can be recorded to obtain processed user operation behavior data, where the processed user operation behavior data includes a processed user operation behavior sequence and a processed time sequence;

[0014] Step S202: Analyze the processed user operation behavior data, identify the pattern and logical relationship of the user operation, and extract the operation node steps of the user operation behavior;

[0015] Step S203: Generate a user operation process according to the operation node steps of the user operation behavior.

[0016] Optionally, the user uses the recording operation monitoring to capture the user's operations in the browser or client page, and uses the viewing operation acquisition to generate an automated process.

[0017] In a second aspect, an automated process generation system based on user operation capture is used to implement any of the automated process generation methods based on user operation capture described in the first aspect, including:

[0018] User operation behavior acquisition module, which monitors and captures user operations in the browser or client page, and obtains user operation behaviors that can be recorded;

[0019] The user operation process acquisition module analyzes the user operation behaviors that can be recorded and generates the user operation process;

[0020] The automated process generation module converts user operation processes into automated process scripts to generate automated processes.

[0021] In a third aspect, a computer-readable storage medium stores instructions, which, when executed on a computer, executes any one of the automated process generation methods based on user operation capture described in the first aspect.

[0022] In a fourth aspect, an automated process generation device based on user operation capture includes a memory and a processor that are communicatively connected, wherein the memory is used to store a computer program, and the processor is used to read the computer program and execute the automated process generation method based on user operation capture as described in any one of the first aspects.

[0023] (1) In the present invention application, first, from user operation capture (step S100) to process analysis (step S200) and then to script generation (step S300), end-to-end automation is achieved, which significantly reduces the manual code development and testing time. Through real-time monitoring and algorithm analysis, user operations are converted into executable scripts, which are suitable for rapid iteration development environments. In addition, by recording operations and viewing operations (such as browser extensions or client tools), non-technical users can also easily capture operations and generate automated processes without in-depth programming knowledge. It is suitable for browsers (Chrome, Firefox, Edge) and client applications, and supports Web and desktop automation. The script generated in step S300 is compatible with a variety of automation tools (such as Selenium, Playwright, UiPath, AutoIt), adapted to different technology stacks, and the generated processes and scripts can be reused for different users or scenarios.

[0024] (2) In the present invention application, multiple operating systems (Windows, macOS, Linux, etc.) and browser environments (Chrome, Firefox, Edge, Safari, etc.) are supported, and seamless recording of user operations and automated process generation are achieved. User operations are captured in real time and automated processes are automatically generated, which significantly improves the development efficiency of automated tasks. Users do not need to write code and can complete the generation of automated processes through simple interface operations. It can adapt to complex IT environments, reduce development and deployment costs, meet the diverse needs of enterprise automated testing and process optimization, and provide technical support for enterprise process automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 This is a flow chart of the method for generating an automated process based on user operation capture applied for by the present invention;

[0027] Figure 2 This is a schematic diagram of the composition of the automatic process generation system based on user operation capture applied for by the present invention;

[0028] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] In the description of the present invention, it should be understood that the terms "upper," "lower," "front," "back," "left," "right," "top," "bottom," "inside," and "outside" and the like, indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. To further clarify the objectives, technical solutions, and advantages of the present invention, embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0031] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means one, two, or more than two, unless otherwise specifically defined.

[0032] In order to make the purpose, technical solutions and advantages of the present invention application clearer, the technical solutions in the embodiments of the present invention application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention application. Obviously, the described embodiments are only part of the embodiments of the present invention application, not all the embodiments; based on the embodiments in the present invention application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention application.

[0033] Automatic process generation method based on user operation capture

[0034] like Figure 1 As shown, in the first aspect, the automated process generation method based on user operation capture includes:

[0035] Step S100: monitoring and capturing the user's operations in the browser or client page, and obtaining the user's operation behaviors that can be recorded;

[0036] Specifically, step S100 includes: a script or plug-in embedded in a browser or client page, which monitors and captures user operation behaviors that can be recorded in real time. User operation behaviors that can be recorded include user clicks, inputs, movements and other operations. The captured user operation behaviors that can be recorded will be marked with a timestamp and stored.

[0037] In some embodiments, before real-time monitoring captures user operation behaviors that are allowed to be recorded, the user can be clearly informed that the user operation behaviors that are allowed to be recorded will be recorded, and the monitoring and acquisition of the user operation behaviors that are allowed to be recorded must comply with local privacy regulations, etc.

[0038] In some embodiments, a JavaScript script or plug-in is embedded in the browser or client page to monitor and capture user operations that can be recorded in real time. The JavaScript script or plug-in can capture user operation events that can be recorded through event monitoring mechanisms such as onclick, oninput, and onchange.

[0039] In some embodiments, user operations that may be recorded include browser operations, which include DOM events (such as clicking buttons and links), keyboard input (such as text box input), mouse operations (such as dragging and hovering), and page navigation (such as page jumping and refreshing).

[0040] In some embodiments, user operations that can be recorded include client interaction operations. Client interaction operations can integrate similar functions through SDK or API to capture client interaction operations. Client operations include clicking buttons, keyboard input (such as text box input), and mouse operations (such as dragging and hovering).

[0041] In some embodiments, each time a user operation behavior that can be recorded is captured, a timestamp (accurate to milliseconds) is attached to the user operation behavior that can be recorded, and the time when the user operation behavior that can be recorded occurs is recorded. The timestamp marks the user operation behavior sequence and time series that can be recorded.

[0042] In some embodiments, the user operation behaviors that can be recorded can be stored in structured format data (such as JSON or XML). The structured format data includes operation type, target element (such as button ID, input box class), operation content (such as input text), timestamp, etc. The data storage method of structured format data can choose local cache (IndexedDB, LocalStorage) or real-time transmission to the back-end database (such as MySQL, MongoDB). An event deduplication mechanism is used to avoid repeated recording of the same operation (such as quick double-clicking). Throttling or debounce technology is used to reduce the recording overhead of high-frequency operations (such as continuous input). Offline operation capture is supported, and data can be cached during network interruption and synchronized after the connection is restored.

[0043] For example, a JavaScript script embedded in a browser monitors the input event of the input box and the click event of the submit button, and records the content entered by the user and the click time each time; the user browses products, adds to the shopping cart, and checks out on the e-commerce platform; the script captures the user's actions such as clicking on the product link, clicking the "Add to Cart" button, entering a discount code, etc., and the monitored captured data is used to analyze user behavior patterns or generate automated test cases.

[0044] In the present invention application, the script / plug-in supports multiple browsers (Chrome, Firefox, Safari) and client environments (Windows, macOS, Linux), monitors and captures allowable user operation behaviors in real time, ensures that the data reflects the user's real behavior and the compliance of data acquisition, and the timestamp can mark the user operation behavior sequence and time series that can be recorded. The captured user operations are subdivided into data to provide input for subsequent process automation generation, reducing the workload of manually writing automation scripts, laying a solid foundation for automated process generation, and taking into account flexibility and user experience.

[0045] Step S200: Analyze the user operation behaviors that can be recorded and generate a user operation process;

[0046] Specifically, step S200 includes: pre-processing the user operation behavior data that can be recorded, identifying the pattern and logical relationship of the user operation, extracting the operation node steps of the user operation behavior, and converting them into a structured user operation process.

[0047] Step S201: pre-processing the user operation behavior data that can be recorded to obtain processed user operation behavior data, where the processed user operation behavior data includes a processed user operation behavior sequence and a processed time sequence;

[0048] In some embodiments, the user operation behavior data that can be recorded in step S100 is preprocessed, valid user operation behavior data records are extracted, irrelevant or repeated operations (such as invalid clicks, redundant inputs) are filtered out, the user operation behavior data is standardized, the operation types (such as normalizing "click" and "tap" to "click") and element identifiers (such as based on DOM path or ID) are unified, and the operation sequence of the user operation behavior data that can be recorded is sorted by timestamp to ensure that the subsequent analysis of the user operation behavior data that can be recorded is based on the correct time sequence, and the processed user operation behavior data includes the processed user operation behavior sequence and the processed time series.

[0049] Step S202: Analyze the processed user operation behavior data, identify the pattern and logical relationship of the user operation, and extract the operation node steps of the user operation behavior;

[0050] In some embodiments, the processed user operation behavior data is analyzed using algorithms such as Apriori, SPADE, or PrefixSpan to identify frequently occurring operation patterns in the processed user operation behavior data. For example, "input → click submit" is detected as a common form interaction pattern, and "enter username → enter password → click login" is detected as a complete login process.

[0051] In some embodiments, a clustering algorithm (such as K-means or DBSCAN) is used on the processed user operation behavior data to group the operations into logically related units, such as grouping operations on the same page into one task.

[0052] In some embodiments, causal relationships are detected on processed user action data, and dependencies between actions are determined through time interval and context analysis. For example, if "click to log in" depends on "enter username and password," a directed acyclic graph (DAG) is constructed, with nodes representing actions and edges representing dependencies. Weights can reflect the importance or frequency of operations, and trigger conditions for actions are identified, such as "click Submit only if the input box is not empty."

[0053] In some embodiments, based on the frequency of user operations, the importance of target elements (such as submit buttons are better than ordinary links) or rules, key operation nodes are screened and arranged in a logical order to form a sequence of operation steps. Each node contains the operation type, target element, parameters and preconditions. Duplicate nodes are merged, redundant or irrelevant nodes (such as page scrolling) are deleted, and concise operation node steps are generated. The extracted operation node steps are converted into a structured format (such as JSON, YAML) to facilitate subsequent process generation or automated execution.

[0054] For example, the following steps are used to extract the operation nodes of the web page login process:

[0055] {"operation":"input","element":"input#username","value":"user123","timestamp":"2025-05-11T10:00:00.123Z"},

[0056] {"operation":"input","element":"input#password","value":"pass456","timestamp":"2025-05-11T10:00:00.456Z"},

[0057] {"operation":"click","element":"button#login","timestamp":"2025-05-11T10:00:01.789Z"}

[0058] An algorithm is used to identify the processed user operation behavior data as "enter username → enter password → click to log in" as the complete login process. Logical relationship analysis confirms that "click to log in" depends on the first two input operations. Irrelevant operations (such as scrolling) are eliminated, and the operation node steps of the user operation behavior are extracted:

[0059] {"step":1,"operation":"input","element":"input#username","value":"user123"},

[0060] {"step":2,"operation":"input","element":"input#password","value":"pass456"},

[0061] {"step":3,"operation":"click","element":"button#login"}

[0062] For example, the following steps are used to extract the operation nodes of the user operation behavior in the checkout process of the e-commerce platform:

[0063] The user's operation sequence on an e-commerce website includes selecting a product, adding it to the shopping cart, entering a discount code, and checking out. Cluster analysis divides these operations into three stages: "product selection," "shopping cart management," and "checkout." A DAG can be used to construct a structure showing that "clicking to check out" depends on "adding to the shopping cart" and "entering a discount code." The key nodes are extracted: select product → add to the shopping cart → enter a discount code → checkout. The operation node steps for extracting user operation behaviors are:

[0064] Step 1:Click product link(a.product-link)

[0065] Step 2:Click add to cart(button#add-to-cart)

[0066] Step 3:Input coupon code(input#coupon,value:"SAVE10")

[0067] Step 4:Click checkout(button#checkout)

[0068] Step S203: Generate a user operation process based on the operation node steps of the user operation behavior:

[0069] The analysis results are converted into executable user operation processes in JSON, XML, or scripting languages ​​(such as Python and JavaScript). Multiple output formats are supported, compatible with different automation tools (such as RPA platforms, Selenium, and Playwright). The generated processes are optimized, repetitive steps are merged, and complex sequences are simplified. The correctness of the processes is verified through simulation or playback to ensure that the instructions can reproduce the user operations.

[0070] In some embodiments, step S203 generates a user operation flow based on the operation node steps extracted in step S202. A user operation flow is a structured, logically complete sequence of operations, typically expressed in a user-readable or intermediate format (such as JSON, YAML, or pseudocode), describing the user's operation logic in a specific task, and has not yet been directly converted into an executable automation script.

[0071] Integrate the operation node steps into a clear and logically consistent user operation process, which can retain the dependencies, trigger conditions and parameters of the operation node steps for subsequent processing or verification. The user operation process is usually structured data (such as JSON), pseudocode or flowchart, which is suitable for user understanding, debugging or further conversion. The operation node steps are serialized in a logical order, embedded with dependencies and conditional logic, and a visual representation (such as a flowchart) is generated to facilitate verification and optimization.

[0072] For example, the user operation process for generating a web page login behavior is as follows:

[0073] step:1,"operation":"input","element":"input#username","value":"{{user123}}",

[0074] step:2,"operation":"input","element":"input#password","value":"{{pass456}}",

[0075] step:3,"operation":"click","element":"button#login", as the middle layer, bridges the operation node steps and the final automation script, focusing on the logical organization and expression of the process.

[0076] For example, the user operation process for generating checkout behavior on an e-commerce platform is as follows:

[0077] step:1, operation: click, element: a.product-l ink

[0078] step:2, operation: click, element: button#add-to-cart

[0079] step:3, operation:input, element:input#coupon, value:

[0080] "{{SAVE10}}"

[0081] step:4,operation:cl ick,element:button#checkout

[0082] In the present invention application, analysis is performed on the user operation behavior data that can be recorded, reducing the time for manual organization and process writing, quickly generating structured user operation processes, adapting to a variety of automation scenarios, identifying key steps and dependencies, avoiding omissions or redundancies, and supporting logical reasoning of complex operation sequences to ensure that the generated process meets the user's intentions. It supports a variety of input data formats and output instruction types, and is adaptable to different platforms and tools. The generated process can be reused for automated testing, RPA, or user behavior analysis. Step S200 can efficiently and accurately convert user operation behaviors into structured operation processes, providing a solid foundation for subsequent automated execution or analysis.

[0083] Step S300: converting the user operation process into an automated process script to generate an automated process;

[0084] In some embodiments, step S300 further converts the user operation process generated in step S203 into a directly executable automated process script, adapts to specific automation tools or platforms (such as Selenium, Playwright, RPA tools), and generates scripts or instructions that can be directly run in the target environment to ensure that the process can automatically execute user operations and complete specific tasks. The executable automation scripts (such as Python, JavaScript) or automation tool configuration files are directly used for automated testing, RPA or other execution environments.

[0085] In some embodiments, step S300 includes:

[0086] Step S301: Convert and generate an automated process script for the user operation process:

[0087] Based on the user operation flow, a complete operation sequence is formed by connecting the flow lines. The operation node steps of the user operation flow can be mapped to API calls of the target automation tool. For example, the "input" operation is mapped to Selenium's send_keys() or Playwright's fill(). Execution environment-specific logic such as page load waiting, error handling, and timeout retries is embedded. Based on the target platform (such as browsers and desktop applications), appropriate tools and element location methods (such as CSS selectors, XPath, and control IDs) are selected. Using a template engine or code generator, the user operation flow is converted into a script (such as Selenium).

[0088] Step S302: Verify and optimize the automation process script and generate an automation process;

[0089] Run the script in the test environment to verify the execution results. For example, map the JSON process to a Python function call, merge continuous operations, reduce the number of script lines, remove redundant waits, and improve execution efficiency. Automated process generation can support multi-platform operating environments.

[0090] In the present invention application, an executable automated process script is generated, which reduces the manual coding time, supports multiple tools (Selenium, Playwright, AutoIt), adapts to different scenarios, embeds a waiting mechanism, error handling and timeout retry, ensures the stability of the script in a dynamic environment, supports parameterized input (such as user name, discount code), enhances the versatility of the script, generates multiple script languages ​​​​(Python, JavaScript, AutoIt), adapts to different development teams, and the modular design facilitates code reuse and updating, providing a reliable technical implementation for automated process generation.

[0091] Optionally, the user uses the recording operation monitoring to capture the user's operations in the browser or client page, and uses the viewing operation acquisition to generate an automated process.

[0092] In some embodiments, users can monitor and capture operational behaviors within a browser or client page by recording operations, and obtain the generated automated processes by viewing the operations. This method provides users with an intuitive and interactive way to capture operations and generate automated processes.

[0093] In some embodiments, a user interface (such as a browser toolbar button, a client floating panel) is provided to support starting, pausing, and stopping recording.

[0094] In some embodiments, the viewing operation can be used to obtain the generation of an automated process, and a "playback" button is provided in the viewing operation interface to simulate the execution of the recorded operation sequence and reproduce the user operation behavior. For example, the web page login process is replayed, and the steps of entering the user name, entering the password, and clicking to log in are executed in sequence. The execution status of each step is displayed, and the playback result is checked to see whether it meets expectations (such as the page jumps to the "dashboard" or a "success" prompt pops up). The user operation process script supports multi-tool output (such as Playwright, Puppeteer) to meet the needs of different technology stacks.

[0095] In the present invention application, the recording operation is implemented through an intuitive interface (buttons, panels), which does not require programming knowledge, lowers the usage threshold, provides visualization for viewing operations (such as flowcharts), facilitates understanding and editing, and enables real-time recording and automated process generation, reducing the time for manual script writing. It supports Web and client applications, is adaptable to multiple platforms (browsers, desktop applications), generates multiple output formats (JSON, scripts), and is adaptable to different automation tools, providing non-technical users with a convenient automation solution.

[0096] Automatic process generation system based on user operation capture

[0097] like Figure 2 As shown, in the second aspect, an automated process generation system based on user operation capture is used to implement any of the automated process generation methods based on user operation capture described in the first aspect, including:

[0098] It should be noted that the automated process generation system based on user operation capture applied for in the present invention is used to implement any of the automated process generation methods based on user operation capture in the first aspect, and accordingly also includes: all the recorded technical problems, technical solutions and technical effects of any of the automated process generation methods based on user operation capture in the first aspect, which will not be repeated here in the present invention application.

[0099] The user operation behavior acquisition module 100 monitors and captures the user's operations in the browser or client page, and obtains the user operation behaviors that can be recorded;

[0100] The user operation process acquisition module 200 analyzes the user operation behaviors that can be recorded and generates a user operation process;

[0101] The automated process generation module 300 converts the user operation process into an automated process script to generate an automated process.

[0102] In some embodiments, the user operation behavior acquisition module monitors and captures the user's operations in the browser or client page, and obtains the specific content of the user operation behavior that can be recorded. It can be the same as all the technical problems, technical solutions, and technical effects recorded in step S100, and the present application will not go into details about this.

[0103] In some embodiments, the user operation process acquisition module analyzes the user operation behaviors that can be recorded and generates the specific content of the user operation process, which can be the same as all the technical problems, technical solutions, and technical effects recorded in step S200. The present invention application will not go into details about this.

[0104] In some embodiments, the automated process generation module converts the user operation process into an automated process script, and generates the specific content of the automated process, which can be the same as all the technical problems, technical solutions, and technical effects recorded in step S300, and the present invention application will not go into details about this.

[0105] In the present invention application, multiple operating systems (Windows, macOS, Linux, etc.) and browser environments (Chrome, Firefox, Edge, Safari, etc.) are supported, and seamless recording of user operations and automated process generation are achieved. User operations are captured in real time and automated processes are automatically generated, significantly improving the development efficiency of automated tasks. Users do not need to write code and can complete the generation of automated processes through simple interface operations. It can adapt to complex IT environments, reduce development and deployment costs, meet the diverse needs of enterprise automated testing and process optimization, and provide technical support for enterprise process automation.

[0106] Computer-readable storage medium

[0107] In a third aspect, the present invention application provides a computer-readable storage medium having instructions stored thereon. When the instructions are run on a computer, the method for generating an automated process based on user operation capture as described in any one of the first aspects is executed.

[0108] Automatic process generation device based on user operation capture

[0109] In a fourth aspect, the present invention application provides an automated process generation device based on user operation capture, comprising a memory and a processor that are communicatively connected, wherein the memory is used to store a computer program, and the processor is used to read the computer program and execute the automated process generation method based on user operation capture as described in any one of the first aspects.

[0110] In some embodiments, an apparatus for generating an automated process based on user operation capture includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any of the methods for generating an automated process based on user operation capture described in the first aspect are implemented.

[0111] Those skilled in the art will understand that the schematic diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0112] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device using various interfaces and lines.

[0113] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0114] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention applies to implement all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0115] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0116] The above is a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. The automated process generation method based on user operation capture is characterized by: include: Step S100: monitoring and capturing the user's operations in the browser or client page, and obtaining the user's operation behaviors that can be recorded; Step S200: Analyze the user operation behaviors that can be recorded and generate a user operation process; Step S300: The user operation process is converted into an automated process script to generate an automated process.

2. The method for generating an automated process based on user operation capture according to claim 1, characterized in that: Step S100 includes: embedding a script or plug-in in the browser or client page to monitor and capture user operation behaviors that can be recorded in real time. User operation behaviors that can be recorded include user clicks, inputs, and movement operations. The captured user operation behaviors that can be recorded will be marked with a timestamp and stored.

3. The method for generating an automated process based on user operation capture according to claim 2, characterized in that: In step S100, the user operation behaviors that may be recorded include browser operations, which include DOM events, keyboard input, mouse operations, and page navigation; and / or, the user operation behaviors that may be recorded include client interaction operations, which include button clicks, keyboard input, and mouse operations.

4. The method for generating an automated process based on user operation capture according to claim 2, characterized in that: The recorded user operation rows may be stored as structured format data, which includes the operation type, target element, operation content, and timestamp.

5. The method for generating an automated process based on user operation capture according to claim 2, characterized in that: Step S200 includes: pre-processing the user operation behavior data that can be recorded, identifying the pattern and logical relationship of the user operation, extracting the operation node steps of the user operation behavior, and converting them into a structured user operation process.

6. The method for generating an automated process based on user operation capture according to claim 4, characterized in that: Step S200 includes: Step S201: pre-processing the user operation behavior data that can be recorded to obtain processed user operation behavior data, where the processed user operation behavior data includes a processed user operation behavior sequence and a processed time sequence; Step S202: Analyze the processed user operation behavior data, identify the pattern and logical relationship of the user operation, and extract the operation node steps of the user operation behavior; Step S203: Generate a user operation process according to the operation node steps of the user operation behavior.

7. The method for generating an automated process based on user operation capture according to claim 6, characterized in that: Users use recording operation monitoring to capture user operations in the browser or client page, and use viewing operation acquisition to generate automated processes.

8. An automated process generation system based on user operation capture, used to implement the automated process generation method based on user operation capture according to any one of claims 1 to 7, characterized in that: include: User operation behavior acquisition module, which monitors and captures user operations in the browser or client page, and obtains user operation behaviors that can be recorded; The user operation process acquisition module analyzes the user operation behaviors that can be recorded and generates the user operation process; The automated process generation module converts user operation processes into automated process scripts to generate automated processes.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method for generating an automated process based on capturing user operations according to any one of claims 1 to 7 is executed.

10. An automated process generation device based on user operation capture, comprising a memory and a processor that are communicatively connected, wherein the memory is used to store a computer program, and the processor is used to read the computer program and execute the automated process generation method based on user operation capture as described in any one of claims 1 to 7.

Images