Execution method and system of webpage operation task, storage medium and computer equipment

Abstract

The invention discloses a webpage operation task execution method and system, a storage medium and computer equipment, and the method comprises the steps that a user interaction interface responds to a task creation instruction, creates a webpage operation task, and adds the webpage operation task into a task queue; monitoring the task queue by each service instance in a back-end service instance cluster, and when monitoring that a newly added webpage operation task exists in the task queue, judging whether to obtain an execution permission of the webpage operation task or not based on a self state and a preemption state of the webpage operation task; and for the target service instance which obtains the execution permission, the target service instance updates the state of the target service instance to a working state, starts a target headless browser in a task execution unit based on the webpage operation task, and executes a target webpage operation through the target headless browser until the webpage operation task is completed. And closing the target headless browser, and updating the state of the target headless browser to be an idle state.

Description

Technical Field

[0001] This application relates to the field of web page automation control technology, and in particular to a method and system for executing web page operation tasks, a storage medium, and a computer device. Background Technology

[0002] With the deep penetration of internet applications and the explosive growth of digital businesses, web page automation technology has become a core means for enterprises to improve operational efficiency and optimize user experience. Its application scenarios cover diverse fields such as financial risk control, e-commerce operations, government services, and scientific research data collection, placing stringent requirements on the real-time performance of task execution, high-concurrency processing capabilities, and system stability.

[0003] Currently, the widespread adoption of distributed computing frameworks and asynchronous task scheduling technologies provides a foundation for web page automation. However, existing solutions often focus on functionality and typically employ static allocation strategies for task allocation. They lack systematic optimization for dynamic resource allocation and intelligent task scheduling in complex business scenarios, making it difficult to meet the reliability requirements of ultra-large-scale task processing and extreme load conditions. Summary of the Invention

[0004] In view of this, this application provides a method and system for executing web page operation tasks, a storage medium, and a computer device. Through the design of a backend service instance cluster, multiple service instances can simultaneously listen to the task queue and obtain task execution permissions, realizing parallel processing of tasks. This enables the system to process multiple web page operation tasks at the same time, greatly improving the overall processing capacity and throughput of the system, and meeting the needs of large-scale web page operation tasks. When obtaining task execution permissions, service instances comprehensively consider their own resource status to ensure that only service instances with sufficient resources can process tasks. This avoids the problem of task execution failure or performance degradation due to insufficient resources, and also makes full use of the system's idle resources, improving resource utilization. Each service instance has the ability to independently process tasks. Even if a service instance fails or is abnormal, other service instances can still continue to process tasks in the task queue, preventing the entire system from paralyzing. This redundancy design improves the reliability and stability of the system, ensuring the continuous and stable execution of web page operation tasks.

[0005] According to one aspect of this application, a method for performing web page operation tasks is provided, comprising:

[0006] The user interaction interface responds to the task creation command, creates a web page operation task, and adds the web page operation task to the task queue;

[0007] Each service instance in the backend service instance cluster listens to the task queue, and when it detects a new web page operation task in the task queue, it determines whether to acquire the execution permission of the web page operation task based on its own state and the preemption state of the web page operation task.

[0008] For a target service instance that has obtained execution permission, the target service instance updates its own status to working status, and based on the web page operation task, opens a target headless browser in the task execution unit, performs the target web page operation through the target headless browser, and closes the target headless browser after the web page operation task is completed, and updates its own status to idle status.

[0009] According to another aspect of this application, a web page operation task execution system is provided, the execution system including a user interaction interface and a backend service instance cluster, the backend service instance cluster including multiple service instances;

[0010] The user interaction interface is used to respond to the task creation command, create a web page operation task, and add the web page operation task to the task queue;

[0011] Each service instance is used to listen to the task queue, and when a new web page operation task is detected in the task queue, it determines whether to obtain the execution permission of the web page operation task based on its own state and the preemption state of the web page operation task.

[0012] The target service instance that obtains execution permission updates its own status to working status, and based on the webpage operation task, opens the target headless browser in the task execution unit, performs the target webpage operation through the target headless browser, and closes the target headless browser after the webpage operation task is completed, and updates its own status to idle status.

[0013] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method for performing web page operation tasks.

[0014] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described web page operation task execution method.

[0015] Using the above technical solution, this application provides a method and system for executing webpage operation tasks, a storage medium, and a computer device. First, a user can initiate a task creation command for a webpage operation task through a user interaction interface. Upon receiving the task creation command, the user interaction interface can create a webpage operation task. Then, this newly created webpage operation task is added to a task queue. Further, multiple service instances deployed in the backend continuously monitor the task queue. When a new webpage operation task is added to the task queue, all listening service instances receive a notification. At this time, each service instance can perform self-evaluation and judgment. On the one hand, the service instance checks its current state; on the other hand, the service instance can check the preemption status of the new task. After comprehensively considering its own state and the preemption status of the new task, it decides whether to acquire the execution permission for the webpage operation task. Once a service instance successfully acquires the execution permission for a webpage operation task (referred to as the target service instance), it first marks its own state as working. Then, the target service instance, based on the acquired webpage operation task, launches a headless browser in a dedicated task execution unit. The target service instance interacts with the headless browser through a programming interface, sequentially controlling the headless browser to complete the content of the webpage operation task. Once all content in the webpage operation task is completed, the target service instance can close the headless browser and release related resources. Then, it updates its own state to idle, so that it can process new webpage operation tasks later. This embodiment of the application, through the design of a backend service instance cluster, allows multiple service instances to simultaneously listen to the task queue and obtain task execution permissions, achieving parallel task processing. This enables the system to process multiple webpage operation tasks simultaneously, greatly improving the overall processing capacity and throughput of the system, and meeting the needs of large-scale webpage operation tasks. When obtaining task execution permissions, service instances comprehensively consider their own resource status, ensuring that only service instances with sufficient resources can process tasks. This avoids problems such as task execution failure or performance degradation due to insufficient resources, while also making full use of the system's idle resources and improving resource utilization. Each service instance has the ability to independently process tasks. Even if one service instance fails or malfunctions, other service instances can still continue to process tasks in the task queue, preventing the entire system from paralyzing. This redundancy design improves the reliability and stability of the system, ensuring the continuous and stable execution of webpage operation tasks.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A flowchart illustrating a method for executing a webpage operation task according to an embodiment of this application is shown.

[0019] Figure 2 A flowchart illustrating another method for executing web page operation tasks provided in an embodiment of this application is shown.

[0020] Figure 3 This illustration shows a structural diagram of an execution system for web page operation tasks provided in an embodiment of this application;

[0021] Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0022] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0023] This embodiment provides a method for executing webpage operation tasks, such as... Figure 1 As shown, the method includes:

[0024] Step 101: The user interaction interface responds to the task creation instruction, creates a web page operation task, and adds the web page operation task to the task queue.

[0025] Step 102: Each service instance in the backend service instance cluster listens to the task queue, and when it detects a new web page operation task in the task queue, it determines whether to acquire the execution permission of the web page operation task based on its own state and the preemption state of the web page operation task.

[0026] Step 103: For the target service instance that has obtained execution permission, the target service instance updates its own status to working status, and based on the web page operation task, opens the target headless browser in the task execution unit, performs the target web page operation through the target headless browser, and closes the target headless browser after the web page operation task is completed, and updates its own status to idle status.

[0027] This application provides a method for executing web page operation tasks, which can automatically execute various web page operation tasks according to user needs. Specifically, this method can be implemented through a user interaction interface and a backend service instance cluster.

[0028] First, users can initiate a task creation command for webpage operation tasks through the user interface. Upon receiving the command, the user interface can extract key information such as the target webpage address, the specific operations to be performed (e.g., clicking a specific element, entering text content), and the order of operations. Based on this extracted information, a complete webpage operation task is generated. This newly created task is then added to a task queue. The task queue uses a suitable data structure (e.g., linked list, queue) for storage to ensure that tasks are processed sequentially.

[0029] Furthermore, the backend deploys multiple service instances (such as the NestJS service), forming a service instance cluster. Specifically, each service instance has the ability to run and process tasks independently. These service instances continuously monitor the task queue, using mechanisms such as polling or event-driven mechanisms to detect changes in the queue's state. When a new webpage operation task is added to the task queue, all listening service instances receive a notification. At this point, each service instance can perform self-evaluation and judgment. On one hand, the service instance checks its current state, including whether it is executing other tasks and whether its remaining resources (such as memory and CPU utilization) are sufficient to handle the new task. On the other hand, the service instance can check the preemption status of the new task, i.e., whether it has been preempted by other service instances. After comprehensively considering its own state and the preemption status of the new task, the service instance decides whether to acquire the execution permission for the webpage operation task.

[0030] Once a service instance successfully obtains execution permissions for a webpage operation task (referred to as the target service instance), it first marks its own state as active. Next, based on the acquired webpage operation task, the target service instance launches a headless browser within a dedicated task execution unit (such as a Pod). A headless browser is a browser without a graphical user interface that runs automatically in the background. The target service instance interacts with the headless browser through a programming interface, controlling the browser to complete the content according to the operation steps defined in the webpage operation task. Once all content in the webpage operation task is complete, the target service instance can close the headless browser, releasing related resources. Then, it updates its own state to idle, ready to process new webpage operation tasks later.

[0031] By applying the technical solution of this embodiment, firstly, the user can initiate a task creation command for a webpage operation task through the user interaction interface. Upon receiving the task creation command, the user interaction interface can create a webpage operation task. Then, this newly created webpage operation task is added to the task queue. Further, multiple service instances deployed in the backend continuously monitor the task queue. When a new webpage operation task is added to the task queue, all listening service instances are notified. At this time, each service instance can perform self-evaluation and judgment. On one hand, the service instance checks its current state; on the other hand, the service instance can check the preemption status of the new task. After comprehensively considering its own state and the preemption status of the new task, it decides whether to acquire the execution permission for the webpage operation task. Once a service instance successfully acquires the execution permission for the webpage operation task (referred to as the target service instance), it first marks its own state as working. Then, based on the acquired webpage operation task, the target service instance launches a headless browser in a dedicated task execution unit. The target service instance interacts with the headless browser through a programming interface, sequentially controlling the headless browser to complete the content of the webpage operation task. Once all content in the webpage operation task is completed, the target service instance can close the headless browser and release related resources. Then, it updates its state to idle so it can process new webpage operation tasks later. This embodiment of the application, through the design of a backend service instance cluster, allows multiple service instances to simultaneously listen to the task queue and obtain task execution permissions, achieving parallel task processing. This enables the system to process multiple webpage operation tasks simultaneously, greatly improving the overall processing capacity and throughput of the system, and meeting the needs of large-scale webpage operation tasks. When obtaining task execution permissions, service instances comprehensively consider their own resource status, ensuring that only service instances with sufficient resources can process tasks. This avoids task execution failures or performance degradation due to insufficient resources, while also making full use of the system's idle resources and improving resource utilization. Each service instance has the ability to independently process tasks. Even if one service instance fails or malfunctions, other service instances can still continue processing tasks in the task queue, preventing the entire system from paralyzing. This redundancy design improves the reliability and stability of the system, ensuring the continuous and stable execution of webpage operation tasks.

[0032] Optionally, in this embodiment of the application, step 103, "execute the target webpage operation through the target headless browser until the webpage operation task is completed, and then close the target headless browser," includes: when the webpage operation task indicates that the webpage operation execution mode is internal execution, identifying the script module identifiers of the script modules to be executed from the webpage operation task, as well as the execution order corresponding to each script module identifier, and based on the execution order corresponding to each script module identifier, sequentially calling each script module to be executed, and executing the script module to be executed through the target headless browser to complete the target webpage operation, until the last script module to be executed is completed, and then closing the target headless browser.

[0033] In this embodiment, if the webpage operation task indicates that the webpage operation execution method is internal execution, the target service instance can identify the script module identifiers of the script modules to be executed from the webpage operation task, as well as the execution order corresponding to each script module identifier. These script modules can be pre-written code snippets used to implement specific webpage operation functions, and each script module identifier corresponds to an independent script module. Then, the target service instance calls each script module to be executed sequentially according to the execution order corresponding to each script module identifier. The calling process can involve reading the code content of the script module from the storage system and loading it into memory.

[0034] Then, the target service instance interacts with the headless browser through a programming interface, passing the script modules loaded into memory to the headless browser for execution. The headless browser can then perform the corresponding target webpage operations according to the code logic in the script modules, such as simulating user clicks on buttons or filling out forms. Once all the script modules have been executed in the preset execution order, i.e., the last script module has finished executing, the target service instance closes the headless browser and releases the relevant resources.

[0035] This application's embodiments break down webpage operation tasks into multiple executable script modules, which are then executed in a specific order. This modular design gives the system excellent scalability. When new webpage operation functions are needed, only new script modules need to be written and their execution order defined, without requiring large-scale modifications to the entire system, thus reducing development and maintenance costs. Simultaneously, the modular design facilitates independent management and optimization of different webpage operation functions. Furthermore, modularizing webpage operation functions can improve module reusability, eliminating the need to repeatedly develop modules with the same webpage operation functions.

[0036] Optionally, in this embodiment, step 103, "execute the target webpage operation through the target headless browser until the webpage operation task is completed, and then close the target headless browser," includes: when the webpage operation task indicates that the webpage operation execution method is third-party service execution, listening for the scheduling instructions of the third-party service; when the scheduling instructions of the third-party service are heard, controlling the target headless browser to execute the script module to be executed called by the third-party service, and after the execution is completed, feeding back the execution result to the third-party service, and listening for the scheduling instructions of the third-party service again until a scheduling end instruction sent by the third-party service is received, and then closing the target headless browser.

[0037] In this embodiment, if the webpage operation task indicates that the webpage operation execution method is executed by a third-party service, the target service instance can start a dedicated listening mechanism to continuously listen for scheduling instructions from the third-party service. This listening mechanism can be implemented by establishing a network connection with the third-party service and setting corresponding callback functions so that it can respond promptly when the third-party service has scheduling instructions.

[0038] When a scheduling instruction from a third-party service is detected, the target service instance can parse the instruction to obtain information about the script module to be executed by the third-party service. Then, the target service instance can control the target headless browser to execute this script module. During execution, the headless browser can perform corresponding operations on the target webpage according to the code logic in the script module, such as simulating user button clicks or form filling.

[0039] After execution, the target service instance can organize and encapsulate the execution results and send them back to the third-party service through predefined communication interfaces (such as API calls, message queues, etc.). After sending the results, the target service instance can restart the listening mechanism to continue listening for scheduling instructions from the third-party service. This process can continue until the target service instance receives a scheduling termination instruction from the third-party service. Upon receiving the termination instruction, the target service instance can close the target headless browser and release related resources.

[0040] This application's embodiments can seamlessly collaborate with third-party services. By listening to the scheduling instructions of third-party services and executing corresponding web page operation tasks according to the instructions, it achieves seamless integration with third-party services. Simultaneously, it promptly feeds back the execution results to the third-party services, ensuring the accuracy and timeliness of information and meeting the business needs for interaction with third-party services.

[0041] In this embodiment of the application, optionally, the "user interaction interface responds to the task creation instruction and creates a web page operation task" in step 101 includes: the user interaction interface responds to the task creation instruction, obtains the task creation requirements, parses the task creation requirements, and determines the page operation items and project operation processes included in the task creation requirements; based on the page operation items and the logical descriptions corresponding to the page operation logic encapsulated by each preset script module, determines the script module identifier of the script module to be executed from multiple preset script modules; according to the project operation process, determines the execution order corresponding to each script module identifier, and creates the web page operation task according to the script module identifier carrying the execution order; and / or, the user interaction module responds to the task creation instruction, calls the script module selection tool, and creates the web page operation task based on the selection instructions of each script module to be executed and the selection order.

[0042] In this embodiment, web page operation tasks can be created in the following manner:

[0043] The process for the first method is as follows:

[0044] Users initiate task creation operations through user interfaces (such as web pages, desktop application interfaces, etc.). The user interface continuously monitors user actions. When a user performs a specific task creation operation (such as a UI testing task, API testing task, compatibility testing task, etc.), the user interface receives the corresponding task creation instruction. After receiving the task creation instruction, the user interface can further obtain the user's input task creation requirements. For example, it can read data entered by the user from a front-end form, or obtain information entered by the user from the corresponding input controls in a desktop application. The task creation requirements contain information related to the operations the user wishes to perform on the target webpage.

[0045] Subsequently, the acquired task creation requirements are parsed to determine the page operation items and operation flow they contain. The parsing process may include breaking down text information and identifying operation steps. For example, if a user wants to complete a test order placement operation on an e-commerce webpage, the parsed page operation items include "open the product page," "add the product to the shopping cart," "enter the shipping address," and "select a payment method," while the operation flow clarifies the sequence of these operations.

[0046] The system pre-maintains a script module library containing multiple preset script modules. Each preset script module encapsulates specific page operation logic and has a corresponding logical description. The user interface can retrieve information about these preset script modules from the script module library, including the script module identifier and its corresponding logical description. Then, based on the parsed page operation items and the logical descriptions corresponding to the page operation logic encapsulated in each preset script module, the user interface can match among the multiple preset script modules to determine the script module identifier of the script module to be executed for each page operation item. For example, for the page operation item "Open product page," by matching with the logical description of the preset script module, the "Open page" preset script module is determined to be used, and its script module identifier is obtained.

[0047] Based on the parsed project operation flow, the user interface can determine the execution order of each script module identifier. For example, if the project operation flow specifies "open the product page" first, then "add the product to the cart," then the execution order of the corresponding script module identifiers is: "open page" script module identifier first, then "add to cart" script module identifier second. Subsequently, based on the script module identifiers carrying the execution order, a web page operation task is created. This web page operation task object contains the identifier information of the script modules to be executed and their execution order information, so that these script modules can be executed in the correct order subsequently.

[0048] The process for the second method is as follows:

[0049] Similarly, users can initiate task creation through the user interface, which then receives the task creation instruction. The user interface then responds to the task creation instruction by invoking the script module selection tool. This script module selection tool can be a visual interface, such as a list of available preset script modules, where users can see all available preset script modules. Users can then select the script module to be executed based on their needs on the script module selection tool interface. Selection methods may include checking boxes, dragging, etc. Each time a user selects a preset script module, a corresponding selection instruction is generated.

[0050] During the selection of script modules to be executed, users can also determine the selection order of the modules using on-screen sorting functions (such as up and down arrow buttons, drag-and-drop sorting, etc.). This selection order represents the execution order of the script modules. Subsequently, the user interface creates a web page operation task based on the selection instructions and order of the script modules. The identifiers of the user-selected script modules and their order information are encapsulated in a web page operation task so that these script modules can be executed in the order specified by the user.

[0051] The first method in this application automatically parses the user's input task creation requirements to determine the script modules to be executed and their execution order, reducing the complexity of manual operation. The second method uses a visual script module selection tool, allowing users to intuitively select and sort script modules. This method is simple and easy to understand, suitable for users with limited technical knowledge. Combining these two methods meets the needs of different user groups and improves user-friendliness.

[0052] In this embodiment of the application, optionally, step 102, "when a new webpage operation task is detected in the task queue, determine whether to acquire the execution permission of the webpage operation task based on its own state and the preemption state of the webpage operation task," includes: when a new webpage operation task is detected in the task queue, determining whether its own state is idle; when its own state is idle, acquiring an idle distributed lock from the lock resource pool, wherein the lock resource pool automatically releases a new idle distributed lock each time a new webpage operation task is detected in the task queue; if an idle distributed lock is successfully acquired from the lock resource pool, it is determined that the execution permission of the webpage operation task has been acquired; otherwise, it is determined that the execution permission of the webpage operation task has not been acquired.

[0053] In this embodiment, each service instance responsible for handling web page operation tasks can maintain a status flag after startup. This status flag records the current working state of the service instance, mainly divided into idle state and working state. When a service instance detects a new web page operation task in the task queue, it can read its own status flag to determine whether it is currently in an idle state. If the service instance is not executing any task and system resources (such as CPU, memory, etc.) are within acceptable limits and have not reached the set resource usage threshold, then the service instance's status flag is in an idle state; conversely, if the service instance is executing a task, or the system resources are close to full load, then the service instance's status flag is in a working state.

[0054] When a service instance determines that its state is idle, it can attempt to acquire an idle distributed lock from the lock resource pool. The lock resource pool is a dedicated resource pool for managing distributed locks, featuring a mechanism for automatically releasing new idle distributed locks. It can also be configured with a listening mechanism; when a new web page operation task is detected in the task queue, the lock resource pool can automatically release an idle distributed lock for the service instance to acquire. Since only one distributed lock is released at a time, only one service instance can acquire the lock at any given time, ensuring that only one service instance can handle a new web page operation task at any given time.

[0055] Each idle service instance can request to acquire a distributed lock from the lock resource pool. Upon receiving the request, the lock resource pool checks if there is an available idle distributed lock. If so (meaning other service instances are slower than this service instance to acquire the webpage operation task), the lock resource pool can allocate the distributed lock to the service instance and mark the distributed lock as occupied; otherwise, if no idle distributed lock is available (meaning other service instances are faster than this service instance to acquire the webpage operation task), the service instance's acquisition request will not be satisfied.

[0056] After successfully acquiring an idle distributed lock from the lock resource pool, the service instance receives a success response. At this point, the service instance confirms that it has obtained the execution permission for the new webpage operation task. The service instance with execution permission can update its status from idle to working, and can also send an acknowledgment message to the task queue, indicating that it has begun processing the task. Simultaneously, the service instance can also retrieve the new webpage operation task from the task queue and begin executing the corresponding operation.

[0057] If a service instance fails to acquire an idle distributed lock from the lock resource pool, it means that another service instance has already acquired execution permission. In this case, the service instance can determine that it has not acquired the execution permission to add a new web page operation task. The service instance that has not acquired execution permission can remain idle and continue listening to the task queue, waiting for the next potential new task to arrive before attempting to acquire execution permission again.

[0058] This application embodiment, through a lock resource pool and distributed lock mechanism, allows multiple service instances to simultaneously listen to the task queue and acquire task execution permissions when conditions are met. This improves the system's concurrent processing capability, enabling the simultaneous handling of multiple web page operation tasks, fully utilizing system resources, and improving task processing efficiency. The distributed lock ensures that only one service instance can acquire execution permissions for a new web page operation task at a time, avoiding task conflicts and data inconsistencies caused by multiple service instances processing the same web page operation task simultaneously, thus guaranteeing the accuracy and reliability of task execution. The lock resource pool's automatic release of new idle distributed locks simplifies the lock management process, reduces manual intervention, and ensures that when a new task arrives, the lock resource pool can promptly provide available locks, guaranteeing that service instances can quickly acquire execution permissions and improving the system's response speed.

[0059] In this embodiment of the application, optionally, after "opening the target headless browser in the task execution unit" in step 103, the method further includes: real-time monitoring of the execution status of the script module to be executed; when an abnormal execution status is detected, calling a target screenshot tool to capture the current view of the target headless browser and obtaining the execution stack information corresponding to the script module to be executed; generating a first exception record based on the current view, the execution stack information, and the script module identifier of the script module to be executed; and sending the first exception record to the target storage space; and / or, during the execution of any script module to be executed, determining the expected web page element corresponding to the script module to be executed and the expected display time corresponding to each expected web page element; when any expected web page element is not displayed on the target headless browser within the expected display time, calling... The target screenshot tool captures the current view of the target headless browser and obtains the execution stack information corresponding to any script module to be executed. Based on the current view, the execution stack information, and the script module identifier of the script module to be executed, a second exception record is generated and sent to the target storage space. Alternatively, during the execution of any script module to be executed, the page display status of the target headless browser is monitored in real time. When an abnormal page display status is detected, the target screenshot tool is invoked to capture the current view of the target headless browser and obtain the execution stack information corresponding to any script module to be executed. Based on the current view, the execution stack information, and the script module identifier of the script module to be executed, a third exception record is generated and sent to the target storage space.

[0060] In this embodiment, during the process of the task execution unit opening the target headless browser to perform web page operation tasks, the execution status can be monitored in real time through various methods. Once an abnormal situation is detected, an abnormal record can be generated accordingly.

[0061] The first method involves real-time monitoring of the execution status of pending script modules. Specifically, when the task execution unit performs web page operation tasks, it executes each pending script module sequentially according to a preset script module identifier order. During the execution of each pending script module, a dedicated listening thread or an event listening mechanism can be used to monitor the execution status of that script module in real time. The execution status can include various situations such as normal execution, script errors, and network request failures. When the listening thread detects an execution status anomaly, such as an unhandled exception thrown during script execution or a network request timeout causing script interruption, the exception handling process is immediately triggered. At this time, a pre-configured target screenshot tool can be invoked. This tool can interact with the target headless browser to capture the page view currently displayed by the headless browser. The screenshot operation can capture the content of the current page as comprehensively as possible, including key information such as existing error messages and exception displays. At the same time, the execution stack information corresponding to the currently executing script module can be obtained. The execution stack information records detailed information such as the function call order and variable status during script execution, which is crucial for locating the specific location and cause of the exception.

[0062] Furthermore, the captured current view, the acquired execution stack information, and the script module identifier of the currently executing script module are integrated to generate the first exception record. This record is stored in a structured format for easy subsequent analysis and processing. The generated first exception record can be sent to a pre-specified target storage space, such as a database, file system, or cloud storage service. The sending method can be direct writing to the database, uploading via API interface, etc., ensuring that the exception record can be stored securely and reliably.

[0063] The second method involves monitoring the expected display time of web page elements. Before executing each script module, the expected web page elements and their expected display times can be determined based on the module's logic and business requirements. Expected web page elements refer to specific elements that should appear on the page during the execution of the script module, such as buttons, text boxes, and images; the expected display time is the time range within which these elements should appear on the page. During the execution of the script module, the page content of the target headless browser can be continuously monitored to check whether the expected web page elements appear within their expected display time. Monitoring can be achieved through periodically querying page elements and listening to page element loading events. When any expected web page element is detected not to appear on the target headless browser within its expected display time, an anomaly is identified, and an exception handling process is triggered.

[0064] At this point, similar to handling exceptions in the execution status of the script module to be executed, a target screenshot tool can be used to capture the current view of the target headless browser, capturing the current display status of the page, including situations such as missing expected elements and slow page loading. Furthermore, the execution stack information corresponding to the currently executing script module can be obtained to analyze the specific circumstances and possible causes of script execution when expected elements are not displayed.

[0065] Furthermore, the captured current view, execution stack information, and script module identifiers are integrated to generate a second exception record. This record focuses on issues where expected elements fail to display on time, which helps in locating problems related to page loading or script logic. The second exception record is sent to the target storage space and stored together with the first exception record, providing comprehensive data support for subsequent troubleshooting.

[0066] The third method involves monitoring page display anomalies. During the execution of any pending script module, the page display status of the target headless browser is monitored in real time. Page display anomalies can include various visual abnormalities such as layout errors, overlapping elements, and color discrepancies. Monitoring can be implemented by periodically checking the style and position attributes of page elements. When the monitoring thread detects an anomaly in the page display status, such as a severely disordered layout or unexpected color variations in certain elements, the anomaly handling process is immediately initiated. At this point, a target screenshot tool can be used to capture the current view of the target headless browser, completely recording the current abnormal display status and providing direct visual evidence for subsequent problem analysis. Simultaneously, the execution stack information corresponding to the currently executing script module is obtained to help determine the specific location and context of script execution when the page display anomaly occurred.

[0067] Furthermore, the captured current view, execution stack information, and script module identifiers are integrated to generate a third exception record. This record focuses on page display-related exceptions, helping to identify and resolve issues related to page rendering and styling. This third exception record is sent to the target storage space, where it, along with other exception records, forms a complete set of exception information, facilitating subsequent comprehensive analysis and processing.

[0068] This application embodiment monitors abnormal situations during task execution in real time through various methods, including execution status abnormalities, expected element display abnormalities, and page display status abnormalities. This can comprehensively cover various possible problems and improve the stability and reliability of the system. When an abnormality is detected, not only is the current view captured, but also the execution stack information is obtained. Detailed abnormality records are generated by combining the script module identifier. These records provide rich information for subsequent problem investigation and localization, which can greatly shorten the time for problem resolution.

[0069] In one specific embodiment, web page operation tasks that failed to execute can also be added back to the task queue and re-executed.

[0070] In this embodiment of the application, optionally, each service instance integrates a browser control tool library; the step 103, "based on the webpage operation task, opening the target headless browser in the task execution unit", includes: based on the webpage operation task, determining the target headless browser from multiple preset headless browsers, and opening the target headless browser in the task execution unit based on the integrated browser control tool library.

[0071] In this embodiment, after a service instance receives a webpage operation task, it needs to launch the target headless browser to execute the task. Each service instance integrates a browser control toolkit (e.g., Playwright) upon startup. This toolkit provides a series of interfaces and functions for interacting with various types of browsers, such as launching various browsers, closing various browsers, controlling page navigation in various browsers, and manipulating page elements in various browsers. It supports cross-browser page operations, high-concurrency execution, and advanced element interaction control. After determining the target headless browser, the service instance can initialize the browser control toolkit to prepare for subsequent browser operations.

[0072] Specifically, when a service instance receives a webpage operation task, it first performs a detailed analysis of the task. This task may include information related to browser selection, such as the task type (e.g., webpage crawling, automated testing) and the specific requirements of the target webpage (e.g., webpages requiring support for specific browser features or specific browser versions to render correctly). By analyzing this information, the service instance can gain a preliminary understanding of the browser requirements of the webpage operation task.

[0073] The service instance can obtain information about multiple pre-configured headless browsers. This information may include the browser version number, supported features (such as whether it supports specific JavaScript APIs or certain browser extensions), performance metrics (such as page load speed and memory usage), and the current resource usage of each headless browser (such as CPU usage and memory usage). This information can be stored in the service instance's local configuration file or obtained by interacting with the configuration management service.

[0074] Next, based on the task requirements analysis and the information of the preset headless browser, the service instance adopts a certain strategy to determine the target headless browser. Common selection strategies may include: (1) Function matching strategy: Prioritize the selection of browsers that can meet the special functional requirements of the web page operation task. For example, if the web page operation task requires the use of a specific JavaScript API, then select a browser version that supports that API. (2) Resource idle strategy: Consider the resource consumption of each headless browser and select a browser with lower current resource consumption to ensure that the task execution will not be affected by resource contention. (3) Performance priority strategy: Based on the performance indicators of the preset headless browser, select a browser with better performance to execute the task in order to improve the task execution efficiency.

[0075] The service instance starts the target headless browser by calling the browser launch interface provided by the browser control library, passing in the path to the target headless browser and the configured launch parameters. The browser control library can interact with the target headless browser and launch it according to the configured parameters. During the launch process, the browser control library can monitor the launch status of the target headless browser to ensure that it starts successfully and enters an operational state. Once the target headless browser starts successfully, the service instance can manage it.

[0076] This application embodiment analyzes task requirements and information from preset headless browsers, employing multiple strategies to select the target headless browser. This allows for flexible adaptation to different types of web page operation tasks, ensuring that web page operation tasks can be executed in a suitable browser environment. The integrated browser control tool library provides rich interfaces and functions, enabling convenient startup, configuration, and management of various preset headless browsers, simplifying the browser operation process and improving task execution efficiency.

[0077] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another method for executing web page operation tasks is provided, such as... Figure 2 As shown, the method includes:

[0078] After receiving a task creation instruction, the user interface can create a web page operation task and insert the created task into the task queue.

[0079] The backend service instance cluster consists of n service instances. Each service instance can listen to the task queue. When it detects a new webpage operation task in the task queue, it can determine whether it has acquired the execution permission for the webpage operation task based on its own status and task preemption status. Based on this, a target service instance can be determined from the n service instances to handle the webpage operation task.

[0080] The target service instance continues to perform subsequent operations. Specifically, the target service instance can first update its own state from idle to working, then open a headless browser to execute the content of the web page operation task, close the headless browser after completion, and restore its own state to idle, waiting to execute a new web page operation task.

[0081] Furthermore, as Figure 1 In terms of specific implementation, this application provides a webpage operation task execution system, such as... Figure 3 As shown, the execution system includes a user interaction interface and a backend service instance cluster, wherein the backend service instance cluster includes multiple service instances;

[0082] The user interaction interface is used to respond to the task creation command, create a web page operation task, and add the web page operation task to the task queue;

[0083] Each service instance is used to listen to the task queue, and when a new web page operation task is detected in the task queue, it determines whether to obtain the execution permission of the web page operation task based on its own state and the preemption state of the web page operation task.

[0084] The target service instance that obtains execution permission updates its own status to working status, and based on the webpage operation task, opens the target headless browser in the task execution unit, performs the target webpage operation through the target headless browser, and closes the target headless browser after the webpage operation task is completed, and updates its own status to idle status.

[0085] Optionally, the target service instance is used for:

[0086] When the webpage operation task indicates that the webpage operation execution mode is internal execution, the script module identifier of the script module to be executed and the execution order corresponding to each script module identifier are identified from the webpage operation task. Based on the execution order corresponding to each script module identifier, each script module to be executed is called in sequence, and the script module to be executed is executed through the target headless browser to complete the target webpage operation, until the last script module to be executed is completed, and the target headless browser is closed.

[0087] Optionally, the target service instance is used for:

[0088] When the webpage operation task indicates that the webpage operation execution method is executed by a third-party service, listen for the scheduling instructions of the third-party service;

[0089] Upon receiving a scheduling instruction from a third-party service, the target headless browser is controlled to execute the script module to be executed by the third-party service. After execution, the execution result is fed back to the third-party service, and the browser resumes listening for scheduling instructions from the third-party service until a scheduling end instruction is received from the third-party service, at which point the target headless browser is closed.

[0090] Optionally, the user interaction interface is used for:

[0091] In response to a task creation instruction, the system obtains the task creation requirements and parses them to determine the page operation items and operation flow included in the requirements. Based on the page operation items and the logical descriptions corresponding to the page operation logic encapsulated in each preset script module, the system determines the script module identifier of the script module to be executed from multiple preset script modules. According to the operation flow, the system determines the execution order corresponding to each script module identifier and creates the webpage operation task based on the script module identifier carrying the execution order. And / or,

[0092] In response to the task creation instruction, the script module selection tool is invoked, and the webpage operation task is created based on the selection instructions of each script module to be executed and the selection order.

[0093] Optionally, the service instance is used for:

[0094] When a new web page operation task is detected in the task queue, the system determines whether its own status is idle.

[0095] When its own state is idle, it obtains an idle distributed lock from the lock resource pool. The lock resource pool automatically releases a new idle distributed lock each time it detects a new web page operation task in the task queue.

[0096] If an idle distributed lock is successfully acquired from the lock resource pool, it is determined that the execution permission for the web page operation task has been obtained; otherwise, it is determined that the execution permission for the web page operation task has not been obtained.

[0097] Optionally, the system further includes a monitoring module; the monitoring module is used for:

[0098] After the target headless browser is opened in the task execution unit, the execution status of the script module to be executed is monitored in real time. When an abnormal execution status is detected, a target screenshot tool is invoked to capture the current view of the target headless browser and obtain the execution stack information corresponding to the script module to be executed. Based on the current view, the execution stack information, and the script module identifier of the script module to be executed, a first exception record is generated and sent to the target storage space; and / or,

[0099] After the target headless browser is opened in the task execution unit, during the execution of any script module to be executed, the expected web page elements corresponding to each script module to be executed, and the expected display time for each expected web page element, are determined. When any expected web page element is not displayed on the target headless browser within the expected display time, a target screenshot tool is invoked to capture the current view of the target headless browser and obtain the execution stack information corresponding to each script module to be executed. Based on the current view, the execution stack information, and the script module identifier of the script module to be executed, a second exception record is generated and sent to the target storage space; and / or,

[0100] After the target headless browser is opened in the task execution unit, the page display status of the target headless browser is monitored in real time during the execution of any script module to be executed. When an abnormal page display status is detected, the target screenshot tool is invoked to capture the current view of the target headless browser and obtain the execution stack information corresponding to any script module to be executed. Based on the current view, the execution stack information and the script module identifier of the script module to be executed, a third exception record is generated and sent to the target storage space.

[0101] Optionally, each service instance integrates a browser control tool library; the target service instance is also used for:

[0102] Based on the webpage operation task, a target headless browser is determined from multiple preset headless browsers, and the target headless browser is opened in the task execution unit based on the integrated browser control tool library.

[0103] It should be noted that other corresponding descriptions of the functional units involved in the web page operation task execution system provided in this application embodiment can be found in the following references. Figures 1 to 2 The corresponding descriptions in the method will not be repeated here.

[0104] This application also provides a computer device, which may specifically be a personal computer, a server, a network device, etc. Figure 4 As shown, the computer device includes a bus, a processor, memory, and a communication interface, and may also include an input / output interface and a display device. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores location information. The network interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0105] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0106] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0107] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0109] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for executing webpage operation tasks, characterized in that, include: The user interaction interface responds to the task creation command, creates a web page operation task, and adds the web page operation task to the task queue; Each service instance in the backend service instance cluster listens to the task queue, and when it detects a new web page operation task in the task queue, it determines whether to acquire the execution permission of the web page operation task based on its own state and the preemption state of the web page operation task. For a target service instance that has obtained execution permission, the target service instance updates its own status to working status, and based on the web page operation task, opens a target headless browser in the task execution unit, performs the target web page operation through the target headless browser, and closes the target headless browser after the web page operation task is completed, and updates its own status to idle status.

2. The method according to claim 1, characterized in that, The step of performing target webpage operations through the target headless browser until the webpage operation task is completed, and then closing the target headless browser, includes: When the webpage operation task indicates that the webpage operation execution mode is internal execution, the script module identifier of the script module to be executed and the execution order corresponding to each script module identifier are identified from the webpage operation task. Based on the execution order corresponding to each script module identifier, each script module to be executed is called in sequence, and the script module to be executed is executed through the target headless browser to complete the target webpage operation, until the last script module to be executed is completed, and the target headless browser is closed.

3. The method according to claim 1, characterized in that, The step of performing target webpage operations through the target headless browser until the webpage operation task is completed, and then closing the target headless browser, includes: When the webpage operation task indicates that the webpage operation execution method is executed by a third-party service, listen for the scheduling instructions of the third-party service; Upon receiving a scheduling instruction from a third-party service, the target headless browser is controlled to execute the script module to be executed by the third-party service. After execution, the execution result is fed back to the third-party service, and the browser resumes listening for scheduling instructions from the third-party service until a scheduling end instruction is received from the third-party service, at which point the target headless browser is closed.

4. The method according to claim 2, characterized in that, The user interaction interface responds to a task creation command and creates a webpage operation task, including: The user interaction interface responds to the task creation instruction, obtains the task creation requirements, parses the requirements, and determines the page operation items and operation flow included in the requirements. Based on the page operation items and the logical descriptions corresponding to the page operation logic encapsulated in each preset script module, it determines the script module identifier of the script module to be executed from multiple preset script modules. According to the operation flow, it determines the execution order corresponding to each script module identifier and creates the webpage operation task based on the script module identifier carrying the execution order; and / or, The user interaction module responds to the task creation instruction by calling the script module selection tool and creating the webpage operation task based on the selection instructions of each script module to be executed and the selection order.

5. The method according to claim 1, characterized in that, When a new webpage operation task is detected in the task queue, the process of determining whether to acquire execution permission for the webpage operation task based on its own state and the preemption state of the webpage operation task includes: When a new web page operation task is detected in the task queue, the system determines whether its own status is idle. When its own state is idle, it obtains an idle distributed lock from the lock resource pool. The lock resource pool automatically releases a new idle distributed lock each time it detects a new web page operation task in the task queue. If an idle distributed lock is successfully acquired from the lock resource pool, it is determined that the execution permission for the web page operation task has been obtained; otherwise, it is determined that the execution permission for the web page operation task has not been obtained.

6. The method according to claim 1, characterized in that, After opening the target headless browser in the task execution unit, the method further includes: The execution status of the script module to be executed is monitored in real time. When an abnormal execution status is detected, a target screenshot tool is invoked to capture the current view of the target headless browser and obtain the execution stack information corresponding to the script module to be executed. Based on the current view, the execution stack information, and the script module identifier of the script module to be executed, a first exception record is generated and sent to the target storage space; and / or, During the execution of any script module to be executed, the expected webpage element corresponding to the script module to be executed, and the expected display time corresponding to each expected webpage element are determined. When any expected webpage element is not displayed on the target headless browser within the expected display time, a target screenshot tool is invoked to capture the current view of the target headless browser and obtain the execution stack information corresponding to the script module to be executed. Based on the current view, the execution stack information, and the script module identifier of the script module to be executed, a second exception record is generated and sent to the target storage space; and / or, During the execution of any script module to be executed, the page display status of the target headless browser is monitored in real time. When an abnormal page display status is detected, the target screenshot tool is invoked to capture the current view of the target headless browser and obtain the execution stack information corresponding to any script module to be executed. Based on the current view, the execution stack information, and the script module identifier of the script module to be executed, a third exception record is generated and sent to the target storage space.

7. The method according to claim 1, characterized in that, Each service instance integrates a browser control tool library; the step of launching the target headless browser in the task execution unit based on the webpage operation task includes: Based on the webpage operation task, a target headless browser is determined from multiple preset headless browsers, and the target headless browser is opened in the task execution unit based on the integrated browser control tool library.

8. A system for executing webpage operation tasks, characterized in that, The execution system includes a user interaction interface and a backend service instance cluster, wherein the backend service instance cluster includes multiple service instances; The user interaction interface is used to respond to the task creation command, create a web page operation task, and add the web page operation task to the task queue; Each service instance is used to listen to the task queue, and when a new web page operation task is detected in the task queue, it determines whether to obtain the execution permission of the web page operation task based on its own state and the preemption state of the web page operation task. The target service instance that obtains execution permission updates its own status to working status, and based on the webpage operation task, opens the target headless browser in the task execution unit, performs the target webpage operation through the target headless browser, and closes the target headless browser after the webpage operation task is completed, and updates its own status to idle status.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images