General message queue task processing system and method under rpa architecture

Abstract

The application discloses a general message queue task processing system and method under an RPA architecture, relates to the technical field of RPA automation and message queue, and comprises the following steps: reading a message queue processing configuration, starting a consumption thread according to queue grouping, and establishing an independent message queue connection and channel for each thread; receiving and analyzing a task message, acquiring a page automation driving instance through a driving instance manager, detecting a login state validity period and instance validity, and performing re-login or reconstruction when the login state is overdue or the instance is invalid; performing a differentiated message confirmation strategy according to a queue type, a processing result and exception information, first restoring the instance and then denying confirmation and re-queuing for driving-related exceptions, and sending the processing result to a downstream queue according to a retry strategy. Through linkage of a message queue consumption mechanism and driving instance state management, the application reduces the queue false death probability, and improves task processing continuity and reliability.

Description

Technical Field

[0001] This invention relates to the field of RPA automated processing and message queue consumption technology, specifically to a general message queue task processing system and method under the RPA architecture. Background Technology

[0002] In RPA architecture, message queues are a key infrastructure for realizing asynchronous distribution, decoupling, and peak shaving of RPA tasks. A typical RPA message queue processing flow is as follows: the upstream business system delivers RPA automation tasks to the message queue in the form of messages, the RPA processing node pulls task messages from the message queue through the consumer thread, calls the page automation driver to execute the automated operation on the browser side, and returns the task processing result to the downstream business system.

[0003] However, in actual production environments, existing RPA message queue task processing solutions generally suffer from the following technical problems: First, in the production environment, there are frequent instances of a "frozen" situation where there are messages to be consumed in the message queue, but the consumer threads have stopped pulling them, causing a large number of tasks to be blocked. One of the root causes of this problem is that in the existing consumption scheme, multiple consumer threads may reuse the same message queue connection or message channel. When a consumer thread is stuck due to RPA page operations, the blockage will spread to other consumer threads through the shared connection or channel. At the same time, after a consumer thread exits abnormally, the associated message queue connection, message channel, and browser resources are not completely cleaned up, resulting in resource residues that prevent newly started consumer threads from establishing a normal consumption chain.

[0004] Second, in the RPA process, the core page automation driver instance is subject to access conflicts in multi-threaded scenarios. Multiple consumer threads operating the same page automation driver instance simultaneously may cause page state crosstalk and element location failure. When the page automation driver instance fails due to browser crashes, session expiration, or driver process freezing, there is a lack of automatic detection and reconstruction mechanisms, requiring manual intervention to restart, which directly leads to the interruption of RPA tasks. In addition, the login state of the target system corresponding to the page automation driver instance has an expiration period. After the login state expires, page operations will inevitably fail, and existing solutions lack automatic control over the validity period of the login state.

[0005] Third, existing solutions typically employ a uniform strategy of either processing before confirmation or confirmation before processing. For RPA architectures, different types of message queues and tasks have different processing characteristics: if long-connection tasks wait for business processing to be completed before confirmation, the message queue connection may time out or messages may be delivered repeatedly due to excessive processing time; if regular page operation tasks confirm before processing, the message has already been removed from the queue after processing failure and cannot be automatically retried; in addition, when RPA-specific exceptions such as page automation-driven session failure or browser unreachability occur, existing solutions lack a mechanism for coordinating exception recovery and message re-queuing.

[0006] Fourth, high-concurrency batch data processing tasks and low-frequency, slow-execution page operation tasks run together in the same consumption channel. Page operation tasks usually require exclusive use of browser instances and have long execution times, which are greatly affected by page loading speed and network conditions. On the other hand, batch data processing tasks have lower browser resource consumption and higher execution efficiency. When the two types of tasks are consumed together, the single-threaded consumption mode cannot adapt to the processing needs of multiple types of business, resulting in low overall processing efficiency. If multi-threaded parallel consumption is simply adopted, it may lead to operation conflicts and page state interference due to multiple page operation tasks competing for the page automation driver instance at the same time.

[0007] Fifth, existing solutions typically employ simple single-transmission or fixed-interval retries, failing to adequately consider the receiving status of downstream queues and network fluctuations. This can easily lead to failures in sending results, causing disruptions to downstream business processes. Furthermore, they lack flexible configuration options for the number of retries and backoff strategies.

[0008] In summary, the technical challenges faced by message queue task processing under the RPA architecture are complex, involving coordination across multiple levels, such as message queue consumption thread management, page automation-driven instance lifecycle management, matching message acknowledgment timing with RPA exception types, resource scheduling for different task attributes, and reliable result delivery, rather than optimization issues at a single stage. Summary of the Invention

[0009] The present invention is mainly intended to provide a general message queue task processing system and method under the RPA architecture, in order to solve the problems existing in the current RPA message queue processing process, such as queue freezing, page automation-driven instance failure leading to task interruption, mismatch between message confirmation strategy and RPA task processing characteristics, low efficiency of running multiple types of tasks together, and unstable result delivery.

[0010] To achieve the above objectives, the technical solution of the present invention is as described in claims 1-10: A general message queue task processing method under RPA architecture, the method includes the following steps: Read the message queue processing configuration, which includes queue grouping information, task mapping relationship and message prefetching quantity; The corresponding consumer thread is started according to the queue grouping information, and an independent message queue connection and message channel are established for each consumer thread; The consumer thread receives task messages, performs unified parsing on the task messages, and determines the RPA business processing logic corresponding to the current message queue based on the task mapping relationship. Before executing the RPA business processing logic, a page automation driver instance is obtained through the driver instance manager. The driver instance manager is used to detect the validity period of the login state and the validity of the page automation driver instance. When the login state expires, a re-login process is triggered. When the page automation driver instance is invalid, a driver instance reconstruction process is triggered. The RPA business processing logic is executed through the page automation-driven instance to obtain task processing results or exception information; Based on the type of the queue to which the task message belongs, the task processing result, and the exception information, a differentiated message confirmation strategy is executed: for regular business queues, a positive confirmation is executed when the task is successfully processed; for page automation driver-related exceptions, the page automation driver instance is first restored through the driver instance manager, then a negative confirmation is executed and the task message is re-enqueued. The task processing result is sent to the downstream result queue corresponding to the upstream message queue, and if the sending fails, it is resent according to the preset retry strategy; When a consumer thread encounters an exception, the message queue connection, message channel, and page automation drive resources corresponding to that consumer thread are cleaned up, and the connection is re-established after a preset delay to continue consumption.

[0011] In the above method of the present invention, by establishing an independent message queue connection and message channel for each consumer thread, the blocking or abnormality of a single consumer thread will not spread to other threads through the shared connection, thereby reducing the probability of queue apparent death from the architectural level.

[0012] Meanwhile, by using the driver instance manager to uniformly check the validity period of the login state and the validity of the page automation driver instance before executing the RPA business processing logic, the state management of the page automation driver is incorporated into the message consumption process. This allows login state expiration and driver instance failure to be proactively detected and automatically recovered before task execution, reducing RPA task execution failures caused by driver instance issues.

[0013] Furthermore, the differentiated message acknowledgment strategy of this invention associates the message acknowledgment timing with the types of exceptions that occur during RPA task processing. For page automation driver-related exceptions, instead of simply performing a negative acknowledgment and re-enqueuing the message, the page automation driver instance is first restored through the driver instance manager. After the driver environment returns to normal, the message is re-enqueued for consumption. This ensures that message retry processing can be performed in an available driver environment, improving the retry success rate, and avoids the problem of repeated message enqueuing and queue blocking caused by continuous driver instance exceptions.

[0014] A general message queue task processing system under the RPA architecture includes: The configuration reading module is used to read the message queue processing configuration; The group consumption module is used to start the corresponding consumption thread according to the queue grouping information, establish an independent message queue connection and message channel for each consumption thread, and clean up the thread's resources and re-establish the connection when an exception occurs in the consumption thread. The task processing module is used to receive and parse task messages through a consumer thread, determine the RPA business processing logic according to the task mapping relationship, and execute the RPA business processing logic through a page automation-driven instance. The driver instance management module is used to provide page automation driver instances to the task processing module, detect the validity period of the login state and the validity of the driver instance, and perform automatic recovery when the login state expires or the driver instance becomes invalid. The message confirmation module is used to execute differentiated message confirmation strategies based on queue type, task processing result, and exception information; The result delivery module is used to send the task processing results to the downstream result queue and to perform a retry if the sending fails.

[0015] An electronic device includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the general message queue task processing method under the RPA architecture described above.

[0016] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the general message queue task processing method under the above-described RPA architecture.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention significantly reduces the probability of queue freeze caused by single-point blocking propagation by using consumer thread-level message queue connections, message channels, and page-driven resource isolation, along with an automatic cleanup and reconnection mechanism after abnormal thread exit.

[0018] This invention automates and systematizes the management of page-based automated driver instances through unified control of the driver instance manager, including global singleton, driver instance mapping table, instance access lock, login status update lock, login status validity detection, driver instance validity detection, and automatic reconstruction. This reduces RPA task execution failures caused by driver failure and login expiration, and lowers the frequency of manual maintenance.

[0019] This invention employs a differentiated message confirmation strategy—confirming success for regular business queues, restoring the driver instance before denying confirmation and re-queuing for page automation-related exceptions, and pre-confirming long-connection queues while combining task status records to achieve failure compensation—to match the message confirmation mechanism with the execution characteristics and exception types of RPA tasks. This ensures the reliability of message processing while avoiding connection timeouts and duplicate message delivery caused by prolonged lack of confirmation.

[0020] This invention enables high-concurrency lightweight tasks to achieve parallel and efficient processing, and low-frequency page tasks to achieve stable serial processing, by configuring queues hierarchically based on RPA task attributes and setting differentiated message prefetching quantities, thus balancing overall processing efficiency and the stability of single-task operations.

[0021] This invention improves the reliability of delivering task processing results to downstream queues and reduces the risk of delivery interruption due to temporary network failures by using exponential backoff retries, queue mapping, and abnormal result queue mechanisms in the result delivery module. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall architecture of a general message queue task processing system under the RPA architecture provided in an embodiment of the present invention; Figure 2 A schematic diagram illustrating the workflow of a driver instance manager provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the overall flow of a general message queue task processing method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an RPA service hierarchical multi-threaded group consumption architecture provided in an embodiment of the present invention.

[0023] In the diagram: 110 is the configuration reading module, 120 is the group consumption module, 130 is the driver instance management module, 140 is the task processing module, 150 is the message confirmation module, and 160 is the result delivery module; S210 receives the driver instance acquisition request, S220 acquires the instance access lock, S230 checks if the corresponding driver instance exists, S240 checks if the login state is valid, S250 refreshes the login state, S260 checks if the driver instance is valid, S270 clears the browser cache and returns the instance, and S280 rebuilds the driver instance; S310 reads the message queue and processes the configuration, and S320 starts the consumption thread and establishes... The connection process is as follows: S330 receives task messages, S340 parses the message body, S350 determines the RPA business processing logic, S360 obtains the page automation driver instance, S370 executes the RPA business processing logic, S380 determines whether the processing is successful, S390 confirms the successful path message, S400 confirms the abnormal path message and restores the driver, S410 sends the result and retryes, and S420 continues to consume the next message; 410 is the first-level group, high-concurrency batch RPA task group, 420 is the second-level group, regular page operation RPA task group, and 430 is the third-level group, low-frequency slow execution RPA task group. Detailed Implementation

[0024] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0025] Example 1 Reference Figure 1 This embodiment provides a general message queue task processing system under the RPA architecture. The system includes a configuration reading module 110, a group consumption module 120, a driver instance management module 130, a task processing module 140, a message confirmation module 150, and a result delivery module 160. The modules interact with each other through internal interfaces.

[0026] The configuration reading module 110 is used to read the message queue processing configuration, which can be stored in a local configuration file, database, or configuration center. The message queue processing configuration includes queue grouping information, task mapping relationships, thread count, message prefetch count, and queue type identifier, etc. Specifically, the queue grouping information defines the queue groups to which different message queues belong; the task mapping relationship defines the correspondence between message queues and RPA business processing logic; the thread count defines the number of consumer threads corresponding to each queue group; the message prefetch count defines the number of messages prefetched by each consumer thread from the message queue; and the queue type identifier indicates whether the queue belongs to a regular business queue or a long-connection type queue.

[0027] The queue grouping information can be set according to the task attributes of the RPA task. The task attributes include at least one of the following: whether a page automation driving instance is required, whether a page automation driving instance needs to be exclusively used, whether page-level operations are involved, average execution time, whether login status is required, and whether concurrent processing is supported. For example, lightweight task queues that only involve data querying, log statistics, etc., and do not depend on page automation driving instances can be divided into the first type of queue group, with a higher number of threads and a larger message prefetch quantity. Task queues that involve page operations, form submissions, etc., and depend on page automation driving instances can be divided into the second type of queue group, with a lower number of threads and a smaller message prefetch quantity. Task queues that require system login or long-term exclusive use of browser instances can be divided into the third type of queue group, with one consumption thread and a message prefetch quantity of 1, and a single-task serial consumption mode is adopted.

[0028] The group consumption module 120 is the system's consumption scheduling module. When the system starts, the group consumption module 120 reads the queue grouping information provided by the configuration reading module 110, creates and starts a corresponding number of consumption threads for each queue group. Each consumption thread independently creates its own required message queue connection object and message channel object; different consumption threads do not share message queue connections and message channels. Each consumption thread contains a consumption loop, continuously pulling task messages from the corresponding message queue and calling the task processing module 140 for processing upon receiving a task message. The consumption thread is also configured with exception handling logic: when a consumption thread exits due to an exception, the exception handling logic is responsible for cleaning up the message queue connection and message channel held by the current consumption thread, and notifying the driver instance management module 130 to release the page automation driver resources associated with the consumption thread. After cleaning, it waits according to a preset reconnection delay, then recreates the message queue connection and message channel, and restarts the consumption loop. Through the above thread-level resource isolation and automatic reconnection mechanism, an exception in a single consumption thread will not propagate to consumption threads in other queue groups, and the abnormal thread can automatically recover its consumption capability after the resources are thoroughly cleaned up.

[0029] The driver instance management module 130 is a global singleton responsible for unified lifecycle management of page automation driver instances. The following description uses SeleniumWebDriver as an example, but those skilled in the art will understand that other types of page automation drivers are also applicable to this invention. The driver instance management module 130 internally maintains a driver instance mapping table, indexed by the consumer thread identifier or queue group identifier, storing the corresponding page automation driver instance and its associated information. This associated information includes the browser process identifier, driver process identifier, and login timestamp. The driver instance management module 130 is configured with an instance access lock and a login status update lock. The instance access lock ensures thread safety when acquiring, creating, rebuilding, or releasing page automation driver instances, preventing multiple consumer threads from simultaneously operating on the same page automation driver instance. The login status update lock, upon detecting an expired login status, allows only one consumer thread to execute the re-login process; other consumer threads reuse the updated login status after the login status update lock is released, avoiding multiple consumer threads concurrently performing duplicate logins.

[0030] The task processing module 140 is used to uniformly process task messages. The core functions of the task processing module 140 include: unified message body parsing—automatically detecting the data type of the message body; if it is a byte type, it is first converted to a string type, and then the string is parsed using JSON to obtain the message body object; if parsing fails, a parsing exception log is recorded and a standardized error format processing result is returned; queue and RPA business logic mapping—based on the task mapping relationship in the configuration reading module 110, the current message queue is mapped to the corresponding RPA business processing function. The RPA business processing function can be any type of RPA automation task such as coupon management operations, promotional activity operations, inventory queries, product listing information updates, form filling, and data crawling; unified exception capture and logging—capturing exceptions at each stage of task processing and recording log information such as message identifier, queue name, exception type, exception stack, and timestamp to ensure that the failure of a single task does not affect the overall queue consumption.

[0031] The message confirmation module 150 is used to execute a differentiated message confirmation strategy. The message confirmation module 150 determines whether to perform a positive or negative confirmation for the current message based on the type identifier of the queue to which the task message belongs, the task processing result returned by the task processing module 140, and exception information. The specific confirmation strategy is as follows: For task messages identified as regular business queues, a positive confirmation is executed after the task processing module 140 returns a successful processing result; if the task processing fails, further judgment is made based on the exception type. If the exception belongs to page automation driver-related exceptions, including driver session failure, browser window unreachable, driver communication timeout, session not created, etc., the driver instance management module 130 is first notified to perform driver instance recovery, which includes closing the browser process and driver process, and recreating the browser instance and driver instance; after the recovery is completed, a negative confirmation is executed and the task message is re-enqueued; if the exception belongs to other types, such as message format error, business logic processing exception, etc., which are unrecoverable exceptions, a positive confirmation is executed to avoid infinite loop of exception messages, and the failure information is recorded in the exception log.

[0032] For task messages identified as long-connection type queues, when preset conditions are met, such as upon receiving the message but before starting business processing, a positive acknowledgment is performed first. Simultaneously, the task identifier, message body summary, upstream queue identifier, and current processing status of the message are written to the task status record. This task status record can be stored in a local file, database, or cache. Then, the task processing module 140 is invoked to execute the RPA business processing logic. If subsequent RPA business processing is successful, the corresponding task status record is cleared. If subsequent RPA business processing fails, a compensation task message is reconstructed based on the information in the task status record and delivered to the corresponding message queue or exception recovery queue, ensuring that tasks in the long-connection type queue are not lost due to premature acknowledgment.

[0033] In the event of message parsing failure, record the parsing exception log, return the processing result in a standardized error format, perform a positive acknowledgment on the message, or transfer the message to a dedicated exception message queue according to the configuration for subsequent manual investigation.

[0034] The result delivery module 160 is used to reliably send the task processing results to the downstream result queue. The workflow of the result delivery module 160 includes: first, determining the downstream result queue corresponding to the current upstream message queue according to the pre-configured queue mapping relationship; for special RPA tasks, the corresponding downstream result queue can also be dynamically matched according to the task type identifier in the task processing result; then, constructing a result message, which includes information such as task identifier, upstream queue identifier, processing status, business result data, exception type, retry count, and log summary. When sending the result message to the downstream result queue, the message confirmation mechanism of the message queue is enabled to ensure reliable message delivery; if the sending fails, it is resent according to the preset exponential backoff retry strategy; if the retry count reaches the preset limit and the message is still not successfully sent, the result message is written to the exception result queue and a detailed failure log is recorded for subsequent manual intervention.

[0035] Example 2 Reference Figure 2 This embodiment further explains the workflow of the driver instance manager 130, which provides a method for securely obtaining driver instances, including the following steps.

[0036] Step S210: Receive a driver instance acquisition request. The driver instance acquisition request is initiated by the task processing module 140 before executing the RPA business processing logic. The request carries the thread identifier of the current consumer thread or the queue group identifier to which it belongs.

[0037] Step S220: Obtain an instance access lock, which is used to ensure the thread safety of subsequent operations.

[0038] Step S230: Check if a page automation driver instance corresponding to the current thread identifier or queue group identifier already exists in the driver instance mapping table; if it does not exist, proceed to step S280 to create a new page automation driver instance; if it exists, continue to step S240.

[0039] Step S240: Detect whether the login status is valid, specifically including: reading the login timestamp corresponding to the instance from the driver instance mapping table, or reading the login time record from a local file or database; calculating the duration between the current time and the login timestamp; comparing the duration with a preset login validity period; if the preset login validity period has not been exceeded, the login status is valid, and continue to step S260; if the preset login validity period has been exceeded, the login status has expired, and proceed to step S250.

[0040] Step S250: Refresh the login state, release the instance access lock, and acquire the login state update lock. After acquiring the login state update lock, check again whether the login state has been refreshed by another thread. If it has been refreshed, release the login state update lock and reacquire the instance access lock, then continue to step S260. If it has not been refreshed, execute the re-login process—automatically drive the instance to open the login page of the target system, fill in the login credentials and submit the login form. After confirming successful login, update the login time record with the current time as the new login timestamp, release the login state update lock, and reacquire the instance access lock. Through the above login state update lock and double-check mechanism, it can be ensured that when multiple consumer threads simultaneously detect that the login state has expired, only one thread actually performs the re-login, while the other threads wait for the login to complete and reuse the updated login state.

[0041] Step S260: Detect whether the page automation driving instance is valid. The validity detection includes a combination of one or more of the following operations: obtain the Uniform Resource Locator (URL) of the current page; if the URL is successfully obtained and the return value is not empty, it is considered valid; search for a specified preset element in the page; if the element exists, it is considered valid; execute a preset lightweight script and check the return value; if the return value meets expectations, it is considered valid; check whether the session identifier of the page automation driving instance exists and has not expired; if all the above checks pass, the page automation driving instance is valid, and continue to step S270; if any check fails or throws an exception, the page automation driving instance becomes invalid, and proceed to step S280.

[0042] Step S270: Clear the browser cache and return the instance. Call the corresponding interface of the page automation driving instance to clear the browser's local storage and session storage to avoid page rendering abnormalities or state crosstalk caused by the accumulation of cached data. Then, return the page automation driving instance to the caller and release the instance access lock.

[0043] Step S280: Rebuild the page automation driver instance, specifically including: detecting and forcibly closing the browser process and driver process associated with the currently failed instance to thoroughly clean up invalid resources; restarting the browser instance and creating a new page automation driver session; updating the newly created page automation driver instance and its related information to the driver instance mapping table, replacing the original failed instance record; after reconstruction, if the reconstruction was triggered by login state expiration, a re-login process must also be executed. Then, execute step S270 to clear the cache and return to the new instance.

[0044] The driver instance manager 130 also provides a forced restart method and a shutdown method. The forced restart method is used to actively execute the process cleanup and instance reconstruction operation in step S280 when a serious abnormality is detected in the page automation driver instance, and notify the associated consumer thread to continue consuming normally after reconstruction. The shutdown method is used to shut down all page automation driver instances, browser processes and driver processes one by one when the system is shut down or under maintenance, to ensure the complete release of resources.

[0045] Example 3 Reference Figure 3 This embodiment illustrates the overall process of general message queue task processing under the RPA architecture.

[0046] Step S310: The system starts and reads the message queue processing configuration, which includes queue grouping information, number of threads, number of messages prefetched, task mapping relationship, and queue type identifier, etc.

[0047] In step S320, the group consumption module 120 creates and starts a corresponding number of consumption threads for each queue group based on the queue grouping information. Each consumption thread independently creates a message queue connection and message channel, and then enters the consumption loop.

[0048] Step S330: The consumer thread receives a task message from the corresponding message queue.

[0049] In step S340, the task processing module 140 performs unified parsing on the received task message, checks the message body type, and converts it to a string type if it is a byte type; it then performs JSON parsing on the string to obtain the message body object. If parsing fails, it records an exception log, performs message confirmation according to the configuration, or transfers the message to the exception message queue, and returns to step S330 to continue receiving the next message.

[0050] Step S350: Determine the RPA business processing function corresponding to the current message queue based on the task mapping relationship.

[0051] Step S360: Obtain an available page automation driver instance through the driver instance manager 130. The driver instance manager 130 then proceeds according to... Figure 2 The process shown checks the login status and the validity of the page-driven instance, and performs re-login or instance reconstruction if necessary.

[0052] Step S370: Call the RPA business processing function, pass in the page automation driver instance and message body object, and execute the actual RPA automation operation.

[0053] Step S380: Determine whether the RPA business processing is successful. If successful, obtain the business processing result data and execute step S390. If it fails and throws an exception, capture the exception information and execute step S400.

[0054] Step S390: Perform successful path processing in the differentiated confirmation strategy for the current task message. Based on the queue type identifier, perform positive confirmation for regular business queues; for long-connection queues, if prior confirmation was not performed before, perform positive confirmation now and clear the corresponding task status record. Then, proceed to step S410 to send the result.

[0055] Step S400: Perform abnormal path processing in the differentiated confirmation strategy for the current task message and determine the abnormal type: If the abnormality is related to page automation driver, call the driver instance manager 130 to force restart the driver instance, then perform negative confirmation and re-enqueue the message; if the abnormality is other unrecoverable types, perform positive confirmation to avoid infinite loop of abnormal messages, and record detailed abnormal logs; for long-connection type queues, if early confirmation has been performed previously, construct a compensation task message according to the task status record and re-deliver it to the corresponding queue.

[0056] In step S410, the result delivery module 160 sends the task processing result to the corresponding downstream result queue. If the sending fails, it will retry according to the exponential backoff strategy. If the number of retries reaches the preset limit, the result message will be written to the abnormal result queue.

[0057] Step S420: Return to step S330 and continue consuming the next message.

[0058] If an uncaught exception occurs in the consumption thread at any stage of the consumption loop, causing the thread to exit, the exception handling logic in the group consumption module 120 will take over: clean up the message queue connection and message channel corresponding to the thread, notify the driver instance management module 130 to release the associated page automation driver resources; after a preset delay, recreate the message queue connection and message channel, and restart the consumption loop of the consumption thread.

[0059] Example 4 Reference Figure 4 This embodiment illustrates the configuration method and operation mechanism of RPA business hierarchical multi-threaded group consumption. In this embodiment, the message queue is divided into three level groups.

[0060] Level 1 Group 410 is a high-concurrency batch RPA task group, including batch data query queues and batch log statistics queues. The characteristics of this type of task are: no or only slight dependence on page automation-driven instances; short execution time; support for concurrent processing of multiple tasks, with no page state dependencies between tasks; for Level 1 Group 410, configure 3 to 5 consumer threads and 10 to 20 message prefetches. Each consumer thread prefetches messages in batches from its bound message queue and processes them in parallel in local memory to improve throughput.

[0061] The second-level group 420 is the regular page operation RPA task group, including form filling queue and data crawling queue. The characteristics of this type of task are: it depends on the page automation driver instance; it involves page element location and operation; the average execution time is in the middle; there may be page state dependencies between different tasks. For the second-level group 420, the number of consumer threads is configured to be 1 to 2, and the number of messages prefetched is 1 to 3. By limiting the number of prefetched messages and the number of threads, state conflicts and element location failures caused by multiple page tasks operating on the page automation driver instance at the same time are avoided.

[0062] Level 3 group 430 is a low-frequency, slow-execution RPA task group, including system login operation queue and complex page flow operation queue. The characteristics of this type of task are: it must exclusively occupy the page automation driver instance; the execution time is relatively long; it involves maintaining the login state or multi-step page jump. For Level 3 group 430, the number of consumer threads is configured to be 1, and the number of messages prefetched is 1. Through the single-threaded, single-task serial consumption mode, it is ensured that only one page task is operating the page automation driver instance at any given time, thereby avoiding resource contention and state crosstalk.

[0063] The division and configuration parameters of the above three level groups can be flexibly adjusted by modifying the configuration information in the configuration reading module 110, without modifying the core consumption logic code. When adding a new type of RPA task queue, it is only necessary to add the corresponding queue grouping and task mapping relationship in the configuration.

[0064] Example 5 The following description, in conjunction with specific application scenarios, further illustrates the implementation methods and beneficial effects of the present invention.

[0065] Scenario 1: An e-commerce platform's RPA system needs to batch process the creation, modification, and removal of thousands of coupons. This task corresponds to the first-level group 410, with 4 consumer threads and 15 prefetched messages. After system startup, the 4 consumer threads pull coupon management task messages from the message queue in parallel. The task processing module 140 parses the message body, extracts parameters such as coupon type, face value, and validity period, calls the corresponding coupon management RPA business processing function, and logs into the merchant backend through a page automation driver instance to execute batch operations. Since the coupon management tasks are independent of each other and do not depend on the page state, the 4 consumer threads share the page automation driver instance pool provided by the driver instance manager. The driver instance manager internally maintains 4 page automation driver instances corresponding to the 4 consumer threads, achieving parallel processing. Through the above mechanism, the system reduces the risk of queue deadlock during long-term operation.

[0066] Scenario 2: A cross-border e-commerce platform's RPA system needs to update information such as the title, description, and price of product details pages based on the operation instructions of the operators. This task corresponds to the second-level group 420, with 2 consumption threads and 1 message prefetch. The system pulls Listing update tasks one by one from the message queue, obtains the page automation driver instance through the driver instance manager, opens the target product editing page, locates each form field and fills in the new content, and submits the update form. During the process, a task processing fails due to a browser page loading timeout, causing a driver communication timeout exception. After the task processing module 140 captures this exception, the message confirmation module 150 determines that it is a page automation driver-related exception and notifies the driver instance manager to perform a forced restart—closing the frozen browser process and driver process, recreating the browser instance and driver session. After recovery, the message is negatively confirmed and re-queued. When the re-queued message is consumed again, the driver instance manager provides a new available driver instance, and the task is successfully executed.

[0067] Scenario 3: In Scenario 2 above, the driver instance manager checks the login status each time it retrieves an automated driver instance for a page. On a certain day, the login validity period has expired since the last login. The driver instance manager detects that the login status has expired, acquires a login status update lock, and executes a re-login process—opening the login page, entering the username and password, submitting and confirming successful login, and updating the login timestamp. During this process, another consumer thread also requests to retrieve a driver instance. The driver instance manager detects that the login status update lock is already held, and this thread enters a waiting state. After the thread holding the login status update lock completes the re-login, it releases the lock, and the waiting thread is awakened and reuses the new login status to continue executing the task. Through the above mechanism, redundant login operations and potential account risk control issues caused by multiple threads re-login simultaneously are avoided.

[0068] The above description is merely a preferred embodiment of the present invention and does not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention's specification and drawings within the spirit and principles of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A general message queue task processing method under RPA architecture, characterized in that, include: Read the message queue processing configuration, which includes queue grouping information, task mapping relationship and message prefetching quantity; The corresponding consumer thread is started according to the queue grouping information, and an independent message queue connection and message channel are established for each consumer thread; The consumer thread receives task messages, performs unified parsing on the task messages, and determines the RPA business processing logic corresponding to the current message queue based on the task mapping relationship. Before executing the RPA business processing logic, the page automation driver instance is obtained through the driver instance manager. The driver instance manager is used to detect the validity period of the login state and the validity of the page automation driver instance. When the login state expires, a re-login process is triggered. When the page automation driver instance is invalid, a driver instance reconstruction process is triggered. The RPA business processing logic is executed through the page automation-driven instance to obtain task processing results or exception information; Based on the type of the queue to which the task message belongs, the task processing result, and the exception information, a differentiated message confirmation strategy is executed: for regular business queues, a positive confirmation is executed when the task is successfully processed; for page automation driver-related exceptions, the page automation driver instance is first restored through the driver instance manager, then a negative confirmation is executed and the task message is re-enqueued. The task processing result is sent to the downstream result queue corresponding to the upstream message queue, and if the sending fails, it is resent according to the preset retry strategy; When a consumer thread encounters an exception, the message queue connection, message channel, and page automation drive resources corresponding to that consumer thread are cleaned up, and the connection is re-established after a preset delay to continue consumption.

2. The general message queue task processing method under the RPA architecture according to claim 1, characterized in that, The driver instance manager is a global singleton, which maintains a driver instance mapping table internally. The driver instance mapping table stores the corresponding page automation driver instances using the consumer thread identifier or queue group identifier as an index. The driver instance manager is configured with an instance access lock and a login status update lock. The instance access lock is used to ensure thread safety when acquiring, creating, rebuilding or releasing page automation driver instances. The login status update lock is used to allow only one consumer thread to execute the re-login process after the login status is detected to have expired, while the other consumer threads wait for the login to be completed and reuse the updated login status.

3. The general message queue task processing method under the RPA architecture according to claim 1, characterized in that, The detection of login status validity and page automation-driven instance validity includes: Read login timestamps recorded in local files, databases, or caches; Determine whether the time between the current time and the login timestamp exceeds the preset login validity period; If the timeout period is exceeded, acquire the login status update lock, execute the re-login process, update the login timestamp, and release the login status update lock. Under the premise that the login state is valid, perform a validity check on the page automation driving instance. The validity check includes at least one of the following: obtaining the current Uniform Resource Locator, checking whether the specified page element exists, executing a preset script and checking the return value, and checking whether the driving session identifier is valid. If the validity check fails, the browser process and driver process associated with the current driver instance are closed, the browser instance and driver instance are recreated, and the browser's local storage and session storage are cleared after recreation.

4. The general message queue task processing method under the RPA architecture according to claim 1, characterized in that, The message queue processing configuration also includes the number of threads corresponding to each queue; the method also includes: For each message queue, its resource consumption level is determined based on the task attributes of the corresponding RPA task. The task attributes include at least one of the following: whether a page automation driver instance is required, whether a page automation driver instance needs to be exclusively used, whether page-level operations are involved, average execution time, whether login status is required, and whether concurrent processing is supported. For message queues of automated drive instances with high resource consumption levels or requiring exclusive page access, configure their message prefetch quantity to 1 and use a single consumption thread for serial consumption; For message queues with low resource consumption levels and support concurrent processing, configure their message prefetch quantity to a value greater than 1, and start multiple consumer threads for parallel consumption.

5. The general message queue task processing method under the RPA architecture according to claim 1, characterized in that, The implementation of the differentiated message confirmation strategy also includes: For long-connection queues, when preset conditions are met, the task message is first affirmed, and the task identifier, message digest, upstream queue identifier and processing status are written into the task status record at the same time or before the affirmative acknowledgment is performed. If the subsequent RPA service processing fails, a compensation task message is generated based on the task status record and re-delivered. If message parsing fails, log the exception and return a standardized failure result, perform a positive acknowledgment on the message or transfer it to the exception message queue.

6. The general message queue task processing method under the RPA architecture according to claim 1, characterized in that, The task messages are parsed uniformly, including: Detect the message body type of the task message; if it is a byte type, convert it to a string type. Parse the converted string using JSON to obtain the message body object; If parsing fails, a parsing exception log is recorded, a standardized error format processing result is returned, and the message is either positively acknowledged or transferred to the exception message queue according to the queue configuration. If the parsing is successful, the task type identifier is extracted from the message body object, and the corresponding RPA business processing function is determined according to the task mapping relationship.

7. The general message queue task processing method under the RPA architecture according to claim 1, characterized in that... The step of sending the task processing result to the downstream result queue corresponding to the upstream message queue includes: Based on the pre-configured queue mapping relationship, determine the downstream result queue corresponding to the current upstream message queue; or, based on the task type identifier in the task processing result, dynamically match the corresponding downstream result queue. Construct a result message, which includes a task identifier, an upstream queue identifier, a processing status, business result data, an exception type, a number of retries, and a log summary; Send the result message to the downstream result queue and enable the message confirmation mechanism; If the transmission fails, a retry is performed according to the exponential backoff strategy, wherein the backoff interval of the exponential backoff strategy increases with the number of retries. If the number of retries reaches the preset limit, the result message will be written to the exception result queue.

8. A general message queue task processing system under RPA architecture, characterized in that, include: A configuration reading module is used to read the message queue processing configuration, which includes queue grouping information, task mapping relationship and message prefetching quantity; The group consumption module is used to start the corresponding consumption thread according to the queue grouping information, and to establish an independent message queue connection and message channel for each consumption thread; The task processing module is used to receive task messages through the consumption thread, perform unified parsing of the task messages, determine the corresponding RPA business processing logic according to the task mapping relationship, and execute the RPA business processing logic through the page automation driver instance. The driver instance management module is used to provide the task processing module with a page automation driver instance before executing the RPA business processing logic. The driver instance management module detects the validity period of the login state and the validity of the page automation driver instance. When the login state expires, a re-login process is triggered. When the page automation driver instance is invalid, a driver instance reconstruction process is triggered. The message confirmation module is used to execute a differentiated message confirmation strategy based on the type of the queue to which the task message belongs, the task processing result, and the exception information. Specifically, for regular business queues, a positive confirmation is executed when the task is successfully processed. For page automation driver-related exceptions, the page automation driver instance is first restored through the driver instance management module before a negative confirmation is executed and the task message is re-enqueued. The result delivery module is used to send the task processing result to the downstream result queue corresponding to the upstream message queue, and resend according to the preset retry strategy when the sending fails. The group consumption module is also used to, when an abnormality occurs in the consumption thread, clean up the message queue connection, message channel and page automation drive resources corresponding to the consumption thread, and re-establish the connection after a preset delay to continue consumption.

9. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the general message queue task processing method under the RPA architecture as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the general message queue task processing method under the RPA architecture as described in any one of claims 1 to 7.

Images