A method and apparatus for processing batch tasks

Abstract

The present disclosure provides a batch task processing method and device, a batch task request for processing a target batch task is sent to a server, a push event channel and an active marker corresponding to the push event channel are established between the server, a sub-task processing result issued by the server is received through the push event channel, a task processing data of a database is queried by a polling device according to a polling frequency corresponding to a task state of the target batch task, if there is a difference between the task processing data and historical task data, whether the push event channel is abnormal is detected based on the active marker, if the push event channel is not abnormal, a view of a client is updated based on the sub-task processing result, and if the push event channel is abnormal, the view of the client is updated based on the task processing data. In this way, the processing accuracy and efficiency of the batch task can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for processing batch tasks. Background Technology

[0002] In generative AI batch task scenarios (such as generating multiple sets of character three-view drawings at once, or batch generating multiple candidate videos), the time taken for a single subtask is typically between 15 and 180 seconds, and a single request outputs multiple sub-results. Users need to see the completion status of each sub-result in real time to improve the experience, and during long process execution, it is essential to ensure that any network anomaly or client interruption does not cause the loss of completed sub-results.

[0003] In related technologies, the server can push result events to the client in real time upon completion of each subtask using the Server Push Events (SSE) or WebSocket protocols. The client listens to the event stream and renders the data immediately. However, if the push connection is lost due to network fluctuations, the client cannot obtain the results of the completed subtasks during the disconnection period, resulting in data loss and affecting the client's data rendering process. Alternatively, the client can periodically query the server for task status at fixed time intervals to obtain the latest results. However, this polling method has a fixed polling interval delay between subtask completion and client awareness, affecting the efficiency of batch task processing. Summary of the Invention

[0004] In view of this, the purpose of this disclosure is to provide a method and apparatus for processing batch tasks. The server determines the processing results of subtasks in real time and persists them to the database. The client can obtain the processing results of subtasks by querying the database through a poller. At the same time, the push event channel and the poller polling channel work together to ensure that the client can accurately obtain the processing results of subtasks even in the event of network abnormalities, which helps to improve the accuracy and efficiency of batch task processing.

[0005] In a first aspect, embodiments of this disclosure provide a method for processing batch tasks, applied to a client; the processing method includes: Send a batch task request to the server to process the target batch tasks, and establish a push event channel with the server and an active tag corresponding to the push event channel; The server sends the subtask processing result through the push event channel; wherein the subtask processing result is sent by the server after it has persisted the subtask processing result to the database. The poller is controlled to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks; If there is a difference between the task processing data and the historical task data, the system detects whether there is an anomaly in the push event channel based on the active flag. If there is no anomaly in the push event channel, the system updates the client's view based on the subtask processing result. If there is an anomaly in the push event channel, the system updates the client's view based on the task processing data.

[0006] In a first aspect, embodiments of this disclosure also provide a method for processing batch tasks, applied to a server; the processing method includes: Receive a batch task request sent by the client to process a target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task; In response to the completion of the persistence of the subtask processing result, the subtask processing result is sent to the client through a push event channel. This allows the client to determine the abnormality of the push event channel based on its activity identifier. If the push event channel is determined to be free of abnormality, the client updates its view based on the subtask processing result. If the push event channel is abnormal, the client updates its view based on the task processing data obtained from the database by the poller.

[0007] Thirdly, embodiments of this disclosure also provide a batch task processing apparatus applied to a client; the processing apparatus includes: The batch task request sending module is used to send batch task requests for processing target batch tasks to the server, and to establish a push event channel with the server and an active flag corresponding to the push event channel. The first subtask processing result distribution module is used to receive the subtask processing result distributed by the server through the push event channel; wherein, the subtask processing result is distributed by the server after persistently writing the subtask processing result into the database; The polling query module is used to control the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch task; The view update module is used to detect whether there is an anomaly in the push event channel based on the active marker if there is a difference between the task processing data and the historical task data. If there is no anomaly in the push event channel, the view of the client is updated based on the subtask processing result; if there is an anomaly in the push event channel, the view of the client is updated based on the task processing data.

[0008] Fourthly, embodiments of this disclosure also provide a batch task processing apparatus applied to a server; the processing apparatus includes: The subtask processing result persistence module is used to receive a batch task request sent by the client to process the target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task; The second subtask processing result distribution module is used to distribute the subtask processing result to the client through a push event channel in response to the end of the subtask processing result persistence. This allows the client to determine the abnormality of the push event channel based on its activity identifier. If the push event channel is determined to be free of abnormality, the client updates its view based on the subtask processing result. If the push event channel is abnormal, the client updates its view based on the task processing data obtained from the database by a poller.

[0009] Fifthly, embodiments of this disclosure also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the batch task processing method as described in the first or second aspect.

[0010] In a sixth aspect, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the batch task processing method as described in the first or second aspect.

[0011] The batch task processing method and apparatus provided in this disclosure send a batch task request to a server to process the target batch task, and establish a push event channel with the server and an activity flag corresponding to the push event channel; receive sub-task processing results from the server through the push event channel; control a poller to query task processing data in the database according to the polling frequency corresponding to the task status of the target batch task; if there is a difference between the task processing data and historical task data, detect whether there is an anomaly in the push event channel based on the activity flag; if there is no anomaly in the push event channel, update the client's view based on the sub-task processing results; if there is an anomaly in the push event channel, update the client's view based on the task processing data. In this way, the server determines in real time that the sub-task processing results are persistently written to the database, and the client can obtain the sub-task processing results by querying the database through the poller. Simultaneously, the push event channel and the poller polling channel work together to ensure that the client can accurately obtain the sub-task processing results even in the event of network anomalies, which helps improve the accuracy and efficiency of batch task processing.

[0012] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this disclosure and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating a batch task processing method provided in an embodiment of this disclosure; Figure 2 A flowchart illustrating another batch task processing method provided in an embodiment of this disclosure; Figure 3 A schematic diagram illustrating the results of a batch task processing system provided in an embodiment of this disclosure; Figure 4 This is a timing diagram illustrating the dual-channel collaborative processing of batch tasks provided in an embodiment of this disclosure. Figure 5 This is a flowchart illustrating the polling frequency decision-making process provided in an embodiment of this disclosure. Figure 6 This is one of the structural schematic diagrams of a batch task processing device provided in an embodiment of the present disclosure; Figure 7 This is a second schematic diagram of a batch task processing device provided in an embodiment of the present disclosure; Figure 8 This is a schematic diagram of the structure of a task processing system provided in an embodiment of the present disclosure. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. Based on the embodiments of this disclosure, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this disclosure.

[0016] First, the technical background of this disclosure will be introduced.

[0017] In generative AI batch task scenarios (such as generating multiple sets of character three-view drawings at once, or batch generating multiple candidate videos), the time taken for a single subtask is typically between 15 and 180 seconds, and a single request outputs multiple sub-results. Users need to see the completion status of each sub-result in real time to improve the experience, and during long process execution, it is essential to ensure that any network anomaly or client interruption does not cause the loss of completed sub-results.

[0018] In related technologies, the server can push result events to the client in real time upon completion of each subtask using the Server Push Events (SSE) or WebSocket protocols. The client listens to the event stream and renders the data immediately. However, if the push connection is lost due to network fluctuations, the client cannot obtain the results of the completed subtasks during the disconnection period, resulting in data loss and affecting the client's data rendering process. Alternatively, the client can periodically query the server for task status at fixed time intervals to obtain the latest results. However, this polling method has a fixed polling interval delay between subtask completion and client awareness, affecting the efficiency of batch task processing. Another approach is to distribute tasks to background processes via message queues, with task status recorded in a cache or database. The client subscribes to status changes through an additional push bridging layer. However, task status is typically updated only when the entire task is completed, and incremental field updates at the subtask level are not supported. Client subscription to message queues requires an additional push proxy service, increasing system architecture complexity and impacting the efficiency of batch task processing.

[0019] Based on this, the present disclosure provides a batch task processing method to improve the accuracy and efficiency of batch task processing.

[0020] Please see Figure 1 , Figure 1 This is a flowchart illustrating a batch task processing method provided in an embodiment of this disclosure. Figure 1 As shown in the embodiments of this disclosure, the batch task processing method includes: S101. Send a batch task request to the server to process the target batch tasks, and establish a push event channel with the server and an active flag corresponding to the push event channel.

[0021] S102. Receive the subtask processing result sent by the server through the push event channel; wherein the subtask processing result is sent by the server after it has persistently written the subtask processing result into the database.

[0022] S103. Control the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch task.

[0023] S104. If there is a difference between the task processing data and the historical task data, detect whether there is an anomaly in the push event channel based on the active marker. If there is no anomaly in the push event channel, update the client's view based on the subtask processing result; if there is an anomaly in the push event channel, update the client's view based on the task processing data.

[0024] The batch task processing method provided in this disclosure involves the server determining the subtask processing results in real time and persistently writing them into the database. The client can obtain the subtask processing results by querying the database through a poller. At the same time, the push event channel and the poller polling channel work together to ensure that the client can accurately obtain the subtask processing results even in the event of network anomalies, which helps to improve the accuracy and efficiency of batch task processing.

[0025] The exemplary steps of the embodiments of this disclosure are described below: S101. Send a batch task request to the server to process the target batch tasks, and establish a push event channel with the server and an active flag corresponding to the push event channel.

[0026] Here, the target batch task can be an AI batch task, such as generating multiple sets of three-view drawings of game characters at once, or generating multiple candidate videos in batches.

[0027] In one alternative implementation, after the client sends a batch task request to the server, the server provides a streaming response endpoint based on the Server Push Event (SSE) protocol. The server returns a streaming response as the main push channel for the subtask processing results, thereby establishing a persistent push event channel.

[0028] Here, after all subtasks of the target batch task have been completed, the server pushes a task completion termination message to the client and closes the push event channel.

[0029] The primary push channel can be either the SSE protocol or the WebSocket protocol.

[0030] However, in this embodiment, the main push channel only requires one-way push capability from the server to the client. The implementation complexity and resource consumption of the SSE protocol in a one-way push scenario are lower than those of WebSocket. Furthermore, the SSE protocol is based on the standard HTTP protocol and is naturally compatible with various proxy servers and load balancers, while WebSocket requires explicit support for protocol upgrades from the proxy server. The native browser support for the SSE protocol is also more extensive, eliminating the need for an additional WebSocket client library. Therefore, in this embodiment, the SSE protocol push event is preferentially selected as the main push channel.

[0031] Furthermore, after establishing a push event channel, a push channel activity flag can be set to determine whether there are any anomalies in the current push channel. Based on the anomaly status of the push channel, the view can be updated according to the task data queried by the poller's query channel or the subtask processing results issued by the main push channel.

[0032] Furthermore, the server can invoke the AI ​​server to process batch task requests and push the subtask processing results to the client.

[0033] S102. Receive the subtask processing result sent by the server through the push event channel; wherein the subtask processing result is sent by the server after it has persistently written the subtask processing result into the database.

[0034] In one alternative implementation, after determining the result of a subtask, the server immediately persists the subtask result to the database. After persistence, the server directly sends the subtask result to the client through the push event channel. This ensures that if the server push channel is interrupted, all completed subtask results are safely stored in the database and can be recovered at any time through polling channels or subsequent sessions.

[0035] Furthermore, the client can use a poller to query the database at the appropriate polling frequency and obtain the corresponding task processing data.

[0036] S103. Control the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch task.

[0037] In one alternative implementation, the poller starts automatically when the component is mounted, and periodically calls the server's task status query interface at a polling frequency calculated based on the task status of the target batch tasks to retrieve all task status records of the current target batch tasks from the database.

[0038] Here, the polling frequency of the poller can be dynamically determined according to the task status. By dynamically setting the polling frequency, different polling frequencies can be achieved under different working conditions, which can greatly reduce invalid polling requests.

[0039] Specifically, the step "controlling the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks" includes: a1: Query the target task table in the database corresponding to the target batch task using a poller.

[0040] a2: If there are tasks in the target task table that are in a running state, control the poller to query the task processing data in the database according to the first polling frequency.

[0041] a3: If there is at least one task in the target task table and at least one task is in a non-running state, control the poller to query the task processing data in the database according to the second polling frequency.

[0042] Here, the status records of all tasks in the current target batch task can be recorded in the target task table. Each record in the target task table includes at least the following fields: project identifier (for isolation of multiple projects), task key (identifying the specific task type and step), status (which can be running, successful, failed, etc.), start time, end time, result summary (readable text summary), structured result data (structured fields that can be incrementally updated), and error message.

[0043] Furthermore, based on whether there is at least one task in the target task table and the task status of each task, it is determined whether the poller queries the database to obtain task processing data at a faster first polling frequency or a slower second polling frequency.

[0044] In one alternative implementation, if there are tasks in the target task table that are in a running state, the poller can be controlled to query the database for task processing data at a faster first polling frequency, providing high-frequency feedback.

[0045] In another alternative implementation, if there is at least one task in the target task table and at least one task is in a non-running state (all tasks are in a completed or failed state), the poller can be controlled to query the task processing data in the database at a faster second polling frequency to maintain low-frequency checks.

[0046] Here, the first polling frequency can be set to 5 seconds; the second polling frequency can be set to 15 seconds.

[0047] In another alternative implementation, if the target task table is empty, the polling process will not be performed.

[0048] Specifically, the processing method further includes: b1: If the target task table is empty, stop the poller's polling operation.

[0049] In this embodiment of the disclosure, if it is determined that there is no empty table of task data in the current target task, the polling frequency is set to 0; the poller stops polling operations, reducing the load pressure on the client.

[0050] In one alternative implementation, the polling frequency of the poller can also be adjusted based on the number of consecutive errors that occur after a poller network failure.

[0051] Specifically, the processing method further includes: c1: If the poller has a network anomaly, the third polling frequency is determined according to the number of consecutive errors, the first polling frequency, and the preset backoff upper limit.

[0052] c2: Controls the poller to query the task processing data in the database according to the third polling frequency.

[0053] In one optional implementation, when a network anomaly is determined in the poller, a third polling frequency can be determined based on the number of consecutive errors, the first polling frequency, and a preset backoff upper limit. Specifically, the third polling frequency can be determined using the following formula: T= min(T_fast × 2^(n-1), T_max); Where T is the third polling frequency; T_fast is the first polling frequency; n is the number of consecutive errors, which can be calculated starting from 1, with an upper limit of 6; and T_max is the preset backoff upper limit.

[0054] Here, as the number of consecutive errors increases, the corresponding frequency of the third round of polling also increases.

[0055] For example, taking a first polling frequency of 5 seconds and a preset backoff limit of 60 seconds as an example: First error: 0 consecutive errors; third polling frequency = min(5 × 2^0, 60) = min(5, 60) = 5 seconds; Second error: 1 consecutive error, third polling frequency = min(5 × 2^1, 60) = min(10, 60) = 10 seconds; Third error: 2 consecutive errors, third polling frequency = min(5 × 2^2, 60) = min(20, 60) = 20 seconds; Fourth error: 3 consecutive errors, third polling frequency = min(5 × 2^3, 60) = min(40, 60) = 40 seconds; Fifth error: 4 consecutive errors, third polling frequency = min(5 × 2^4, 60) = min(80, 60) = 60 seconds (reaching the limit).

[0056] In one alternative implementation, if the poller network recovers after an error occurs, the accumulated number of consecutive errors is cleared to zero, and then the corresponding polling frequency is determined according to the task status to query the database normally.

[0057] Furthermore, when the poller performs polling operations, a concurrency flag can be maintained. Each time a polling request is initiated, it is necessary to determine whether there are any incomplete requests. If so, the current request should be skipped to prevent request backlog and reduce client load.

[0058] Specifically, the step "controlling the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks" includes: d1: For each polling query request, check if there are any historical query requests that have not returned results.

[0059] d2: If there are no historical query requests that have not returned results, query the task processing data in the database.

[0060] d3: If there are historical query requests that have not returned results, skip the current polling query request until the historical query request is completed.

[0061] In one optional implementation, before each polling query request, it is checked whether there is a flag in progress in the concurrency flag. If there is a flag in progress, it indicates that the previous historical query request has not returned a result, that is, the previous historical query request has not been completed, and the current polling query request needs to be skipped until the historical query request is completed. After the historical query request is completed (completed normally or failed), the corresponding flag is cleared.

[0062] In another alternative implementation, if there is no in-process marker, it indicates that the previous historical query request has returned a result, that is, the previous historical query request has been completed, and the task processing data in the database can be queried directly.

[0063] In this embodiment of the disclosure, the client may execute different projects to perform different batch tasks. Therefore, the poller needs to maintain another project identifier. After the query data is returned, it is necessary to determine whether the project identifier corresponding to the previous task data is consistent with the current project identifier, and then consider whether to discard the task processing data queried this time.

[0064] Specifically, after the step of "controlling the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks", the processing method further includes: e1: Detect whether the project identifier corresponding to the task processing data is consistent with the project identifier of the current client.

[0065] e2: If the project identifier corresponding to the task processing data is inconsistent with the project identifier of the current client, discard the task processing data.

[0066] In one optional implementation, after obtaining the task processing data, it is checked whether the project identifier corresponding to the task processing data is consistent with the project identifier of the current client. If they are inconsistent, it is determined that the user may have switched projects after the batch task request was issued. At this time, the project identifier corresponding to the task processing data is inconsistent with the project identifier of the current client. The poller discards the current task processing data to avoid old project data from polluting the state of the new project.

[0067] Furthermore, if the project identifier corresponding to the task processing data is consistent with the project identifier of the current client, then the method of updating the client's view can be determined based on the differences between the obtained task processing data and historical task data, as well as the abnormal situation of the push channel.

[0068] S104. If there is a difference between the task processing data and the historical task data, detect whether there is an anomaly in the push event channel based on the active marker. If there is no anomaly in the push event channel, update the client's view based on the subtask processing result; if there is an anomaly in the push event channel, update the client's view based on the task processing data.

[0069] In one alternative implementation, after the poller retrieves new task processing data, to ensure the accuracy of the view update, the view update is not triggered directly. Instead, the currently obtained task processing data is compared with the historical task data to determine if there is a difference. Only when a difference is found will the subsequent view update process be executed.

[0070] Specifically, the following steps are used to determine the difference between the task processing data and historical task data: f1: The task processing data and historical task data are compared one by one using a shallow comparison function.

[0071] f2: If the data in at least one target field of the task processing data is different from that in the historical task data, it is determined that there is a difference between the task processing data and the historical task data.

[0072] In one alternative implementation, as described above, the acquired task processing data can be recorded in the target task table. A shallow comparison function can be used to compare the task key, status, end time, error message, result summary, timeout flag, and other fields of the task processing data with those of historical task data. For structured result fields (including the length of the completed sub-result array and the completion count), the shallow comparison function does not rely on reference equality judgment but delves into the business field level, comparing at least the values ​​of the length of the completed sub-result array and the completion count. Only when these two business fields have actually changed is it determined that the data has been updated. When the shallow comparison function determines that there is a difference between the task processing data and the historical task data, the poller will trigger a view state update to notify the business components that the data has changed.

[0073] Furthermore, after changes occur in the business component data, it is necessary to determine the abnormal situation of the current push event channel based on the activity flag, and then update the view on the client.

[0074] Specifically, the step "detecting whether there is an anomaly in the push event channel based on the active marker; if there is no anomaly in the push event channel, updating the client's view based on the subtask processing result; if there is an anomaly in the push event channel, updating the client's view based on the task processing data" includes: g1: If the active flag is a valid value, determine that there is no abnormality in the push event channel, and update the client's view based on the subtask processing result.

[0075] g2: If the active flag is empty, it is determined that there is an anomaly in the push event channel, and the view of the client is updated based on the task processing data.

[0076] Here, a reference object that can be asynchronously terminated can be maintained on the client as a marker of the active push channel. Specifically, when the push event connection is successfully established, the reference is set to a valid value (pointing to the termination controller of the current connection); when the push event connection is closed (normal completion, client-initiated termination, or network disconnection), the reference is set to null.

[0077] In one optional implementation, if the active flag is a valid value, it indicates that the push channel is active and there is no anomaly. At this time, the poller queries data and performs auxiliary functions such as timeout detection according to the corresponding polling frequency. After receiving the status change notification from the poller, the business component detects the existence of the active flag and does not trigger the data reload caused by the incremental change of the structured result data. Instead, it transfers the data triggering right to the push event channel and updates the client's view based on the subtask processing results issued by the push event channel.

[0078] In another optional implementation, if the active flag is empty, it indicates that there is an anomaly in the push event channel, such as a network interruption: after the business component receives the poller's status change notification, it detects that the active flag is empty, and then automatically takes over the data triggering rights, updates the client's view based on the task processing data found by polling, performs data reload, and refreshes the view.

[0079] This avoids the problem of simultaneous refresh of both channels and ensures the fault tolerance capability that the other channel can automatically take over when any one channel fails; the poller continues to perform queries (instead of stopping completely) while the push channel is active in order to keep auxiliary functions such as timeout detection running continuously.

[0080] Specifically, the processing method further includes: h1: Determine the start time of at least one target task that is in the running state in the task processing data.

[0081] h2: Determine the target timeout threshold based on the task type of the target batch task.

[0082] h3: For each target task, if the time difference between the current time and the start time of the target task is greater than the target timeout threshold, add a timeout flag to the target task.

[0083] In one alternative implementation, the time taken to generate batch tasks varies significantly depending on the type of task. An independent timeout threshold can be configured for each task type. That is, the target timeout threshold can be determined based on the task type of the target task.

[0084] For example, the task type of the target batch task is determined based on the task key of the target batch task. If the task key contains a video generation identifier, the timeout threshold can be set to 10 minutes; if the task key contains an image generation identifier, the timeout threshold can be set to 5 minutes; for other types of tasks (such as text generation), the timeout threshold can be set to 3 minutes.

[0085] In one optional implementation, for each target task in the target batch tasks, the start time of the target task is determined, and then it is determined whether the time difference between the current time and the start time of the target task is greater than the target timeout threshold. If the time difference between the current time and the start time of the target task is greater than the target timeout threshold, a timeout flag is added to the task. After the timeout flag is set, the poller automatically calls the task deletion interface to remove the task from the database, the task status is restored to idle, and the user can re-initiate it.

[0086] Here, after adding a timeout flag, the client's polling query operation will no longer query the target task, but the server will still perform the operation of persistently storing the processing results of the subtasks corresponding to the target task.

[0087] In this way, timeout protection prevents tasks from being permanently suspended due to failures in external AI services, avoiding the user interface from displaying "loading" for an extended period and becoming inoperable; thresholds are configured differently according to task type to prevent timeouts from being accidentally triggered for long-running tasks, thereby reducing client load.

[0088] In another alternative implementation, client view updates can be triggered and deduplicated independently through dual channels. Specifically, the push channel and the polling channel trigger view updates independently, and duplicate updates are filtered out at the business component layer through deduplication logic.

[0089] However, deduplication logic increases the complexity of business components, as each component needs to implement deduplication checks; independent triggering of dual channels may result in two consecutive view refreshes within a short period (one triggered by push and one by polling), causing visual flickering; by implementing a mutual exclusion scheme for active marking at the poller layer, business components do not need to be aware of the existence of dual channels, reducing the complexity of upper-layer code. Therefore, in this embodiment, the dual-channel view update method with mutual exclusion detection of push channel and poller polling is preferred.

[0090] Furthermore, for the client side, if the user closes the page corresponding to the target batch task, the poller's polling operation can be stopped, and the poller's polling operation can be restarted after the page is restored. The polling behavior is dynamically adjusted according to the page status, which further reduces the network consumption of invisible pages. In addition, the real-time query when switching back to the visible page ensures that the user can see the latest status immediately after switching tabs and returning, thus improving the interactive experience.

[0091] Specifically, the processing method further includes: i1: In response to the view corresponding to the target batch task becoming invisible, stop the poller's polling operation.

[0092] i2: In response to the view corresponding to the target batch task becoming visible, resume the polling operation of the poller and query the task processing data of the database according to the polling frequency corresponding to the task status of the target batch task.

[0093] Here, the view change corresponding to the target batch task becoming invisible can be caused by the user switching to another tab or minimizing the browser window. At this time, the poller's polling operation is stopped. Specifically, the poller clears the current polling timer and pauses all periodic queries to avoid generating unnecessary network requests to the invisible page.

[0094] Furthermore, if the view corresponding to the target batch task becomes visible, the poller immediately triggers a task status query (without waiting for the next polling cycle) to ensure that the user sees the latest data; after the query is completed, the periodic polling at the corresponding polling frequency is resumed.

[0095] In another alternative implementation, the batch task processing is completed through collaborative processing between the client and the server. Specifically, after receiving the batch task request from the client, the server determines the processing results of each subtask, persists the subtask processing results, and sends them to the client. The client updates the view according to the subtask processing. The following will describe the batch task processing process with the server as the execution entity.

[0096] Please see Figure 2 , Figure 2 A flowchart illustrating another batch task processing method provided in an embodiment of this disclosure. Figure 2 As shown in the embodiments of this disclosure, the batch task processing method includes: S201. Receive a batch task request sent by the client to process the target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task.

[0097] S202. In response to the completion of the persistence of the subtask processing result, the subtask processing result is sent to the client through the push event channel, so that the client can determine the abnormality of the push event channel based on the activity identifier of the push event channel. When it is determined that there is no abnormality in the push event channel, the client's view is updated based on the subtask processing result. When there is an abnormality in the push event channel, the client's view is updated based on the task processing data obtained by querying the database by the poller.

[0098] In step S201, if the target task table corresponding to the target batch task does not exist in the current database, the corresponding target task table needs to be created, and the target task table is dynamically updated after the server obtains the subtask processing results.

[0099] Each record in the task status table must include at least the following fields: project identifier (for isolation of multiple projects), task key (identifying the specific task type and steps), status (with values ​​of running, success, and failure), start time, end time, result summary (human-readable text summary), structured result data (structured fields that can be incrementally updated), and error message. Here, the structured result data fields adopt a nestable structured type (such as the JSON type in the database), which includes at least the following: an array of completed sub-results (each element contains a sub-result identifier, tag, and resource address), total count, completion count, and error array.

[0100] In another alternative implementation, a message queue can be used instead of a database persistent table. Specifically, a message queue (such as a Redis stream or RabbitMQ) can be used instead of a database task status table as the persistent carrier for the subtask processing results.

[0101] However, message queues typically default to "consume and delete," making them unsuitable as persistent storage that can be repeatedly queried. Introducing message queues increases system architecture complexity and operational costs, requiring the additional setup of queue services and consumer processes. Database persistent tables support standard SQL queries, and pollers can directly obtain task status through ORM queries without additional message consumption logic. Furthermore, database persistent tables inherently support cross-session recovery (users can directly query after closing and reopening the browser), while message queues require additional message replay mechanisms. Therefore, in this embodiment, persistent storage based on the database's task table is preferred.

[0102] In one alternative implementation, the server implements four independent task status write interfaces, each of which independently opens a database session, does not occupy the caller's transaction context, and ensures atomicity.

[0103] Specifically, the task start interface sets the status of the specified task to "Running," records the start time, and clears the end time; if the task record already exists, it is updated; otherwise, it is created. Inputs: Project ID, Task Key; Output: Persistent task record. The task success interface sets the status of the specified task to "Success," writes the end time and final result data; Inputs: Project ID, Task Key, Result Summary, Structured Result Data; Output: Updated task record. The task failure interface sets the status of the specified task to "Failure," writes the error message and end time; Inputs: Project ID, Task Key, Error Message; Output: Updated task record. The incremental result update interface updates only the structured result data fields without changing the status value when the task is in the "Running" state; this interface is the underlying support for incremental persistent storage; Inputs: Project ID, Task Key, New Structured Result Data, Optional Result Summary; Output: Updated task record.

[0104] Furthermore, after determining the subtask processing result, the database is directly persisted, and the subtask processing result is directly written to the structured result data field of the target task table.

[0105] Specifically, the step "persisting the subtask processing result corresponding to the subtask to the target task table corresponding to the target batch task in the database" includes: j1: Append the subtask processing result corresponding to the subtask to the sub-result array corresponding to the structured result data field.

[0106] In this embodiment of the disclosure, the newly completed subtask processing result may be appended to the completed sub-result array to update the completion count.

[0107] For example, after the second subtask is completed, the structured result data becomes "The array of completed sub-results contains subtask processing result 1 and subtask processing result 2, with a completion count of 2 and a total of 4".

[0108] Furthermore, the defined incremental result update interface can be called to write the accumulated structured result data into the database, and the forced dirty marking function can be called to submit the database update event.

[0109] Specifically, after the step of "persisting the subtask processing result corresponding to the subtask to the target task table corresponding to the target batch task in the database", the processing method further includes: k1: Call the forced dirty marking function, submit the database update event to indicate the database update, and complete the persistence of the subtask processing results.

[0110] Here, when dealing with structured fields (such as JSON types), object-relational mapping (ORM) frameworks typically determine whether a database update statement needs to be issued by comparing whether the field reference has changed. When a developer makes in-place modifications to a structured field (such as appending an element to an array) without replacing the field reference, the ORM framework will determine that the field has "not changed" and skip the update operation, resulting in the loss of the modification.

[0111] In this embodiment of the disclosure, after writing the subtask processing result to the database, the forced dirty marking function provided by the ORM framework is explicitly called to inform the framework that the field has been modified and an update statement must be issued. After calling the forced dirty marking, the current database transaction is immediately committed so that the update is immediately visible to other database sessions. The immediate commit ensures that the polling can read the latest sub-result in the next query, thereby ensuring the accuracy of data synchronization.

[0112] Furthermore, after persisting the subtask processing results, the subtask processing results are sent to the client through a push event channel. The client determines the abnormal situation of the push event channel based on the activity identifier of the push event channel. When it is determined that there is no abnormality in the push event channel, the client's view is updated based on the subtask processing results. When there is an abnormality in the push event channel, the client's view is updated based on the task processing data obtained by querying the database by the poller. The specific update process is consistent with the batch task processing process with the client as the execution subject provided in the embodiments of this disclosure, and will not be described again here.

[0113] The following specific examples will illustrate the batch task processing procedure in the embodiments of this disclosure: Please see Figure 3 , Figure 3 This is a schematic diagram of the results of the batch task processing system provided in the embodiments of this disclosure, as shown below. Figure 3 As shown, the processing system includes a client, a server, and a persistence layer. The client's business components initiate batch generation requests to the push event client and send batch task requests to the server via the push event connection. The server responds to push event breakpoints in a streaming manner, calling the batch task concurrent execution engine. For each subtask completed, streaming incremental persistence is performed: forced dirty marking + immediate submission to the persistence layer, updating the task table's status and structured result data, and simultaneously pushing the subtask processing results to the client. A task status poller is maintained on the client, periodically querying the task status according to the corresponding polling frequency. The server's task status query endpoint accesses the persistence layer to obtain the task table. On the client, an active flag is set for the push event connection, and the component starts immediately upon mounting. Incremental checks are performed based on shallow comparisons, mutual exclusion references are used through the active flag of the push channel, and the task status poller silently provides fallback when the push channel is determined to be active.

[0114] Please see Figure 4 , Figure 4 This is a timing diagram illustrating the dual-channel collaborative processing of batch tasks provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, a user triggers a batch generation request on the client (e.g., generating 4 videos); the client sends a streaming request to the server; the server calls the task start interface, sets the status to running, updates the task status table, and concurrently submits multiple sub-tasks to the AI ​​generation server. After the AI ​​generation server completes processing sub-task 1, it returns the result of sub-task 1 to the server. The server updates the task status table of sub-task 1 through incremental persistence and then pushes the sub-result 1 event to the client. The client renders sub-result 1. At this time, if the user switches tabs on the client, the push channel remains silent. After the AI ​​generation server completes processing sub-task 2, it returns the result of sub-task 2 to the server. The server updates the task status table of sub-task 2 through incremental persistence and then pushes the sub-result 2 event to the client. The client renders sub-result 1. At this time, if the user switches tabs on the client, the push channel remains silent. If the client does not process the request, it determines that the current push channel network is interrupted, the push channel is disconnected, and the active flag of the push channel is set to null. At this time, if the polling cycle of the task status poller has arrived, it queries the server for the task status. The server reads the task status table, and the task status table returns structured data results (containing 3 sub-results). The server returns the task table to the task status poller, detects changes in the length of the sub-result array through shallow comparison, triggers a client view update, and renders sub-result 1 + sub-result 2 + sub-result 3. After the AI ​​generation server completes processing sub-task 4, it returns the completion result of sub-task 4 to the server. The server calls the task success interface, sets the status to success, and the task status poller queries the server for the task status. The server returns a message indicating that the task status is successful, and the status poller displays a complete completion message to the client.

[0115] For further details, please refer to Figure 5 , Figure 5 The polling frequency decision flowchart provided in the embodiments of this disclosure is as follows: Figure 5As shown, after the polling trigger point triggers polling, it checks whether the current consecutive error count is greater than 0. If it is not greater than 0, it determines whether there is a running task in the target task table. If there is no running task, it determines whether the target task table is not empty. If the target task table is not empty, it queries according to the second polling frequency (15 seconds) and sets the next polling timer. If the target task table is empty, it sets the polling interval to 0 and waits for external wake-up or a new task to be initiated. If there is a running task, it queries according to the first polling frequency (5 seconds) and sets the next polling timer. If the consecutive error count is greater than 0, it determines the third polling frequency based on the consecutive error count, the first polling frequency, and the preset backoff upper limit value, queries, and sets the next polling timer. It checks whether the view corresponding to the current target task is visible. If it is not visible, it pauses the timer and queries immediately when switching back during the visibility change event. If it is visible, it executes the task status query process.

[0116] The batch task processing method provided in this disclosure sends a batch task request to a server to process the target batch task, and establishes a push event channel with the server and an activity flag corresponding to the push event channel; it receives the sub-task processing results issued by the server through the push event channel; it controls a poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch task; if there is a difference between the task processing data and historical task data, it checks whether there is an anomaly in the push event channel based on the activity flag; if there is no anomaly in the push event channel, it updates the client's view based on the sub-task processing results; if there is an anomaly in the push event channel, it updates the client's view based on the task processing data. In this way, the server determines in real time that the sub-task processing results are persistently written to the database, and the client can obtain the sub-task processing results by querying the database through the poller. At the same time, the push event channel and the poller polling channel work together to ensure that the client can accurately obtain the sub-task processing results even in the event of network anomalies, which helps to improve the accuracy and efficiency of batch task processing.

[0117] Based on the same inventive concept, this disclosure also provides a batch task processing device corresponding to the batch task processing method. Since the principle of the device in this disclosure for solving the problem is similar to the batch task processing method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0118] Please see Figure 6 , Figure 7 , Figure 6 This is one of the structural schematic diagrams of a batch task processing device provided in an embodiment of this disclosure. Figure 7 This is a second schematic diagram of a batch task processing apparatus provided in an embodiment of this disclosure. Figure 6 As shown, the processing device 600 includes: The batch task request sending module 610 is used to send a batch task request for processing target batch tasks to the server, and to establish a push event channel with the server and an active flag corresponding to the push event channel. The first subtask processing result sending module 620 is used to receive the subtask processing result sent by the server through the push event channel; wherein, the subtask processing result is sent by the server after it has persistently written the subtask processing result into the database; The polling query module 630 is used to control the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch task; The view update module 640 is used to detect whether there is an anomaly in the push event channel based on the active marker when there is a difference between the task processing data and the historical task data; if there is no anomaly in the push event channel, update the view of the client based on the subtask processing result; if there is an anomaly in the push event channel, update the view of the client based on the task processing data.

[0119] In an optional implementation, the view update module 640 is used to determine a difference between the task processing data and historical task data through the following steps: The task processing data and historical task data are compared one by one using a shallow comparison function. If the data in at least one target field of the task processing data is different from that in the historical task data, it is determined that there is a difference between the task processing data and the historical task data.

[0120] In an optional implementation, when the view update module 640 is used to detect whether there is an anomaly in the push event channel based on the active marker, and if there is no anomaly in the push event channel, update the client's view based on the subtask processing result; and if there is an anomaly in the push event channel, update the client's view based on the task processing data, the view update module 640 is used to: If the activity flag is a valid value, it is determined that there is no anomaly in the push event channel, and the view of the client is updated based on the subtask processing result; If the activity flag is empty, it is determined that there is an anomaly in the push event channel, and the view of the client is updated based on the task processing data.

[0121] In an optional implementation, when the polling module 630 controls the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks, the polling module 630 is used to: The poller queries the target task table in the database corresponding to the target batch task. If there are tasks in the target task table that are running, control the poller to query the task processing data in the database according to the first polling frequency; If the target task table contains at least one task but all at least one task is in a non-running state where no task is running, control the poller to query the task processing data in the database according to the second polling frequency; The first polling frequency is greater than the second polling frequency.

[0122] In an optional implementation, the polling module 630 is further configured to: If the target task table is empty, stop the poller's polling operation.

[0123] In an optional implementation, the polling module 630 is further configured to: If the poller has a network anomaly, the third polling frequency is determined according to the number of consecutive errors, the first polling frequency, and the preset backoff upper limit. The poller is controlled to query the database for task processing data at a third polling frequency.

[0124] In an optional embodiment, the processing device 600 further includes a timeout detection module (not shown in the figure), the timeout detection module being used for: Determine the start time of at least one target task that is in a running state in the task processing data; Based on the task type of the target batch tasks, determine the target timeout threshold; For each target task, if the time difference between the current time and the start time of the target task is greater than the target timeout threshold, a timeout flag is added to the target task.

[0125] In an optional implementation, when the polling module 630 controls the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks, the polling module 630 is used to: For each polling query request, check if there are any historical query requests that have not returned results. If there are no historical query requests that did not return results, query the task processing data in the database; If there are historical query requests that have not returned results, skip the current polling query request until the historical query request is completed.

[0126] In an optional embodiment, the processing device 600 further includes an item detection module (not shown in the figure), the item detection module being used for: Check whether the project identifier corresponding to the task processing data is consistent with the project identifier of the current client; If the project identifier corresponding to the task processing data is inconsistent with the project identifier of the current client, the task processing data is discarded.

[0127] In an optional embodiment, the processing device 600 further includes a view visibility state polling control module (not shown in the figure), which is used to: In response to the view corresponding to the target batch task becoming invisible, the poller's polling operation is stopped. In response to the view corresponding to the target batch task becoming visible, the poller resumes its polling operation and queries the database for task processing data according to the polling frequency corresponding to the task status of the target batch task.

[0128] Furthermore, such as Figure 7 As shown, the processing device 700 further includes: The subtask processing result persistence module 650 is used to receive a batch task request sent by the client to process the target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task. The second subtask processing result distribution module 660 is used to distribute the subtask processing result to the client through a push event channel in response to the end of the subtask processing result persistence. This allows the client to determine the abnormality of the push event channel based on its activity identifier. If the push event channel is determined to be normal, the client updates its view based on the subtask processing result. If the push event channel is abnormal, the client updates its view based on the task processing data obtained from the database by the poller.

[0129] In one optional implementation, the target task table includes at least a structured result data field; the structured result data field is used to record the processing results of completed subtasks; when the subtask processing result persistence module 650 persists the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task, the subtask processing result persistence module 650 is used to: The subtask processing result corresponding to the subtask is appended to the sub-result array corresponding to the structured result data field.

[0130] In an optional implementation, the subtask processing result persistence module 650 is further configured to: Call the forced dirty marking function to submit a database update event to indicate the database update and complete the persistence of the subtask processing results.

[0131] The batch task processing method and apparatus provided in this embodiment sends a batch task request to a server to process a target batch task, and establishes a push event channel with the server and an activity flag corresponding to the push event channel; receives sub-task processing results from the server through the push event channel; controls a poller to query task processing data in the database according to the polling frequency corresponding to the task status of the target batch task; if there is a difference between the task processing data and historical task data, it checks whether there is an anomaly in the push event channel based on the activity flag; if there is no anomaly in the push event channel, it updates the client's view based on the sub-task processing results; if there is an anomaly in the push event channel, it updates the client's view based on the task processing data. In this way, the server determines in real time that the sub-task processing results are persistently written to the database, and the client can obtain the sub-task processing results by querying the database through the poller. Simultaneously, the push event channel and the poller polling channel work together to ensure that the client can accurately obtain the sub-task processing results even in the event of network anomalies, which helps improve the accuracy and efficiency of batch task processing.

[0132] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a task processing system provided in an embodiment of this disclosure. Figure 8 As shown, the task processing system 800 includes a client 810 and a server 830.

[0133] The client 810 executes the following instructions during runtime: Send a batch task request to the server to process the target batch tasks, and establish a push event channel with the server and an active tag corresponding to the push event channel; The server sends the subtask processing result through the push event channel; wherein the subtask processing result is sent by the server after it has persisted the subtask processing result to the database. The poller is controlled to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks; If there is a difference between the task processing data and the historical task data, the system detects whether there is an anomaly in the push event channel based on the active flag. If there is no anomaly in the push event channel, the system updates the client's view based on the subtask processing result. If there is an anomaly in the push event channel, the system updates the client's view based on the task processing data.

[0134] In one optional implementation, the instructions executed by client 810 determine the difference between the task processing data and historical task data through the following steps: The task processing data and historical task data are compared one by one using a shallow comparison function. If the data in at least one target field of the task processing data is different from that in the historical task data, it is determined that there is a difference between the task processing data and the historical task data.

[0135] In one optional implementation, the instructions executed by client 810 include: detecting whether there is an anomaly in the push event channel based on the activity marker; if there is no anomaly in the push event channel, updating the client's view based on the subtask processing result; if there is an anomaly in the push event channel, updating the client's view based on the task processing data, including: If the activity flag is a valid value, it is determined that there is no anomaly in the push event channel, and the view of the client is updated based on the subtask processing result; If the activity flag is empty, it is determined that there is an anomaly in the push event channel, and the view of the client is updated based on the task processing data.

[0136] In one optional implementation, the instruction executed by the client 810 includes the control poller querying the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks, including: The poller queries the target task table in the database corresponding to the target batch task. If there are tasks in the target task table that are running, control the poller to query the task processing data in the database according to the first polling frequency; If the target task table contains at least one task but all at least one task is in a non-running state where no task is running, control the poller to query the task processing data in the database according to the second polling frequency; The first polling frequency is greater than the second polling frequency.

[0137] In an optional implementation, the instructions executed by the client 810 further include: If the target task table is empty, stop the poller's polling operation.

[0138] In an optional implementation, the instructions executed by the client 810 further include: If the poller has a network anomaly, the third polling frequency is determined according to the number of consecutive errors, the first polling frequency, and the preset backoff upper limit. The poller is controlled to query the database for task processing data at a third polling frequency.

[0139] In one optional implementation, the instructions executed by the client 810 further include: Determine the start time of at least one target task that is in a running state in the task processing data; Based on the task type of the target batch tasks, determine the target timeout threshold; For each target task, if the time difference between the current time and the start time of the target task is greater than the target timeout threshold, a timeout flag is added to the target task.

[0140] In one optional implementation, the instruction executed by the client 810 includes the control poller querying the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks, including: For each polling query request, check if there are any historical query requests that have not returned results. If there are no historical query requests that did not return results, query the task processing data in the database; If there are historical query requests that have not returned results, skip the current polling query request until the historical query request is completed.

[0141] In an optional implementation, the instructions executed by the client 810 further include: Check whether the project identifier corresponding to the task processing data is consistent with the project identifier of the current client; If the project identifier corresponding to the task processing data is inconsistent with the project identifier of the current client, the task processing data is discarded.

[0142] In an optional implementation, the instructions executed by the client 810 further include: In response to the view corresponding to the target batch task becoming invisible, the poller's polling operation is stopped. In response to the view corresponding to the target batch task becoming visible, the poller resumes its polling operation and queries the database for task processing data according to the polling frequency corresponding to the task status of the target batch task.

[0143] The server 820 executes the following instructions during operation: Receive a batch task request sent by the client to process a target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task; In response to the completion of the persistence of the subtask processing result, the subtask processing result is sent to the client through a push event channel. This allows the client to determine the abnormality of the push event channel based on its activity identifier. If the push event channel is determined to be free of abnormality, the client updates its view based on the subtask processing result. If the push event channel is abnormal, the client updates its view based on the task processing data obtained from the database by the poller.

[0144] In one optional implementation, the instructions executed by server 820 include at least a structured result data field in the target task table; the structured result data field is used to record the processing results of completed subtasks; persisting the subtask processing results corresponding to the subtasks to the target task table in the database corresponding to the target batch task includes: The subtask processing result corresponding to the subtask is appended to the sub-result array corresponding to the structured result data field.

[0145] In one optional implementation, the instructions executed by the server 820 further include: Call the forced dirty marking function to submit a database update event to indicate the database update and complete the persistence of the subtask processing results.

[0146] In this way, the server determines the subtask processing results in real time and persists them to the database. The client can retrieve the subtask processing results by querying the database through a poller. At the same time, the push event channel and the poller polling channel work together to ensure that the client can accurately obtain the subtask processing results even in the event of network anomalies, which helps to improve the accuracy and efficiency of batch task processing.

[0147] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the following instructions: Send a batch task request to the server to process the target batch tasks, and establish a push event channel with the server and an active tag corresponding to the push event channel; The server sends the subtask processing result through the push event channel; wherein the subtask processing result is sent by the server after it has persisted the subtask processing result to the database. The poller is controlled to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks; If there is a difference between the task processing data and the historical task data, the system detects whether there is an anomaly in the push event channel based on the active flag. If there is no anomaly in the push event channel, the system updates the client's view based on the subtask processing result. If there is an anomaly in the push event channel, the system updates the client's view based on the task processing data.

[0148] In one optional implementation, the instructions executable by the computer-readable storage medium determine a difference between the task processing data and historical task data through the following steps: The task processing data and historical task data are compared one by one using a shallow comparison function. If the data in at least one target field of the task processing data is different from that in the historical task data, it is determined that there is a difference between the task processing data and the historical task data.

[0149] In one optional implementation, the instructions executable by the computer-readable storage medium include: detecting whether there is an anomaly in the push event channel based on the active tag; updating the client's view based on the subtask processing result if there is no anomaly in the push event channel; and updating the client's view based on the task processing data if there is an anomaly in the push event channel. If the activity flag is a valid value, it is determined that there is no anomaly in the push event channel, and the view of the client is updated based on the subtask processing result; If the activity flag is empty, it is determined that there is an anomaly in the push event channel, and the view of the client is updated based on the task processing data.

[0150] In one optional implementation, the instructions executable by the computer-readable storage medium include a control poller querying task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks, comprising: The poller queries the target task table in the database corresponding to the target batch task. If there are tasks in the target task table that are running, control the poller to query the task processing data in the database according to the first polling frequency; If the target task table contains at least one task but all at least one task is in a non-running state where no task is running, control the poller to query the task processing data in the database according to the second polling frequency; The first polling frequency is greater than the second polling frequency.

[0151] In one optional implementation, the instructions executed by the computer-readable storage medium further include: If the target task table is empty, stop the poller's polling operation.

[0152] In one optional implementation, the instructions executed by the computer-readable storage medium further include: If the poller has a network anomaly, the third polling frequency is determined according to the number of consecutive errors, the first polling frequency, and the preset backoff upper limit. The poller is controlled to query the database for task processing data at a third polling frequency.

[0153] In one optional implementation, the instructions executed by the computer-readable storage medium further include: Determine the start time of at least one target task that is in a running state in the task processing data; Based on the task type of the target batch tasks, determine the target timeout threshold; For each target task, if the time difference between the current time and the start time of the target task is greater than the target timeout threshold, a timeout flag is added to the target task.

[0154] In one optional implementation, the instructions executable by the computer-readable storage medium include a control poller querying task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks, comprising: For each polling query request, check if there are any historical query requests that have not returned results. If there are no historical query requests that did not return results, query the task processing data in the database; If there are historical query requests that have not returned results, skip the current polling query request until the historical query request is completed.

[0155] In one optional implementation, the instructions executed by the computer-readable storage medium further include: Check whether the project identifier corresponding to the task processing data is consistent with the project identifier of the current client; If the project identifier corresponding to the task processing data is inconsistent with the project identifier of the current client, the task processing data is discarded.

[0156] In one optional implementation, the instructions executed by the computer-readable storage medium further include: In response to the view corresponding to the target batch task becoming invisible, the poller's polling operation is stopped. In response to the view corresponding to the target batch task becoming visible, the poller resumes its polling operation and queries the database for task processing data according to the polling frequency corresponding to the task status of the target batch task.

[0157] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the following instructions: Receive a batch task request sent by the client to process a target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task; In response to the completion of the persistence of the subtask processing result, the subtask processing result is sent to the client through a push event channel. This allows the client to determine the abnormality of the push event channel based on its activity identifier. If the push event channel is determined to be free of abnormality, the client updates its view based on the subtask processing result. If the push event channel is abnormal, the client updates its view based on the task processing data obtained from the database by the poller.

[0158] In one optional implementation, the instructions executable by the computer-readable storage medium include at least a structured result data field in the target task table; the structured result data field is used to record the processing results of completed subtasks; persisting the subtask processing results corresponding to the subtasks to the target task table in the database corresponding to the target batch task includes: The subtask processing result corresponding to the subtask is appended to the sub-result array corresponding to the structured result data field.

[0159] In one optional implementation, the instructions executed by the computer-readable storage medium further include: Call the forced dirty marking function to submit a database update event to indicate the database update and complete the persistence of the subtask processing results.

[0160] In this way, the server determines the subtask processing results in real time and persists them to the database. The client can retrieve the subtask processing results by querying the database through a poller. At the same time, the push event channel and the poller polling channel work together to ensure that the client can accurately obtain the subtask processing results even in the event of network anomalies, which helps to improve the accuracy and efficiency of batch task processing.

[0161] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0162] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0163] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0165] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for processing batch tasks, characterized in that, Applied to the client side; the processing method includes: Send a batch task request to the server to process the target batch tasks, and establish a push event channel with the server and an active tag corresponding to the push event channel; The server sends the subtask processing result through the push event channel; wherein the subtask processing result is sent by the server after it has persisted the subtask processing result to the database. The poller is controlled to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks; If there is a difference between the task processing data and the historical task data, the system detects whether there is an anomaly in the push event channel based on the active flag. If there is no anomaly in the push event channel, the system updates the client's view based on the subtask processing result. If there is an anomaly in the push event channel, the system updates the client's view based on the task processing data.

2. The processing method according to claim 1, characterized in that, The following steps are used to determine the difference between the task processing data and historical task data: The task processing data and historical task data are compared one by one using a shallow comparison function. If the data in at least one target field of the task processing data is different from that in the historical task data, it is determined that there is a difference between the task processing data and the historical task data.

3. The processing method according to claim 1, characterized in that, The system detects whether there is an anomaly in the push event channel based on the active flag. If there is no anomaly in the push event channel, the view of the client is updated based on the subtask processing result. If the push event channel is abnormal, the client's view is updated based on the task processing data, including: If the activity flag is a valid value, it is determined that there is no anomaly in the push event channel, and the view of the client is updated based on the subtask processing result; If the activity flag is empty, it is determined that there is an anomaly in the push event channel, and the view of the client is updated based on the task processing data.

4. The processing method according to claim 1, characterized in that, The control poller queries the database for task processing data according to the polling frequency corresponding to the task status of the target batch tasks, including: The poller queries the target task table in the database corresponding to the target batch task. If there are tasks in the target task table that are running, control the poller to query the task processing data in the database according to the first polling frequency; If there is at least one task in the target task table and all at least one task is in a non-running state, control the poller to query the task processing data in the database according to the second polling frequency; The first polling frequency is greater than the second polling frequency.

5. The processing method according to claim 4, characterized in that, The processing method further includes: If the target task table is empty, stop the poller's polling operation.

6. The processing method according to claim 4, characterized in that, The processing method further includes: If the poller has a network anomaly, the third polling frequency is determined according to the number of consecutive errors, the first polling frequency, and the preset backoff upper limit. The poller is controlled to query the database for task processing data at a third polling frequency.

7. The processing method according to claim 1, characterized in that, The processing method further includes: Determine the start time of at least one target task that is in a running state in the task processing data; Based on the task type of the target batch tasks, determine the target timeout threshold; For each target task, if the time difference between the current time and the start time of the target task is greater than the target timeout threshold, a timeout flag is added to the target task.

8. The processing method according to claim 1, characterized in that, The control poller queries the database for task processing data according to the polling frequency corresponding to the task status of the target batch tasks, including: For each polling query request, check if there are any historical query requests that have not returned results. If there are no historical query requests that did not return results, query the task processing data in the database; If there are historical query requests that have not returned results, skip the current polling query request until the historical query request is completed.

9. The processing method according to claim 1, characterized in that, After the control poller queries the task processing data in the database according to the polling frequency corresponding to the task status of the target batch tasks, the processing method further includes: Check whether the project identifier corresponding to the task processing data is consistent with the project identifier of the current client; If the project identifier corresponding to the task processing data is inconsistent with the project identifier of the current client, the task processing data is discarded.

10. The processing method according to claim 1, characterized in that, The processing method further includes: In response to the view corresponding to the target batch task becoming invisible, the poller's polling operation is stopped. In response to the view corresponding to the target batch task becoming visible, the poller resumes its polling operation and queries the database for task processing data according to the polling frequency corresponding to the task status of the target batch task.

11. A method for processing batch tasks, characterized in that, Applied to a server; the processing method includes: Receive a batch task request sent by the client to process a target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task; In response to the completion of the persistence of the subtask processing result, the subtask processing result is sent to the client through a push event channel. This allows the client to determine the abnormality of the push event channel based on its activity identifier. If the push event channel is determined to be free of abnormality, the client updates its view based on the subtask processing result. If the push event channel is abnormal, the client updates its view based on the task processing data obtained from the database by the poller.

12. The processing method according to claim 11, characterized in that, The target task table includes at least a structured result data field; the structured result data field is used to record the processing results of completed subtasks; The step of persisting the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task includes: The subtask processing result corresponding to the subtask is appended to the sub-result array corresponding to the structured result data field.

13. The processing method according to claim 11, characterized in that, After persisting the subtask processing result corresponding to the subtask to the target task table corresponding to the target batch task in the database, the processing method further includes: Call the forced dirty marking function to submit a database update event to indicate the database update and complete the persistence of the subtask processing results.

14. A batch task processing apparatus, characterized in that, Applied to the client; the processing device includes: The batch task request sending module is used to send batch task requests for processing target batch tasks to the server, and to establish a push event channel with the server and an active flag corresponding to the push event channel. The first subtask processing result distribution module is used to receive the subtask processing result distributed by the server through the push event channel; wherein, the subtask processing result is distributed by the server after persistently writing the subtask processing result into the database; The polling query module is used to control the poller to query the task processing data in the database according to the polling frequency corresponding to the task status of the target batch task; The view update module is used to detect whether there is an anomaly in the push event channel based on the active marker if there is a difference between the task processing data and the historical task data. If there is no anomaly in the push event channel, the view of the client is updated based on the subtask processing result; if there is an anomaly in the push event channel, the view of the client is updated based on the task processing data.

15. A batch task processing apparatus, characterized in that, Applied to a server; the processing device includes: The subtask processing result persistence module is used to receive a batch task request sent by the client to process the target batch task, and after each subtask in the target batch task is completed, persist the subtask processing result corresponding to the subtask to the target task table in the database corresponding to the target batch task; The second subtask processing result distribution module is used to distribute the subtask processing result to the client through a push event channel in response to the end of the subtask processing result persistence. This allows the client to determine the abnormality of the push event channel based on its activity identifier. If the push event channel is determined to be free of abnormality, the client updates its view based on the subtask processing result. If the push event channel is abnormal, the client updates its view based on the task processing data obtained from the database by a poller.

Images