Task processing method and device, equipment, medium and program product

Abstract

The invention provides a task processing method and device, equipment, a storage medium and a program product, and can be applied to the technical field of distributed computing. The method comprises the steps that in response to a processing request of a first target task, task information of the first target task and the node state of each task processing node in a task management server are obtained, and the node state comprises at least one of the real-time processing task number, the load state and the historical task processing success rate of the task processing nodes; based on the task information, determining a target resource required for executing the first target task; in response to the fault state of the first task processing node, determining at least one second task processing node from each task processing node to process the first target task based on the node state and the target resource; wherein the first task processing node is a node, which has processed the first target task, in each task processing node.

Description

Technical Field

[0001] This application relates to the field of distributed computing technology, and can be applied to the field of financial technology. More specifically, it relates to a task processing method, apparatus, device, medium, and program product. Background Technology

[0002] In distributed computing scenarios, task management servers need to dynamically allocate massive amounts of tasks to multiple task processing nodes (such as virtual machines, containers, or physical servers) to achieve efficient resource utilization and rapid task execution.

[0003] Existing task scheduling methods typically rely on pre-allocating fixed resources based on task type. On the one hand, task characteristics are not static. As business logic progresses, some tasks may require more resources than pre-allocated due to factors such as a sudden increase in data volume or computational complexity. On the other hand, resource status is also dynamic. Task processing nodes may experience a reduction in available resources due to the simultaneous running of multiple tasks or unforeseen system events.

[0004] Therefore, the task processing methods in related technologies are difficult to adapt to the dynamic changes in tasks and resources in distributed computing scenarios. Summary of the Invention

[0005] In view of the above problems, embodiments of this application provide a task processing method, apparatus, device, medium, and program product.

[0006] According to a first aspect of this application, a task processing method is provided, applied to a task management server. The method includes: in response to a processing request for a first target task, obtaining task information of the first target task and the node status of each task processing node in the task management server, wherein the node status includes at least one of the following: the number of tasks processed in real time, load status, and historical task processing success rate of the task processing node; determining the target resources required to execute the first target task based on the task information; and in response to a first task processing node being in a fault state, determining at least one second task processing node from among the task processing nodes to process the first target task based on the node status and the target resources; wherein the first task processing node is a node among the task processing nodes that has processed the first target task.

[0007] According to an embodiment of this application, the method further includes: during the process of the second task processing node processing the first target task, in response to the interruption of the processing of the first target task, updating the node status of other task processing nodes in the task management server; based on the updated node status of other task processing nodes, determining a third task processing node from the other task processing nodes, and migrating the first target task to the third task processing node.

[0008] According to an embodiment of this application, determining at least one second task processing node from among the various task processing nodes to process the first target task based on node status and target resources includes: determining the idle resources of each task processing node according to the node status; if the idle resources of any task processing node satisfy the target resources, calling any task processing node to process the first target task; if the idle resources of any task processing node do not satisfy the target resources, calling at least two task processing nodes to process the first target task.

[0009] According to an embodiment of this application, calling at least two task processing nodes to process a first target task includes: splitting the first target task into at least two sub-target tasks according to task information; allocating a corresponding task processing node to each sub-target task according to the idle resources of each task processing node; and establishing a communication link between each task processing node processing the sub-target tasks, the communication link being used to synchronize the processing progress of each sub-target task.

[0010] According to an embodiment of this application, determining at least one second task processing node from among the various task processing nodes to process the first target task based on node status and target resources further includes: determining the priority of each task processing node in processing the first target task based on node status; wherein the priority decreases as the number of real-time processing tasks increases, the priority decreases as the load represented by the load status increases, and the priority increases as the historical task processing success rate increases; when multiple task processing nodes meet the conditions for processing the first target task, determining at least one second task processing node from among the multiple task processing nodes to process the first target task based on the priority.

[0011] According to an embodiment of this application, after determining the target resources required to execute the first target task based on task information, the method further includes: determining whether the first task processing node is in a target state based on the target resources, wherein the target state indicates that the task processing node is configured with the target resources required to execute the first target task; in response to the first task processing node being in a target state, directly calling the first task processing node to process the first target task; in response to the first task processing node not being in a target state, determining that the first task processing node is in a fault state.

[0012] According to embodiments of this application, based on task information, the target resources required to execute the first target task are determined, including at least one of the following: if the task information indicates that the first target task is a data processing type task, the node computing power required to execute the first target task is determined; if the task information indicates that the first target task is a data storage type task, the node storage space required to execute the first target task is determined; if the task information indicates that the first target task is a data transmission type task, the node network bandwidth required to execute the first target task is determined.

[0013] According to a second aspect of this application, a task processing apparatus is provided, comprising: a status acquisition module, configured to acquire task information of the first target task and node status of each task processing node in a task management server in response to a processing request of a first target task; the node status includes at least one of the following: the number of tasks processed in real time, load status, and historical task processing success rate of the task processing node; a resource determination module, configured to determine the target resources required to execute the first target task based on the task information; and a node invocation module, configured to determine at least one second task processing node from among the task processing nodes to process the first target task in response to a first task processing node being in a fault state, based on the node status and the target resources; wherein the first task processing node is a node among the task processing nodes that has processed the first target task.

[0014] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0015] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0016] According to a fifth aspect of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0017] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0018] Figure 1 The illustrations depict application scenarios of task processing methods, apparatuses, devices, media, and program products according to embodiments of this application.

[0019] Figure 2This illustration schematically shows one of the flowcharts of a task processing method according to an embodiment of this application;

[0020] Figure 3 A second flowchart illustrating a task processing method according to an embodiment of this application is shown schematically;

[0021] Figure 4 A flowchart of a task processing method according to an embodiment of this application is illustrated schematically;

[0022] Figure 5 This schematic diagram illustrates a structural block diagram of a task processing apparatus according to an embodiment of the present application;

[0023] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a task processing method according to an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0028] Figure 1An exemplary distributed system architecture 100 is schematically illustrated and can be applied to a task processing method according to an embodiment of this application. It should be noted that... Figure 1 The examples shown are merely examples of the architecture that can be applied to the methods according to the embodiments of this application, in order to help those skilled in the art understand the technical content of this application, but do not imply any limitation on the application scenarios of the embodiments of this application.

[0029] like Figure 1 As shown, the distributed system architecture 100 according to an embodiment of this application may include multiple nodes N1 to N9 and a task management server 110.

[0030] For example, nodes N1 to N9 can be computers or software modules, applications, or applications running on computers. Nodes N1 to N9 can communicate and connect in various ways, such as, but not limited to, wired connections, wireless connections, signaling connections, and connections via the Internet. Multiple nodes N1 to N9 can be used to implement one or more services, such as, but not limited to, financial services, and each service can correspond to a transaction. Assume service A is accessed from node N1 and processed sequentially through nodes N4 and N7. Node N7 returns the processing result to node N1 via node N4. Thus, the service link for service A is N1→N4→N7. In the service link for service A, N1 is the upstream application, and N4 and N7 are downstream applications. In a similar manner, each service is processed through corresponding nodes, and these nodes form corresponding service links. For example, the service link for service B is N2→N5→N7, the service link for service C is N3→N6→N7, the service link for service D is N2→N5→N8, and the service link for service E is N3→N6→N9.

[0031] Task management server 110 is used to configure and manage each node N1 to N9. Task management server 110 may include one or more computers. Task management server 110 can communicate with each node N1 to N9 in various ways, including but not limited to wired connection, wireless connection, signaling connection, and Internet connection.

[0032] The method according to the embodiments of this application can be executed by a task management server 110. Accordingly, the apparatus according to one aspect of the embodiments of this application can be disposed in the task management server 110. Optionally, the method according to the embodiments of this application can be executed by any one of a plurality of nodes. Accordingly, the apparatus according to the embodiments of this application can be disposed in any one of the nodes.

[0033] In some embodiments, the method according to the present application can be executed by a computer or computer cluster that is different from the task management server 110 and is capable of communicating with nodes N1 to N9 and / or the task management server 110. Accordingly, the apparatus provided in the present application can also be located in a computer or computer cluster that is different from the task management server 110 and is capable of communicating with nodes N1 to N9 and / or the task management server 110.

[0034] It should be understood that Figure 1 The number of nodes and task management servers shown is merely illustrative. Depending on implementation needs, any number of nodes and task management servers can be used.

[0035] Figure 2 The flowchart illustrating one of the task processing methods according to an embodiment of this application is shown schematically.

[0036] like Figure 2 As shown, a task processing method 200 according to an embodiment of this application may include steps S210 to S230.

[0037] In step S210, in response to the processing request of the first target task, the task information of the first target task and the node status of each task processing node in the task management server are obtained. The node status includes at least one of the following: the number of tasks processed in real time, the load status, and the historical task processing success rate of the task processing node.

[0038] In the embodiments of this application, the task management server is used to manage multiple task processing nodes in a distributed system and perform task allocation and scheduling operations.

[0039] In the embodiments of this application, the processing request for the target task may be initiated by an external system or a user.

[0040] In embodiments of this application, the task information may be various descriptive data about the first target task, such as task type, task size, and task importance.

[0041] In the embodiments of this application, the node status is a set of information reflecting the current working status of the task processing node, including at least one of the following: the number of tasks processed in real time, the load status, and the historical task processing success rate. The number of tasks processed in real time is the number of tasks currently being processed simultaneously by the task processing node, indicating the current workload of the node. The load status describes the resource utilization of the task processing node, such as processor utilization and memory utilization, reflecting the burden on the node in processing tasks. The historical task processing success rate is the proportion of tasks successfully processed by the task processing node in the past, serving as an indicator of the node's task processing capability and reliability.

[0042] In step S220, based on the task information, the target resources required to execute the first target task are determined.

[0043] In step S230, in response to the first task processing node being in a fault state, based on the node state and target resources, at least one second task processing node is determined from each task processing node to process the first target task; wherein, the first task processing node is the node among the task processing nodes that has processed the first target task.

[0044] Specifically, in a typical distributed system, some task processing nodes have previously processed or have processed specific types of tasks. When the task management server receives a processing request for the first target task, it determines the node resources required to execute the first target task by using the task information carried in the request. However, the task management server does not directly select the node to execute the first target task from among the various task processing nodes; instead, it prioritizes selecting the first task processing node that has previously processed the first target task or has processed tasks very similar to it. Once the first task processing node is determined to be capable of executing the first target task, it is no longer necessary to select the node from among the various task processing nodes, thus improving the speed of task allocation and enabling the task management server to respond to task processing requests as quickly as possible.

[0045] Furthermore, when the first task processing node is in a faulty state or cannot execute the first target task, it is necessary to determine which task processing nodes can be used to execute the first target task based on the node status of other task processing nodes besides the first task processing node and the target resources required to execute the first target task. In determining which task processing nodes can be used to execute the first target task, there may be a situation where multiple task processing nodes can be used to execute the first target task.

[0046] By employing the above method, task information for the first target task and the real-time status of each task processing node are obtained, enabling dynamic task allocation based on actual task requirements and the current status of nodes. When the first task processing node that has processed the task fails, a replacement second task processing node can be quickly determined from other nodes to ensure continuous task execution; this improves the ability to cope with unexpected situations such as node failures and the reliability of task processing in distributed computing scenarios.

[0047] Figure 3 The flowchart illustrating a task processing method according to an embodiment of this application is shown in the illustration.

[0048] like Figure 3As shown, a task processing method 300 according to an embodiment of this application may include steps S310 to S320.

[0049] In step S310, during the process of the second task processing node processing the first target task, in response to the interruption of the processing of the first target task, the node status of other task processing nodes in the task management server is updated.

[0050] In step S320, based on the updated node status of other task processing nodes, a third task processing node is determined from the other task processing nodes, and the first target task is migrated to the third task processing node.

[0051] Specifically, if the second task processing node stops executing the first target task due to some reason, such as insufficient resources or program error, it is necessary to call other task processing nodes to continue processing the target task. Therefore, it is necessary to re-acquire the node status of other task processing nodes. After the node status of other tasks is updated, the third task processing node to execute the first target task is re-determined based on the updated node status.

[0052] In some embodiments, when the processing of the first target task is interrupted, the task processing progress is guaranteed to a redundant or backup node in the distributed system. After the third task processing node is determined, the task processing progress of the first target task is migrated to the third task processing node.

[0053] For example, in an enterprise-level distributed application system, a task management server coordinates various business tasks, with multiple distributed application servers serving as task processing nodes. For instance, application server X is currently executing a critical business process task (the first objective task), such as integrating order processing and shipping processes. Simultaneously, application server Y is executing a customer data update task (the second objective task). During execution, a sudden change in the data format from an external data provider causes application server Y's customer data update task to be interrupted. Upon detecting this interruption in real-time, the task management server immediately updates the node status of other application servers (such as application server Z, application server W, etc.). For example, application server Z, which originally processed three tasks in real-time, now processes only two tasks due to application server Y's task interruption releasing some system resources, thus reducing the system resource load. Based on the updated node status, the task management server, after comprehensive evaluation, determines application server Z as the third task processing node and migrates the order processing and shipping process integration task from application server X to application server Z for continued processing.

[0054] By employing the above method, the resource and task scheduling mechanisms during task processing are optimized. If the processing of the second target task is interrupted while the second task processing node is handling the first target task, the status of other task processing nodes is updated promptly, reflecting the reallocation of system resources and changes in task load in real time. Based on the updated status, a third task processing node is determined and the first target task is migrated, enabling the task management server to allocate tasks in complex task execution environments.

[0055] According to an embodiment of this application, in operation S230, based on the node status and target resources, at least one second task processing node is determined from each task processing node to process the first target task, including the following steps: determining the idle resources of each task processing node according to the node status; if the idle resources of any task processing node meet the target resources, calling any task processing node to process the first target task; if the idle resources of any task processing node do not meet the target resources, calling at least two task processing nodes to process the first target task.

[0056] In the embodiments of this application, idle resources are resources in the task processing node that are not currently in use and can be used to execute new tasks, such as idle processor cores, unused memory space, remaining storage space, or network bandwidth.

[0057] Specifically, when a task processing node in the distributed cluster has sufficient idle resources to execute the first target task, that node is prioritized for execution, and the search for another task processing node to execute the first target task is stopped. If none of the task processing nodes in the distributed cluster can handle the first target task independently, multiple task processing nodes are invoked to execute the first target task synchronously.

[0058] For example, in a large-scale data processing cluster, a task management server is responsible for distributing massive data processing tasks to multiple data processing nodes. These nodes can be of different functional types, such as high-performance computing nodes and distributed storage nodes. For instance, when a large-scale log data analysis task is submitted to the task management server, the server obtains the node status of each data processing node and determines its available resources accordingly. Computing node A has 10 idle processor cores, 32GB of idle memory, and 1TB of idle storage capacity; storage node B has 5 idle data transmission channels for data reading and writing. If, based on log data, the target resources required for the task are determined to be 8 processor cores, 20GB of memory, and 500GB of storage capacity, and the idle resources of computing node A fully meet these requirements, then the server directly calls computing node A to process the task. If the target resources required for the task are 15 processor cores, 50GB of memory, and 2TB of storage capacity, the idle resources of a single node cannot meet these requirements. In this case, after resource planning, the server calls on computing node A and storage node B to jointly process the task, utilizing the computing resources of computing node A and the storage and data transmission resources of storage node B to collaboratively complete the task.

[0059] By adopting the above method, task processing nodes are flexibly selected based on the availability of node resources, effectively solving the problem of mismatch between task resource requirements and node resources. Whether a single node can meet the demand or multiple nodes need to collaborate, tasks can be allocated reasonably, adapting to dynamic changes in task resource requirements and improving the resource utilization rate and parallel processing capability of nodes in the distributed system.

[0060] According to an embodiment of this application, calling at least two task processing nodes to process a first target task includes: splitting the first target task into at least two sub-target tasks according to task information; allocating a corresponding task processing node to each sub-target task according to the idle resources of each task processing node; and establishing a communication link between each task processing node processing the sub-target tasks, the communication link being used to synchronize the processing progress of each sub-target task.

[0061] In the embodiments of this application, the sub-target task is obtained by splitting the first target task into multiple smaller task units. Each sub-target task is processed relatively independently, and together they constitute the complete processing flow of the first target task.

[0062] In the embodiments of this application, the communication link is used as a connection channel established between various task processing nodes that process sub-target tasks, for data transmission and information interaction between nodes, so as to synchronize the processing progress of each sub-target task and ensure the consistency and coordination of task processing.

[0063] The following describes the specific method of splitting the target task into multiple sub-target tasks in the embodiments of this application.

[0064] For example, image recognition tasks can be divided into sub-tasks such as image preprocessing (e.g., grayscale conversion, noise reduction), feature extraction, and classification. Each stage has different requirements for resources and algorithms. Assigning tasks to appropriate processing nodes can improve overall processing efficiency. For instance, image preprocessing may have relatively low computational resource requirements and can be assigned to nodes with slightly lower performance; while feature extraction and classification require more powerful computing capabilities and should be assigned to high-performance nodes.

[0065] For example, data storage tasks can be broken down by data type or data time range. For instance, breaking down by data type: The target task requires storing a multimedia database containing different types of data such as images, videos, and audio. Image storage, video storage, and audio storage can be treated as separate sub-tasks. Different types of data have different storage requirements; for example, video data may require more storage space and higher data transmission bandwidth, while image data has specific requirements for storage formats and compression algorithms. By breaking down by type, storage node resources can be allocated according to the characteristics of each sub-task. For instance, breaking down by data time range: For a target task storing historical business data, if the data volume is large, it can be broken down by time intervals. For example, business data from the past 10 years can be divided by year or quarter, with the data storage for each time period treated as a sub-task.

[0066] For example, in the compilation task of a software development project, there are steps such as compiling source files, linking library files, and generating executable files, and these steps have sequential dependencies. These steps can be treated as sub-tasks, with the source file compilation task completed first, followed by the library file linking and executable file generation tasks. Each task processing node is processed sequentially according to the task order.

[0067] Figure 4 The flowchart of a processing method according to an embodiment of this application is shown schematically as third.

[0068] like Figure 4 As shown, in operation S230, based on the node status and the target resource, at least one second task processing node is determined from the various task processing nodes to process the first target task, and operations S410 to S420 are also included.

[0069] In step S410, based on the node status, the priority of each task processing node in processing the first target task is determined; wherein, the priority decreases as the number of real-time processing tasks increases, the priority decreases as the load represented by the load status increases, and the priority increases as the historical task processing success rate increases.

[0070] In the embodiments of this application, the priority of a task processing node is a quantitative representation of the priority of the task processing node in processing the first target task. Its value is affected by the number of tasks processed in real time, the load status, and the historical task processing success rate, and is used to determine which nodes are selected to prioritize processing the first target task when multiple task processing nodes meet the processing conditions.

[0071] In step S420, if multiple task processing nodes meet the conditions for processing the first target task, at least one second task processing node is determined from the multiple task processing nodes to process the first target task based on priority.

[0072] Specifically, the priority of each task processing node for handling the first target task is determined based on the node status. Factors such as the number of tasks processed in real time, load status, and historical task success rate are considered, enabling a comprehensive evaluation of the node's task processing capability and reliability. When multiple nodes meet the processing conditions, the second task processing node is determined based on priority, prioritizing the selection of the better node to handle the task. This helps improve task processing efficiency and success rate, while avoiding inefficient or failed tasks due to random or unreasonable selection.

[0073] For example, after a user submits a business system upgrade task as the first target task, the task management server obtains the node status of each task processing node and calculates the priority of each node in handling the task. For instance, task processing node M processes 10 tasks in real time, with an overall resource load of 80%, and a historical task success rate of 90% for similar critical business system upgrade tasks; task processing node N processes 4 tasks in real time, with an overall resource load of 50%, and a historical task success rate of 95%. According to the priority calculation rules (priority decreases as the number of tasks processed in real time increases, decreases as the load represented by the load status increases, and increases as the historical task success rate increases), task processing node N has a higher priority than task processing node M. If both task processing node N and task processing node M meet the conditions for handling the business system upgrade task, then task processing node N is determined as the second task processing node to handle the first target task.

[0074] According to the embodiments of this application, after operation S210, which determines the target resources required to execute the first target task based on task information, the task processing method further includes: determining whether the first task processing node is in a target state based on the target resources, wherein the target state indicates that the task processing node is configured with the target resources required to execute the first target task; in response to the first task processing node being in a target state, directly calling the first task processing node to process the first target task; and in response to the first task processing node not being in a target state, determining that the first task processing node is in a fault state.

[0075] Specifically, since the first task processing node has previously executed the first target task, when the task management server receives the processing request for the first target task again, it can compare the historical node status of the first task processing node when it previously processed the first target task with the current node status of the first task processing node, thereby quickly determining whether the first task processing node is in the target state. If the current node status of the first task processing node is similar to the historical node status, it is determined that the first task processing node is in the target state, and the first task processing node can be directly invoked to process the first target task.

[0076] Furthermore, when the first task processing node is not in the target state, it indicates that the resource level available for processing tasks is low and cannot meet the requirements of the first target task. Therefore, it is determined that the first task processing node is in a fault state, and the task management server can continue to execute operation S230.

[0077] For example, in a general distributed task execution framework, the task management server is responsible for scheduling various types of nodes, which may be servers with specific functions, edge computing devices, etc.

[0078] For example, the task management server receives a data acquisition and preprocessing task (the first target task). Based on the task information, it determines the target resources required to execute the task, including a specific sensor data acquisition interface, sufficient computing power for data preprocessing, and appropriate storage resources for temporary storage of the acquired data. The task management server checks the status of the edge computing device E (the first task processing node) responsible for this type of task to see if it is configured with the necessary target resources to execute the task, i.e., whether it has the corresponding sensor data acquisition interface, sufficient computing power, and storage resources. If the edge computing device E has the required target resources and is in the target state, the task management server directly calls upon the edge computing device E to process the task. If the sensor data acquisition interface of the edge computing device E malfunctions and lacks complete target resources, the task management server determines that the edge computing device E is in a fault state and then re-determines a node from other edge computing devices or server nodes capable of processing the task to execute the data acquisition and preprocessing task.

[0079] By employing the above method, after determining the target resources required to execute the first target task, the decision to directly invoke the node to process the task is made by checking whether the first task processing node is in the target state. If the node is in the target state, direct invocation reduces intermediate steps in task allocation, allows for rapid response to task requirements, enhances adaptability to changes in the task execution environment, and ensures that tasks are processed promptly and effectively under various circumstances.

[0080] According to an embodiment of this application, in operation S220, the target resources required to perform the first target task are determined based on task information, including at least one of the following situations.

[0081] Scenario 1: If the task information indicates that the first target task is a data processing type task, determine the node computing power required to execute the first target task.

[0082] Scenario 2: If the task information indicates that the first target task is a data storage type task, determine the node storage space required to execute the first target task.

[0083] Scenario 3: If the task information indicates that the first target task is a data transmission type task, determine the node network bandwidth required to execute the first target task.

[0084] Specifically, different types of tasks have different resource requirements. After clarifying the task type represented by the task information, the resources required to execute the task can be determined accordingly. For example, data processing tasks require node computing power, data storage tasks require node storage space, and data transmission tasks require node network bandwidth. This can determine the resource requirements of the target task and provide a basis for allocating tasks to task processing nodes.

[0085] Based on the above task processing method, embodiments of this application also provide a task processing apparatus. The following will be combined with... Figure 5 The device is described in detail.

[0086] Figure 5 A schematic block diagram of a task processing apparatus according to an embodiment of this application is shown.

[0087] like Figure 5 As shown, the task processing device 500 in this embodiment includes a status acquisition module 510, a resource determination module 520, and a node invocation module 530.

[0088] The status acquisition module 510 is used to respond to the processing request of the first target task, acquire the task information of the first target task, and the node status of each task processing node in the task management server. The node status includes at least one of the following: the number of tasks processed in real time, the load status, and the historical task processing success rate. In one embodiment, the status acquisition module 510 can be used to execute step S210 described above, which will not be repeated here.

[0089] The resource determination module 520 is used to determine the target resources required to execute the first target task based on the task information. In one embodiment, the resource determination module 520 can be used to execute the step S220 described above, which will not be repeated here.

[0090] The node invocation module 530 is used to, in response to a first task processing node being in a fault state, determine at least one second task processing node from among the various task processing nodes to process the first target task, based on the node state and the target resource; wherein, the first task processing node is a node among the various task processing nodes that has processed the first target task. In one embodiment, the node invocation module 530 can be used to execute step S230 described above, which will not be repeated here.

[0091] According to an embodiment of this application, the node invocation module 530 is further configured to, in response to the interruption of the processing of the first target task during the process of the second task processing node processing the first target task, update the node status of other task processing nodes in the task management server; based on the updated node status of other task processing nodes, determine a third task processing node from the other task processing nodes, and migrate the first target task to the third task processing node.

[0092] According to an embodiment of this application, the node invocation module 530 is further configured to determine the idle resources of each task processing node based on the node status; if the idle resources of any task processing node satisfy the target resource, invoke any task processing node to process the first target task; if the idle resources of any task processing node do not satisfy the target resource, invoke at least two task processing nodes to process the first target task.

[0093] According to an embodiment of this application, the node invocation module 530 is further configured to: divide the first target task into at least two sub-target tasks according to the task information; allocate a corresponding task processing node to each sub-target task according to the idle resources of each task processing node; and establish a communication link between each task processing node processing the sub-target task, wherein the communication link is used to synchronize the processing progress of each sub-target task.

[0094] According to an embodiment of this application, the node invocation module 530 is further configured to determine the priority of each task processing node in processing the first target task based on the node status; wherein, the priority decreases as the number of real-time processing tasks increases, the priority decreases as the load represented by the load status increases, and the priority increases as the historical task processing success rate increases; when multiple task processing nodes meet the conditions for processing the first target task, at least one second task processing node is determined from the multiple task processing nodes to process the first target task based on the priority.

[0095] According to an embodiment of this application, the status acquisition module 510 is further configured to determine whether the first task processing node is in a target state based on the target resources, wherein the target state indicates that the task processing node is configured with the target resources required to execute the first target task. The node invocation module 530 is further configured to directly invoke the first task processing node to process the first target task in response to the first task processing node being in the target state; the status acquisition module 510 is further configured to determine that the first task processing node is in a fault state in response to the first task processing node not being in the target state.

[0096] According to an embodiment of this application, the resource determination module 520 is further configured to determine the node computing power required to execute the first target task when the task information indicates that the first target task is a data processing type task; determine the node storage space required to execute the first target task when the task information indicates that the first target task is a data storage type task; and determine the node network bandwidth required to execute the first target task when the task information indicates that the first target task is a data transmission type task.

[0097] According to embodiments of this application, any multiple modules among the state acquisition module 510, resource determination module 520, and node invocation module 530 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the state acquisition module 510, resource determination module 520, and node invocation module 530 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays, programmable logic arrays, systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits, or other reasonable means of integrating or packaging circuits, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the state acquisition module 510, resource determination module 520, and node invocation module 530 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0098] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a task processing method according to an embodiment of this application.

[0099] like Figure 6 As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage portion 608 into a random access memory 603. The processor 601 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0100] Random access memory 603 stores various programs and data required for the operation of electronic device 600. Processor 601, read-only memory 602, and random access memory 603 are interconnected via bus 604. Processor 601 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 602 and / or random access memory 603. It should be noted that the programs may also be stored in one or more memories other than read-only memory 602 and random access memory 603. Processor 601 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0101] According to embodiments of this application, the electronic device 600 may further include an input / output interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card, such as a local area network card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0102] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0103] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 602 described above, and / or random access memory 603, and / or one or more memories other than read-only memory 602 and random access memory 603.

[0104] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0105] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0106] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0107] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A task processing method, characterized in that, Applied to a task management server, the method includes: In response to the processing request of the first target task, the task information of the first target task and the node status of each task processing node in the task management server are obtained. The node status includes at least one of the following: the number of tasks processed in real time, the load status, and the historical task processing success rate of the task processing node. Based on the task information, determine the target resources required to execute the first target task; In response to the first task processing node being in a fault state, based on the node state and the target resource, at least one second task processing node is determined from the respective task processing nodes to process the first target task; The first task processing node is the node among the various task processing nodes that has processed the first target task.

2. The method according to claim 1, characterized in that, The method further includes: During the process of the second task processing node processing the first target task, in response to the interruption of the processing of the first target task, the node status of other task processing nodes in the task management server is updated. Based on the updated node status of the other task processing nodes, a third task processing node is determined from the other task processing nodes, and the first target task is migrated to the third task processing node.

3. The method according to claim 1, characterized in that, Based on the node status and the target resource, determining at least one second task processing node from the various task processing nodes to process the first target task includes: Based on the node status, determine the idle resources of each task processing node; If the idle resources of any task processing node meet the target resources, the first target task is processed by the aforementioned task processing node. If the idle resources of any task processing node do not meet the target resource requirements, at least two task processing nodes shall be invoked to process the first target task.

4. The method according to claim 3, characterized in that, Invoke at least two task processing nodes to process the first target task, including: Based on the task information, the first target task is divided into at least two sub-target tasks; Based on the available resources of each task processing node, a corresponding task processing node is allocated to each sub-target task; and a communication link is established between each task processing node processing the sub-target task, the communication link being used to synchronize the processing progress of each sub-target task.

5. The method according to any one of claims 1 to 4, characterized in that, Based on the node status and the target resource, determining at least one second task processing node from among the task processing nodes to process the first target task further includes: Based on the node status, the priority of each task processing node in processing the first target task is determined; wherein, the priority decreases as the number of real-time processing tasks increases, the priority decreases as the load represented by the load status increases, and the priority increases as the historical task processing success rate increases. If multiple task processing nodes meet the conditions for processing the first target task, at least one second task processing node is determined from the multiple task processing nodes to process the first target task based on the priority.

6. The method according to claim 1, characterized in that, After determining the target resources required to execute the first target task based on the task information, the method further includes: Based on the target resources, it is determined whether the first task processing node is in a target state, wherein the target state indicates that the task processing node is configured with the target resources required to execute the first target task; In response to the first task processing node being in the target state, the first task processing node is directly invoked to process the first target task. In response to the first task processing node not being in the target state, it is determined that the first task processing node is in a fault state.

7. The method according to claim 1, characterized in that, Based on the task information, the target resources required to perform the first target task are determined, including at least one of the following: If the task information indicates that the first target task is a data processing type task, determine the node computing power required to execute the first target task; If the task information indicates that the first target task is a data storage type task, determine the node storage space required to execute the first target task; If the task information indicates that the first target task is a data transmission type task, determine the node network bandwidth required to execute the first target task.

8. A task processing device, characterized in that, The device includes: The status acquisition module is used to respond to the processing request of the first target task, acquire the task information of the first target task, and the node status of each task processing node in the task management server. The node status includes at least one of the following: the number of tasks processed in real time, the load status, and the historical task processing success rate of the task processing node. The resource determination module is used to determine the target resources required to execute the first target task based on the task information. The node invocation module is used to respond to a first task processing node being in a fault state, and based on the node state and the target resource, determine at least one second task processing node from the various task processing nodes to process the first target task; wherein, the first task processing node is a node among the various task processing nodes that has processed the first target task.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images