Task processing method and device

Abstract

According to the task processing method and device provided by the embodiment of the invention, the target sub-task is allocated to the target slave node corresponding to the distributed system based on the corresponding relationship between the sub-task of the to-be-processed task and the task model; the execution result of the target sub-task returned by the slave node is monitored through the master node, and the monitored execution result is transmitted to the corresponding dependent slave node through the master node based on the model dependency relationship between the task models of the slave nodes, so that asynchronous non-blocking scheduling of the master node on a large number of tasks can be realized; therefore, a plurality of tasks can be processed by efficiently utilizing a plurality of models, and the resource utilization rate of the distributed system is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a task processing method and a task processing device. Background Art

[0002] With the development of generative artificial intelligence technology, various large machine learning models have emerged that can handle more complex tasks, such as large models for generating videos from text.

[0003] More complex task processing generally requires the assistance of multiple models to generate videos. Taking Wensheng video as an example, in the relevant technical solution, multiple machine learning models are combined into a video generation link, and the video is generated through the video generation link. For example, the video is generated through the following video generation link: text prompt word -> optimization model -> image prompt word generation model -> image generation model -> video generation model -> quality detection model -> super-resolution model. However, due to the complex task processing flow of multiple models, the efficiency of processing tasks is low.

[0004] Therefore, how to improve the task processing efficiency of multiple model processing tasks has become a technical problem that needs to be solved urgently.

[0005] The content of the background technology section is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the invention

[0006] The present specification provides a task processing method and a task processing device, which can efficiently utilize multiple models to process tasks and improve resource utilization.

[0007] In a first aspect, the present specification provides a task processing method, which is applied to a distributed system, wherein the distributed system includes a master node and multiple slave nodes, wherein the slave nodes are deployed with a model instance of a task model, wherein the task model is used to execute at least one subtask of multiple subtasks of a task to be processed, and the method includes:

[0008] Acquire at least one of the tasks to be processed through the master node;

[0009] Allocating the target subtask of the task to be processed to the corresponding target slave node through the master node, the target slave node being deployed with a model instance of the target task model for executing the target subtask;

[0010] Monitoring, by the master node, the execution result of the target subtask returned by the target slave node; and

[0011] Based on the model dependency relationship between the task models of each of the slave nodes, the monitored execution result is transmitted to the corresponding dependent slave node through the master node, and the dependent slave node is a slave node that depends on the execution result of the target slave node.

[0012] In some example embodiments, based on the above solution, a plurality of the tasks to be processed are stored in a message queue, and obtaining at least one of the tasks to be processed through the master node includes:

[0013] At least one of the to-be-processed tasks is obtained from the message queue through the master node.

[0014] In some example embodiments, based on the above solution, obtaining at least one of the to-be-processed tasks from the message queue through the master node includes:

[0015] Based on a preset queuing strategy of the message queue, at least one of the to-be-processed tasks is obtained from the message queue through the master node, and the preset queuing strategy includes a priority strategy.

[0016] In some example embodiments, based on the above solution, before obtaining at least one of the to-be-processed tasks from the message queue through the master node, the method further includes:

[0017] Determining remaining instance resources of a model instance of a target task model of the target slave node;

[0018] The number of the to-be-processed tasks obtained from the message queue is determined based on the remaining instance resources of the model instance of the target task model.

[0019] In some example embodiments, based on the above solution, the method further includes:

[0020] Counting the node instance resources of the model instance of the task model of each of the slave nodes;

[0021] The node instance resources of the slave node are expanded or reduced based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system.

[0022] In some example embodiments, based on the above solution, scaling the node instance resources of the slave node based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system includes:

[0023] The node instance resources of the slave node are scaled up or down in proportion based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system.

[0024] In some example embodiments, based on the above solution, scaling the node instance resources of the slave node in proportion based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system includes:

[0025] If the remaining resource ratio of the node instance resources is less than the first predetermined ratio threshold corresponding to the slave node, the node instance resources of the slave node are proportionally expanded based on the remaining system resources of the distributed system;

[0026] If the remaining resource ratio of the node instance resources is greater than the second predetermined ratio threshold corresponding to the slave node, the node instance resources of the slave node are scaled down in proportion based on the remaining system resources of the distributed system.

[0027] In some example embodiments, based on the above solution, allocating the target subtask of the task to be processed to the corresponding target slave node by the master node includes:

[0028] Determining a target task model corresponding to the target subtask based on the corresponding relationship between the subtask of the task to be processed and the task model;

[0029] Based on the corresponding relationship between the task model and the slave node, determining a target slave node corresponding to the target task model;

[0030] The target subtask is allocated to the target slave node through the master node.

[0031] In some example embodiments, based on the above scheme, the model dependency relationship is a directed acyclic graph, and the model dependency relationship between the task models of each of the slave nodes is based on the model dependency relationship between the task models of each of the slave nodes, and the master node transmits the monitored execution result to the corresponding dependent slave node, including:

[0032] Determine the dependent slave node corresponding to the target slave node based on the directed acyclic graph corresponding to the task model of each slave node;

[0033] The monitored execution result is transmitted to the dependent slave node through the master node.

[0034] In some example embodiments, based on the above solution, the method further includes:

[0035] If the master node fails, the backup master node is determined as the master node;

[0036] If the slave node fails, the task that was processed by the slave node when the failure occurred is reassigned to the slave node.

[0037] In some example embodiments, based on the above solution, the step of monitoring, by the master node, the execution result of the target subtask returned by the target slave node includes:

[0038] The scheduling process of the master node monitors the execution result of the target subtask returned by the target slave node,

[0039] The step of transmitting the monitored execution result to the corresponding dependent slave node through the master node includes:

[0040] The monitored execution result is transmitted to the corresponding dependent slave node through the scheduling process of the master node.

[0041] In a second aspect, the present specification also provides a task processing device, comprising: at least one storage medium storing at least one instruction set for performing task processing; and at least one processor communicatively connected to the at least one storage medium, wherein when the task processing device is running, the at least one processor reads the at least one instruction set and executes the task processing method described in the first aspect of this specification according to the instructions of the at least one instruction set.

[0042] It can be seen from the above technical solutions that the task processing method and device provided in the embodiments of this specification, on the one hand, allocate the target subtask to the target slave node corresponding to the distributed system based on the correspondence between the subtask of the task to be processed and the task model, and can efficiently and accurately allocate the subtask of the task to be processed to the slave nodes of the distributed system; on the other hand, the master node monitors the execution results of the target subtask returned by the slave node, and based on the model dependency relationship between the task models of each slave node, the master node transmits the monitored execution results to the corresponding dependent slave nodes, which can realize asynchronous non-blocking scheduling of a large number of tasks by the master node, thereby being able to efficiently utilize multiple models to process multiple tasks, thereby improving the resource utilization of the distributed system.

[0043] Other functions of the task processing method and device provided in this specification will be partially listed in the following description. Based on the description, the content introduced by the following figures and examples will be obvious to those of ordinary skill in the art. The creative aspects of the task processing method and device provided in this specification can be fully explained by practicing or using the methods, devices and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram showing the dependency relationship between models in the related technical solution;

[0046] Figure 2 A schematic diagram of a multi-model orchestration scenario provided by a related technical solution is shown;

[0047] Figure 3 A schematic diagram showing an implementation environment of a task processing method provided in an embodiment of this specification;

[0048] Figure 4 shows a hardware structure diagram of a task processing device 200 provided according to an embodiment of this specification;

[0049] Figure 5 A flowchart of a task processing method provided according to some embodiments of this specification is shown;

[0050] Figure 6 A schematic diagram showing an implementation environment of a task processing method provided according to some embodiments of this specification;

[0051] Figure 7 A schematic diagram of a process of scaling resources provided according to some embodiments of this specification is shown; and

[0052] Figure 8 A flowchart of a task processing method provided according to some other embodiments of the present specification is shown. DETAILED DESCRIPTION

[0053] The following description provides specific application scenarios and requirements of this specification, with the purpose of enabling those skilled in the art to make and use the contents of this specification. Various local modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but to the widest scope consistent with the claims.

[0054] The terms used herein are only used for the purpose of describing specific example embodiments and are not restrictive. For example, unless the context clearly indicates otherwise, as used herein, the singular forms "a", "an" and "the" may also include plural forms. When used in this specification, the terms "include", "comprise" and / or "contain" mean that the associated integers, steps, operations, elements and / or components exist, but do not exclude the existence of one or more other features, integers, steps, operations, elements, components and / or groups or that other features, integers, steps, operations, elements, components and / or groups may be added in the system / method.

[0055] In view of the following description, these and other features of the present specification, as well as the operation and function of the related elements of the structure, and the economy of the combination and manufacture of the parts can be significantly improved. Reference is made to the accompanying drawings, all of which form a part of this specification. However, it should be clearly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0056] The flowcharts used in this specification illustrate the operations implemented by the system according to some embodiments in this specification. It should be clearly understood that the operations of the flowcharts may not be implemented in sequence. On the contrary, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0057] First, the terms involved in one or more embodiments of this specification are explained.

[0058] Large Language Model (LLM): A large model is an artificial intelligence model that is trained on a large amount of text data and can understand and generate natural language text.

[0059] Multi-model orchestration: that is, multi-model serial and parallel orchestration, which means that one model cannot solve the scene problem, and usually multiple models are required for serial and parallel processing. For example, the video generation link usually needs to be processed by multiple models such as a large language model, image generation model, video generation model, detection model, video super-resolution, etc. For example, the video is generated through the following video generation link: text prompt word -> optimization model -> image prompt word generation model -> image generation model -> video generation model -> quality detection model -> super-resolution model.

[0060] Non-blocking scheduling: means that after a task is started, even if the task needs to wait for some resources, the thread or process executing the task will not be blocked, but will continue to execute other tasks or operations.

[0061] Non-blocking task processing: Usually a more complex task processing flow includes multiple subtask processing nodes, each of which processes a subtask. Non-blocking task processing can ensure that when a program or system executes a task, it will not stop the execution of other tasks because it is waiting for the task to be completed. For example, if a task is received from a user and the request is to wait for the first node to process, the processing time of the first node takes 10 seconds. The user's second task is not processed within these 10 seconds, and other subsequent nodes are also idle. This is not only a huge waste of resources, but also blocks the user's task processing progress. Therefore, in non-blocking task processing, there is a master control logic. After reading multiple tasks, it requests the first node to process the first task, and can process the second task without waiting for the return result.

[0062] Dynamic scaling: A more complex task processing flow usually includes multiple subtask processing nodes, and each node has different processing capabilities. For example, the prompt optimization subtask may be very fast and can return within 5 seconds, but the video super-molecule task usually takes 1 minute. In this way, if throughput alignment is to be achieved, the number of nodes for the prompt optimization subtask and the task resources for the super-molecule task, such as the number of nodes, are set to 1:12 during deployment. However, during actual operation, the ratio may be unbalanced, depending on the time processing duration. 1:12 is not necessarily the best ratio, and the ratio needs to be adjusted dynamically and periodically, for example, expanding the super-resolution node or shrinking the prompt optimization node.

[0063] In the related technical solution, in order to adapt to more complex task processing scenarios, refer to Figure 1 As shown, the abstraction is as follows Figure 1 The task processing flow shown includes 4 subtasks. Subtask 1 is processed by model 1, and the processing result is sent to model 2 and model 3. Model 2 and model 3 process subtask 2 and subtask 3, and send the model processing results of subtask 2 and subtask 3 to model 4. Model 4 processes subtask 4.

[0064] Further, refer to Figure 2 As shown in the figure, for the above technical solution, the user's task request is received through the online centralized gateway and the task request is forwarded to the model service. For the multi-model orchestration scenario, a task processing orchestration is performed in the centralized gateway, and the processing model link of the task request is configured, as shown above. Figure 1As shown, the processing result of model 1 for subtask 1 is sent to model 2 and model 3; the processing results of model 2 and model 3 for subtask 2 and subtask 3 are sent to model 4, and the processing result of model 4 for subtask 4 is returned to the user. However, in the above technical solution, on the one hand, if the gateway adopts asynchronous polling to obtain the model processing result, due to the large number of tasks and model nodes, the thread overhead will be very large, resulting in unnecessary waste of resources; on the other hand, if the gateway adopts a long link to synchronously wait for the model result, since there is no data transmission in the link most of the time, it will waste network link resources.

[0065] Based on the above content, the embodiments of this specification provide a task processing method and a task processing device. On the one hand, based on the correspondence between the subtasks of the task to be processed and the task model, the target subtask is assigned to the target slave node of the corresponding distributed system, which can efficiently and accurately assign the subtasks of the task to be processed to the slave nodes of the distributed system; on the other hand, the master node monitors the execution results of the target subtask returned by the target slave node, and based on the model dependency relationship between the task models of each slave node, the master node transmits the monitored execution results to the corresponding dependent slave nodes, which can realize asynchronous non-blocking scheduling of a large number of tasks by the master node, avoiding the performance overhead caused by continuous polling, so that multiple models can be efficiently used to process multiple tasks, thereby improving the resource utilization of the distributed system.

[0066] The technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings.

[0067] Figure 3 A schematic diagram showing an implementation environment of a task processing method provided in an embodiment of this specification is shown.

[0068] See also Figure 3 As shown, the implementation environment 300 may include a terminal 310 , a server 330 , and a database 340 .

[0069] The terminal 310 is connected to the server 330 via a wireless network or a wired network 320. The terminal 310 may be a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto.

[0070] The terminal 310 may store data or instructions for executing the task processing method described in this specification. The terminal 310 may include a hardware device with data information processing capabilities and a necessary program to drive the hardware device to work.

[0071] The server 330 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms, etc. The server 330 provides background services for applications running on the terminal 310.

[0072] The server 330 is equipped with an integrated development platform. The integrated development platform, also known as an integrated development environment (IDE), is an application program for providing a program development environment, generally including tools such as a code editor, a compiler, a debugger, and a graphical user interface. Developers can write program codes (i.e., program development) on the integrated development platform. The integrated development platform server can be a computing device specifically used by the integrated development platform to implement the task processing method. The server 330 can communicate data with the terminal 310 and the database 340 respectively.

[0073] In addition, the server 330 may store data or instructions for executing the task processing method described in this specification. The server 330 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to work. Of course, the server 330 may also be only a hardware device with data processing capabilities, or only a program running in the hardware device. In some embodiments, the server 330 may also be used as a plug-in and deployed in the terminal 310. In this case, the server 330 stores data or instructions for executing the task processing method corresponding to the terminal 310 described in this specification.

[0074] Database 340 can store data and / or instructions. In some embodiments, database 340 can store message queues and model instances, etc. In some embodiments, database 340 can store data and / or instructions executed by server 330 or used to execute the task processing method described in this specification. Terminal 310 and server 330 have access to database 340, and terminal 310 and server 330 can access data or instructions stored in database 340 through a network. In some embodiments, database 340 can be directly connected to terminal 310 and server 330. In some embodiments, database 340 can be part of server 330. In some embodiments, database 340 can include mass storage, removable storage, volatile read-write memory, read-only memory (ROM) or the like, or any combination thereof. Exemplary mass storage may include non-transitory storage mediums such as disks, optical disks, solid-state drives, etc. Example removable storage may include flash drives, floppy disks, optical disks, memory cards, zip disks, tapes, etc. Typical volatile read-write memory may include random access memory (RAM). Example RAMs may include dynamic RAM (DRAM), double date rate synchronous dynamic RAM (DDRSDRAM), static RAM (SRAM), thyristor RAM (T-RAM), zero capacitance RAM (Z-RAM), etc. Exemplary ROMs may include mask ROM (MROM), programmable ROM (PROM), virtually programmable ROM (PEROM), electronically programmable ROM (EEPROM), compact disk (CD ROM), digital versatile disk ROM, etc.

[0075] Those skilled in the art will appreciate that the number of the above terminals may be more or less. For example, the above terminal may be only one, or the above terminals may be dozens or hundreds, or a greater number, in which case the above implementation environment may also include other terminals. The embodiments of this specification do not limit the number of terminals and device types.

[0076] After introducing the implementation environment of the embodiments of this specification, the application scenarios of the embodiments of this specification will be introduced in combination with the above implementation environment. In the following description, the terminal is the terminal 310 in the above implementation environment, and the server is the server 330 in the above implementation environment. The technical solution provided by the embodiments of this specification can be applied to multi-model or multi-task processing scenarios, such as multi-task processing scenarios of video generation, multi-task processing scenarios of personal travel assistants, etc.

[0077] Taking the application of the technical solution provided in the embodiments of this specification in a multi-task processing scenario of a personal travel assistant as an example, the scenario includes a travel subtask responsible for transportation and a recommendation subtask responsible for recommending attractions. The task processing method is applied to a distributed system, and the distributed system includes a master node and multiple slave nodes. The slave node is deployed with a model instance of a task model, and the task model is used to execute a subtask of the task to be processed, such as a travel subtask or a recommendation subtask of a travel task. The task processing method includes: obtaining at least one task to be processed through the master node; based on the correspondence between the subtasks of the task to be processed and the task model, assigning the target subtask to the corresponding target slave node through the master node, and the target slave node is deployed with a model instance of a target task model for executing the target subtask, such as a model instance of a travel planning model for processing the travel subtask or a model instance of a scenic spot recommendation model for processing the recommendation subtask; monitoring the execution result of the target subtask returned by the target slave node through the master node; and based on the model dependency relationship between the task models of each slave node, transmitting the monitored execution result to the corresponding dependent slave node through the master node, and the dependent slave node is a slave node that depends on the execution result of the target slave node.

[0078] It should be noted that the above is explained by taking the application of the technical solution provided in the embodiments of this specification in the multi-tasking scenario of a personal travel assistant application as an example. The technical solution provided in the embodiments of this specification can also be applied in other appropriate scenarios, for example, a shopping assistant application or a multi-tasking scenario, etc. The implementation process belongs to the same inventive concept as the above description and will not be repeated here.

[0079] It should be noted that the steps in the task processing method in the example embodiment of this specification may be partially executed by the client, partially executed by the server, or entirely executed by the server or entirely by the client, and this specification does not specifically limit this.

[0080] based on Figure 3 The implementation environment shown below will be combined with Figure 4-Figure 8 , the task processing method and task processing device provided by the embodiments of this specification are introduced in detail. It should be noted that the above implementation environment is only shown to facilitate understanding of the spirit and principle of this specification, and the embodiments of this specification are not limited in this regard. On the contrary, the embodiments of this specification can be applied to any applicable scenario.

[0081] Figure 4is a schematic diagram of a task processing device 400 provided according to some embodiments of this specification. The task processing device 400 can execute the task processing method described in this specification. The task processing method is introduced in other parts of this specification. The task processing device 400 can be a general-purpose computer or a special-purpose computer. For example, the task processing device 400 can be a server, a personal computer, a portable computer (such as a notebook computer, a tablet computer, etc.), or other task processing devices with computing capabilities. Of course, the task processing device can be Figure 3 The terminal 310 or the server 330 may also be a terminal device used by multiple developers to develop programs on an integrated development platform.

[0082] The task processing device in this specification may include one or more of the following components: a processor 410, a memory 420, an input device 430, an output device 440, and a bus 450. The processor 410, the memory 420, the input device 430, and the output device 440 may be connected via a bus 450.

[0083] The processor 410 may include one or more processing cores. The processor 410 uses various interfaces and lines to connect the various parts of the entire task processing device, and executes the task processing method described in this specification by running or executing instructions, programs, code sets or instruction sets stored in the memory 420, and calling data stored in the memory 420. Optionally, the processor 410 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 410 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 410, but may be implemented separately through a communication chip.

[0084] The memory 420 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 420 includes a non-transitory computer-readable storage medium. The memory 420 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 420 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system, including a system deeply developed based on the IOS system, or other systems.

[0085] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0086] The input device 430 is used to receive input instructions or data, and the input device 430 includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 440 is used to output instructions or data, and the output device 440 includes but is not limited to a display device and a speaker. In one example, the input device 430 and the output device 440 can be combined, and the input device 430 and the output device 440 are touch screen displays.

[0087] In addition, those skilled in the art can understand that the structure of the task processing device shown in the above figures does not constitute a limitation on the task processing device, and the task processing device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the task processing device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0088] Figure 5A flowchart of a task processing method provided according to an embodiment of the present specification is shown. As before, the task processing device 400 can execute the task processing method of the embodiment of the present specification. Specifically, the processor 410 can read the instruction set stored in its local storage medium, and then execute the task processing method of the embodiment of the present specification according to the provisions of the instruction set. Below, steps S510 to S540 in the task processing method will be described in detail in conjunction with the accompanying drawings.

[0089] like Figure 5 As shown, in step S510, at least one task to be processed is obtained through the master node.

[0090] In an example embodiment, the task processing method is applied to a distributed system, which includes a master node and multiple slave nodes, and the slave nodes are deployed with a model instance of a task model, and the task model is used to execute at least one of multiple subtasks of the task to be processed. The task model can be a task processing tool or a task processing model. The task processing tool is a tool for performing a specific task, such as a search tool, a database search tool, and a calculation tool. The interface of the task processing tool can be a function with a string as input and a string as output. The task processing model can be a large language model such as a GPT model, or it can be other appropriate machine learning models such as a Transformer model.

[0091] Furthermore, the task to be processed includes multiple subtasks, and the task model is used to execute a subtask of the task to be processed. The task processing device 400 of the master node obtains at least one task to be processed from the received task request. For example, the distributed system can be a distributed cluster, and the distributed cluster includes a master node and multiple slave nodes, each node is a computing resource, which can be a physical machine, a resource container, etc. Among them, the master node is used to receive task requests and perform task scheduling, and the slave node is a unit for executing tasks. The task processing device 400 of the master node obtains at least one task to be processed.

[0092] Reference Figure 6As shown, the distributed system includes a master node and slave nodes 1 to 4. Slave node 1 is deployed with a model instance of task model 1, slave node 2 is deployed with a model instance of task model 2, slave node 3 is deployed with a model instance of task model 3, and slave node 4 is deployed with a model instance of task model 4. Assume that the task to be processed is a tourism recommendation task, and the tourism recommendation task includes a weather search task, a traffic planning task, a budget calculation task, and a travel route recommendation task. Among them, slave node 1 is deployed with a model instance of a weather search model, slave node 2 is deployed with a model instance of a traffic planning model, slave node 3 is deployed with a model instance of a budget calculation model, and slave node 4 is deployed with a model instance of a travel route recommendation model. Among them, the weather search model is used to search for corresponding weather information based on the departure time, departure place, and destination input by the user; the transportation planning model is used to recommend travel vehicles and corresponding flights to users based on weather information, departure time, departure place, and destination; the budget calculation model is used to calculate the travel budget based on information such as transportation, travel duration, and number of people, for example, the travel budget required for 3 people, taking the high-speed rail, and traveling for 3 days; the travel route recommendation model is used to recommend travel routes to users based on the destination, travel budget, and travel duration.

[0093] It should be noted that although the task to be processed is a travel recommendation task as an example, those skilled in the art should understand that the task to be processed may also be other appropriate tasks such as a video generation task or a shopping assistant task, which is also within the scope of the embodiments of this specification.

[0094] In step S520, the target subtask of the task to be processed is allocated to the corresponding target slave node through the master node.

[0095] In the example embodiment, each subtask of the task to be processed is processed by the corresponding task model. Therefore, there is a corresponding relationship between each subtask of the task to be processed and the task model. The slave node is deployed with a model instance of the task model, and the target slave node is deployed with a model instance of the target task model for executing the target subtask. The model instance of the target task model is used to execute the target subtask. Figure 6 As shown, the task to be processed is a tourism recommendation task, which includes a weather search task, a traffic planning task, a budget calculation task, and a travel route recommendation task. Among them, a model instance of a weather search model is deployed from node 1, a model instance of a traffic planning model is deployed from node 2, a model instance of a budget calculation model is deployed from node 3, and a model instance of a travel route recommendation model is deployed from node 4.

[0096] Further, the correspondence between the subtasks of the task to be processed and the task model is pre-configured, for example, the correspondence between the subtasks of the task to be processed and the task model is pre-stored in the form of key-value pairs. The task processing device 400 of the master node allocates the target subtask to the corresponding target slave node through the master node based on the pre-configured correspondence between the subtasks of the task to be processed and the task model. For example, referring to Figure 6 As shown, assuming that the target subtask is a weather search task, the weather search task is assigned to slave node 1 by remote calling through the task processing device 400 of the master node, and slave node 1 is deployed with a model instance of the weather search model.

[0097] For example, the task processing device 400 of the master node determines the target task model corresponding to the target subtask based on the correspondence between the subtasks of the task to be processed and the task model; determines the target slave node corresponding to the target task model based on the correspondence between the task model and the slave node; and assigns the target subtask to the target slave node by remote calling through the master node. Taking the distributed system as a Ray-based distributed system as an example, the task model of the target subtask is changed into a task model that can be distributed and called through the @ray.remote decorator, and the target subtask is submitted through the task model name.remote method, and the target subtask is submitted to the corresponding target slave node.

[0098] By allocating the target subtask to the target slave node through remote calling, the distributed calling of the task model of the subtask can be realized simply and efficiently.

[0099] In step S530, the master node monitors the execution result of the target subtask returned by the target slave node.

[0100] In the example embodiment, the execution result of the target subtask is the result of executing the target subtask through the model instance of the target task model. After the target slave node receives the target subtask assigned by the master node, it executes the target subtask through the model instance of the target task model deployed by the target slave node, and returns the execution result of the target subtask to the master node. The task processing device 400 of the master node monitors the execution result of the target subtask returned by the target slave node through the scheduling process of the master node.

[0101] For example, let the target subtask be subtask 1. The execution result of the target subtask can be represented by the object reference ObjectRef1. The target slave node returns the execution result ObjectRef1 of the target subtask to the master node. The task processing device 400 of the master node monitors the execution result ObjectRef1 of the target subtask returned by the target slave node through the scheduling process of the master node.

[0102] For example, assuming that the distributed system is a Ray-based distributed system, the target slave node executes the target subtask through the model instance of the deployed target task model, and the execution result ObjectRef1 of the target subtask is returned by Ray.wait(). The task processing device 400 of the master node monitors the execution result ObjectRef1 of the target subtask returned by the target slave node through Ray.wait() through the scheduling process of the master node. Ray.wait() can return after waiting for the execution result of the current target subtask to be satisfied, without waiting for the execution results of each subtask to be satisfied before returning, thereby improving the resource utilization of the distributed system.

[0103] In step S540, based on the model dependency relationship between the task models of the various slave nodes, the master node transmits the monitored execution result of the target subtask to the corresponding dependent slave node.

[0104] In the exemplary embodiment, there is a dependency relationship between each subtask of the task to be processed, and therefore, there is also a dependency relationship between the task models of each slave node, and the dependent slave node is a slave node that depends on the execution result of the target slave node. Figure 1 and Figure 6 As shown, model 2 and model 3 depend on the task processing results of model 1, and model 4 depends on the task processing results of model 2 and model 3. Then slave node 2 and slave node 3 are dependent slave nodes of slave node 1, and slave node 4 is a dependent slave node of slave node 2 and slave node 3.

[0105] For example, the model dependency relationship between the task models of each slave node is preset, for example, the model dependency relationship between the task models of each slave node is stored in the form of a key-value pair. The task processing device 400 of the master node transmits the execution result of the monitored target subtask to the corresponding dependent slave node through the scheduling process of the master node based on the model dependency relationship between the task models of each slave node. Figure 1 and 6 As shown, let the target subtask be task 1, the target slave node be node 1, slave nodes 2 and 3 be dependent slave nodes of slave node 1, and the task processing device 400 of the master node transmits the execution result monitored from slave node 1 to slave nodes 2 and 3 through the master node based on the model dependency relationship.

[0106] Further, in the exemplary embodiment, the model dependency is a directed acyclic graph, and the task processing device 400 of the master node determines the dependent slave node corresponding to the target slave node based on the directed acyclic graph corresponding to the task model of each slave node; and transmits the monitored execution result to the dependent slave node through the master node. Figure 1As shown in Figure 1, model dependencies are organized into a directed acyclic graph (DAG). Each node in the DAG represents a model, and the edges represent the dependencies between models. If model B needs to be executed after model A is completed, it is represented in the DAG as a directed edge from A to B.

[0107] For example, the task processing device 400 of the master node analyzes the directed acyclic graph corresponding to the task model of each slave node, determines the dependent slave node corresponding to the target slave node based on the analysis result, and transmits the monitored execution result to the dependent slave node through the master node. Figure 1 , the target slave node is slave node 1, the task processing device 400 of the master node parses the directed acyclic graph through the master node analysis method, and determines that the dependent slave nodes corresponding to slave node 1 are slave node 2 and slave node 3, and transmits the execution result monitored from slave node 1 to slave node 2 and slave node 3 through the master node.

[0108] According to the technical solution in the above-mentioned example embodiments, on the one hand, based on the correspondence between the subtasks of the task to be processed and the task model, the target subtask is assigned to the corresponding target slave node, so that the subtasks of the task to be processed can be efficiently assigned to the slave nodes of the distributed system; on the other hand, the master node monitors the execution results of the target subtasks returned by the slave node, and based on the model dependency relationship between the task models of each slave node, the master node transmits the monitored execution results to the corresponding dependent slave nodes, so that the master node can realize asynchronous non-blocking scheduling of a large number of tasks, avoiding the performance overhead caused by continuous polling; thereby, multiple models can be efficiently used to process multiple tasks, thereby improving resource utilization.

[0109] In addition, in the exemplary embodiment, multiple tasks to be processed are stored in a message queue, and at least one task to be processed is obtained from the message queue by the task processing device 400 of the master node. The message queue is a container for queuing multiple tasks to be processed, and is used to sort and dequeue multiple tasks to be processed in order according to a predetermined queuing strategy.

[0110] For example, the received multiple pending tasks are stored in the message queue, and the task processing device 400 of the master node pulls at least one pending task from the message queue based on the predetermined queuing strategy. The predetermined queuing strategy includes a priority strategy. For example, if the processing time of the task is longer, the priority of the task is lower, and if the processing time of the task is shorter, the priority of the task is higher. For example, suppose that under normal circumstances, 5 users submit task requests at the same time within a predetermined time period, such as 1 second, and the task processing device 400 obtains 5 pending tasks from the message queue. If a hot spot situation suddenly occurs, 100 users submit task requests at the same time within 1 second. The task requests of the 100 users are placed in the MQ through the message queue MQ, and at least one pending task is obtained from the message queue according to the priority of the 100 pending tasks.

[0111] According to the technical solution in the above-mentioned example embodiment, on the one hand, the received multiple tasks to be processed are stored in the message queue, and the tasks can be received in the form of task queue MQ; on the other hand, at least one task to be processed is obtained from the message queue based on a predetermined queuing strategy, and the traditional task push mode can be changed to a task pull mode as needed, so that the master node can autonomously control the speed of consuming tasks and has the ability to smooth out peaks and fill valleys.

[0112] Further, in the exemplary embodiment, the task processing device 400 of the node determines the remaining instance resources of the model instance of the target task model of the target slave node; based on the remaining instance resources of the model instance of the target task model, the number of pending tasks obtained from the message queue is determined. For example, suppose the target slave node, such as slave node 1, is deployed with 5 model instances of the target task model. If it is used for the first time, the remaining instance resources of the model instance of the target task model of the target slave node are determined to be 5, and the number of pending tasks obtained from the message queue is determined to be 3; if 2 model instances have been used, the remaining instance resources of the model instance of the target task model of the target slave node are determined to be 3, and the number of pending tasks obtained from the message queue is determined to be 3.

[0113] According to the technical solution in the above-mentioned example embodiment, based on the remaining instance resources of the model instance of the target task model, a corresponding number of tasks to be processed are obtained from the message queue. The corresponding number of tasks to be processed can be obtained according to the task resources of the slave node, thereby improving the task processing efficiency.

[0114] In addition, in the example embodiment, if the master node fails, the backup master node is determined as the master node; if the slave node fails, the tasks that the slave node was processing when the failure occurred are reassigned to the slave node. Taking the distributed system as a Ray-based distributed system as an example, Ray's tasks have the ability of lineage backtracking, that is, when an object that a task depends on is lost, the task corresponding to the object can be found through lineage and re-executed to restore the object.

[0115] According to the technical solution in the above example embodiment, when a node fails, the node is restored through the self-healing capability of the distributed system, and the failure of the node will not affect the unconsumed data, thereby improving the robustness of task processing of the distributed system.

[0116] Figure 7 A schematic diagram of a process for scaling resources provided according to some embodiments of this specification is shown.

[0117] In step S710, the node instance resources of the model instance of the task model of each slave node are counted.

[0118] In the exemplary embodiment, the task processing device 400 of the master node periodically collects the node instance resources of the model instances of the task models of each slave node, such as the used instance resources and the remaining instance resources, and updates them to the global control service unit after statistics. For example, suppose that there are 5 model instances deployed on slave node 1, and the number of used model instances is 2 and the number of remaining model instances is 3 after statistics.

[0119] In step S720, the node instance resources of the slave node are expanded or reduced based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system.

[0120] In an example embodiment, the system of the distributed system can be a CPU or GPU resource, and the remaining system resources of the distributed system can be represented by a key-value pair, for example, the remaining GPU resources are {"GPU": 20} to represent the remaining 20 GPU resources. The task processing device 400 of the master node scales the node instance resources of the slave node based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system. For example, suppose that one instance resource of the target slave node requires one GPU resource, the total node instance resources of the target slave node are 5, and the remaining system resources of the distributed system are 20. If the remaining node instance resources corresponding to the target slave node are less than the first predetermined resource threshold, for example, less than 2, then the node instance resources corresponding to the target slave node are expanded, for example, 2 GPU resources are expanded to the target slave node, and the node instance resources of the target slave node are expanded to 4; if the remaining node instance resources corresponding to the target slave node are greater than the second predetermined resource threshold, for example, greater than 4, then the node instance resources corresponding to the target slave node are scaled down. The first predetermined resource threshold and the second predetermined resource threshold are determined according to the total number of node instance resources of the slave node and the processing performance.

[0121] Further, in the exemplary embodiment, the task processing device 400 of the master node scales up or down the node instance resources of the slave node in proportion based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system. For example, if the remaining resources of the node instance resources corresponding to the slave node are small, the node instance resources of the slave node are scaled up in proportion, for example, by 0.25, based on the remaining system resources of the distributed system; if the remaining resources of the node instance resources corresponding to the slave node are large, the node instance resources of the slave node are scaled down in proportion, for example, by 0.25, based on the remaining system resources of the distributed system.

[0122] For example, if the remaining resource ratio of the node instance resources is less than the first predetermined ratio threshold corresponding to the slave node, for example, less than 0.3, then the node instance resources of the slave node are expanded proportionally, for example, by 0.25 based on the remaining system resources of the distributed system; if the remaining resource ratio of the node instance resources is greater than the second predetermined ratio threshold corresponding to the slave node, for example, 0.9, then the node instance resources of the slave node are reduced proportionally, for example, by 0.25 based on the remaining system resources of the distributed system.

[0123] According to the technical solution in the above-mentioned example embodiment, the node instance resources of the slave node are scaled up or down proportionally based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system. The node instance resources of the slave node can be dynamically scaled up or down according to the actual situation of the node resources and system resources, thereby improving resource utilization and task processing efficiency.

[0124] Figure 8A flowchart of a task processing method provided according to some other embodiments of the present specification is shown.

[0125] Reference Figure 8 As shown, in step S810, at least one to-be-processed task is obtained from the message queue through the master node.

[0126] In an example embodiment, a message queue is a container for queuing multiple tasks to be processed, and is used to sort and dequeue multiple tasks to be processed in order according to a predetermined queuing strategy. The task processing device 400 of the master node pulls at least one task to be processed from the message queue based on the predetermined queuing strategy, and the predetermined queuing strategy includes a priority strategy.

[0127] For example, assuming that under normal circumstances, 5 users submit task requests at the same time within a predetermined time period, such as 1 second, the task processing device 400 obtains 5 pending tasks from the message queue. If a hot spot situation suddenly occurs, 100 users submit task requests at the same time within 1 second. The task requests of the 100 users are placed in the message queue MQ, and at least one pending task is obtained from the message queue according to the priority of the 100 pending tasks.

[0128] In step S820, based on the correspondence between the subtasks of the task to be processed and the task model, the target subtask is allocated to the corresponding target slave node through the master node.

[0129] In the example embodiment, the task processing device 400 of the master node determines the target task model corresponding to the target subtask based on the correspondence between the subtask of the task to be processed and the task model; determines the target slave node corresponding to the target task model based on the correspondence between the task model and the slave node; and assigns the target subtask to the target slave node by remote calling through the master node. In other words, the model instance of the target task model for processing the target subtask is called by remote calling.

[0130] Taking the Ray-based distributed system as an example, the task model of the target subtask is changed into a distributed callable task model through the @ray.remote decorator, and the target subtask is submitted through the task model name.remote method, and the target subtask is submitted to the corresponding target slave node. The target subtask is assigned to the target slave node through remote calls, which can simply and efficiently implement distributed calls to the task model of the subtask.

[0131] In step S830, the master node monitors the execution result of the target subtask returned by the target slave node.

[0132] In an example embodiment, after the target slave node receives the target subtask assigned by the master node, it executes the target subtask through the model instance of the target task model deployed by the target slave node, and returns the execution result of the target subtask to the master node. The task processing device 400 of the master node monitors the execution result of the target subtask returned by the target slave node through the scheduling process of the master node.

[0133] For example, assuming that the distributed system is a Ray-based distributed system, the target slave node executes the target subtask through the model instance of the deployed target task model, and the execution result ObjectRef1 of the target subtask is returned by Ray.wait(). The task processing device 400 of the master node monitors the execution result ObjectRef1 of the target subtask returned by the target slave node through Ray.wait() through the scheduling process of the master node. Ray.wait() can return after waiting for the execution result of the current target subtask to be satisfied, without waiting for the execution results of each subtask to be satisfied before returning, thereby improving the resource utilization of the distributed system.

[0134] In step S840, based on the model dependency relationship between the task models of the slave nodes, the master node transmits the monitored execution result of the target subtask to the corresponding dependent slave node.

[0135] In the exemplary embodiment, the model dependency relationship between the task models of the slave nodes is preset, for example, the model dependency relationship between the task models of the slave nodes is stored in the form of key-value pairs. The task processing device 400 of the master node transmits the execution result of the monitored target subtask to the corresponding dependent slave node through the scheduling process of the master node based on the model dependency relationship between the task models of the slave nodes. Figure 1 and 6 As shown, let the target subtask be task 1, the target slave node be node 1, slave nodes 2 and 3 be dependent slave nodes of slave node 1, and the task processing device 400 of the master node transmits the execution result monitored from slave node 1 to slave nodes 2 and 3 through the master node based on the model dependency relationship.

[0136] For example, the task processing device 400 of the master node analyzes the directed acyclic graph corresponding to the task model of each slave node, determines the dependent slave node corresponding to the target slave node based on the analysis result, and transmits the monitored execution result to the dependent slave node through the master node. Figure 1 , the target slave node is slave node 1, the task processing device 400 of the master node parses the directed acyclic graph through the master node analysis method, and determines that the dependent slave nodes corresponding to slave node 1 are slave node 2 and slave node 3, and transmits the execution result monitored from slave node 1 to slave node 2 and slave node 3 through the master node.

[0137] In step S840, the node instance resources of the slave node are expanded or reduced based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system.

[0138] In an exemplary embodiment, the task processing device 400 of the master node scales up or down the node instance resources of the slave node based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system. For example, suppose that an instance resource of the target slave node requires a GPU resource, the total node instance resources of the target slave node are 5, and the remaining system resources of the distributed system are 20. If the remaining node instance resources corresponding to the target slave node are less than the first predetermined resource threshold, for example, less than 2, then the node instance resources corresponding to the target slave node are expanded, for example, 2 GPU resources are expanded to the target slave node, and the node instance resources of the target slave node are expanded to 4; if the remaining node instance resources corresponding to the target slave node are greater than the second predetermined resource threshold, for example, greater than 4, then the node instance resources corresponding to the target slave node are scaled down. The first predetermined resource threshold and the second predetermined resource threshold are determined according to the total number of node instance resources of the slave node and the processing performance.

[0139] According to the technical solution in the above-mentioned example embodiments, on the one hand, based on the remote call and result reference monitoring capabilities provided by distributed systems such as Ray, the master node can realize asynchronous non-blocking scheduling of a large number of tasks, avoiding the performance overhead caused by continuous polling and simplifying the task scheduling process; on the other hand, task reception adopts the message to MQ method, which can change the traditional push task mode into a mode of pulling tasks as needed, so that the master node can independently control the speed of consuming tasks and has the ability to smooth out peaks and fill valleys; on the other hand, since the model orchestration processing systems of each task are independent and the tasks do not affect each other, the master node and the slave node are uniformly managed and maintained by the system, which can conveniently dynamically expand and shrink the instance resources of each node.

[0140] On the other hand, the present specification provides a non-transitory storage medium storing at least one set of executable instructions for task processing. When the executable instructions are executed by the processor, the executable instructions instruct the processor to implement the steps of the task processing method described in the present specification. In some possible implementations, various aspects of the present specification can also be implemented in the form of a program product, which includes a program code. When the program product is run on the task processing device 200, the program code is used to enable the task processing device 200 to perform the steps of the task processing method described in the present specification. The program product for implementing the above method can use a portable compact disk read-only memory (CD-ROM) to include program code and can be run on the task processing device 200. However, the program product of the present specification is not limited to this. In the present specification, the readable storage medium can be any tangible medium containing or storing a program, which can be used by the instruction execution system or used in combination with it. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media include: an electrical connection with one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. Program code for performing the operations of the present specification may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the task processing device 200, partially on the task processing device 200, as a stand-alone software package, partially on the task processing device 200 and partially on a remote computing device, or entirely on a remote computing device.

[0141] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require a specific order or a continuous order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0142] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented only by way of example and may not be limiting. Although not explicitly stated herein, those skilled in the art will appreciate that this specification requires various reasonable changes, improvements and modifications to the embodiments. These changes, improvements and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0143] In addition, certain terms in this specification have been used to describe embodiments of this specification. For example, "one embodiment", "an embodiment" and / or "some embodiments" mean that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of this specification. Therefore, it can be emphasized and should be understood that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics may be appropriately combined in one or more embodiments of this specification.

[0144] It should be understood that in the foregoing description of the embodiments of this specification, in order to help understand a feature and for the purpose of simplifying this specification, this specification combines various features in a single embodiment, figure or its description. However, this does not mean that the combination of these features is necessary. When reading this specification, it is entirely possible for a person skilled in the art to mark out some of the devices as separate embodiments. In other words, the embodiments in this specification can also be understood as the integration of multiple secondary embodiments. This is also true when the content of each secondary embodiment is less than all the features of a single aforementioned disclosed embodiment.

[0145] Each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated herein by reference in its entirety for all purposes, except for any prosecution document history related thereto, any equivalent that may be inconsistent or conflicting with this document, or any equivalent prosecution document history that may have a limiting effect on the broadest scope of the claims now or later associated with this document. For example, if there is any inconsistency or conflict between the description, definition, and / or use of a term associated with any of the incorporated materials and the term, description, definition, and / or use associated with this document, the term in this document shall control.

[0146] Finally, it should be understood that the embodiments of the application disclosed herein are explanations of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are only used as examples and not as limitations. Those skilled in the art can adopt alternative configurations according to the embodiments in this specification to implement the applications in this specification. Therefore, the embodiments of this specification are not limited to the embodiments accurately described in the application.

Claims

1. A task processing method, applied to a distributed system, the distributed system comprising a master node and a plurality of slave nodes, the slave nodes being deployed with a model instance of a task model, the task model being used to execute at least one of a plurality of subtasks of a task to be processed, the method comprising: Acquire at least one of the tasks to be processed through the master node; Allocating the target subtask of the task to be processed to the corresponding target slave node through the master node, the target slave node being deployed with a model instance of the target task model for executing the target subtask; Monitoring, by the master node, the execution result of the target subtask returned by the target slave node; and Based on the model dependency relationship between the task models of each of the slave nodes, the monitored execution result is transmitted to the corresponding dependent slave node through the master node, and the dependent slave node is a slave node that depends on the execution result of the target slave node.

2. The method according to claim 1, wherein: A plurality of the tasks to be processed are stored in a message queue, and obtaining at least one of the tasks to be processed through the master node includes: At least one of the to-be-processed tasks is obtained from the message queue through the master node.

3. The method according to claim 2, wherein: The acquiring at least one of the to-be-processed tasks from the message queue through the master node includes: Based on a preset queuing strategy of the message queue, at least one of the to-be-processed tasks is obtained from the message queue through the master node, and the preset queuing strategy includes a priority strategy.

4. The method according to claim 2, wherein: Before acquiring at least one of the to-be-processed tasks from the message queue through the master node, the method further includes: Determining remaining instance resources of a model instance of a target task model of the target slave node; The number of the to-be-processed tasks obtained from the message queue is determined based on the remaining instance resources of the model instance of the target task model.

5. The method according to claim 1, wherein: The method further comprises: Counting the node instance resources of the model instance of the task model of each of the slave nodes; The node instance resources of the slave node are expanded or reduced based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system.

6. The method according to claim 5, wherein: The scaling of the node instance resources of the slave node based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system includes: The node instance resources of the slave node are scaled up or down in proportion based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system.

7. The method according to claim 6, wherein: The scaling up or down the node instance resources of the slave node in proportion based on the node instance resources corresponding to the slave node and the remaining system resources of the distributed system includes: If the remaining resource ratio of the node instance resources is less than the first predetermined ratio threshold corresponding to the slave node, the node instance resources of the slave node are proportionally expanded based on the remaining system resources of the distributed system; If the remaining resource ratio of the node instance resources is greater than the second predetermined ratio threshold corresponding to the slave node, the node instance resources of the slave node are scaled down in proportion based on the remaining system resources of the distributed system.

8. The method according to claim 1, wherein: The step of allocating the target subtask of the task to be processed to the corresponding target slave node through the master node includes: Determining a target task model corresponding to the target subtask based on the corresponding relationship between the subtask of the task to be processed and the task model; Based on the corresponding relationship between the task model and the slave node, determining a target slave node corresponding to the target task model; The target subtask is allocated to the target slave node through the master node.

9. The method according to claim 1, wherein: The model dependency relationship is a directed acyclic graph, and the model dependency relationship between the task models based on each of the slave nodes is transmitted to the corresponding dependent slave node through the master node by monitoring the execution result, including: Determine the dependent slave node corresponding to the target slave node based on the directed acyclic graph corresponding to the task model of each slave node; The monitored execution result is transmitted to the dependent slave node through the master node.

10. The method according to claim 1, wherein: The method further comprises: If the master node fails, the backup master node is determined as the master node; If the slave node fails, the task that was processed by the slave node when the failure occurred is reassigned to the slave node.

11. The method according to claim 1, wherein: The monitoring, by the master node, of the execution result of the target subtask returned by the target slave node includes: The scheduling process of the master node monitors the execution result of the target subtask returned by the target slave node, The step of transmitting the monitored execution result to the corresponding dependent slave node through the master node includes: The monitored execution result is transmitted to the corresponding dependent slave node through the scheduling process of the master node.

12. A task processing device, comprising: at least one storage medium storing at least one instruction set for performing task processing; as well as at least one processor, in communication with the at least one storage medium, Wherein, when the task processing device is running, the at least one processor reads the at least one instruction set and executes the task processing method according to any one of claims 1 to 11 according to the instructions of the at least one instruction set.

Images