Task execution method and system

Abstract

The invention provides a task execution method and system, and belongs to the technical field of computers. According to the task execution method, a plurality of models needing to be called for task processing are deployed in a plurality of working nodes in a distributed mode with the models as the granularity; the management node can issue the task data of the task in the processing link to the target working node capable of executing the processing link to be processed based on the arrangement information of the model to be called by each processing link of the indication model, the processing progress of the task and the deployment condition of the model on the plurality of working nodes. When the processing flow of the task changes, only the arrangement information needs to be modified, a large number of business logic codes do not need to be modified or rewritten, and the task execution flexibility is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a task execution method and system. Background Art

[0002] With the development of artificial intelligence technology, related technologies have begun to use distributed systems to process tasks under specified services issued by users. The distributed system includes management nodes and working nodes. The management node is used to receive tasks from users and send tasks to the working nodes, which then process the received tasks.

[0003] Currently, for any business, operations personnel write extensive business logic code based on the multiple models required to process tasks within that business, as well as the order in which these models are called. This business logic code is used to call these multiple models to process tasks within the corresponding business. Operations personnel deploy the business logic code corresponding to a business on a worker node in the distributed system. The corresponding task execution method is as follows: the management node receives tasks from users and, based on the business to which the task belongs, sends the task to a worker node deployed with the corresponding business logic code. The worker node then receives and executes the task by running the business logic code.

[0004] The above task execution method relies on fixed business logic code. When business requirements change, resulting in changes in the processing flow of tasks under the business, the business logic code corresponding to the business needs to be modified or rewritten. That is, the above task execution method has low flexibility. Summary of the Invention

[0005] The present disclosure provides a task execution method and system for improving the flexibility of task execution. The technical solution is as follows.

[0006] According to one aspect of an embodiment of the present disclosure, a task execution method is provided. The method is applied to a distributed system, where the distributed system includes a management node and multiple worker nodes, each of which has one or more models deployed thereon. The method includes: The management node determines a target work node from multiple work nodes based on the task's corresponding orchestration information, the task's processing progress, and the deployment relationship between the model and the work node. The task's processing flow includes multiple processing links. The processing progress indicates the processing link the task is in, and the orchestration information indicates the model to be called by the task at each processing link. The target work node is used to execute the processing link indicated by the processing progress for the above task. The management node sends task data to the target working node. Task data is the data of the task at the processing stage indicated by the processing progress; The target working node executes the processing steps indicated by the processing progress based on the model and task data that conform to the processing progress.

[0007] In some embodiments, the process of obtaining the above arrangement information includes: The management node generates orchestration information based on at least one of the calling order, input-output relationship, parameter mapping information, and conditional judgment logic of multiple models corresponding to the business. The above-mentioned task belongs to the business, and the multiple models are the models to be called to execute the task.

[0008] In some embodiments, the management node determines the target working node from the plurality of working nodes based on the orchestration information corresponding to the task, the processing progress of the task, and the deployment relationship between the model and the working node, including: The management node determines the target model to be called at the processing link indicated by the task processing progress based on the task's corresponding orchestration information and the task's processing progress; The management node determines, from a plurality of working nodes, a target working node for executing a processing link indicated by the processing progress based on the deployment relationship and the target model.

[0009] In some embodiments, the management node determines, based on the deployment relationship and the target model, from a plurality of working nodes a target working node for executing the processing link indicated by the processing progress, including: The management node determines, based on the deployment relationship and the target model, at least one working node on which the target model is deployed from the plurality of working nodes; The management node determines a target working node from the at least one working node based on a routing policy and a load of the at least one working node.

[0010] In some embodiments, the above method further comprises: The target working node returns the processing result and the updated processing progress to the management node. The processing result is the processing result of the processing link indicated by the processing progress.

[0011] In some embodiments, multiple inference engines are deployed on each of the multiple working nodes, and different inference engines are used to load different types of models.

[0012] According to another aspect of an embodiment of the present disclosure, there is provided a task execution device, the device comprising: a node determination module configured to cause a management node in a distributed system to determine a target working node from a plurality of working nodes in the distributed system based on orchestration information corresponding to a task, a processing progress of the task, and a deployment relationship between a model and a working node, wherein the processing flow of the task includes a plurality of processing links, the processing progress indicates the processing link in which the task is located, the orchestration information indicates the model to be called by the task in each processing link, and the target working node is used to execute the processing link indicated by the processing progress on the task; A data sending module is configured to execute and enable the management node to send task data to the target working node, wherein the task data is data of the processing link of the task indicated by the processing progress; The execution module is configured to enable the target working node to execute the processing link indicated by the processing progress based on the model that conforms to the processing progress and the task data.

[0013] In some embodiments, the apparatus further comprises: The orchestration module is configured to cause the management node to generate the orchestration information based on at least one of a calling sequence, input-output relationships, parameter mapping information, and conditional judgment logic of multiple models corresponding to the business, wherein the task belongs to the business, and the multiple models are models to be called to execute the task.

[0014] In some embodiments, the node determination module includes: a target model determining unit configured to enable the management node to determine a target model to be called by the task at a processing link indicated by the processing progress based on the orchestration information corresponding to the task and the processing progress of the task; The node determination unit is configured to enable the management node to determine, based on the deployment relationship and the target model, a target working node for executing the processing link indicated by the processing progress from the multiple working nodes.

[0015] In some embodiments, the node determination unit is configured to execute: enabling the management node to determine, based on the deployment relationship and the target model, at least one working node on which the target model is deployed from the plurality of working nodes; The management node is enabled to determine the target working node from the at least one working node based on a routing policy and a load of the at least one working node.

[0016] In some embodiments, the apparatus further comprises: The return module is configured to execute the target working node to return the processing result and the updated processing progress to the management node, where the processing result is the processing result of the processing link indicated by the processing progress.

[0017] In some embodiments, multiple inference engines are deployed on each of the multiple working nodes, and different inference engines are used to load different types of models.

[0018] According to another aspect of an embodiment of the present disclosure, a distributed system is provided, which includes the management node and the working node provided in the above-mentioned first aspect or various optional implementations of the first aspect.

[0019] According to another aspect of an embodiment of the present disclosure, a computer device or a computer device cluster is provided, wherein the computer device cluster includes at least one computer device, wherein the computer device includes a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the task execution method provided in the above-mentioned first aspect or various optional implementations of the first aspect.

[0020] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the task execution method provided in the above-mentioned first aspect or various optional implementations of the first aspect.

[0021] According to another aspect of an embodiment of the present disclosure, a computer program product or computer program is provided, which includes computer program code, and the computer program code is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the operations performed by the task execution method provided in the above-mentioned first aspect or various optional implementations of the first aspect.

[0022] The task execution method provided by the disclosed embodiment uses models as the granularity, distributing the multiple models needed to process the task across multiple work nodes. During task execution, the management node, based on the orchestration information indicating the models to be called for each processing link of the model, the task's processing progress, and the deployment of the models on multiple work nodes, can send the task data for the task at the processing link to the target work node capable of executing the processing link. When the task processing flow changes, only the orchestration information needs to be modified, without having to modify or rewrite a large amount of business logic code, thereby improving the flexibility of task execution.

[0023] Based on the implementations provided in the above aspects, the present disclosure can be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 is a schematic diagram showing an implementation environment according to an exemplary embodiment; Figure 2 is a data interaction diagram illustrating a task execution method according to an exemplary embodiment; Figure 3 is an example diagram showing an arrangement diagram according to an exemplary embodiment; Figure 4 is a data interaction diagram illustrating another task execution method according to an exemplary embodiment; Figure 5 is a flowchart showing a task execution method according to an exemplary embodiment; Figure 6 is a structural diagram of a task execution device according to an exemplary embodiment; Figure 7 The figure is a schematic structural diagram of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0027] In the present disclosure, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0028] In the present disclosure, the term "at least one" means one or more, and "plurality" means two or more.

[0029] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, and display, etc.), and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the task data involved in this disclosure was obtained with full authorization.

[0030] Figure 1 is a schematic diagram of an implementation environment according to an exemplary embodiment, see Figure 1 The implementation environment includes a client 101 and a distributed system 102. The client 101 is directly or indirectly connected to the distributed system 102 via a wireless network or a wired network, which is not limited in the embodiments of the present disclosure.

[0031] Client 101 is used to provide business services for users, such as text-generated video services, image-generated video services, question-and-answer services, machine translation services, or intelligent assistant services. A text-generated video service generates a video based on user-entered text, while an image-generated video service generates text based on user-entered images. For example, a user enters text through client 101, triggering client 101 to send a text-generated video task to distributed system 102. The text-generated video task carries the user-entered text and instructs distributed system 102 to generate a video based on the user-entered text.

[0032] The client 101 is deployed on a portable mobile terminal, such as at least one of a smartphone, tablet computer, smartwatch, desktop computer, laptop computer, MP3 (Moving Picture Experts Group Audio Layer III) player, MP4 (Moving Picture Experts Group Audio Layer IV) player, and laptop computer. Alternatively, the client 101 is deployed on a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), big data, and artificial intelligence platforms.

[0033] The client 101 generally refers to one of multiple clients. The present disclosure uses client 101 as an example. Those skilled in the art will appreciate that the number of clients may be greater or lesser. For example, there may be a few clients, or dozens, hundreds, or even more. The present disclosure does not limit the number of clients or the type of device on which the client resides.

[0034] Distributed system 102 is used to provide background services for multiple businesses. That is, it receives and processes tasks under any business from client 101 and returns the processing results of the task to client 101. For example, distributed system 102 provides background services for the Vincent video business. Accordingly, distributed system 102 receives a Vincent video task from client 101. The Vincent video task generates a video based on the text carried in the task. Distributed system 102 processes the task and returns the generated video to client 101.

[0035] The distributed system 102 includes multiple nodes, each of which can be implemented as a computer device (such as a physical server) or a virtual machine running on a computer device, although this is not limited in the present embodiment. The multiple nodes include a management node and a worker node. The management node is used to receive tasks from the client 101 and send the received tasks to the worker node used to process the tasks. Each worker node is a basic computing unit used to process the received tasks and return the processing results to the management node. One or more models are deployed on each worker node. Accordingly, one or more containers are running on each worker node, each of which is used to run a model. The deployment of different worker nodes is independent of each other. Tasks under any of the above services require the collaborative processing of multiple models running in the distributed system 102. Accordingly, the management node stores the deployment relationship between the models and the worker nodes. After receiving a task from the client 101, the management node determines the worker node where the model to be called is deployed based on the model to be called to process the task and the above deployment relationship. The received task is sent to the worker node, which then processes the received task using the deployed model.

[0036] In some embodiments, the above-mentioned distributed system 102 adopts a microservice architecture, and each working node is used to provide one or more microservices based on the model deployed on the working node. The working nodes interact with each other through a lightweight communication protocol, which is not limited in the embodiments of the present disclosure.

[0037] It should be noted that Figure 1 The number of management nodes and working nodes shown is only an example. The number of management nodes and working nodes in a distributed system can be one or more, respectively, and the embodiments of the present disclosure are not limited to this.

[0038] Figure 2 is a data interaction diagram of a task execution method according to an exemplary embodiment, which is applied to the above-mentioned distributed system, which includes a management node and multiple working nodes, and one or more models are deployed on each working node, such as Figure 2 As shown, the method includes the following steps.

[0039] In step 201, the management node determines a target working node from multiple working nodes based on the orchestration information corresponding to the task, the processing progress of the task, and the deployment relationship between the model and the working node. The processing flow of the task includes multiple processing links. The processing progress indicates the processing link in which the task is located. The orchestration information indicates the model to be called by the task in each processing link. The target working node is used to execute the processing link indicated by the above processing progress for the above task.

[0040] Among them, the above-mentioned tasks are also called reasoning tasks, which can be text-generated video tasks, image-generated video tasks or machine translation tasks, etc., and the embodiments of the present disclosure do not limit this. Text-generated video tasks refer to generating videos based on the text carried in the task, and image-generated video tasks refer to generating videos based on the pictures carried in the task. The orchestration information corresponding to the task is the orchestration information corresponding to the business to which the task belongs. The orchestration information corresponding to each business is pre-set based on the processing flow of the task under the business, including multiple processing links of the task, the model to be called by each processing link, and the dependency relationship between the multiple processing links, etc. The dependency relationship between the multiple processing links includes the execution order between the multiple processing links and the input-output relationship between the multiple processing links. The input-output relationship indicates which models the intermediate processing results output by any model in the multiple processing links should be input into for processing. There are one or more models to be called for each processing link, and the embodiments of the present disclosure do not limit this. The orchestration information can be implemented as an orchestration graph, which includes multiple nodes, each node corresponds to a processing link of the task, the node information of the node indicates the model to be called by the processing link, and the connection relationship between any two nodes indicates the dependency relationship between the processing links corresponding to the two nodes. The following Figure 3 is an example diagram of an arrangement diagram according to an exemplary embodiment. Figure 3 As shown, Figure 3Each box in the diagram corresponds to a processing step. Each circle in the box represents the model to be called by the corresponding processing step. The lines connecting the circles indicate the calling order and input-output relationship between the corresponding models. The deployment relationship between models and worker nodes indicates which worker nodes in the distributed system have deployed each model. The model on each worker node is pre-deployed. There are one or more target worker nodes, and each target worker node is deployed with at least one model to be called by the processing step indicated by the processing progress. The model can be used to execute the processing step indicated by the processing progress.

[0041] In an embodiment of the present disclosure, the management node determines the orchestration information corresponding to the task based on the business to which the task belongs, determines the model to be called for the processing link from the orchestration information based on the processing link indicated by the processing progress of the task, and then determines the target working node for executing the above processing link for the above task from multiple working nodes based on the deployment relationship between the model and the working node.

[0042] In step 202, the management node sends task data to the target working node. The task data is the data of the task at the processing link indicated by the processing progress.

[0043] Among them, the task data can be the input data of the task, such as text, pictures or parameters input by the user, etc. The task data can also be the intermediate processing results output by any preceding processing link of the above processing links, such as the encoding results of text or pictures, etc. The task data can also be a combination of the above two types of task data, that is, the task data includes both the input data of the task and the intermediate processing results of the task. The embodiments of the present disclosure do not limit this. When the task is in the first processing link, the task data is the input data of the task. When the task is in the hth processing link, the task data includes at least one of the input data of the task and the intermediate processing results of the task, where h is an integer greater than 1.

[0044] In an embodiment of the present disclosure, the management node determines the task data of the above-mentioned task in the processing link based on the processing link indicated by the above-mentioned processing progress, and sends the task data to the target working node.

[0045] In step 203, the target working node executes the processing steps indicated by the processing progress based on the model that conforms to the processing progress and the task data.

[0046] In an embodiment of the present disclosure, the target working node calls the model to be called for the processing link indicated by the above processing progress through an interface pre-set for the model to process the received task data so as to execute the processing link indicated by the above processing progress.

[0047] The task execution method provided by the disclosed embodiment uses models as the granularity, distributing the multiple models needed to process the task across multiple work nodes. During task execution, the management node, based on the orchestration information indicating the models to be called for each processing link of the model, the task's processing progress, and the deployment of the models on multiple work nodes, can send the task data for the task at the processing link to the target work node capable of executing the processing link. When the task processing flow changes, only the orchestration information needs to be modified, without having to modify or rewrite a large amount of business logic code, thereby improving the flexibility of task execution.

[0048] above Figure 2 What is shown is only the basic process of the present disclosure. Before further elaborating on the solution provided by the present disclosure, the deployment process of the distributed system involved in the present disclosure is explained.

[0049] During the deployment of a distributed system, for any model, the relevant technicians deploy the model on a working node in the distributed system, or repeatedly deploy the model on multiple working nodes in the distributed system. After the deployment is completed, the working node automatically sends the deployment information of the working node to the management node. The deployment information carries the model identifier of the model deployed on the working node. The management node receives the deployment information sent by each working node, and based on the received deployment information, generates and stores the deployment relationship between the model and the working node.

[0050] In some embodiments, multiple inference engines are deployed on each of the above-mentioned multiple working nodes, and different inference engines are used to load different types of models. For example, the first inference engine is used to load text generation models, image processing models, and speech recognition models, etc., the second inference engine is used to load language models, and the third inference engine is used to load embedding models and reward models, etc. The embodiments of the present disclosure do not limit the inference engines deployed on the working nodes. By deploying multiple inference engines on the working nodes to adapt to diverse models, flexible access to diverse models can be achieved. Moreover, when business needs change, resulting in a change in the type of model to be called by the business, relevant technical personnel can directly load the model of that type onto the working node through the corresponding inference engine based on the changed type, thereby quickly achieving the replacement of the model type.

[0051] In some embodiments, for any working node, the relevant technical personnel perform offline deployment of the working node, that is, before the working node goes online to provide services, the relevant technical personnel debug each model deployed on the working node until the performance of the model reaches the preset performance conditions. By deploying the working node offline, problems that may be encountered in the process of reasoning service can be put in front of the scenes, making the modification requirements of the model clearer and more specific, reducing the communication costs between relevant technical personnel, and solving problems that may be encountered in the process of reasoning service in advance, ensuring that the performance of the working node after going online is consistent with the performance during offline deployment, ensuring that the service finally delivered can achieve the expected results in the business, and effectively reducing the verification cost and communication cost before going online.

[0052] In some embodiments, the working node provides a remote function call interface. During the debugging of the model deployed on the working node, relevant technical personnel call the remote function call interface through the working node to quickly build and verify the model deployed on the working node.

[0053] For any business, the relevant technical personnel, based on the processing flow of the tasks under the business and in accordance with the pre-set configuration specifications, input the model calling order, input-output relationship between models, parameter mapping information and at least one configuration information of the conditional judgment logic during the task execution through the management node. Based on the configuration information input by the relevant technical personnel, the management node generates and stores the orchestration information corresponding to the above business. Accordingly, the model orchestration module in the management node provides a highly configurable orchestration interface. The management node responds to the configuration information input by the user, uses the configuration information input by the user as the input parameter of the orchestration interface, and generates the orchestration information corresponding to the above business based on the configuration information input by the user by calling the interface. Among them, the input-output relationship between models refers to which models the intermediate processing results output by any model during the task execution should be input into for processing, the parameter mapping information refers to which models the parameters carried in the task issued by the user should be input into for processing, and the conditional judgment logic refers to how to determine which branch processing flow should be selected to process the intermediate processing result output by any model when it corresponds to multiple branch processing flows.

[0054] In some embodiments, the above-mentioned model orchestration module provides multiple configuration templates, each configuration template includes pre-set configuration information, and each configuration module corresponds to a template identifier. The management node responds to the template identifier input by the relevant technical personnel, uses the template identifier as the input parameter of the above-mentioned orchestration interface, and generates orchestration information based on the configuration information in the configuration template indicated by the template identifier by calling the orchestration interface, thereby improving the convenience of orchestration information generation and accelerating the iteration speed of the business and the launch speed of new businesses.

[0055] The above process is one possible implementation method for a management node to generate orchestration information based on at least one of the call order, input-output relationships, parameter mapping information, and conditional judgment logic of multiple models corresponding to a business. These multiple models are the models to be called to execute tasks under the aforementioned business. This possible implementation method allows relevant technical personnel to perform simple configuration operations (such as entering configuration information) to implement the orchestration and combination of multiple models through configuration. This allows the management node to quickly construct complex processing flows based on business requirements, achieving configurable service links and eliminating the need for relevant personnel to manually write lengthy business logic code, thus reducing their workload. Furthermore, if business requirements and other factors change, resulting in changes to the processing flow of tasks under the business, relevant technical personnel can simply modify the above configuration information to have the management node regenerate the orchestration information, eliminating the need to modify or rewrite business logic code, thus providing greater flexibility.

[0056] After explaining the deployment process of the distributed system, the solution provided by the present disclosure will be further elaborated based on a specific implementation method, taking the Vincent video task as an example. This specific implementation method is explained by taking the Vincent video task as an example. The multiple processing links of the Vincent video task are the encoding link, the video generation link and the decoding link in sequence. The models to be called in each link are the encoding model, the video generation module and the decoding model. It should be noted that this specific implementation method is only an example. The above-mentioned task can be a task under any business. The processing flow of the task may include more or fewer processing links, and the dependency relationship between the processing links may also be simpler or more complex. The embodiments of the present disclosure do not limit this.

[0057] Figure 4 is a data interaction diagram of a task execution method according to an exemplary embodiment. The method is applied to a distributed system. The distributed system includes a management node and multiple working nodes. One or more models are deployed on each working node, such as Figure 4 As shown, the method includes the following steps.

[0058] In step 401, the management node receives a Vincent video task from the client, and based on the arrangement information corresponding to the Vincent video task, determines that the first processing link of the Vincent video task is the encoding link, and determines from the arrangement information that the model to be called in the encoding link is the encoding model. The Vincent video task includes text entered by the user.

[0059] The Vincent video task carries a business identifier, which indicates the business to which the Vincent video task belongs.

[0060] In the disclosed embodiment, a management node receives a Vincent video task from a client and, based on the service identifier carried by the task, determines that the task belongs to the Vincent video service. The management node then determines from the orchestration information corresponding to the Vincent video service that the first processing step of the task is the encoding step, and that the model to be invoked in the encoding step is the encoding model.

[0061] The above content is explained by taking one model to be called in a processing link as an example. In some embodiments, there are multiple models to be called in a processing link, and the multiple models are respectively used to perform different processing on the task data, or the multiple models are respectively used to process different parts of the task data. The embodiments of the present disclosure do not limit this. Accordingly, the management node determines the multiple models to be called in the processing link of the task based on the scheduling information corresponding to the task. After determining the multiple models to be called in the processing link, the management node sends the task data processed by the model to the multiple models based on the input-output relationship in the scheduling information in a manner similar to the following step 402. For example, the management node determines that the above encoding link needs to call multiple encoding models based on the scheduling information corresponding to the Wensheng video service, and different encoding models are respectively used to encode the partial text carried by the above task. The management node sends the partial text processed by the encoding model to each of the multiple encoding models.

[0062] In the above process, the management node can determine the target model to be called for the task at the processing stage indicated by the processing progress based on the corresponding orchestration information of the task and the task's processing progress. Relevant technical personnel no longer need to write complex business logic code to indicate the target model to be called for each processing stage, which greatly reduces the workload of relevant technical personnel. Moreover, if the processing flow of the task changes, the relevant staff only needs to configure the configuration information so that the management node generates new orchestration information based on the configured configuration information, and then processes the task based on the newly generated orchestration information. There is no need to modify or rewrite the business logic code, which improves the flexibility of task execution.

[0063] The processing progress of the task indicates the processing link in which the task is located. In the above exemplary process, the processing link indicated by the processing progress of the task is the above encoding link, and accordingly, the target model is the above encoding model.

[0064] In step 402, the management node determines the target working node 1 for executing the encoding step from multiple working nodes based on the encoding model and the deployment relationship between the model and the working node, and sends the text in the Wensheng video task and the model identifier of the encoding model to the target working node 1.

[0065] Among them, the above encoding model is deployed on the target working node, which can encode the text carried by the above-mentioned Wensheng video task.

[0066] In the embodiment of the present disclosure, the management node determines at least one working node on which the above-mentioned encoding model is deployed from multiple working nodes of the distributed system based on the encoding model and the deployment relationship between the model and the working node, and then determines the target working node 1 for executing the above-mentioned encoding link from the working node, and sends the model identifier of the text and encoding model in the Wensheng video task to the target working node 1.

[0067] In some embodiments, in the process of determining the target working node 1, the management node randomly determines one of the at least one working nodes as the target working node 1, or the management node maintains the load of each working node, and the management node determines the target working node 1 from the at least one working node based on a pre-set routing policy and the load of each working node in the at least one working node. The load of the working node is reported to the management node by the working node at regular intervals. Accordingly, the working node sends a heartbeat message to the management node at preset intervals. The heartbeat message carries the load of the working node, which indicates the number of tasks to be processed by the working node. The more tasks to be processed by the working node, the greater the load of the working node, and the fewer tasks to be processed by the working node, the smaller the load of the working node. For each working node, after receiving the heartbeat message sent by the working node, the management node uses the load carried in the heartbeat message to update the load maintained for the working node in the management node.

[0068] In some embodiments, in the process of the management node determining the target working node 1 based on the routing strategy and the load of at least one working node, the routing strategy is to prioritize the working node with the smallest load as the target working node, then the management node determines the working node with the smallest load among the above-mentioned at least one working node as the target working node 1, or, the Vincent video task received by the management node carries the priority of the task, and the routing strategy is that for tasks with the first priority, the working node with the smallest load is prioritized as the target working node, and for tasks with the second priority, the target working node is randomly determined from the above-mentioned at least one working node, and the first priority is higher than the second priority, then the management node determines the working node with the smallest load among the above-mentioned at least one working node as the target working node 1 based on the priority of the Vincent video task being the first priority, and randomly determines one of the above-mentioned at least one working node as the target working node 1 based on the priority of the Vincent video task being the second priority.

[0069] In some embodiments, when there are multiple coding models to be invoked during the encoding phase, for each coding model to be invoked, the management node determines, from among the multiple work nodes, a target work node corresponding to the coding model, using a method similar to the above process. The target work node is deployed with the coding model and is used to execute the portion of the coding task corresponding to the coding model during the encoding phase. Based on the input-output relationship in the orchestration information, the management node sends the text to be processed by the coding model and the model identifier of the coding model to the target work node corresponding to the coding model.

[0070] In the above process, the management node determines the target worker node 1 for encoding from among multiple worker nodes based on the deployment relationship and encoding model. It then sends the task data (i.e., the text contained in the aforementioned Wensheng video task) to the target worker node 1 so that it can process the task data. During this process, the management node utilizes intelligent routing services to dynamically adjust the scheduling of worker nodes based on the perceived worker node load. This ensures that task data is routed to the appropriate worker node for processing, improving resource utilization, reducing model inference costs, and addressing performance bottlenecks in high-concurrency and large-scale deployment scenarios.

[0071] The above process is illustrated by taking the example of the management node dynamically adjusting the scheduling of the working node based on the load of the working node. In some embodiments, the heartbeat information sent by the working node to the management node also carries the hardware resource usage of the working node. The management node can also determine the target working node based on the hardware resource usage of each working node in the above at least one working node, or determine the target working node based on the load and hardware resource usage of each working node in the above at least one working node, thereby realizing automatic allocation of tasks according to resource conditions, thereby improving resource utilization and reducing task processing costs. The embodiments of the present disclosure are not limited to this.

[0072] In some embodiments, when the management node determines that the load of each of the at least one working node exceeds a preset load threshold, the management node returns an error message to the client, indicating that the distributed system is busy and temporarily unable to process the received task. That is, the management node controls the intelligent routing service to downgrade the service so that the workflow corresponding to the task processing process returns to normal as soon as possible. The workflow includes multiple working nodes involved in the processing process. The above process is described by downgrading the intelligent routing service to return an error message as an example. Of course, the management node can also control the intelligent routing service to downgrade to random routing, that is, randomly determine the target working node. This is not limited in the embodiments of the present disclosure.

[0073] In step 403, the target working node 1 encodes the received text through the encoding model, and returns the encoding result of the text and the updated processing progress to the management node. The updated processing progress indicates that the Wensheng video task enters the next processing link.

[0074] The text encoding result is the processing result of the above encoding step. The updated processing progress is implemented as completion information. This completion information carries a model identifier, indicating that the model corresponding to the model identifier has completed the task to be processed by the model. This model identifier is sent by the management node to the target working node 1. Alternatively, each model deployed on the target working node 1 corresponds to a model identifier. Accordingly, the model identifier carried in the above completion information is the model identifier of the model executing the task in the target working node 1. This is not limited in the present embodiment.

[0075] In an embodiment of the present disclosure, the target working node 1 receives a model identifier and text from a management node, encodes the received text using the encoding model indicated by the model identifier, obtains the encoding result of the text, and returns the encoding result of the text to the management node. In addition, the target working node 1 adds the model identifier indicating the encoding model to the completion information and returns the completion information to the management node.

[0076] In some embodiments, in the process of the target working node 1 encoding the received text through the encoding model, the target working node 1 uses the received text as the input parameter of the model calling interface corresponding to the encoding model, and calls the encoding model through the model calling interface to process the received text, wherein the model calling interface is also called the local service interface of the target working node, and the operation of the encoding model depends on the computing resources and operating environment provided by the inference engine corresponding to the model type to which the encoding model belongs.

[0077] The above process is a possible implementation method in which the target work node executes the processing link indicated by the above processing schedule based on the model that conforms to the above processing schedule and the received task data. This possible implementation method is explained by taking the target work node directly processing the received task data through the model that conforms to the above processing schedule as an example. In some embodiments, before processing the task data through the model, the target work node pre-processes the received task data. For example, the task data received by the target work node indicates the storage space of the model's to-be-processed data. The target work node then pre-processes the task data, that is, the target work node obtains the model's to-be-processed data from the storage space. Subsequently, the target work node processes the obtained to-be-processed data through the model to execute the corresponding processing link. In some embodiments, after processing the task data through the model, the target work node post-processes the processing result. For example, the target work node stores the processing result in a pre-set storage space. Accordingly, in the process of returning the processing result to the management node, the target work node can return the processing result output by the model to the management node, or can return an address identifier to the management node, the address identifier indicating the storage space of the processing result. The embodiments of the present disclosure are not limited to this.

[0078] The above description uses the example of a working node returning an updated processing progress to a management node. In some embodiments, the working node returns a processing result to the management node, which maintains the execution progress of the aforementioned task. After receiving the processing result from the working node, the management node updates the processing progress of the task based on the received processing result and continues to execute the task based on the updated processing progress. In addition, when a distributed system includes multiple management nodes, the multiple working nodes synchronize the execution progress of the task through information exchange, which is not limited in the present embodiments.

[0079] In step 404, the management node determines that the Wensheng video task is in the video generation phase based on the scheduling information and the received processing progress, and determines from the scheduling information that the model to be called in the video generation phase is the video generation model.

[0080] The processing progress received by the management node indicates that the encoding phase has been completed, and the Wensheng video task is in the next processing phase of the encoding phase, namely, the video generation phase. It should be noted that, if there is only one encoding model to be called in the encoding phase, the management node determines that the encoding phase has been completed when it receives completion information carrying the model identifier of the encoding model, and the Wensheng video task is in the video generation phase. If there are multiple encoding models to be called in the encoding phase, the management node determines that the encoding phase has been completed when it receives completion information corresponding to each of the multiple encoding models. In addition, in some embodiments, the distributed system does not begin processing the video generation phase until the encoding phase is completely completed. If there are multiple encoding models to be called in the encoding phase, after receiving the encoding result output by any encoding model, the management node can directly determine which model to input the encoding result output by the encoding model based on the input-output relationship in the orchestration information, and send the encoding result to the target working node corresponding to the model, without waiting for the encoding results output by each of the multiple encoding models to be received before sending.

[0081] In the embodiment of the present disclosure, the management node receives the encoding results and processing progress from the target working node 1, and based on the received processing progress indication that the encoding link has been completed, determines the next processing link of the encoding link from the scheduling information, that is, the processing link where the Wensheng video task is located is the video generation link, and the model to be called by the video generation link is the video generation model.

[0082] The above process is described by taking the example of a single model to be called in the video generation link. In some embodiments, there are multiple models to be called in the video generation link, which is not limited in the embodiments of the present disclosure.

[0083] In step 405, the management node determines the target working node 2 for executing the video generation link from multiple working nodes based on the video generation model and the deployment relationship between the model and the working node, and sends the text encoding result and the model identifier of the video generation model to the target working node 2.

[0084] Among them, the encoding result obtained through the encoding stage is the task data of the video generation stage.

[0085] This process is similar to the related process of step 402 above, and will not be described in detail in this embodiment of the present disclosure.

[0086] It should be noted that the above content is explained based on the example that the model to be called in the encoding link and the video generation link is only one. In some embodiments, there are multiple encoding models to be called in the encoding link, and there is only one video generation model to be called in the video generation link. Accordingly, the processing result of the encoding link includes the encoding results output by multiple encoding models. Based on the input-output relationship between the multiple encoding models and the video generation model in the orchestration information, the management node sends the encoding results output by the multiple encoding models to the target working node corresponding to the video generation model, and the video generation model deployed in the target working node processes the encoding results output by the multiple encoding models.

[0087] In some embodiments, there is one encoding model to be called in the encoding stage, and there are multiple video generation models to be called in the video generation stage. The management node sends the encoding results to be processed by the video generation model to the target working node corresponding to each video generation model in the multiple video generation models based on the input-output relationship between the encoding model and the multiple video generation models in the orchestration information.

[0088] In some embodiments, there are multiple models to be called in the encoding stage, and there are multiple models to be called in the video generation stage. Accordingly, the processing results of the encoding stage include the encoding results output by multiple encoding models. The management node sends the encoding results to be processed by the video generation model to the target working node corresponding to each video generation model based on the input-output relationship between the multiple encoding models and the multiple video generation models in the orchestration information.

[0089] The above description uses the example of the task data for any processing link being the processing result of the previous processing link. In some embodiments, the task data for that processing link can be the processing result of any preceding processing link, or it can be the input data for the task, such as parameters input by the user for the task. It can also include both the processing result of any preceding processing link and the input data for the task, but this disclosure does not limit this. Accordingly, for any processing link, the management node determines the task data for that processing link based on the input-output relationship in the orchestration information and sends the task data for that processing link to the target work node corresponding to that processing link.

[0090] In step 406, the target working node 2 processes the received encoding result through the video generation model, and returns the video generation result and the updated processing progress to the management node. The updated processing progress indicates that the Wensheng video task enters the next processing link.

[0091] Among them, the video generation result is the processing result of the above-mentioned video generation link.

[0092] This process is similar to the related process of the above-mentioned step 403, and the embodiment of the present disclosure will not be repeated here. It should be noted that, in the embodiment of the present disclosure, the target working node 2 processes the received encoding result through the video generation model indicated by the received model identifier, obtains the video generation result, returns the video generation result to the management node, and adds the model identifier indicating the video generation model to the completion information, and returns the completion information to the management node.

[0093] In step 407, the management node determines that the Vincent video task is in the decoding phase based on the scheduling information and the received processing progress, and determines from the scheduling information that the model to be called in the decoding phase is the decoding model.

[0094] In step 408 , the management node determines a target working node 3 for executing the decoding step from multiple working nodes based on the decoding model and the deployment relationship between the model and the working nodes, and sends the video generation result and the model identifier of the decoding model to the target working node 3 .

[0095] In step 409, the target working node 3 decodes the received video generation result through the decoding model, and returns the decoded video and the updated processing progress to the management node. The updated processing progress indicates that the execution of the Wensheng video task is completed.

[0096] The above steps 407 to 409 are similar to the above steps 404 to 406, and will not be repeated here in this embodiment of the present disclosure.

[0097] In step 410 , the management node receives the video and processing progress from the target working node 3 , and returns the received video to the client.

[0098] In the embodiment of the present disclosure, the management node receives the video and processing progress from the target working node 3, determines based on the processing progress that the above-mentioned Wensheng video task has been completed, and returns the received video to the client.

[0099] The task execution method provided by the disclosed embodiment uses models as the granularity, distributing the multiple models needed to process the task across multiple work nodes. During task execution, the management node, based on the orchestration information indicating the models to be called for each processing link of the model, the task's processing progress, and the deployment of the models on multiple work nodes, can send the task data for the task at the processing link to the target work node capable of executing the processing link. When the task processing flow changes, only the orchestration information needs to be modified, without having to modify or rewrite a large amount of business logic code, thereby improving the flexibility of task execution.

[0100] The above content provides an exemplary description of the task execution method provided in the embodiment of the present disclosure. The following provides an exemplary description of a task processing link based on the architecture of a distributed system. Figure 5 is a flowchart of a task execution method provided by an embodiment of the present disclosure, such as Figure 5 As shown, the distributed system applied by the task execution method includes a management node and a working node. The management node includes a model orchestration module and an intelligent routing module. The working node includes an agent part and a local service part. The agent part is responsible for receiving, parsing and transferring task data, including an input and output module, a control module and a pipeline module. The local service part is used to execute tasks, including a local service module and an inference engine.

[0101] The model orchestration module is used to determine the target model to be invoked for the processing link of the task based on the orchestration information corresponding to the task received by the management node and the task's processing progress. The intelligent routing module is used to determine the target work node from multiple work nodes based on the target model and the deployment relationship between the model and the work node, and to send the task data for the task in the aforementioned processing link to the target work node. By integrating an intelligent routing mechanism into the management node, the management node can automatically allocate tasks based on at least one of the load and hardware resource usage of each work node, thereby improving the overall resource utilization of the distributed system and significantly reducing inference costs.

[0102] The input / output module in the aforementioned working node is the interface for communication between the working node and the management node. It is used to receive task data from the intelligent routing module and send this task data to the control module. In addition, the input / output module is used to output the processing results of the aforementioned task in the aforementioned processing steps to the management node. The control module is the control center of the proxy part, which is used to control the flow of task data within the proxy part. Specifically, it sends the received task data to the pipeline module. After the working node completes the aforementioned processing steps of the aforementioned task based on the task data, it uploads the processing results to the aforementioned input / output module, which then returns the processing results to the management node through the aforementioned input / output module. The pipeline module is used to manage multiple groups of work pipelines, each group of work pipelines corresponding to a business, and is used to execute the specified processing steps of the tasks under that business based on the received task data. Each work pipeline includes an inference module. The inference module is used to call the model call interface corresponding to the model in the local service module based on the model to be called in the aforementioned processing step. The inference module calls the model call interface corresponding to the model in the operating environment provided by the corresponding inference engine to process the received task data and obtain the processing results of the aforementioned processing steps. By deploying the model calling interface in the local service module, the flexibility and efficiency of model deployment are significantly improved.

[0103] In some embodiments, the workflow also includes a preprocessing module, which is used to perform the preprocessing process of the above-mentioned processing link. The preprocessing process is the same as the above-mentioned related content, and the embodiments of the present disclosure will not be repeated here.

[0104] In some embodiments, the workflow also includes a post-processing module, which is used to perform the post-processing process of the above-mentioned processing link. The post-processing process is the same as the above-mentioned related content, and the embodiments of the present disclosure will not be repeated here.

[0105] In some embodiments, the input-output module is further used to output at least one of the load and hardware resource usage of the working node to the management node, which is not limited in the embodiments of the present disclosure.

[0106] In some embodiments, the agent portion of the aforementioned working node further includes a monitoring module, which is configured to monitor the working status of the working node, such as business indicators such as the load of the working node, hardware resource usage, and the processing status of the processing link. The monitoring module is also configured to restart the processing of the processing link if an error is reported in the processing link.

[0107] The above-mentioned distributed system provides functions such as automated deployment, fault tolerance, and monitoring of model services, and can support efficient debugging, development, and operation and maintenance.

[0108] Figure 6 This is a block diagram of a task execution device according to an exemplary embodiment. The device is used to execute the steps of the above-mentioned task execution method. Figure 6 , the task execution device includes the following modules.

[0109] According to another aspect of an embodiment of the present disclosure, there is provided a task execution device, the device comprising: The node determination module 601 is configured to cause a management node in the distributed system to determine a target working node from multiple working nodes in the distributed system based on the orchestration information corresponding to the task, the processing progress of the task, and the deployment relationship between the model and the working node, wherein the processing flow of the task includes multiple processing links, the processing progress indicates the processing link of the task, the orchestration information indicates the model to be called by the task in each processing link, and the target working node is used to execute the processing link indicated by the processing progress on the task; A data sending module 602 is configured to execute the management node to send task data to the target working node, wherein the task data is data of the task at the processing stage indicated by the processing progress; The execution module 603 is configured to enable the target working node to execute the processing link indicated by the processing progress based on the model that conforms to the processing progress and the task data.

[0110] In some embodiments, the apparatus further comprises: The orchestration module is configured to cause the management node to generate the orchestration information based on at least one of a calling sequence, input-output relationships, parameter mapping information, and conditional judgment logic of multiple models corresponding to the business, wherein the task belongs to the business, and the multiple models are models to be called to execute the task.

[0111] In some embodiments, the node determination module 601 includes: a target model determining unit configured to enable the management node to determine a target model to be called by the task at a processing link indicated by the processing progress based on the orchestration information corresponding to the task and the processing progress of the task; The node determination unit is configured to enable the management node to determine, based on the deployment relationship and the target model, a target working node for executing the processing link indicated by the processing progress from the multiple working nodes.

[0112] In some embodiments, the node determination unit is configured to execute: enabling the management node to determine, based on the deployment relationship and the target model, at least one working node on which the target model is deployed from the plurality of working nodes; The management node is enabled to determine the target working node from the at least one working node based on a routing policy and a load of the at least one working node.

[0113] In some embodiments, the apparatus further comprises: The return module is configured to execute the target working node to return the processing result and the updated processing progress to the management node, where the processing result is the processing result of the processing link indicated by the processing progress.

[0114] In some embodiments, multiple inference engines are deployed on each of the multiple working nodes, and different inference engines are used to load different types of models.

[0115] It should be noted that the above embodiments provide devices that perform tasks using the aforementioned functional modules as examples. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, i.e., the internal structure of the device can be divided into different functional modules to perform all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments share the same concept. The specific implementation process is detailed in the method embodiments and will not be further elaborated here.

[0116] Figure 7 This is a schematic diagram of the structure of a computer device provided according to an embodiment of the present disclosure. The computer device 700 may vary significantly due to different configurations or performance, and may include one or more CPUs (Central Processing Units, processors) 701 and one or more memories 702, wherein the memory 702 stores at least one computer program, which is loaded and executed by the processor 701 to implement the task execution methods provided in the above-mentioned various method embodiments. Of course, the computer device may also have components such as a wired or wireless network interface, a keyboard, and input and output interfaces for input and output. The computer device may also include other components for implementing device functions, which will not be described in detail here.

[0117] The present disclosure also provides a computer-readable storage medium that stores at least one computer program. The at least one computer program is loaded and executed by a processor of a computer device to implement the operations performed by the management node or the working node in the task execution method of the above embodiment. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0118] In some embodiments, the computer program involved in the embodiments of the present disclosure may be deployed and executed on one computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network may constitute a blockchain system.

[0119] The present disclosure also provides a computer program product or computer program, which includes computer program code stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium and executes the computer program code, causing the computer device to perform the task execution method provided in the various optional implementations described above.

[0120] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0121] The above description is merely an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A task execution method, characterized in that: The method is applied to a distributed system, the distributed system including a management node and multiple working nodes, each of which has one or more models deployed thereon, and the method includes: The management node determines a target working node from the multiple working nodes based on the orchestration information corresponding to the task, the processing progress of the task, and the deployment relationship between the model and the working node. The processing flow of the task includes multiple processing links. The processing progress indicates the processing link of the task. The orchestration information indicates the model to be called by the task in each processing link. The target working node is used to execute the processing link indicated by the processing progress on the task. The management node sends task data to the target working node, where the task data is data of the processing link of the task indicated by the processing progress; The target working node executes the processing link indicated by the processing progress based on the model that conforms to the processing progress and the task data.

2. The method according to claim 1, characterized in that The process of obtaining the arrangement information includes: The management node generates the orchestration information based on at least one of a calling order, input-output relationships, parameter mapping information, and conditional judgment logic of multiple models corresponding to the business, the task belongs to the business, and the multiple models are models to be called to execute the task.

3. The method according to claim 1, characterized in that The management node determines, based on the scheduling information corresponding to the task, the processing progress of the task, and the deployment relationship between the model and the working node, a target working node from the multiple working nodes, including: The management node determines, based on the orchestration information corresponding to the task and the processing progress of the task, a target model to be called by the task at the processing link indicated by the processing progress; The management node determines, from the plurality of working nodes, a target working node for executing the processing link indicated by the processing progress based on the deployment relationship and the target model.

4. The method according to claim 3, characterized in that The management node determines, based on the deployment relationship and the target model, from the plurality of working nodes a target working node for executing the processing link indicated by the processing progress, including: The management node determines, based on the deployment relationship and the target model, at least one working node on which the target model is deployed from the plurality of working nodes; The management node determines the target working node from the at least one working node based on a routing policy and a load of the at least one working node.

5. The method according to claim 1, wherein The method further comprises: The target working node returns the processing result and the updated processing progress to the management node, where the processing result is the processing result of the processing link indicated by the processing progress.

6. The method according to claim 1, characterized in that Multiple inference engines are deployed on each of the multiple working nodes, and different inference engines are used to load different types of models.

7. A task execution device, characterized in that: The device comprises: a node determination module configured to cause a management node in a distributed system to determine a target working node from a plurality of working nodes in the distributed system based on orchestration information corresponding to a task, a processing progress of the task, and a deployment relationship between a model and a working node, wherein the processing flow of the task includes a plurality of processing links, the processing progress indicates the processing link in which the task is located, the orchestration information indicates the model to be called by the task in each processing link, and the target working node is used to execute the processing link indicated by the processing progress on the task; A data sending module is configured to execute and enable the management node to send task data to the target working node, wherein the task data is data of the processing link of the task indicated by the processing progress; The execution module is configured to enable the target working node to execute the processing link indicated by the processing progress based on the model that conforms to the processing progress and the task data.

8. A distributed system, characterized in that: The distributed system includes the management node and the working node according to any one of claims 1 to 6.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor to execute the method according to any one of claims 1 to 6.

10. A computer equipment cluster, characterized in that: comprising at least one computer device, each computer device comprising a processor and a memory; The processor of the at least one computer device is configured to execute instructions stored in a memory of the at least one computer device, so that the computer device cluster executes the method according to any one of claims 1 to 6.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store at least one computer program, and the at least one computer program is used to execute the method according to any one of claims 1 to 6.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images