Task processing method and device based on size model cooperation, equipment and medium

Abstract

The application discloses a task processing method and device based on size model cooperation, equipment and medium, and relates to the technical field of computer. The method comprises the following steps: task decomposition is performed on a to-be-processed task to obtain a plurality of subtasks, a candidate processing model set corresponding to each subtask is determined according to the task attribute of each subtask, and a first model allocation result is obtained; the first model allocation result is adjusted according to the task quantity of each subtask to obtain a second model allocation result; an association relationship between each subtask is determined, a processing order between each subtask is determined based on the association relationship between each subtask, a cooperative processing mechanism is determined based on the processing order between each subtask and the second model allocation result, and the plurality of subtasks are processed based on the cooperative processing mechanism to obtain a final task processing result. The application can establish an efficient cooperative processing mechanism, and improves the rational use of resources and the task execution processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a task processing method and device based on size model cooperation, equipment and medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, models play an increasingly important role in processing various tasks. Especially in the face of complex tasks, multiple models often need to be cooperatively processed to fully exert their respective advantages and achieve more efficient and more accurate results.

[0003] However, in the existing model-based cooperative task processing mode, the selection of models is relatively fixed, and resources cannot be flexibly allocated according to the characteristics of subtasks, resulting in serious waste of resources. For example, for some simple subtasks that do not require high precision, large models are still used, which not only fails to fully exert their advantages, but also occupies valuable computing resources, increasing the overall task processing cost. In addition, the association between tasks is not fully explored to determine the cooperative processing mode, and there is a lack of effective cooperative processing mechanism, so that the cooperation between models is not efficient enough when multiple tasks are processed in parallel, resulting in low task processing efficiency. SUMMARY

[0004] The present application provides a task processing method and device based on size model cooperation, equipment and medium, which solves the problem of low task processing efficiency and unreasonable resource utilization in the existing model-based cooperative task processing mode, and establishes an efficient cooperative processing mechanism, improving the rational use of resources and task execution processing efficiency.

[0005] In a first aspect, the present application provides a task processing method based on size model cooperation, comprising:

[0006] Obtaining a to-be-processed task, performing task decomposition on the to-be-processed task to obtain a plurality of subtasks, determining a candidate processing model set corresponding to each subtask according to the task attribute of each subtask, and obtaining a first model allocation result;

[0007] Adjusting the first model allocation result according to the task amount of each subtask to obtain a second model allocation result;

[0008] Determining the association between each subtask, determining the processing order between each subtask based on the association between each subtask, determining a cooperative processing mechanism based on the processing order between each subtask and the second model allocation result, and processing a plurality of subtasks based on the cooperative processing mechanism to obtain a final task processing result.

[0009] Optionally, the adjusting the first model allocation result according to the task amount of each subtask to obtain a second model allocation result comprises:

[0010] quantitatively evaluating the task amount of each subtask to obtain a task quantitative value of each subtask;

[0011] determining a processing capability threshold of each candidate processing model in a candidate model set corresponding to use of each subtask;

[0012] calculating a matching degree between each subtask and each candidate processing model in the candidate model set corresponding to use of the subtask based on the task quantitative value and the processing capability threshold;

[0013] determining a final processing model corresponding to each subtask from the candidate processing model set corresponding to use of each subtask based on the matching degree to obtain a second model allocation result.

[0014] Optionally, the determining the association relationship between each subtask comprises:

[0015] determining an overall target of the to-be-processed task and a target of each subtask;

[0016] determining an execution precondition of each subtask based on the overall target of the to-be-processed task and the target of each subtask;

[0017] determining a dependent association relationship between each subtask based on the execution precondition.

[0018] Optionally, the determining the processing order between each subtask based on the association relationship between each subtask comprises:

[0019] constructing a subtask relationship directed graph based on the dependent association relationship between each subtask; wherein, a node in the directed graph is the subtask, and a directed edge is a dependent direction;

[0020] topologically sorting each node of the directed graph to obtain the processing order between each subtask.

[0021] Optionally, the determining the cooperative processing mechanism based on the processing order between each subtask and the second model allocation result comprises:

[0022] obtaining a processing model corresponding to each subtask in the second model allocation result to establish a mapping relationship table between the subtask and the processing model;

[0023] constructing a subtask processing time sequence chain based on the processing order between each subtask;

[0024] Based on the mapping table and the subtask processing sequence chain, determine the model interaction relationship between the processing model corresponding to the preceding subtask and the processing model corresponding to the following subtask.

[0025] Based on the subtask processing sequence chain and the model interaction relationship, determine the calling order of the processing model corresponding to each subtask;

[0026] Obtain the current resource status of the system, and determine the collaborative processing mechanism based on the resource status and the calling order of the processing model.

[0027] Optionally, determining the calling order of the processing model corresponding to each subtask based on the subtask processing sequence chain and the model interaction relationship includes:

[0028] Based on the subtask processing sequence chain, the temporal order of each subtask is determined;

[0029] Based on the model interaction relationship, the data flow direction between the processing model corresponding to the preceding subtask and the processing model corresponding to the subsequent subtask is determined.

[0030] Based on the temporal sequence of the subtasks and the data flow between models, the calling order of the processing model corresponding to each subtask is determined.

[0031] Optionally, determining the collaborative processing mechanism based on the resource status and the calling order of the processing model includes:

[0032] Determine the model type for each of the processing models, wherein different model types correspond to different resource requirement parameters;

[0033] Based on the calling order, determine whether the current resource status of the system meets the resource requirement parameters required when processing is performed according to the calling order of the processing model;

[0034] If the current resource status of the system meets the resource requirement parameters required to execute processing according to the calling order of the processing model, a collaborative processing mechanism is determined based on the calling order of the processing model.

[0035] If the current resource status of the system does not meet the resource requirements for executing processing according to the calling order of the processing model, the calling order of the processing model is adjusted according to the model type and the current resource status of the system, and a collaborative processing mechanism is determined based on the adjustment result.

[0036] Secondly, this application provides a task processing device based on a size model for collaborative processing, comprising:

[0037] The first allocation module is used to obtain the task to be processed, decompose the task to be processed into multiple sub-tasks, determine the candidate processing model set corresponding to each sub-task according to the task attributes of each sub-task, and obtain the first model allocation result.

[0038] The second allocation module is used to adjust the first model allocation result according to the task quantity of each sub-task to obtain the second model allocation result;

[0039] The task processing module is used to determine the relationship between each of the subtasks, determine the processing order between each subtask based on the relationship between each subtask, determine a collaborative processing mechanism based on the processing order between each subtask and the second model allocation result, and process multiple subtasks based on the collaborative processing mechanism to obtain the final task processing result.

[0040] Thirdly, this application provides a task processing device based on a size model for collaborative processing, comprising:

[0041] One or more processors;

[0042] A memory that stores one or more programs that, when executed by one or more processors, enable the one or more processors to implement the size-model-based collaborative task processing method as described in the first aspect.

[0043] Fourthly, this application provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the size-model-based collaborative task processing method as described in the first aspect.

[0044] In summary, compared with the prior art, the effective effects of the technical solution provided in this application include at least the following:

[0045] In the present application, a task to be processed is obtained, task decomposition is performed on the task to be processed to obtain a plurality of subtasks, a candidate processing model set corresponding to each subtask is determined according to the task attribute of each subtask, a first model allocation result is obtained; the first model allocation result is adjusted according to the task amount of each subtask to obtain a second model allocation result; the association relationship between each subtask is determined, the processing order between each subtask is determined based on the association relationship between each subtask, the collaborative processing mechanism is determined based on the processing order between each subtask and the second model allocation result, and the plurality of subtasks are processed based on the collaborative processing mechanism to obtain a final task processing result. In the above technical means, the model allocation is flexibly adjusted according to the task amount of different subtasks, the model resources can be more accurately allocated, the processing capacity of the model is matched with the task amount, and the rationality of resource utilization is improved; and the association relationship between the subtasks is determined, the processing order between the subtasks is determined based on the association relationship, and then the collaborative processing mechanism is determined in combination with the second model allocation result, so that the association relationship between the tasks can be fully tapped, the processing order can be reasonably arranged, and the plurality of subtasks can be processed in an orderly and efficient manner, thereby significantly improving the task execution processing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a flowchart of a task processing method based on size model collaboration provided by an embodiment of the present application;

[0047] Figure 2 is a flowchart of adjusting the first model allocation result to obtain the second model allocation result provided by an embodiment of the present application;

[0048] Figure 3 is a flowchart of determining the association relationship between each subtask provided by an embodiment of the present application;

[0049] Figure 4 is a flowchart of determining the processing order between each subtask provided by an embodiment of the present application;

[0050] Figure 5 is a flowchart of determining the collaborative processing mechanism based on the processing order between each subtask and the second model allocation result provided by an embodiment of the present application;

[0051] Figure 6 is a timing diagram provided by an embodiment of the present application;

[0052] Figure 7 is a flowchart of determining the calling order of the processing model corresponding to each subtask provided by an embodiment of the present application;

[0053] Figure 8 is a flowchart of determining the collaborative processing mechanism according to the resource state and the calling order of the processing model provided by an embodiment of the present application;

[0054] Figure 9 is a structural schematic diagram of a task processing device based on size model cooperation provided by an embodiment of the present application;

[0055] Figure 10 is a structural schematic diagram of a task processing device based on size model cooperation provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purposes, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in combination with the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only parts related to the present application are shown in the drawings, but not all contents. Before discussing the example embodiments in more detail, it should be mentioned that some example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0057] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a class, not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in an "or" relationship.

[0058] In the existing model-based collaborative task processing mode, the selection of the model is relatively fixed, and resources cannot be flexibly allocated according to the characteristics of the subtasks, resulting in serious waste of resources. For example, for some simple subtasks with low precision requirements, large models are still used, which not only fails to fully play their advantages, but also occupies valuable computing resources, increasing the overall task processing cost. In addition, the association between tasks is not fully exploited to determine the collaborative processing mode, and there is a lack of effective collaborative processing mechanism, so that the cooperation between models is not efficient when multiple tasks are processed in parallel, resulting in low task processing efficiency.

[0059] To solve the problems of low task processing efficiency and unreasonable resource utilization in the existing model-based collaborative task processing mode, the embodiments of the present application provide a task processing method based on size model collaboration. By flexibly adjusting model allocation according to the task amount of different subtasks, the model resources can be more accurately allocated, so that the processing capacity of the model matches the task amount, and the rationality of resource utilization is improved. By determining the processing order between subtasks based on the association relationship between the subtasks, and then determining the collaborative processing mechanism combined with the second model allocation result, the association relationship between tasks can be fully tapped, the processing order can be reasonably arranged, and multiple subtasks can be processed in an orderly and efficient manner, thereby significantly improving the task execution processing efficiency.

[0060] The task processing device based on size model collaboration can be installed with an operating system, where the operating system includes but is not limited to an Android system, a Linux system or a Windows system. The task processing device based on size model collaboration can be installed with an application based on the operating system, where the application can be an application provided by the operating system, or an application downloaded from a third-party device or a server. In this embodiment, the task processing device based on size model collaboration has at least an application that can execute the task processing method based on size model collaboration.

[0061] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0062] Figure 1 A flowchart of the task processing method based on size model collaboration provided by the embodiments of the present application is given. Please refer to Figure 1 The task processing method based on size model collaboration specifically includes:

[0063] S110, obtaining a to-be-processed task, performing task decomposition on the to-be-processed task to obtain a plurality of subtasks, determining a candidate processing model set corresponding to each subtask according to the task attribute of each subtask, and obtaining a first model allocation result.

[0064] Wherein, the to-be-processed task is a task that needs to be processed. For example, the to-be-processed task can be an image recognition task, a natural language processing task, a data processing task, etc.

[0065] In order to more efficiently process the to-be-processed task, it is necessary to decompose the to-be-processed task. The purpose of task decomposition is to split a complex task into multiple subtasks so that subsequent processing models can better process each subtask. For example, the task decomposition of the to-be-processed task in this application can be based on different decomposition dimensions. First, the decomposition dimension of the to-be-processed task is determined, and the to-be-processed task is decomposed into multiple preliminary subtasks based on the decomposition dimension. The decomposition dimension refers to different angles, standards or features that are used when decomposing a complex task, and is the basis for dividing a whole task into multiple subtasks. For example, the decomposition dimension can include the functional module dimension of the task, the task type dimension, the task data feature dimension, or the task execution stage dimension, etc.

[0066] After decomposing the to-be-processed task into multiple subtasks, it is also necessary to determine the corresponding candidate processing model according to the task attributes of the subtasks.

[0067] For example, the step of determining the candidate processing model corresponding to each subtask includes:

[0068] Obtaining the task attributes of each subtask and the model attributes of each processing model in the model library; determining the matching relationship between each subtask and each processing model based on the task attributes of each subtask and the model attributes of each processing model; and determining the candidate processing model corresponding to each subtask based on the matching relationship.

[0069] The task attributes refer to various information that can describe the characteristics, requirements or execution requirements of the subtasks, which are used to select the appropriate processing model. For example, the task attributes can be data type attributes (such as data type attributes, text data, image data, audio data, etc.), and can also be task target attributes (such as classification tasks, regression tasks, clustering tasks, detection tasks, etc.). The model attributes refer to information about the characteristics or application scope of the processing model. For example, the model attributes can be model applicability attributes, which indicate which tasks the model is suitable for or good at (such as image tasks or text tasks).

[0070] By determining the candidate processing model according to the task attributes, the application can quickly filter out the models that do not match the characteristics of the subtasks, reducing the range of candidate processing models. For example, when the data type of the subtask is image data, models that are only suitable for text or numerical data can be excluded, and models suitable for image processing can be focused on, thereby reducing the time cost of model selection and improving the efficiency of the preparation stage of task processing.

[0071] S120, adjusting the first model allocation result to obtain a second model allocation result according to the task amount of each subtask.

[0072] In the embodiment of the present application, the task amount refers to a processing scale of a data object required for processing a subtask. The size of the task amount directly affects the selection adaptability of the model. For a subtask with a large task amount, a large model is selected from the candidate processing models for processing, thereby preferentially guaranteeing the processing efficiency and stability of the model. For a subtask with a small task amount, a small model is selected from the candidate processing models for processing. The large model has a large parameter scale and high computational complexity, and can efficiently and stably complete the processing of large-scale complex tasks. The small model has a small parameter scale, low computational complexity, and less resource demand, and is suitable for processing small-scale simple tasks. The embodiment of the present application can flexibly and reasonably adjust the model allocation according to the task amount of different subtasks, so that the large model processes large-scale complex tasks and the small model processes small-scale simple tasks, which can more accurately match the processing capability of the model with the task demand, avoid resource waste and performance bottleneck, and improve the rationality of resource utilization.

[0073] Optionally, Figure 2 is a flowchart of the step of adjusting the first model allocation result to obtain the second model allocation result provided by the embodiment of the present application. As shown in Figure 2 , the step of adjusting the first model allocation result to obtain the second model allocation result specifically includes S1201-S1204:

[0074] S1201, quantitatively evaluate the task amount of each subtask to obtain a task quantitative value of each subtask.

[0075] In the embodiment of the present application, the task quantitative value is a quantitative parameter for representing the number scale of the data object processed by the subtask. The higher the task quantitative value is, the more data objects or the higher complexity the subtask needs to process, and the larger the processing scale is.

[0076] In the embodiment of the present application, the step of quantitative evaluation can include: determining the data object type and the counting unit processed by the subtask, for example, for an image classification subtask, the data object type is an image, and the counting unit can be the number of images, and for a data processing subtask, the data object can be a data record or a data point, etc. Since the processing complexity of different data objects is different, a complexity weight needs to be allocated for each data object type, which can be determined based on the intrinsic characteristics of the data object (such as data size, structural complexity, processing difficulty), or can be obtained through historical data. Based on the number of data objects and the complexity weight, the task amount of each subtask is quantitatively evaluated to obtain the task quantitative value of each subtask, which can be calculated by the following formula: task quantitative value = ∑(data object number × complexity weight).

[0077] S1202, determine the processing capability threshold of each candidate processing model in the candidate model set corresponding to each subtask.

[0078] The processing capability threshold is a quantitative index for representing the maximum tolerable workload of the processing model in a specific task scenario, which can stably and efficiently process the subtask (such as the upper limit of the data volume that the processing model can effectively process). For example, the step of determining the processing capability threshold of each candidate processing model in the candidate model set corresponding to each subtask can include: first, obtaining the model attributes of the candidate processing model, including the model type, architecture parameters and performance data; second, based on the processing model attributes and the task data processed by the processing model in the past, evaluating the maximum stable processing capability of each processing model when processing similar subtasks, for example, by determining the maximum data input amount or the maximum concurrent processing number of the processing model under the condition of maintaining stable operation, to determine the processing capability threshold of the processing model.

[0079] S1203, based on the task quantitative value and the processing capability threshold, calculate the matching degree between each subtask and each candidate processing model in the candidate processing model set corresponding to the subtask.

[0080] Further, the task quantitative value and the processing capability threshold are normalized respectively, and the matching degree between each subtask and each candidate processing model in the candidate processing model set corresponding to the subtask is determined according to the normalized task quantitative value and the normalized processing capability threshold. Specifically, the absolute value of the difference between the normalized task quantitative value and the normalized processing capability threshold is calculated, and the matching degree is determined according to the absolute value, and the formula is: matching degree = 1- absolute value; the more matched the subtask demand and the model capability are, the higher the matching degree is.

[0081] S1204, based on the matching degree, determine the final processing model corresponding to each subtask from the candidate processing model set corresponding to each subtask, and obtain the allocation result of the second model.

[0082] S130, determine the association relationship between each subtask, determine the processing order between each subtask based on the association relationship between each subtask, determine the cooperative processing mechanism based on the processing order between each subtask and the second model allocation result, and process multiple subtasks based on the cooperative processing mechanism to obtain the final task processing result.

[0083] In the embodiments of the present application, in order to fully exploit the association relationship between tasks, the processing order between subtasks is reasonably arranged based on the association relationship, so that multiple subtasks can be processed in an orderly and efficient manner, avoiding the problem of low cooperation efficiency in parallel processing of multiple tasks due to the lack of effective cooperative processing mechanism in the prior art. The cooperative processing mechanism is determined by the processing order between each subtask and the second model allocation result, thereby significantly improving the task execution processing efficiency.

[0084] Optionally, Figure 3 is a flowchart for determining the association relationship between each subtask provided by the embodiments of the present application. As shown in Figure 3 , the step of determining the association relationship between each subtask specifically includes S1301-S1303:

[0085] S1301, determining the overall goal of the to-be-processed task and the goal of each subtask.

[0086] The overall goal of the to-be-processed task refers to the result or purpose ultimately expected to be achieved by the entire task, reflecting the achievement ultimately expected to be achieved by the to-be-processed task. For example, in an image processing task, the overall goal is to classify and label image data. After the to-be-processed task is decomposed into multiple subtasks, the goal of each subtask is the specific task or responsibility assumed by each subtask, i.e., the subtask goal focuses on a certain aspect or stage of the task; for example, in the above image processing task, the goal of one subtask can be to classify image data, and the goal of another subtask can be to label image data.

[0087] S1302, determining the execution precondition of each subtask based on the overall goal of the to-be-processed task and the goal of each subtask.

[0088] The execution precondition refers to a condition or state that must be met when a subtask starts execution (such as a subtask depending on the output result of another subtask); specifically, the specific responsibilities and contributions assumed by each subtask in achieving the overall goal are identified; secondly, the input data required by each subtask in the execution process and the output data generated are analyzed, i.e., for each subtask, the data source on which its input data depends is determined, and the destination of the output data generated after the execution of the subtask is determined; based on the data source on which the input data of each subtask depends and the destination of the output data generated after the execution of the subtask, the execution precondition of each subtask can be derived, for example, if the processing of subtask B needs to depend on the output data of subtask A, the execution precondition of subtask B is defined as subtask A has completed processing and output valid results, and similarly, if a subtask needs the output of multiple previous subtasks as input, its execution precondition is that all these previous subtasks have been completed.

[0089] S1303, determine the dependency association relationship between each of the sub-tasks based on the execution precondition.

[0090] The dependency association relationship refers to logical association of prior execution and subsequent execution or achievement supply and demand between multiple sub-tasks due to constraints of the execution precondition, and reflects mutual restrictive relationship of the sub-tasks in time sequence or data flow.

[0091] According to the determined dependency association relationship between each of the sub-tasks, the processing sequence between each of the sub-tasks is determined. Figure 4 is a flowchart for determining the processing sequence between each of the sub-tasks provided by the embodiment of the present application. As shown in the figure, the step of determining the processing sequence between each of the sub-tasks specifically includes S1304-S1305: Figure 4

[0092] S1304, construct a sub-task relationship directed graph based on the dependency association relationship between each of the sub-tasks; wherein the node in the directed graph is the sub-task, and the directed edge is the dependency direction.

[0093] The directed graph is a graph composed of nodes and directed edges (connection lines with direction), and is used to intuitively present the directional association relationship between elements.

[0094] In the embodiment of the present application, the step of constructing the sub-task relationship directed graph includes: generating a unique corresponding node for each sub-task to form a node set; secondly, based on the determined dependency association relationship, traversing all the sub-tasks, identifying the sub-task combination with dependency relationship; for each sub-task combination with dependency relationship, creating a directed edge, i.e. from the prior node providing output to the subsequent node depending on the output, obtaining a directed edge set, combining the node set and the directed edge set to construct a complete directed graph.

[0095] The node is used to uniquely identify each sub-task, and the directed edge represents the one-way dependency relationship between the nodes, and the direction of the edge is from the prior sub-task to the subsequent sub-task. For example, if the sub-task B depends on the output of the sub-task A, there is a directed edge from A to B (denoted as A→B).

[0096] S1305, topologically sort each node of the directed graph to obtain the processing sequence between each of the sub-tasks.

[0097] ​In the embodiment of the present application, the topological sorting refers to a method of linearly sorting the vertices of a directed graph to ensure that each node in the sequence is arranged before all its postorder nodes. The steps are as follows: traversing the directed graph, counting the in-degree (i.e. the number of directed edges pointing to the node) of each node (subtask), and the in-degree of 0 represents the starting task that does not depend on any other subtask; scanning all nodes and adding all nodes with a current in-degree value of 0 to a preset processable queue. When the processable queue is not empty, the following operations are continuously executed in a loop: taking a current node from the head of the processable queue and adding the node to the end of the topological sequence list; traversing all direct postorder nodes of the current node and performing the operation of reducing the in-degree value of each postorder node by 1; if the in-degree value of a postorder node becomes 0 after reduction, the postorder node is immediately added to the processable queue. The process is iterated continuously until the processable queue becomes an empty queue. When the processable queue becomes an empty queue, the current topological sequence list is obtained, the entire topological sequence list is divided into several processing levels according to the arrangement order of the nodes in the topological sequence list, and the first node is analyzed from the starting position of the topological sequence list as the first processing level. For each node in the topological sequence list, if all the preorder nodes of the node are in the current processing level or an earlier processing level, the node is included in the current processing level, otherwise a new processing level needs to be established. The multiple nodes in the same processing level are identified as being able to be executed in parallel, and the nodes in different processing levels need to be executed in series in order of levels. For example, for four nodes [A, B, C, D] included in the topological sequence, if the level relationship obtained by analysis is that the first processing level includes node A, the second processing level includes nodes B and C, and the third processing level includes node D, the final determined processing order is: first, execute the subtask corresponding to node A; after node A is completed, execute the subtasks corresponding to nodes B and C in parallel (i.e. simultaneously); after nodes B and C are both completed, execute the subtask corresponding to node D.

[0098] Optionally, Figure 5 is a flowchart of a cooperative processing mechanism based on the processing order between each subtask and the second model allocation result provided by the embodiment of the present application. As shown in Figure 5 , the steps of determining the cooperative processing mechanism based on the processing order between each subtask and the second model allocation result specifically include S1306-S1310:

[0099] S1306, obtain the processing model corresponding to each subtask in the second model allocation result, and establish a mapping relationship table between the subtask and the processing model.

[0100] The mapping table is a structured data table used to record and manage the one-to-one or many-to-one relationship between each subtask and the corresponding processing model in the second model allocation result. It can intuitively show which processing model is responsible for processing each subtask.

[0101] S1307. Based on the processing order between each of the subtasks, construct a subtask processing sequence chain.

[0102] Processing order refers to the sequential arrangement of multiple subtasks during execution. Determining the processing order of subtasks is crucial for the correct execution and efficient completion of a task. For example, processing order can be sequential (subtasks have dependencies on each other, and the result of a previous subtask is the input prerequisite for a subsequent subtask, so they must be executed in order; for example, in a data processing task containing multiple subtasks, if a subtask depends on the output of another subtask as input, then these two subtasks must be processed in the order of first executing the subtask that provides the input result, and then executing the subtask that depends on the result), parallel (subtasks have no direct dependencies on each other, and can be executed independently and simultaneously without affecting each other), etc.

[0103] A time-series chain is an ordered sequence of multiple subtasks organized according to their processing order, which intuitively shows the execution order of each subtask in the time dimension.

[0104] For example, Figure 6 This is a schematic diagram of a timing chain provided in an embodiment of this application. Assume there are three subtasks A, B, and C. Based on the relationships between these subtasks, it is determined that subtasks A and B have no direct dependency and can be executed independently and simultaneously; this is a parallel processing order. Subtasks A and C, and B and C, have dependencies and are processed sequentially. Therefore, based on this processing order, a timing chain is constructed, as follows: Figure 6 The timing chain diagram shown is shown.

[0105] S1308. Based on the mapping relationship table and the subtask processing sequence chain, determine the model interaction relationship between the processing model corresponding to the preceding subtask and the processing model corresponding to the subsequent subtask.

[0106] Among them, model interaction relationship refers to the association pattern or interaction logic between processing models corresponding to different sub-tasks. It is used to describe whether there are dependencies or data transfer relationships between models (for example, the interaction relationship formed between the processing model corresponding to the preceding sub-task and the processing model corresponding to the subsequent sub-task due to data dependency or control dependency). Its function is to clarify which models need to interact.

[0107] S1309, determine the calling sequence of the processing model corresponding to each subtask according to the subtask processing time sequence chain and the model interaction relationship.

[0108] The calling sequence of the processing model refers to the order of starting the processing model corresponding to each subtask, which ensures that the processing model is executed in order according to the logical dependency and data flow requirements, and avoids data loss, resource conflict or abnormal processing results caused by incorrect starting time.

[0109] Optionally, Figure 7 is a flowchart provided by the embodiment of the present application for determining the calling sequence of the processing model corresponding to each subtask. As shown in the figure, Figure 7 the step of determining the calling sequence of the processing model corresponding to each subtask specifically includes S13091-S13093:

[0110] S13091, determine the order of each subtask in time sequence based on the subtask processing time sequence chain.

[0111] S13092, determine the data flow between the processing model corresponding to the previous subtask and the processing model corresponding to the subsequent subtask based on the model interaction relationship.

[0112] The data flow refers to the specific form of the model interaction relationship, which refers to the direction and path of data transmission from one model to another between models with interaction relationship; for example, the output data of model Y is transmitted to model Z.

[0113] S13093, determine the calling sequence of the processing model corresponding to each subtask based on the time sequence order of the subtask and the data flow between the models.

[0114] For example, assume that a task is divided into three subtasks A, B and C, subtask A corresponds to processing model X, subtask B corresponds to processing model Y, and subtask C corresponds to processing model Z; based on the subtask processing time sequence chain, it is determined that A and B are in parallel relationship (no sequence dependency), and C is a subsequent task of A and B (needs to be executed after A and B are completed); the data flow is "output of X→input of Z" and "output of Y→input of Z", that is, the input of Z depends on the output data of X and Y, while there is no data flow between X and Y (no mutual dependency), based on which it can be obtained that X and Y can be executed synchronously under the condition of resource permission, that is, X and Y can be called simultaneously, and Z can be called only after X and Y have completed data output, that is, the calling sequence of Z needs to lag behind the completion time of X and Y; finally, the determined model calling sequence is: start X and Y synchronously, and start Z after X and Y have completed execution and output data.

[0115] S13010, acquire the current resource state of the system, and determine the cooperative processing mechanism according to the resource state and the calling sequence of the processing model.

[0116] The system resource state refers to the current availability of hardware and software resources in the computing device environment when performing a task. For example, the resource state can include computing power resources, storage space resources, network resources, and the like.

[0117] Optionally, Figure 8 is a flowchart provided by the embodiment of the present application for determining the cooperative processing mechanism according to the resource state and the calling sequence of the processing model. As shown in the figure, the steps of determining the cooperative processing mechanism according to the resource state and the calling sequence of the processing model specifically include S130101-S130104: Figure 8

[0118] S130101, determine the model type of each processing model, wherein different types of models correspond to different resource demand parameters.

[0119] The model type refers to a category classified according to the structural size, functional complexity, resource consumption characteristics, and applicable scenarios of the processing model. Different model types have different demands for system resources (such as computing power, storage space, network bandwidth, etc.) when processing tasks. Therefore, in order to reasonably allocate system resources and optimize task processing efficiency, it is necessary to determine the model type of each processing model and its corresponding resource demand parameter. The resource demand parameter refers to the quantitative demand of the processing model for system resources when performing a task (such as computing power demand parameter, storage space demand parameter, network bandwidth demand parameter).

[0120] S130102, judge whether the current resource state of the system meets the resource demand parameters required when processing according to the calling sequence of the processing model.

[0121] Specifically, after determining the calling sequence of the processing model (such as synchronously starting models X and Y→starting model Z after both are completed), the system needs to compare the resource demand of each stage under this sequence with the actual callable resource state, and verify whether the system has the resource support capability to complete the processing task of all models according to this sequence. This step can identify resource gaps in advance, and avoid task interruption, processing delay or abnormal results due to insufficient resources.

[0122] S130103, in the case where the current resource state of the system meets the resource demand parameters required when processing according to the calling sequence of the processing model, determine the cooperative processing mechanism based on the calling sequence of the processing model.

[0123] ​The collaborative processing mechanism refers to a model scheduling strategy designed to efficiently execute multiple sub-tasks, maximizing task execution efficiency while ensuring the rational utilization of system resources and the correct completion of tasks. For example, the final determined model invocation order might be: simultaneously start models X and Y, and start model Z after both models X and Y have completed execution and output data; if the current system resource status meets the resource requirements for processing according to the model invocation order, then the model scheduling strategy will be based on this model invocation order.

[0124] S130104. If the current resource status of the system does not meet the resource requirement parameters required for processing according to the calling order of the processing model, the calling order of the processing model is adjusted according to the model type and the current resource status of the system, and a collaborative processing mechanism is determined based on the adjustment result.

[0125] Specifically, when the current resource status of the system cannot support the original processing model calling order, dynamic adjustments need to be made based on the model type and the current real-time resource status. The adjustment process includes: analyzing the resource requirements of each processing model under the original calling order, comparing the currently available computing resources, storage resources and network resources of the system, determining the specific processing stage and corresponding processing model with insufficient resources (resource bottleneck level), and adjusting the original processing model calling order based on the resource bottleneck level and the resource requirement parameters required for the processing model to run. For example, a serialization adjustment is implemented, changing the original parallel processing order to serial processing. For example, the original parallel execution of model X and model Y is changed to executing X first and then Y.

[0126] In summary, the task processing method based on big-small model collaboration provided in this application involves: acquiring a task to be processed; decomposing the task into multiple subtasks; determining a set of candidate processing models for each subtask based on its task attributes to obtain a first model allocation result; adjusting the first model allocation result based on the workload of each subtask to obtain a second model allocation result; determining the relationships between each subtask; determining the processing order of each subtask based on these relationships; determining a collaborative processing mechanism based on the processing order and the second model allocation result; and processing multiple subtasks based on the collaborative processing mechanism to obtain the final task processing result. In this technical approach, flexibly adjusting model allocation according to the workload of different subtasks allows for more precise allocation of model resources, ensuring that the model's processing capacity matches the workload and improving the rationality of resource utilization. Furthermore, by identifying the relationships between subtasks and determining their processing order, and then combining this with the second model allocation result to determine the collaborative processing mechanism, the method fully explores the relationships between tasks, rationally arranges the processing order, and enables multiple subtasks to be processed in an orderly and efficient manner, thereby significantly improving task execution efficiency.

[0127] Figure 9 This is a schematic diagram of a task processing device based on a size model collaboration, provided as an embodiment of this application. (Reference) Figure 9 The task processing device based on size model collaboration provided in this embodiment specifically includes: a first allocation module 21, a second allocation module 22, and a task processing module 23.

[0128] The first allocation module 21 is used to obtain the task to be processed, decompose the task to be processed into multiple sub-tasks, determine the candidate processing model set corresponding to each sub-task according to the task attributes of each sub-task, and obtain the first model allocation result.

[0129] The second allocation module 22 is used to adjust the first model allocation result according to the task quantity of each sub-task to obtain the second model allocation result;

[0130] The task processing module 23 is used to determine the relationship between each of the sub-tasks, determine the processing order between each sub-task based on the relationship between each of the sub-tasks, determine a collaborative processing mechanism based on the processing order between each sub-task and the second model allocation result, and process multiple sub-tasks based on the collaborative processing mechanism to obtain the final task processing result.

[0131] Based on the above embodiments, the second allocation module 22 includes:

[0132] A quantitative evaluation unit is used to quantitatively evaluate the task quantity of each sub-task and obtain a quantitative task value for each sub-task.

[0133] A threshold determination unit is used to determine the processing capability threshold of each candidate processing model in the candidate model set used for each sub-task;

[0134] The matching degree calculation unit is used to calculate the matching degree between each subtask and each candidate processing model in the corresponding candidate processing model set based on the task quantization value and the processing capacity threshold.

[0135] The second allocation unit is used to determine the final processing model corresponding to each subtask from the set of candidate processing models used by each subtask based on the matching degree, and to obtain the allocation result of the second model.

[0136] Based on the above embodiments, the task processing module 23 includes an association relationship determination unit, which includes:

[0137] The target determination subunit is used to determine the overall target of the task to be processed and the target of each of the subtasks.

[0138] The condition determination subunit is used to determine the execution preconditions for each subtask based on the overall goal of the task to be processed and the goal of each subtask.

[0139] The relationship determination subunit is used to determine the dependency relationships between the subtasks based on the execution preconditions.

[0140] Based on the above embodiments, the task processing module 23 includes a processing order determination unit, which includes:

[0141] A directed graph construction subunit is used to construct a directed graph of subtask relationships based on the dependency relationships between the subtasks; wherein, the nodes in the directed graph are the subtasks, and the directed edges are the dependency directions;

[0142] The processing order determination subunit is used to perform topological sorting on each node of the directed graph to obtain the processing order between each subtask.

[0143] Based on the above embodiments, the task processing module 23 includes a collaborative processing mechanism determination unit, which includes:

[0144] A relationship table is established as a sub-unit to obtain the processing model corresponding to each sub-task in the second model allocation result, and to establish a mapping relationship table between the sub-task and the processing model.

[0145] The timing chain construction subunit is used to construct a subtask processing timing chain based on the processing order between each of the subtasks;

[0146] The interaction relationship determination subunit is used to determine the model interaction relationship between the processing model corresponding to the preceding subtask and the processing model corresponding to the subsequent subtask based on the mapping relationship table and the subtask processing sequence chain.

[0147] The calling order determination subunit is used to determine the calling order of the processing model corresponding to each subtask based on the subtask processing sequence chain and the model interaction relationship;

[0148] A collaborative processing mechanism subunit is defined to obtain the current resource status of the system and determine the collaborative processing mechanism based on the resource status and the calling order of the processing model.

[0149] Based on the above embodiments, determining the calling order subunit includes:

[0150] Based on the subtask processing sequence chain, the temporal order of each subtask is determined;

[0151] Based on the model interaction relationship, the data flow direction between the processing model corresponding to the preceding subtask and the processing model corresponding to the subsequent subtask is determined.

[0152] Based on the temporal sequence of the subtasks and the data flow between models, the calling order of the processing model corresponding to each subtask is determined.

[0153] Based on the above embodiments, the collaborative processing mechanism subunit is determined to include:

[0154] Determine the model type for each of the processing models, wherein different model types correspond to different resource requirement parameters;

[0155] Based on the calling order, determine whether the current resource status of the system meets the resource requirement parameters required when processing is performed according to the calling order of the processing model;

[0156] If the current resource status of the system meets the resource requirement parameters required to execute processing according to the calling order of the processing model, a collaborative processing mechanism is determined based on the calling order of the processing model.

[0157] If the current resource status of the system does not meet the resource requirements for executing processing according to the calling order of the processing model, the calling order of the processing model is adjusted according to the model type and the current resource status of the system, and a collaborative processing mechanism is determined based on the adjustment result.

[0158] In summary, the task processing apparatus based on large and small model collaboration provided in this application obtains a task to be processed, decomposes the task into multiple subtasks, determines a set of candidate processing models for each subtask based on its task attributes, and obtains a first model allocation result; adjusts the first model allocation result based on the workload of each subtask to obtain a second model allocation result; determines the correlation between each subtask, determines the processing order of each subtask based on the correlation, determines a collaborative processing mechanism based on the processing order of each subtask and the second model allocation result, and processes multiple subtasks based on the collaborative processing mechanism to obtain the final task processing result. In the above technical means, flexibly adjusting model allocation according to the workload of different subtasks allows for more precise allocation of model resources, ensuring that the model's processing capacity matches the workload and improving the rationality of resource utilization; and by determining the processing order of subtasks based on their correlation, and then combining the second model allocation result to determine the collaborative processing mechanism, the apparatus can fully explore the correlation between tasks, rationally arrange the processing order, and enable multiple subtasks to be processed in an orderly and efficient manner, thereby significantly improving task execution efficiency.

[0159] The task processing apparatus based on size model collaboration provided in this application can be used to execute the task processing method based on size model collaboration provided in the above embodiments, and has corresponding functions and beneficial effects.

[0160] Figure 10 This is a schematic diagram of the structure of a task processing device based on a size model collaboration provided in an embodiment of this application. (Refer to...) Figure 10 The task processing device based on size-model collaboration includes a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35. The number of processors 31 and the number of memories 32 in this task processing device can be one or more. The processor 31, memory 32, communication device 33, input device 34, and output device 35 of this task processing device can be connected via a bus or other means.

[0161] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the size-model-based collaborative task processing method in any embodiment of this application (e.g., the first allocation module 21, the second allocation module 22, and the task processing module 23 in the size-model-based collaborative task processing device). The memory 32 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory 32 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0162] The communication device 33 is used for data transmission.

[0163] The processor 31 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 32, thereby realizing the above-mentioned task processing method based on size model collaboration.

[0164] Input device 34 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 35 may include display devices such as a display screen.

[0165] The task processing device based on size model collaboration provided above can be used to execute the task processing method based on size model collaboration provided in the above embodiments, and has corresponding functions and beneficial effects.

[0166] This application also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a task processing method based on a size-model collaborative approach. The task processing method based on a size-model collaborative approach includes: acquiring a task to be processed; decomposing the task to be processed into multiple subtasks; determining a set of candidate processing models for each subtask based on the task attributes of each subtask to obtain a first model allocation result; adjusting the first model allocation result according to the task volume of each subtask to obtain a second model allocation result; determining the correlation between each subtask; determining the processing order between each subtask based on the correlation between each subtask; determining a collaborative processing mechanism based on the processing order between each subtask and the second model allocation result; and processing multiple subtasks based on the collaborative processing mechanism to obtain a final task processing result.

[0167] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” can include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0168] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the task processing method based on size model collaboration as described above, but can also execute related operations in the task processing method based on size model collaboration provided in any embodiment of this application.

[0169] The task processing apparatus, storage medium, and task processing device based on size-model collaboration provided in the above embodiments can execute the task processing method based on size-model collaboration provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the task processing method based on size-model collaboration provided in any embodiment of this application.

[0170] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application. The scope of this application is determined by the scope of the claims.

Claims

1. A task processing method based on size model collaboration, characterized in that, include: Obtain the task to be processed, decompose the task to be processed into multiple sub-tasks, determine the candidate processing model set corresponding to each sub-task according to the task attributes of each sub-task, and obtain the first model allocation result. The second model allocation result is obtained by adjusting the first model allocation result based on the task volume of each sub-task; The association between each subtask is determined, the processing order between each subtask is determined based on the association between each subtask, the collaborative processing mechanism is determined based on the processing order between each subtask and the allocation result of the second model, and the multiple subtasks are processed based on the collaborative processing mechanism to obtain the final task processing result; The step of determining the collaborative processing mechanism based on the processing order between each subtask and the second model allocation result includes: obtaining the processing model corresponding to each subtask in the second model allocation result, and establishing a mapping relationship table between the subtasks and the processing models; constructing a subtask processing sequence chain based on the processing order between each subtask; determining the model interaction relationship between the processing model corresponding to the preceding subtask and the processing model corresponding to the following subtask according to the mapping relationship table and the subtask processing sequence chain; determining the calling order of the processing model corresponding to each subtask according to the subtask processing sequence chain and the model interaction relationship; obtaining the current resource status of the system, and determining the collaborative processing mechanism according to the resource status and the calling order of the processing models. The step of determining the calling order of the processing model corresponding to each subtask based on the subtask processing time sequence chain and the model interaction relationship includes: determining the temporal order of each subtask based on the subtask processing time sequence chain; determining the data flow between the processing model corresponding to the preceding subtask and the processing model corresponding to the following subtask based on the model interaction relationship; and determining the calling order of the processing model corresponding to each subtask based on the temporal order of the subtasks and the data flow between the models.

2. The task processing method based on size model collaboration according to claim 1, characterized in that, The step of adjusting the first model allocation result according to the task volume of each sub-task to obtain the second model allocation result includes: The workload of each subtask is quantitatively evaluated to obtain a quantitative value for each subtask. Determine the processing capability threshold of each candidate processing model in the candidate model set corresponding to each sub-task; Based on the task quantification value and the processing capacity threshold, calculate the matching degree between each subtask and each candidate processing model in the corresponding candidate processing model set; Based on the matching degree, the final processing model corresponding to each sub-task is determined from the set of candidate processing models used by each sub-task, and the allocation result of the second model is obtained.

3. The task processing method based on size model collaboration according to claim 1, characterized in that, Determining the relationship between each of the subtasks includes: Determine the overall objective of the task to be processed and the objective of each of the subtasks; The execution preconditions for each subtask are determined based on the overall goal of the task to be processed and the goal of each subtask. Based on the aforementioned execution preconditions, the dependency relationships between each of the subtasks are determined.

4. The task processing method based on size model collaboration according to claim 3, characterized in that, Determining the processing order of each subtask based on the association between each subtask includes: Based on the dependency relationships between the subtasks, a directed graph of subtask relationships is constructed; wherein, the nodes in the directed graph are the subtasks, and the directed edges are the dependency directions; The nodes of the directed graph are topologically sorted to obtain the processing order between each subtask.

5. The task processing method based on size model collaboration according to claim 1, characterized in that, The step of determining the collaborative processing mechanism based on the resource status and the calling order of the processing model includes: Determine the model type for each of the processing models, wherein different model types correspond to different resource requirement parameters; Based on the calling order, determine whether the current resource status of the system meets the resource requirement parameters required when processing is performed according to the calling order of the processing model; If the current resource status of the system meets the resource requirement parameters required to execute processing according to the calling order of the processing model, a collaborative processing mechanism is determined based on the calling order of the processing model. If the current resource status of the system does not meet the resource requirements for executing processing according to the calling order of the processing model, the calling order of the processing model is adjusted according to the model type and the current resource status of the system, and a collaborative processing mechanism is determined based on the adjustment result.

6. A task processing device based on size model collaboration, characterized in that, include: The first allocation module is used to obtain the task to be processed, decompose the task to be processed into multiple sub-tasks, determine the candidate processing model set corresponding to each sub-task according to the task attributes of each sub-task, and obtain the first model allocation result. The second allocation module is used to adjust the first model allocation result according to the task quantity of each sub-task to obtain the second model allocation result; The task processing module is used to determine the relationship between each of the subtasks, determine the processing order between each subtask based on the relationship between each of the subtasks, determine a collaborative processing mechanism based on the processing order between each of the subtasks and the second model allocation result, and process multiple subtasks based on the collaborative processing mechanism to obtain the final task processing result. The task processing module is specifically used to obtain the processing model corresponding to each subtask in the second model allocation result and establish a mapping relationship table between the subtask and the processing model; Based on the processing order between each of the subtasks, a subtask processing sequence chain is constructed. Based on the mapping table and the subtask processing sequence chain, determine the model interaction relationship between the processing model corresponding to the preceding subtask and the processing model corresponding to the following subtask; based on the subtask processing sequence chain and the model interaction relationship, determine the calling order of the processing model corresponding to each subtask. Obtain the current resource status of the system, and based on the resource status and the calling order of the processing model, determine the collaborative processing mechanism, and... Specifically, it is used to determine the temporal order of each subtask based on the subtask processing sequence chain; to determine the data flow between the processing model corresponding to the preceding subtask and the processing model corresponding to the following subtask based on the model interaction relationship; and to determine the calling order of the processing model corresponding to each subtask based on the temporal order of the subtasks and the data flow between the models.

7. A task processing device based on a size-model collaborative architecture, characterized in that, include: One or more processors; A memory that stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the size-model-based collaborative task processing method as described in any one of claims 1-5.

8. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the task processing method based on size model collaboration as described in any one of claims 1-5.

Images