Task processing method and device based on large model, electronic equipment and storage medium

Abstract

The invention provides a task processing method and device based on a large model, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical field of task scheduling. According to the specific implementation scheme, the method comprises the steps of receiving a to-be-processed task, and analyzing the task to obtain task key information of the task; determining a target model corresponding to the task from the large model and the small model according to the task key information of the task; and sending the task to the target model to obtain an execution result of the task.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of task scheduling technology, and in particular to a task processing method, device, electronic device and storage medium based on a large model. Background Art

[0002] In recent years, the development of artificial intelligence (AI) models has been extremely rapid. As deep learning models become more and more complex, large-scale pre-trained models have significantly improved the processing effect of various tasks. Large models, with their massive parameters and excellent generalization capabilities, can solve complex problems well, but the accompanying computational costs are also relatively high. Relatively small models, such as those that use optimization methods such as model distillation, pruning, and quantization, although they respond quickly and consume low energy, their accuracy is often slightly insufficient in complex tasks.

[0003] Existing intelligent task scheduling systems often find it difficult to flexibly and effectively switch and combine the advantages of large models and small models, especially in scenarios involving complex tasks and high real-time requirements. Current technologies find it difficult to simultaneously strike a balance between accuracy and efficiency. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device and storage medium for task processing based on a large model.

[0005] According to one aspect of the present disclosure, a task processing method based on a large model is provided, including: receiving a task to be processed, and parsing the task to obtain task critical information of the task; determining a target model corresponding to the task from a large model and a small model based on the task critical information of the task; and sending the task to the target model to obtain the execution result of the task.

[0006] According to another aspect of the present disclosure, a task processing device based on a large model is provided, including: a parsing module, used to receive a task to be processed, and parse the task to obtain task critical information of the task; a determination module, used to determine a target model corresponding to the task from a large model and a small model according to the task critical information of the task; and a sending module, used to send the task to the target model to obtain the execution result of the task.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large model-based task processing method described in the above-mentioned one aspect embodiment.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, on which a computer program / instructions are stored, and the computer instructions are used to enable the computer to execute the large model-based task processing method described in the above-mentioned embodiment.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the large model-based task processing method described in the above-mentioned embodiment is implemented.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 A flowchart of a task processing method based on a large model provided in an embodiment of the present disclosure;

[0013] Figure 2 A flowchart of another large model-based task processing method provided in an embodiment of the present disclosure;

[0014] Figure 3 A schematic diagram of a flow chart of a strategy for adjusting task scheduling provided in an embodiment of the present disclosure;

[0015] Figure 4 A flowchart of another large model-based task processing method provided in an embodiment of the present disclosure;

[0016] Figure 5 A schematic diagram of a process for determining a target model provided in an embodiment of the present disclosure;

[0017] Figure 6 A flowchart of another large model-based task processing method provided in an embodiment of the present disclosure;

[0018] Figure 7A schematic diagram of a scheduling process based on a scheduler provided in an embodiment of the present disclosure;

[0019] Figure 8 A flowchart of another large model-based task processing method provided in an embodiment of the present disclosure;

[0020] Fig. 9 A schematic diagram of a process for implementing a heat exchange mechanism provided in an embodiment of the present disclosure;

[0021] Fig.10 A timing diagram for performing task processing provided in an embodiment of the present disclosure;

[0022] Fig.11 A timing diagram of processing tasks in an autonomous driving scenario provided in an embodiment of the present disclosure;

[0023] Fig.12 A schematic diagram of the structure of a task processing device based on a large model provided in an embodiment of the present disclosure;

[0024] Fig.13 The present invention is a block diagram of an electronic device for implementing the large model-based task processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] The following describes a large model-based task processing method, device, and electronic device according to an embodiment of the present disclosure with reference to the accompanying drawings.

[0027] Artificial Intelligence (AI) is a discipline that studies how computers can simulate certain thought processes and intelligent behaviors of human beings (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technologies and software-level technologies. AI hardware technologies generally include computer vision technology, speech recognition technology, natural language processing technology, as well as learning / deep learning, big data processing technology, knowledge graph technology, and other aspects.

[0028] Figure 1 A flowchart of a large model-based task processing method provided in an embodiment of the present disclosure.

[0029] like Figure 1As shown, the task processing method based on the large model may include:

[0030] S101, receiving a task to be processed, and parsing the task to obtain key task information of the task.

[0031] It should be noted that the execution subject of the task processing method based on the large model in the embodiment of the present disclosure may be a hardware device with data processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a server, a user terminal and other intelligent devices. Optionally, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, etc. Optionally, the server includes but is not limited to a network server, an application server, and may also be a server of a distributed system, or a server combined with a blockchain, etc. The embodiment of the present disclosure is not specifically limited.

[0032] In some implementations, input information from sensors, user requests, or predefined scripts may be received and used as tasks to be processed. Optionally, the tasks may be parsed based on natural language processing (NLP) technology to obtain key fields related to the tasks.

[0033] Furthermore, feature extraction can be performed on key fields to obtain information such as task priority, complexity, and task type as key information of the task. Optionally, pre-trained analysis models, such as task difficulty classification models and text semantic complexity analysis models, can be used to extract features from key fields.

[0034] In some implementations, the task type may indicate whether the task is a real-time task or a non-real-time task. Optionally, a machine learning model may be used to analyze the real-time nature of the task to determine the task type. Optionally, the priority of the task may also be calculated based on the time requirements of the task and the tolerance for real-time response delay. For example, in autonomous driving, the task of avoiding obstacles should have a higher priority than other visual analysis tasks.

[0035] S102, determining a target model corresponding to the task from the large model and the small model according to the key task information of the task.

[0036] In some implementations, the target model corresponding to the task can be determined from the large model and the small model according to any one of the priority, complexity, and task type of the task included in the task key information. For example, if the complexity of the task is greater than the complexity threshold, the large model can be selected as the target model. For another example, if the task type indicates that the task is a real-time task, the small model can be selected as the target model.

[0037] It can be understood that a large model is a model with a large number of parameters, which can better solve complex problems; a small model refers to a model that has been optimized by means of model distillation, pruning, quantization, etc. In other words, the model parameters of a large model are greater than those of a small model.

[0038] In some implementations, the target model can be initially determined based on the task type, and then the target model can be determined from the large model and the small model according to the priority and complexity of the task. For example, if the task type indicates that the task is a real-time task, the target model is a small model. If the task type indicates that the task is a non-real-time task and the complexity of the task is high, the target model is a large model.

[0039] S103, sending the task to the target model to obtain the execution result of the task.

[0040] In some implementations, after determining the target model required to execute the task, the task to be processed is sent to the target model, and the target model executes the task to obtain the execution result of the task. Optionally, after obtaining the execution result of the task, the process of determining the target model can be optimized and adjusted based on the execution result.

[0041] Optionally, based on the execution results of the task, the performance of the task execution process, such as latency, energy consumption, and accuracy, can be analyzed, and reinforcement learning can be performed based on the performance to optimize the process of determining the target model.

[0042] According to the large model-based task processing method provided by the embodiment of the present disclosure, the task to be processed is parsed to obtain the key information of the task, and the target model is determined from the large model and the small model according to the key information of the task. The task is sent to the target model to execute the task and obtain the execution result of the task. The present disclosure can intelligently schedule different models according to the key information of the task, is applicable to a variety of scenarios, and optimizes the resources of the system to improve the efficiency of task processing. For tasks with lower complexity, a small model is selected for processing, which can save computing costs while ensuring performance.

[0043] Figure 2 A flowchart of a large model-based task processing method provided in an embodiment of the present disclosure.

[0044] like Figure 2 As shown, the task processing method based on the large model may include:

[0045] S201, receiving a task to be processed, and parsing the task to obtain key task information of the task.

[0046] S202, determining a target model corresponding to the task from the large model and the small model according to the key task information of the task.

[0047] The relevant contents of steps S201 - S202 can be found in the above embodiment and will not be described again here.

[0048] S203, determining the priority, complexity and task type of the task according to the key information of the task.

[0049] S204, determining a target model corresponding to the task from the large model and the small model according to at least one of the priority, complexity and task type of the task.

[0050] In some implementations, the mission critical information includes the priority, complexity, and task type of the mission. The target model may be determined based on at least one of the priority, complexity, and task type of the mission. In other words, the target model may be determined based on any one of the priority, complexity, and task type, or may be determined based on a combination of multiple information.

[0051] For example, taking the task type and complexity as an example, the target model can be determined based on the task type or based on the complexity. For example, if the task type indicates that the task is a real-time task, the target model is a small model; if the task complexity is high, the target model is a large model. The target model can also be determined based on the task type and complexity. For example, if the task type indicates that the task is a non-real-time task and the task complexity is low, the target model is determined to be a small model.

[0052] S205, sending the task to the target model to obtain the execution result of the task.

[0053] The relevant contents of step S205 can be found in the above embodiment and will not be described again here.

[0054] In some implementations, after obtaining the execution result of the task, reinforcement learning can be performed on the task scheduling based on the execution result to optimize the scheduling rules, thereby providing a better processing method in the subsequent task scheduling process and improving the overall performance.

[0055] In some implementations, the execution results can be analyzed according to the execution results of the task to obtain the execution evaluation index. Further, the execution requirement index of the task is obtained. Optionally, the execution evaluation index and the execution requirement index can include: accuracy, latency, energy consumption, and precision.

[0056] Furthermore, sample data for task scheduling adjustment can be generated based on the execution requirement index and execution evaluation index of the task. Optionally, different sample tasks can be generated based on the execution requirement index and execution evaluation index to obtain sample data. Then, reinforcement learning can be performed on task scheduling based on the sample data to adjust the task scheduling strategy.

[0057] like Figure 3 The flowchart of the strategy for adjusting task scheduling is shown in FIG. By obtaining the execution result of the task and analyzing the execution result, the execution evaluation index is determined, and then the execution requirement index of the task can be obtained, such as accuracy, latency, energy consumption and other information. Furthermore, reinforcement learning can be performed based on the index information to adjust the task scheduling strategy, and the adjusted task scheduling strategy can be used to select the model to obtain new execution results and adjust the task scheduling strategy again. In other words, a cycle of new task input and feedback adjustment can be realized.

[0058] According to the task processing method based on a large model provided by the embodiment of the present disclosure, the task to be processed is parsed to obtain the key information of the task, and the target model is determined from the large model and the small model according to at least one item of the key information of the task, and then the task can be sent to the target model to execute the task and obtain the execution result of the task. The present disclosure can intelligently schedule different models according to the key information of the task, and is applicable to a variety of scenarios, so as to optimize the resources of the system to improve the efficiency of task processing. For tasks with lower complexity, a small model is selected for processing, which can save computing costs while ensuring performance. By intelligently judging the optimal configuration between large and small models, and giving priority to calling the small model with low power consumption and fast response speed for execution, a flexible balance is made between computing cost and accuracy to ensure high-efficiency processing; at the same time, through in-depth analysis of the large model, it is ensured that real-time demand is not delayed, and energy consumption is reduced and responsiveness is improved.

[0059] Figure 4 A flowchart of a large model-based task processing method provided in an embodiment of the present disclosure.

[0060] like Figure 4 As shown, the task processing method based on the large model may include:

[0061] S401, receiving a task to be processed, and parsing the task to obtain key task information of the task.

[0062] The relevant contents of step S401 can be found in the above embodiment and will not be described again here.

[0063] S402: In response to the task type indicating that the task is a real-time task, determining that the target model corresponding to the task is a small model.

[0064] Optionally, when it is confirmed that the task type indicates that the task is a real-time task, which means that the task needs to be processed with low latency, a small model can be selected as the target model to process the task in real time.

[0065] S403: In response to the task type indicating that the task is a non-real-time task, determine a target model corresponding to the task according to key task information of the task.

[0066] In some implementations, when a task is determined to be a non-real-time task, a target model can be determined based on the priority or complexity in the key information of the task, and the target model can be used to process the task to improve task processing efficiency and achieve rational use of resources.

[0067] That is, in response to the task type indicating that the task is a non-real-time task, and the priority of the task is higher than or equal to the set priority, the target model corresponding to the task is determined to be a small model. In response to the task type indicating that the task is a non-real-time task, and the priority of the task is lower than the set priority, the target model corresponding to the task is determined to be a large model.

[0068] In some implementations, the target model corresponding to the task can also be determined from the large model and the small model according to the model resources to achieve reasonable use of resources, improve the processing efficiency of the task, and expand the applicable scenarios of the model. Optionally, the model resources include but are not limited to: central processing unit (CPU) utilization, graphics processing unit (GPU) utilization, memory and storage capacity, network bandwidth, etc.

[0069] Optionally, by acquiring model resources available for candidate models, the candidate models include a large model and a small model, and according to task key information and model resources of the task, determining a target model corresponding to the task from the large model and the small model.

[0070] In some implementations, the target model can be determined based on the task complexity in the key information of the task and the model resources. The required resources of the task can be determined based on the complexity of the task, and based on the required resources of the task, it can be determined whether the model resources available for the large model are sufficient.

[0071] Optionally, in response to the model resources available for the large model being greater than or equal to the required resources, the target model corresponding to the task is determined to be the large model; in response to the model resources available for the large model being less than the required resources, the target model corresponding to the task is determined to be the small model, so as to achieve rational utilization of model resources and avoid wasting model resources.

[0072] like Figure 5The flowchart for determining the target model is shown in the figure. By extracting features from the tasks to be processed, the priority and complexity of the tasks can be determined, and then the tasks can be analyzed in real time to determine the task type. When the task type is determined to be a real-time task, the small model can be used as the target model to quickly execute the task. When the task type is determined to be a non-real-time task, the scheduling process can be entered. By determining the required resources for the task based on the complexity of the task, and judging whether the model resources available for the large model are sufficient based on the required resources for the task. If the resources are sufficient, the large model can be determined as the target model, and if the resources are supplementary components, the small model can be determined as the target model.

[0073] In some implementations, the target model can also be determined by determining the degree of match between the task and the model. When the task and the model are highly matched, the model can more accurately capture the key features and information of the task, thereby providing more accurate processing results, helping to reduce errors and unnecessary duplication of work, and improving task execution efficiency.

[0074] Optionally, the matching matrix between the task and the model can be obtained based on the key information of the task and the model resources, and the target model corresponding to the task can be determined from the large model and the small model based on the matching matrix. Optionally, each element in the matching matrix is ​​the matching value between the task and the model, and the element with the largest matching value in the matching matrix is ​​obtained, and the model corresponding to the element is used as the target model.

[0075] In some implementations, the mission-critical information and model resources may be weighted and scored to determine the matching value between the task and the model. Optionally, a first weight corresponding to each item of mission-critical information and a second weight corresponding to each item of model resources may be determined.

[0076] Furthermore, the matching value between the task and the model can be determined based on the task key information and the corresponding first weight, and the model resource and the corresponding second weight. The matching value between each task and a model can be calculated based on the first weight and the second weight, and then the matching degree matrix can be generated based on the matching value between the task and the model.

[0077] For example, taking M tasks and N models as an example, an M*N matching matrix can be determined, which represents the matching values ​​between M tasks and N models. Each element of the matching matrix represents the matching value of the mth task when using the nth model, that is, it reflects the matching between the model and the task in terms of model resources, priority and complexity.

[0078] As shown in Table 1, the matching matrix:

[0079] Mission\Model Model 1 Model 2 Model 3 Task 1 85 90 60 Task 2 90 70 95

[0080] According to Table 1, the matching value between Task 1 and Model 2 is the highest, so Model 2 is selected as the target model corresponding to Task 1. The matching value between Task 2 and Model 3 is the highest, so Model 3 is selected as the target model corresponding to Task 2.

[0081] S404, sending the task to the target model to obtain the execution result of the task.

[0082] The relevant contents of step S404 can be found in the above embodiment and will not be described again here.

[0083] According to the task processing method based on a large model provided by the embodiment of the present disclosure, the task to be processed is parsed to obtain the key information of the task, and the target model is determined from the large model and the small model according to at least one item of the key information of the task, and then the task can be sent to the target model to execute the task and obtain the execution result of the task. The present disclosure can intelligently schedule different models according to the key information of the task, and is applicable to a variety of scenarios, so as to optimize the resources of the system to improve the efficiency of task processing. For tasks with lower complexity, a small model is selected for processing, which can save computing costs while ensuring performance. By intelligently judging the optimal configuration between large and small models, and giving priority to calling the small model with low power consumption and fast response speed for execution, a flexible balance is made between computing cost and accuracy to ensure high-efficiency processing; at the same time, through in-depth analysis of the large model, it is ensured that real-time demand is not delayed, and energy consumption is reduced and responsiveness is improved.

[0084] Figure 6 A flowchart of a large model-based task processing method provided in an embodiment of the present disclosure.

[0085] like Figure 6 As shown, the task processing method based on the large model may include:

[0086] S601, receiving a task to be processed, and parsing the task to obtain key task information of the task.

[0087] S602, determining a target model corresponding to the task from the large model and the small model according to the key task information of the task.

[0088] The relevant contents of steps S601-S602 can be found in the above embodiment and will not be repeated here.

[0089] S603 , for any task set in the first task set and the second task set, in response to any task set including two or more tasks, sort the tasks in any task set according to task key information of the tasks to determine an execution order of the tasks.

[0090] In some implementations, when it is determined that there are multiple tasks to be processed, the multiple tasks can be divided according to the key information of the tasks to determine the first task set corresponding to the large model and the second task set corresponding to the small model, and then the tasks to be processed can be sent to the target model according to different task sets to improve the execution efficiency of the tasks. Optionally, the multiple tasks can be divided according to the priority of the tasks to obtain a first task set with a low priority and a second task set with a high priority.

[0091] In some implementations, for any of the first task set and the second task set, the tasks in any of the task sets may be sorted to obtain an execution order of the tasks. By determining the execution order of the tasks, the waiting time between tasks and resource idleness may be reduced, thereby improving task processing efficiency.

[0092] Optionally, the execution order of the tasks may be determined according to the priorities of the tasks, that is, the higher the priority of the task, the earlier the execution order of the task.

[0093] S604, sending the task to the target model according to the execution order of the task to obtain the execution result of the task.

[0094] In some implementations, the first task queue corresponding to the first task set and the second task queue corresponding to the second task set can be determined according to the execution order of the tasks, and the first task queue and the second task queue can be sent to the target model, which helps to better allocate and schedule resources for the model and avoid wasting model resources.

[0095] That is to say, a first task queue can be generated based on the first task set, the first task queue includes sorted tasks, and the tasks in the first task queue are sent to the large model in sequence, and a second task queue can be generated based on the second task set, the second task queue includes sorted tasks, and the tasks in the second task queue are sent to the small model in sequence.

[0096] Optionally, a scheduler may be used to send the first task queue and the second task queue to the target model. Figure 7 The diagram shows a schematic diagram of scheduling based on a scheduler. Figure 7 The low priority task queue in is the first task queue, and the high priority task queue is the second task queue. The low priority task queue and the high priority task queue are input into the scheduler, and the scheduler sends the task. The scheduler sends the low priority task queue to the large model and the high priority task queue to the small model, and then the execution result of the task can be obtained. In other words, the target model of the low priority task queue is the large model, and the target model of the high priority task queue is the small model.

[0097] According to the task processing method based on a large model provided by an embodiment of the present disclosure, when there are multiple tasks to be processed, the multiple tasks can be divided according to the key information of the tasks, and the execution order of the tasks can be determined according to the division results, so that the tasks can be sent to the target model according to the execution order, and then the tasks can be executed to obtain the execution results of the tasks. The present disclosure can intelligently schedule different models according to the key information of the tasks, and is suitable for a variety of scenarios, so as to optimize the resources of the system to improve the efficiency of task processing. For tasks with lower complexity, a small model is selected for processing, which can save computing costs while ensuring performance. By intelligently judging the optimal configuration between large and small models, and giving priority to calling small models with low power consumption and fast response speed for execution, a flexible balance is made between computing cost and accuracy to ensure high-efficiency processing; at the same time, through in-depth analysis of the large model, it is ensured that real-time needs are not delayed, and energy consumption is reduced and responsiveness is improved.

[0098] Figure 8 A flowchart of a large model-based task processing method provided in an embodiment of the present disclosure.

[0099] like Figure 8 As shown, the task processing method based on the large model may include:

[0100] S801, receiving a task to be processed, and parsing the task to obtain key task information of the task.

[0101] The relevant contents of step S801 can be found in the above embodiment and will not be described again here.

[0102] S802, predict at least one large model required for the task, and perform an asynchronous preloading operation on the at least one large model required for the task.

[0103] In some implementations, in order to achieve seamless switching between large models and small models and reduce delays caused by model switching and loading, the large model can be preloaded asynchronously to cache the large model in memory, thereby reducing the loading time of the model.

[0104] Optionally, you can use a pre-trained prediction model to make predictions for the large model required for the task.

[0105] Optionally, a hot swap mechanism can be used to asynchronously preload large models. The main goal of the hot swap mechanism is to ensure fast switching between different types of models and reduce waiting time by optimizing the model loading, retention and switching processes.

[0106] In some implementations, the large model can be preloaded while the task is using the small model to increase the speed of switching from the small model to the large model and reduce model delay. Optionally, a first subtask that needs to use the large model and a second subtask that uses the small model can be determined from the task, so that the large model corresponding to the first subtask can be preloaded.

[0107] Optionally, the task can be decomposed to obtain multiple subtasks, and according to the task feature information of the subtasks, a first subtask that needs to call the large model and a second subtask that needs to call the small model are determined from the multiple subtasks. Furthermore, during the execution of the second subtask by the small model, an asynchronous preloading operation is performed on the large model corresponding to the first subtask.

[0108] For example, in an autonomous driving scenario, a small model is used to perform emergency obstacle avoidance tasks. The preloading mechanism ensures that the small model is always loaded locally on the device for quick access; when a more complex task (such as planning a new path task) requires a large model, the working time of the model can be detected in advance and the model can be loaded from the cloud to the memory in advance.

[0109] In some implementations, the large model required for a task can be predicted based on the historical data of the model, thereby improving the accuracy of the predicted large model. Seamless switching of models can be achieved by obtaining the load data of the model when executing historical tasks, predicting at least one large model required for the task based on the load data and the execution data of the historical tasks, and performing asynchronous preloading operations on at least one large model required for the task.

[0110] like Fig. 9 The flowchart of implementing the hot exchange mechanism is shown in the figure. By obtaining the task to be processed and determining at least one large model required for the second subtask in the task, the large model is asynchronously preloaded. When the large model is loaded, it can be seamlessly switched to the large model to provide high-precision processing. When the task contains a real-time second subtask, the small model is started for rapid response to obtain the task execution result.

[0111] S803, determining a target model corresponding to the task from the large model and the small model according to the key task information of the task.

[0112] S804, sending the task to the target model to obtain the execution result of the task.

[0113] The relevant contents of steps S803-S804 can be found in the above embodiment and will not be repeated here.

[0114] According to the large model-based task processing method provided by the embodiment of the present disclosure, by performing asynchronous preloading operations on the large model required for the task, seamless switching between the large model and the small model can be achieved, reducing the delay caused by model switching and loading, thereby improving the task processing efficiency.

[0115] Fig.10 Shown is a timing diagram for performing task processing. Fig.10 It includes: task input module, task management module, task scheduling module, model selection engine, feedback optimization module, Fig.10 The models in include large models and small models.

[0116] By receiving multiple task requirements, the task management module parses the input tasks and determines the priority of the tasks. It can then make a preliminary judgment on whether to schedule a large model or a small model for processing based on the priority of the task, and analyze the real-time and complexity of the task. The model selection engine receives the task, as well as the priority, real-time and complexity of the task, and can schedule a large model or a small model for the task. If the task is a real-time task, the model selection engine can schedule a small model to process the task, that is, if the task is a real-time task, the target model corresponding to the task is a small model; if the task is a non-real-time task and the complexity of the task is high, that is, the task is a high-precision task, the target model can be determined based on the required resources of the task and the available resources of the large model. If the available resources of the large model are greater than or equal to the required resources, the model selection engine schedules the large model to process the task.

[0117] Furthermore, after using the model to process the task, the execution result of the task can be obtained, and the execution result can be fed back to the task scheduling module. The task scheduling module analyzes the execution result, and can obtain the execution evaluation index and the execution requirement index of the task as the execution information, and feeds it back to the feedback optimization module. The feedback optimization module generates sample data for task scheduling adjustment based on the execution information, and performs reinforcement learning on the task scheduling based on the sample data to adjust the task scheduling strategy, and then the task management module can use the adjusted task scheduling strategy to perform the next task scheduling.

[0118] Fig.10 It is shown that the target model is determined according to the real-time and complexity of the task key information of the task. It is also possible to determine the matching matrix between the task and the model according to the task key information and the model resources, and then determine the target model corresponding to the task according to the matching matrix. By obtaining the first weight of the task key information and the second weight of the model resource, the matching value between the model and the task can be calculated to generate a matching matrix according to the matching value, and then the element with the largest matching value can be obtained from the matching matrix, and the model corresponding to the element is used as the target model.

[0119] By way of example, the task processing method based on a large model proposed in the present disclosure is explained by taking an autonomous driving scenario as an example. Fig.11 The timing diagram of the autonomous driving scenario processing tasks is shown. Fig.11 It includes: sensor input module, task management module, scheduling module, model selection engine, Fig.11 The models in include large models and small models.

[0120] The driving environment collected by the sensor is input, wherein the sensor includes a camera and a radar sensor. The task management module determines the priority, real-time performance and complexity of the task carried by the input driving environment. Fig.11 The tasks to be processed include obstacle avoidance tasks and path planning tasks. Among them, the obstacle avoidance task is a real-time task, the path planning task is a non-real-time task, and the obstacle avoidance task has a higher priority than the path planning task. When it is determined from the driving environment that a real-time obstacle avoidance task needs to be performed, the scheduling module can schedule the real-time small model through the scheduling model selection engine so that the small model can identify obstacles, and then the obstacle avoidance result can be returned. In other words, the target model corresponding to the obstacle avoidance task is the small model. If it is determined from the driving environment that a path planning task needs to be performed and the complexity of the task is high, the large model that can be used for deep understanding is prioritized for scheduling. The scheduling module schedules the large model through the scheduling model selection engine for environmental detection and semantic analysis, and then the large model can be started for deep semantic understanding to return the planned route as the execution result. In other words, the target model corresponding to the path planning task is the large model.

[0121] Fig.11 The target model is determined according to the real-time nature of the task. Fig.11 The large model contains multiple tasks, and the tasks can be divided into a first task set corresponding to the large model and a second task set corresponding to the small model according to the priority of the tasks. That is to say, the tasks in the first task set are path planning tasks, and the tasks in the second task set are obstacle avoidance tasks. Then, the second task set can be sent to the small model to perform the obstacle avoidance task, and the first task set can be sent to the large model to perform the path planning task. When performing the obstacle avoidance task, the large model can be asynchronously preloaded to achieve seamless switching between the large model and the small model, reducing the delay caused by model switching and loading.

[0122] The embodiments of the present disclosure can be applied to autonomous driving scenarios, smart manufacturing scenarios, smart home scenarios, etc.

[0123] 1. Autonomous driving scenarios:

[0124] The autonomous driving scenario is a typical complex application field, which involves a variety of perception tasks, such as target detection, environmental perception, route planning, obstacle avoidance, etc. In order to provide a safe and reliable driving experience, high-frequency, millisecond-level perception responses to external environmental information are required (such as red light recognition, pedestrian detection, etc.), in which the system needs to quickly call small models to meet low-latency response requirements. Complex environmental understanding, semantic segmentation, and long-distance path optimization tasks require large models with strong computing power to provide high-precision results.

[0125] For example, when the traffic light status is detected in an autonomous driving scenario, a small low-power model can be directly dispatched to perform rough object classification (such as traffic light color detection) first to quickly perform necessary operations on the vehicle (such as braking or accelerating); at the same time, the large model is started to refine and verify the depth of the environment information to ensure that the system can adjust the vehicle's subsequent path planning at any time.

[0126] 2. Smart manufacturing scenarios:

[0127] In smart manufacturing, there are multiple tasks with different time spans and different data levels, such as equipment operation monitoring (fast analysis of sensor data), machine maintenance anomaly detection, fault cause tracing, etc. These system scenarios also require powerful capabilities to handle complex working condition monitoring, and some scenarios (such as some specific error handling) may only be implemented by empirical formulas or basic feature judgments, so there is no need to occupy the resources of large models to complete, so small models can fully handle the task. When it comes to complex process problems such as process prediction and quality management analysis, pre-trained large models must be used to ensure decision quality.

[0128] Through real-time judgment by the scheduling system, the most suitable model is assigned according to the equipment workload or status, which can significantly optimize the manufacturing process and reduce the total power consumption, while also ensuring that the core tasks in the factory can be accurately detected. For example, the task of detecting abnormal fluctuations is given priority to the small model for rough detection, and the correction program is started in time for timely response; when the relationship between the waveform and the equipment status is further refined, the large model is responsible for finding out.

[0129] 3. Smart home scenarios:

[0130] In smart homes, the processing of user voice, action, and habit data requires the support of models with different precision and reaction time. For example, user voice control commands can be responded to quickly, but the semantic understanding of complex behaviors, such as in-depth contextual conversations, multi-room data joint recognition (camera, sound), and other complex behavior processing, requires additional large models for comprehensive reasoning. The system needs to ensure that it responds to simple commands immediately, and then combine the real-time feedback from the surrounding environment sensors to decide whether to mobilize more complex models in the future.

[0131] For example, a smart lighting control system can use a small model to process simple switch commands. In abnormal environments (fire alarms, long periods of absence, etc.), it can integrate monitoring data and call complex models to obtain the best scenario response, thereby improving processing efficiency and ensuring user experience.

[0132] Corresponding to the big model-based task processing methods provided in the above-mentioned embodiments, an embodiment of the present disclosure also provides a big model-based task processing device. Since the big model-based task processing device provided in the embodiment of the present disclosure corresponds to the big model-based task processing methods provided in the above-mentioned embodiments, the implementation methods of the above-mentioned big model-based task processing methods are also applicable to the big model-based task processing device provided in the embodiment of the present disclosure, and will not be described in detail in the following embodiments.

[0133] Fig.12 A schematic diagram of the structure of a task processing device based on a large model provided in an embodiment of the present disclosure.

[0134] like Fig.12 As shown, the large model-based task processing device 1200 of the embodiment of the present disclosure includes a parsing module 1201 , a determining module 1202 and a sending module 1203 .

[0135] The parsing module 1201 is used to receive a task to be processed, and parse the task to obtain key task information of the task;

[0136] A determination module 1202 is used to determine a target model corresponding to the task from the large model and the small model according to the task key information of the task;

[0137] The sending module 1203 is used to send the task to the target model to obtain the execution result of the task.

[0138] In one embodiment of the present disclosure, the determination module 1202 is also used to: determine the priority, complexity and task type of the task based on the task key information; and determine the target model corresponding to the task from the large model and the small model based on at least one of the priority, complexity and task type of the task.

[0139] In one embodiment of the present disclosure, the determination module 1202 is also used to: in response to the task type indicating that the task is a real-time task, determine that the target model corresponding to the task is a small model; in response to the task type indicating that the task is a non-real-time task, determine the target model corresponding to the task based on task key information of the task.

[0140] In one embodiment of the present disclosure, the determination module 1202 is also used to: obtain model resources available for candidate models, the candidate models including large models and small models; and determine the target model corresponding to the task from the large model and the small model based on task key information of the task and the model resources.

[0141] In one embodiment of the present disclosure, the determination module 1202 is also used to: determine the required resources for the task based on the complexity of the task; determine whether the model resources available for the large model are sufficient based on the required resources for the task; in response to the model resources available for the large model being greater than or equal to the required resources, determine that the target model corresponding to the task is a large model; in response to the model resources available for the large model being less than the required resources, determine that the target model corresponding to the task is a small model.

[0142] In one embodiment of the present disclosure, the determination module 1202 is also used to: obtain a matching matrix between the task and the model based on the task key information and the model resources; and determine the target model corresponding to the task from the large model and the small model based on the matching matrix.

[0143] In one embodiment of the present disclosure, the determination module 1202 is also used to: determine the first weight corresponding to each item of information in the task key information and the second weight corresponding to each resource in the model resource; determine the matching value between the task and the model based on the task key information and the corresponding first weight, and the model resource and the corresponding second weight; and generate the matching degree matrix based on the matching value between the task and the model.

[0144] In one embodiment of the present disclosure, the determination module 1202 is further used to: in response to the task type indicating that the task is a non-real-time task and the priority of the task is higher than or equal to the set priority, determine that the target model corresponding to the task is a small model; or, in response to the task type indicating that the task is a non-real-time task and the priority of the task is lower than the set priority, determine that the target model corresponding to the task is a large model.

[0145] In one embodiment of the present disclosure, the sending module 1203 is further used to: divide the multiple tasks according to the task key information, and determine a first task set corresponding to the large model and a second task set corresponding to the small model.

[0146] In one embodiment of the present disclosure, the sending module 1203 is also used to: for any task set in the first task set and the second task set, in response to any task set including two or more tasks, sort the tasks in any task set according to task key information of the tasks to determine the execution order of the tasks.

[0147] In one embodiment of the present disclosure, the sending module 1203 is also used to: generate a first task queue according to the first task set, the first task queue including sorted tasks, and send the tasks in the first task queue to the large model in sequence; generate a second task queue according to the second task set, the second task queue including sorted tasks, and send the tasks in the second task queue to the small model in sequence.

[0148] In one embodiment of the present disclosure, the device further includes: a preloading module, which is used to predict at least one large model required for the task and perform an asynchronous preloading operation on the at least one large model required for the task.

[0149] In one embodiment of the present disclosure, the preloading module is also used to: decompose the task to obtain multiple subtasks, and determine, from the multiple subtasks, a first subtask that needs to call a large model and a second subtask that needs to call a small model based on task feature information of the subtasks; and perform an asynchronous preloading operation on the large model corresponding to the first subtask during execution of the second subtask through the small model.

[0150] In one embodiment of the present disclosure, the preloading module is also used to: obtain load data when the model executes historical tasks; predict at least one large model required for the task based on the load data and the execution data of the historical tasks, and perform asynchronous preloading operations on the at least one large model required for the task.

[0151] In one embodiment of the present disclosure, the sending module 1203 is also used to: analyze the execution result according to the execution result of the task to obtain an execution evaluation index; obtain the execution requirement index of the task; generate sample data for task scheduling adjustment according to the execution requirement index of the task and the execution evaluation index; and perform reinforcement learning on task scheduling based on the sample data to adjust the task scheduling strategy.

[0152] According to the large model-based task processing device provided by the embodiment of the present disclosure, the task to be processed is parsed to obtain the key information of the task, and the target model is determined from the large model and the small model according to the key information of the task. The task is sent to the target model to execute the task and obtain the execution result of the task. The present disclosure can intelligently schedule different models according to the key information of the task, is applicable to a variety of scenarios, and optimizes the resources of the system to improve the efficiency of task processing. For tasks with lower complexity, a small model is selected for processing, which can save computing costs while ensuring performance.

[0153] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0154] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0155] Fig.13 A schematic block diagram of an example electronic device 1300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0156] like Fig.13 As shown, the device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program / instruction stored in a read-only memory (ROM) 1302 or a computer program / instruction loaded from a storage unit 1306 to a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the device 1300 can also be stored. The computing unit 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0157] A number of components in the device 1300 are connected to the I / O interface 1305, including: an input unit 1306 such as a keyboard, a mouse, etc.; an output unit 1307 such as various types of displays, speakers, etc.; a storage unit 1308 such as a disk, an optical disk, etc.; and a communication unit 1309 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1309 allows the device 1300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0158] The computing unit 1301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1301 performs the various methods and processes described above, such as a task processing method based on a large model. For example, in some embodiments, the task processing method based on a large model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1306. In some embodiments, part or all of the computer program / instructions may be loaded and / or installed on the device 1300 via the ROM 1302 and / or the communication unit 1309. When the computer program / instructions are loaded into the RAM 1303 and executed by the computing unit 1301, one or more steps of the task processing method based on the large model described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured to execute the large model-based task processing method in any other appropriate manner (eg, by means of firmware).

[0159] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs / instructions that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0160] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0161] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0163] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0164] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs / instructions running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0165] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in the disclosure can be achieved, and this document does not limit them here.

[0166] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A task processing method based on a large model, wherein: The method comprises: Receiving a task to be processed, and parsing the task to obtain key task information of the task; Determine a target model corresponding to the task from the large model and the small model according to the key task information of the task; The task is sent to the target model to obtain the execution result of the task.

2. The method according to claim 1, wherein: Determining the target model corresponding to the task from the large model and the small model according to the task key information of the task includes: Determine the priority, complexity and task type of the task according to the key information of the task; According to at least one of the priority, complexity and task type of the task, a target model corresponding to the task is determined from the large model and the small model.

3. The method according to claim 2, wherein: The method further comprises: In response to the task type indicating that the task is a real-time task, determining that the target model corresponding to the task is a small model; In response to the task type indicating that the task is a non-real-time task, a target model corresponding to the task is determined according to task critical information of the task.

4. The method according to claim 3, wherein: Determining the target model corresponding to the task according to the task key information of the task includes: Acquire model resources available for candidate models, where the candidate models include large models and small models; According to the task key information of the task and the model resources, a target model corresponding to the task is determined from the large model and the small model.

5. The method according to claim 4, wherein: The step of determining a target model corresponding to the task from the large model and the small model according to the task key information of the task and the model resources includes: Determine the required resources for the task according to the complexity of the task; According to the required resources of the task, determining whether the model resources available for the large model are sufficient; In response to the model resources available to the large model being greater than or equal to the required resources, determining that the target model corresponding to the task is the large model; In response to the model resources available for the large model being less than the required resources, determining that the target model corresponding to the task is a small model.

6. The method according to claim 4, wherein: The step of determining a target model corresponding to the task from the large model and the small model according to the task key information of the task and the model resources includes: Acquire a matching matrix between the task and the model according to the task key information and the model resources; According to the matching degree matrix, a target model corresponding to the task is determined from the large model and the small model.

7. The method according to claim 6, wherein: The obtaining, according to the task key information and the model resource, a matching degree matrix between the task and the model comprises: Determine a first weight corresponding to each item of the mission-critical information and a second weight corresponding to each item of the model resource; Determining a matching value between the task and the model according to the task key information and the corresponding first weight, and the model resource and the corresponding second weight; The matching degree matrix is ​​generated based on the matching value between the task and the model.

8. The method according to claim 3, wherein: The method further comprises: In response to the task type indicating that the task is a non-real-time task and the priority of the task is higher than or equal to the set priority, determining that the target model corresponding to the task is a small model; or, In response to the task type indicating that the task is a non-real-time task and the priority of the task is lower than the set priority, it is determined that the target model corresponding to the task is a large model.

9. The method according to claim 1, wherein: When there are multiple tasks to be processed, the method further includes: The multiple tasks are divided according to the key information of the tasks to determine a first task set corresponding to the large model and a second task set corresponding to the small model.

10. The method according to claim 9, wherein: The sending the task to the target model comprises: For any task set in the first task set and the second task set, in response to any task set including two or more tasks, the tasks in any task set are sorted according to task critical information of the tasks to determine the execution order of the tasks.

11. The method according to claim 10, wherein: The method further comprises: Generate a first task queue according to the first task set, the first task queue including sorted tasks, and send the tasks in the first task queue to the large model in sequence; A second task queue is generated according to the second task set, the second task queue includes sorted tasks, and the tasks in the second task queue are sent to the small model in sequence.

12. The method according to any one of claims 1 to 11, wherein: The method further comprises: At least one large model required for the task is predicted, and an asynchronous preloading operation is performed on the at least one large model required for the task.

13. The method according to claim 12, wherein: The predicting at least one large model required for the task and performing an asynchronous preloading operation on the at least one large model required for the task includes: Decomposing the task to obtain a plurality of subtasks, and determining, according to task feature information of the subtasks, a first subtask that needs to call a large model and a second subtask that needs to call a small model from the plurality of subtasks; During the execution of the second subtask by the small model, an asynchronous preloading operation is performed on the large model corresponding to the first subtask.

14. The method according to claim 12, wherein: The predicting at least one large model required for the task and performing an asynchronous preloading operation on the at least one large model required for the task includes: Obtain the load data when the model executes historical tasks; At least one large model required for the task is predicted based on the load data and the execution data of the historical task, and an asynchronous preloading operation is performed on the at least one large model required for the task.

15. The method according to any one of claims 1 to 11, wherein: After obtaining the execution result of the task, the method further includes: Analyze the execution results of the tasks to obtain execution evaluation indicators; Obtaining the execution requirement indicators of the task; Generating sample data for task scheduling adjustment according to the execution requirement index of the task and the execution evaluation index; Reinforcement learning is performed on task scheduling based on the sample data to adjust the task scheduling strategy.

16. A task processing device based on a large model, wherein: The device comprises: The parsing module is used to receive the task to be processed, and parse the task to obtain the key information of the task; A determination module, used to determine a target model corresponding to the task from a large model and a small model according to key task information of the task; The sending module is used to send the task to the target model to obtain the execution result of the task.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 15.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-15.

19. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images