Large model screening method, device, equipment, medium and product

Abstract

The invention discloses a large model screening method, device and equipment, a medium and a product. The large model screening method comprises the steps of performing scene classification on large model tasks to obtain task scene types, and performing importance classification on the large model tasks to obtain task importance levels; based on the task scene type, the task importance level and the large model task, predicting the answer accuracy of each candidate large model for the large model task and the user satisfaction; obtaining the task queuing state of each candidate large model, and determining the waiting duration of the large model task under each candidate large model based on the task queuing state and the task scene type; and based on the answer accuracy, the user satisfaction, the waiting duration and the task importance level of each candidate large model for the large model task, determining a target large model for processing the large model task in the plurality of candidate large models. According to the technical scheme provided by the embodiment of the invention, balance between user experience and resource cost can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a large model screening method, device, equipment, medium and product. BACKGROUND

[0002] With the rapid development of artificial intelligence large model technology, more and more fields begin to use large models for transaction processing, the large model industry ecology is gradually perfect, and the number of models on record online is gradually rising.

[0003] The technical characteristics of large models also bring serious computing resource challenges. Training and inference face high consumption, and a single training of a hundred billion parameter model consumes millions of dollars of computing resources. Daily inference also relies on massive calculations, for example, financial institutions deploying a hundred billion models need to build dedicated computing clusters. How to balance between technology and computing power shortage has attracted widespread attention. To solve the above problems, existing technologies mainly focus on framework and training optimization, lightweight inference and local deployment, computing resource sharing and scheduling, and model compression and efficient storage.

[0004] However, due to the constraints of existing technologies, in model services, especially in general-purpose model services, there are still a large number of use cases that small-sized large models cannot handle, so deploying small models alone may cause the experience of some scenarios to decline, thereby leading to user loss; deploying large models is also prone to resource constraints, which makes it difficult to respond to customer needs in a timely manner, affecting the expansion speed of services in the market. SUMMARY

[0005] The present application provides a large model screening method, device, equipment, medium and product to solve the problem that the existing technology cannot balance user experience and resource cost.

[0006] According to an aspect of the present application, a large model screening method is provided, comprising:

[0007] In response to receiving a large model task, classifying the large model task by scene to obtain a task scene type, and classifying the large model task by importance to obtain a task importance level;

[0008] Based on the task scene type, task importance level and large model task, predicting the answer accuracy of each candidate large model in a plurality of candidate large models of different scales for the large model task, and user satisfaction;

[0009] Obtaining the task queuing state of each candidate large model, and based on the task queuing state and the task scene type, determining the waiting time of the large model task under each candidate large model;

[0010] Based on the accuracy of each candidate large model's response to the large model task, user satisfaction, waiting time, and task importance level, a target large model for handling the large model task is determined from among multiple candidate large models.

[0011] According to another aspect of the present invention, a large model screening device is provided, comprising:

[0012] The task classification module is used to respond to the receipt of a large model task, classify the large model task into scenarios to obtain task scenario types, and classify the large model task into importance levels to obtain task importance levels.

[0013] The accuracy prediction module is used to predict the accuracy of each candidate large model's response to the large model task, as well as user satisfaction, based on the task scenario type, task importance level, and large model task.

[0014] The waiting time estimation module is used to obtain the task queuing status of each candidate large model, and determine the waiting time of the large model task under each candidate large model based on the task queuing status and the task scenario type.

[0015] The large model screening module is used to determine the target large model for processing the large model task from multiple candidate large models based on the accuracy of each candidate large model's response to the large model task, user satisfaction, waiting time, and task importance level.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the large model screening method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the large model screening method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the large model screening method of any embodiment of the present disclosure.

[0022] The technical solution of the embodiment of the present application, in response to receiving a large model task, classifies the large model task according to the scene, obtains the task scene type, and classifies the large model task according to the importance, obtains the task importance level, based on the task scene type, task importance level and large model task, predicts the answer accuracy of each candidate large model in multiple different scales of candidate large models for the large model task, and the user satisfaction, and then obtains the task queuing state of each candidate large model, and based on the task queuing state and the task scene type, determines the waiting time of the large model task under each candidate large model, based on the answer accuracy of each candidate large model for the large model task, user satisfaction, waiting time and task importance level, determines the target large model for processing the large model task in multiple candidate large models, which can dynamically select the matched large model in multiple different scales of candidate large models according to the current received large model task, and balances in user experience and resource cost.

[0023] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is a flowchart of a large model screening method according to the first embodiment of the present application;

[0026] Figure 2a is a flowchart of a large model screening method according to the second embodiment of the present application;

[0027] Figure 2b is a structural schematic diagram of a prediction model according to the second embodiment of the present application;

[0028] Figure 3 is a structural schematic diagram of a large model screening device according to the third embodiment of the present application;

[0029] Figure 4 is a structural schematic diagram of an electronic device for implementing the large model screening method of the embodiment of the present application. DETAILED DESCRIPTION

[0030] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work should fall within the protection scope of the present application.

[0031] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device including a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0032] Embodiment one

[0033] Figure 1 A flowchart of a large model screening method is provided for the first embodiment of the present application. The present embodiment can be applicable to selecting a target large model from candidate large models of different scales to perform a current large model task. The method can be performed by a large model screening device, which can be realized in the form of hardware and / or software, and can be configured in various general-purpose computing devices. As shown in the figure, the method comprises: Figure 1

[0034] S110, in response to receiving a large model task, classifying the large model task according to a scene, obtaining a task scene type, and classifying the large model task according to importance, obtaining a task importance level.

[0035] The large model task is a successive specific call request initiated by a user to a large model service system, requesting the large model to process. The large model task can be a piece of natural language text, for example, a question, an instruction, or a piece of text to be processed, etc.

[0036] The scene classification is a process of dividing the large model task into a plurality of predefined task scene types according to the content, purpose or business field of the large model task, wherein the task scene types can be customized according to the application field. For example, the task scene type can be general text generation, semantic understanding and analysis, language conversion, real-time data acquisition, custom tool invocation, mathematical reasoning, etc.​

[0037] Importance classification is a process of dividing large model tasks into several task importance levels defined according to the severity of business risks or consequences that may be caused by errors or delays in the processing results of large model tasks. For example, the task importance levels can include general importance, medium importance, and very important, and the specific granularity can be flexibly defined according to actual needs.

[0038] In the embodiments of the present application, in order to select a large model suitable for a large model task, after receiving the large model task, the large model task is first classified according to the scene to obtain the task scene type, and the large model task is classified according to the importance to obtain the task importance level.

[0039] Specifically, a large number of large model task samples labeled with task scene types can be used to train a basic classification model to obtain a scene classification model. Similarly, a large number of large model task samples labeled with task importance levels can be used to train a basic classification model to obtain a task importance classification model. For example, the basic classification model can be a Bidirectional Encoder Representations from Transformers (BERT) model or a Bert+TextCNN model.

[0040] After receiving the large model task, the large model task is input into the scene classification model and the importance classification model respectively, and the task scene type and the task importance level output by the model are obtained respectively.

[0041] In addition, a set of keywords, phrase patterns or regular expressions can be defined for each task scene type in advance. When a new large model task arrives, whether the content of the large model task contains specific pattern words is detected by a text matching algorithm. For example, the appearance of words such as “translation” and “translated into” in the task can be classified as “language conversion”; the appearance of “summary” and “abstract” can be classified as “text summary”. The task importance level can also be determined based on the task source and business context metadata of the current large model task.

[0042] S120, based on the task scene type, the task importance level and the large model task, predicting the answer accuracy and user satisfaction of each candidate large model in a plurality of candidate large models of different scales for the large model task.

[0043] Different scale candidate large models refer to a series of large models with different parameter scales, computational complexity, inference speed and ability. For example, the candidate large models include 7B, 32B, 72B and 671B large models. Generally, larger scale models have stronger capabilities but consume more resources and respond more slowly.

[0044] The answer accuracy is a prediction probability or score of the correctness, logical rigor, and fit degree of the generated answer of the large model for the task. The user satisfaction is a prediction probability or score of the subjective satisfaction of the user for the understandability, completeness, format friendliness, and interaction experience of the generated answer of the model. The user satisfaction and the answer accuracy can not be completely consistent.

[0045] In the embodiment of the application, the large model task, the task scene type, and the task importance level are respectively coded and input into a pre-trained prediction model. The prediction model outputs two parallel multi-label heads, which are an accuracy prediction head and a satisfaction prediction head. The accuracy prediction head outputs a first vector, and each dimension of the first vector corresponds to a candidate large model. Each value in the vector represents the predicted answer accuracy of the corresponding candidate large model when processing the current large model task. Similarly, the satisfaction prediction head outputs a second vector, and each dimension of the first vector corresponds to a candidate large model. Each value in the vector represents the predicted user satisfaction of the corresponding candidate large model when processing the current large model task.

[0046] The training data of the above prediction model needs to be constructed in the following way: collect historical large model tasks, submit them to all candidate large models to obtain multiple answers, and manually label the “accuracy” and “satisfaction” labels for each answer independently. The model learns samples by minimizing the joint loss function of the two prediction heads. The loss function can be obtained by weighting the answer accuracy loss and the user satisfaction loss.

[0047] S130, acquire the task queuing state of each candidate large model, and determine the waiting time of the large model task under each candidate large model based on the task queuing state and the task scene type.

[0048] The task queuing state is the real-time state of the task queue held by each candidate large model and waiting to be processed. The task queuing state includes the number of tasks to be processed by the current candidate large model and the type of each task.

[0049] The waiting time is the time that the current large model task is expected to wait if it is allocated to a certain candidate large model for processing.

[0050] In the embodiment of the application, the historical running data of each candidate large model is obtained, and the historical average running duration of each candidate large model for processing various task scene types is calculated. Then, the task queuing state of each candidate large model is obtained, and for each candidate large model, the first duration for processing the current queued task is calculated according to the task scene type of the task to be processed contained in the task queuing state and the average running time of the candidate large model for processing the task scene type. Then, the average running time of the candidate large model for processing the target task scene type is determined as the second duration according to the target task scene type to which the current large model task belongs. Finally, the first duration and the second duration are summed to obtain the waiting duration of the current large model task under the candidate large model.

[0051] In S140, the target large model for processing the large model task is determined from the plurality of candidate large models based on the answer accuracy, the user satisfaction, the waiting duration and the task importance level of each candidate large model for the large model task.

[0052] In the embodiment of the application, a model cost score is calculated for each candidate large model based on the answer accuracy, the user satisfaction, the waiting duration and the task importance level of each candidate large model for the large model task. The target large model for processing the current large model task is determined from the plurality of candidate large models according to the model cost score corresponding to each candidate large model. The model cost score can be composed of a quality cost part and a time cost part.

[0053] Specifically, the quality cost score is calculated according to the predicted answer accuracy and the user satisfaction, and the time cost score is determined according to the waiting duration. Finally, the quality cost score and the time cost score are weighted and summed based on the importance weight corresponding to the task importance level of the current large model task to obtain the model cost score. The target large model for processing the large model task is selected from the plurality of candidate large models as the candidate large model with the highest model cost score.

[0054] The technical scheme of the embodiment of the present application, in response to receiving a large model task, classifies the large model task by scene to obtain a task scene type, classifies the large model task by importance to obtain a task importance level, predicts the answer accuracy of each candidate large model in multiple candidate large models of different scales for the large model task based on the task scene type, the task importance level and the large model task, and the user satisfaction, and then obtains the task queuing state of each candidate large model, and determines the waiting time of the large model task under each candidate large model based on the task queuing state and the task scene type, and determines the target large model for processing the large model task in multiple candidate large models based on the answer accuracy of each candidate large model for the large model task, the user satisfaction, the waiting time and the task importance level, which can dynamically select a large model that matches the current received large model task in multiple candidate large models of different scales, and balances the user experience and resource cost.

[0055] Embodiment two

[0056] Figure 2a A flowchart of a large model screening method provided by the second embodiment of the present application, which is further refined on the basis of the above-mentioned embodiment, provides specific steps of predicting the answer accuracy of each candidate large model in multiple candidate large models of different scales for the large model task based on the task scene type, the task importance level and the large model task, the user satisfaction, determining the waiting time of the large model task under each candidate large model based on the task queuing state and the task scene type, and determining the target large model for processing the large model task in multiple candidate large models based on the answer accuracy of each candidate large model for the large model task, the user satisfaction, the waiting time and the task importance level. As shown in the figure, Figure 2a The method comprises:

[0057] S210, in response to receiving a large model task, classifying the large model task by scene to obtain a task scene type, and classifying the large model task by importance to obtain a task importance level.

[0058] Optionally, classifying the large model task by scene to obtain a task scene type comprises:

[0059] inputting the large model task into a scene classification model to obtain the task scene type output by the scene classification model;

[0060] The scene classification model is obtained by training a large model task sample and a corresponding scene classification label based on a classification model as a base model; and the output category of the scene classification model corresponds to a plurality of task scene types set in advance.

[0061] In this optional embodiment, a specific method is provided for classifying large-scale model tasks to obtain task scene types: A classification model is pre-trained as the base model, using large-scale model task samples and corresponding scene classification labels to obtain a scene classification model for scene classification. The output categories of the scene classification model correspond to multiple pre-defined task scene types, and the base model can be a BERT text classification model or a BERT+TextCNN model, etc.

[0062] Then, after receiving the large model task, the large model task is input into the scene classification model to obtain the probability of the current large model task belonging to each candidate task scene type output by the scene classification model, thereby determining the candidate task scene type with the highest probability as the task scene type to which the current large model task belongs.

[0063] Optionally, the importance of large model tasks can be classified to obtain task importance levels, including:

[0064] The large model task is input into the importance classification model to obtain the task importance level output by the importance classification model;

[0065] The importance classification model is based on a classification model and is trained using large model task samples and corresponding importance level labels; the output category of the importance classification model corresponds to multiple pre-set task importance levels.

[0066] In this optional embodiment, a specific method is provided for classifying the importance of large model tasks to obtain task importance levels: A classification model is pre-trained as the base model, using large model task samples and corresponding importance level labels to obtain an importance classification model for importance classification. The output categories of the importance classification model correspond to multiple pre-set task importance levels, and the base model can be a BERT text classification model or a BERT+TextCNN model, etc.

[0067] Then, after receiving the large model task, the large model task is input into the importance classification model to obtain the probability of the current large model task belonging to each candidate task importance level as output by the importance classification model, thereby determining the importance level of the candidate task with the highest probability as the task importance level to which the current large model task belongs.

[0068] By classifying large-scale model tasks across different dimensions, we can achieve refined and structured analysis of task attributes. This data serves as the basis for subsequent large-scale model selection, enabling resource scheduling strategies to shift from extensive to precise. This allows us to identify high-importance tasks and prioritize their resource allocation, or to match more suitable models to different scenario tasks, thereby improving the precision of model matching.

[0069] S220, encode the task scene type, task importance level and large model task respectively, input to the prediction model, and obtain the answer accuracy and user satisfaction of each candidate large model for the large model task output by the prediction model.

[0070] The prediction model is trained with historical large model tasks and corresponding task scene types and task importance levels as training samples, and the answer accuracy and user satisfaction labeled by the user for the answer result output by the prediction model as real labels; the prediction model comprises a feature fusion layer, an attention layer and a fully connected layer.

[0071] In the embodiment of the application, the structure of the prediction model is as shown in Figure 2b The current received large model task text is converted into a dense text feature vector through an embedding layer or a pre-trained language model (for example, a Bert encoder). At the same time, the two discrete classification variables of the task scene type and the task importance level are respectively one-hot encoded and converted into sparse category feature vectors. Through a feature fusion layer (for example, feature splicing operation, element-wise addition or a small fully connected network), the above text feature vector and the category feature vector are integrated into a unified joint feature vector rich in semantic and attribute information.

[0072] Further, the fused joint feature vector is input to the attention layer to learn the importance of different parts of the joint feature vector to the final prediction target, capture long-distance dependency relationships, and extract core feature representations for distinguishing different model performances. The feature identification refined by the attention layer is input to a multi-task prediction head composed of a fully connected layer and a Sigmoid function. Specifically, the network structure comprises two parallel and symmetric fully connected network branches, specifically an accuracy prediction head and a satisfaction prediction head.

[0073] Finally, an N-dimensional vector output by the accuracy prediction head is obtained, where N is the number of candidate large models, and the value of the i-th element in the N-dimensional vector represents the predicted answer accuracy of the i-th candidate large model processing the current task (normalized by the Sigmoid function to a probability between 0 and 1). Similarly, an N-dimensional vector output by the satisfaction prediction head is obtained, where N is the number of candidate large models, and the value of the i-th element in the N-dimensional vector represents the predicted user satisfaction of the i-th candidate large model processing the current task (normalized by the Sigmoid function to a probability between 0 and 1). By predicting the answer accuracy and user satisfaction through the prediction model, the complex qualitative evaluation is quantified, the fuzzy experience of model performance is converted into a calculable probability value, and data basis is provided for subsequent accurate cost accounting.

[0074] The training manner of the prediction model is as follows: taking historical large model tasks and corresponding task scene types and task importance levels as training samples, and taking the answer accuracy and user satisfaction labeled by the user according to the answer result of the prediction model output as real labels for training. The prediction model includes a feature fusion layer, an attention layer and a full connection layer.

[0075] In each training process, the accuracy loss is calculated according to the answer accuracy output by the model and the labeled answer accuracy. The satisfaction loss is calculated according to the user satisfaction output by the model and the labeled user satisfaction. The training loss value is determined according to the accuracy loss and the satisfaction loss. The specific loss function is as follows:

[0076]

[0077] wherein, is the accuracy loss, is the satisfaction loss, and are the accuracy loss weight and the satisfaction loss weight, respectively.

[0078]

[0079]

[0080] wherein, C is the number of candidate large models, and are the answer accuracy and the user satisfaction output by the ith candidate large model, and are the labeled labels for the answer accuracy and the user satisfaction.

[0081] S230, according to the historical running data, determine the average running time of each candidate large model when processing the task of each task scene type.

[0082] In the embodiment of the application, the log data of the large model task processing process recorded in the past running process is obtained. The log data at least includes task identification, candidate large model identification for processing the task, task scene type to which the task belongs and running time consumed from starting processing the task to returning the result. According to the above log data, the mean value of the running time of each candidate large model when processing the large model task of each task scene type, i.e. the average running time, is calculated.

[0083] S240, obtain the task queuing state of each candidate large model at present, and the task queuing state includes the number of tasks of each task scene type.

[0084] In the embodiment of the application, the task queuing state of each candidate large model is obtained, including the number of tasks to be processed of the current candidate large model and the task scene type of each task, and the to-be-processed task in the task queue can also be input to the scene classification model to obtain the task scene type of each task.

[0085] In S250, the waiting time of the large model task under each candidate large model is calculated according to the task scene type of the large model task, the average running time and the task queuing state.

[0086] In the embodiment of the application, for each candidate large model, the first time length for processing the current queued task is calculated according to the task scene type of the to-be-processed task and the average running time of the candidate large model for processing tasks of each task scene type. Then, the average running time of the candidate large model for processing tasks of the target task scene type to which the current large model task belongs is determined as the second time length according to the target task scene type of the current large model task. Finally, the first time length and the second time length are summed to obtain the waiting time of the current large model task under the candidate large model, and the specific calculation formula is as follows:

[0087]

[0088] Among them, is the waiting time required for the i th candidate large model to process the current large model task, is the average running time of the i th candidate large model for processing tasks of the j th task scene type, is the number of tasks of the j th task scene type in the task queue of the i th candidate large model, is the average running time of the i th candidate large model for processing the m th task scene type to which the current large model task belongs.

[0089] 260, for each candidate large model, the complement of the answer accuracy and the user satisfaction are calculated respectively, and the complement of the answer accuracy and the user satisfaction are weighted and summed to obtain a first evaluation score.

[0090] In the embodiment of the application, the answer accuracy and the user satisfaction for the current large model task are obtained from the output result of the prediction model, the difference between 1 and the answer accuracy and the difference between 1 and the user satisfaction are calculated respectively to obtain the complement of the answer accuracy and the user satisfaction. Then, based on the complement of the answer accuracy and the complement of the answer accuracy, the weighted sum calculation is performed to obtain the first evaluation score. The sum of the weight of the user satisfaction and the weight of the answer accuracy is 1.

[0091] In S270, the waiting time and the first evaluation score are weighted and summed based on the preset waiting time weight, the first evaluation score weight and the importance weight corresponding to the task importance level to obtain a model cost score.

[0092] In the embodiment of the application, the waiting time length and the first evaluation score are weighted and summed based on the preset waiting time weight, the first evaluation score weight and the importance weight corresponding to the task importance level to obtain a model cost score. Specifically, the product of the waiting time weight and the waiting time is calculated, and the product of the first evaluation score weight, the importance weight and the first evaluation score is calculated, and the two products are summed to obtain the model cost score. The specific calculation formula is as follows:

[0093]

[0094] wherein, is the model cost score of the i th candidate large model, and are the waiting time weight and the first evaluation score weight respectively, is the importance weight, is the answer accuracy weight, is the answer accuracy, is the user satisfaction.

[0095] Optionally, the importance weight corresponding to the task importance level is positively correlated with the task importance level.

[0096] In the optional embodiment, the importance weight corresponding to the task importance level is positively correlated with the task importance level, for example, the importance weight is 0.3 for general importance, the importance weight is 0.6 for moderate importance, and the importance weight is 1 for very important.

[0097] S280, according to the model cost score corresponding to each candidate large model, determining a target large model for processing the large model task from the plurality of candidate large models.

[0098] In the embodiment of the application, after the model cost score of each candidate large model in processing the current large model task is calculated, the candidate large model with the minimum model cost score is selected as the target large model for processing the large model task. By calculating the model cost score containing multiple dimensions such as waiting time, answer accuracy, task importance level and user satisfaction, the target large model is selected, which can balance the user experience and resource cost compared with deploying a single large model service.

[0099] The technical scheme of the embodiment of the application jointly calculates the model cost score of each candidate large model based on the waiting time, the predicted answer accuracy and the predicted user satisfaction of each candidate large model in processing the current large model task, and finally selects the candidate large model with the minimum model cost score to process the current large model task, which can comprehensively consider the reply quality and speed of the task and can balance the user experience and resource cost.

[0100] Embodiment three

[0101] Figure 3 A structural schematic diagram of a large model screening device provided for embodiment three of the application. As shown in the figure, the device comprises: Figure 3

[0102] The task classification module 310 is configured to perform scene classification on the large model task in response to receiving the large model task, to obtain a task scene type, and to perform importance classification on the large model task, to obtain a task importance level.

[0103] The accuracy prediction module 320 is configured to predict, based on the task scene type, the task importance level and the large model task, the answering accuracy of each candidate large model in the plurality of candidate large models for the large model task, and the user satisfaction.

[0104] The waiting time estimation module 330 is configured to obtain the task queuing state of each candidate large model, and to determine, based on the task queuing state and the task scene type, the waiting time of the large model task under each candidate large model.

[0105] The large model screening module 340 is configured to determine, based on the answering accuracy of each candidate large model for the large model task, the user satisfaction, the waiting time and the task importance level, a target large model for processing the large model task from the plurality of candidate large models.

[0106] The technical solution of the embodiment of the application is configured to perform scene classification on the large model task in response to receiving the large model task, to obtain a task scene type, and to perform importance classification on the large model task, to obtain a task importance level, to predict, based on the task scene type, the task importance level and the large model task, the answering accuracy of each candidate large model in the plurality of candidate large models for the large model task, and the user satisfaction, to further obtain the task queuing state of each candidate large model, and to determine, based on the task queuing state and the task scene type, the waiting time of the large model task under each candidate large model, and to determine, based on the answering accuracy of each candidate large model for the large model task, the user satisfaction, the waiting time and the task importance level, a target large model for processing the large model task from the plurality of candidate large models, so that a large model that matches the currently received large model task can be dynamically selected from the plurality of candidate large models of different scales, and a balance between user experience and resource cost is achieved.

[0107] Optionally, the task classification module 310 is specifically configured to:

[0108] ​input the large model task into a scene classification model to obtain a task scene type output by the scene classification model;

[0109] The scene classification model is a classification model as a base model, and is trained by using a large model task sample and a corresponding scene classification label; and an output category of the scene classification model corresponds to a plurality of task scene types preset in advance.

[0110] Optionally, the task classification module 310 is further configured to:

[0111] input the large model task into an importance classification model to obtain a task importance level output by the importance classification model;

[0112] The importance classification model is a classification model as a base model, and is trained by using a large model task sample and a corresponding importance level label; and an output category of the importance classification model corresponds to a plurality of task importance levels preset in advance.

[0113] Optionally, the accuracy prediction module 320 is specifically configured to:

[0114] encode the task scene type, the task importance level and the large model task respectively, and input the encoded task scene type, the task importance level and the large model task into a prediction model to obtain an answer accuracy and a user satisfaction degree of each candidate large model for the large model task output by the prediction model;

[0115] The prediction model is trained by using historical large model tasks and corresponding task scene types and task importance levels as training samples, and by using an answer accuracy and a user satisfaction degree annotated by a user for an answer result output by the prediction model as real labels; and the prediction model comprises a feature fusion layer, an attention layer and a fully connected layer.

[0116] Optionally, the waiting time estimation module 330 is specifically configured to:

[0117] determine an average running time of each candidate large model when processing a task of each task scene type according to historical running data;

[0118] obtain a task queuing state of each candidate large model at present, wherein the task queuing state comprises a task quantity of each task scene type;

[0119] calculate a waiting time of the large model task under each candidate large model according to the task scene type of the large model task, the average running time and the task queuing state.

[0120] Optionally, the large model screening module 340 is specifically configured to:

[0121] For each candidate large model, the complement of the answer accuracy and the user satisfaction is calculated respectively, and the complement of the answer accuracy and the user satisfaction is weighted and summed to obtain a first evaluation score;

[0122] Based on the preset waiting time weight, the first evaluation score weight and the importance weight corresponding to the task importance level, the waiting time length and the first evaluation score are weighted and summed to obtain a model cost score;

[0123] According to the model cost score corresponding to each candidate large model, a target large model for processing the large model task is determined from the plurality of candidate large models.

[0124] Optionally, the importance weight corresponding to the task importance level is positively correlated with the task importance level.

[0125] The large model screening device provided by the embodiment of the application can execute the large model screening method provided by any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0126] In the technical solution of the application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of countries and regions, necessary security measures are taken, public order and good customs are not violated, and corresponding operation entrances are provided for the user to select authorization or refusal.

[0127] Embodiment four

[0128] According to the embodiments of the application, the application further provides an electronic device, a readable storage medium and a computer program product.

[0129] Figure 4 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, appliances, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the applications described and / or claimed in this document.

[0130] As Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0132] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the large model screening method.

[0133] In some embodiments, the large model screening method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large model screening method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the large model screening method by any other appropriate means, such as by means of firmware.

[0134] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0135] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program

[0136] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0138] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data application server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0139] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing application system, and solves the defects of large management difficulty and weak business scalability in traditional physical host and VPS application.

[0140] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, the steps recited in the present invention can be performed in parallel, in series, or in a different order, without limitation herein, as long as the desired results of the technical solutions of the present invention can be achieved.

[0141] The specific embodiments described above are not intended to be limiting, but rather to illustrate a few possible forms in which the present invention can be implemented. Many modifications, combinations, sub-combinations and alternatives are possible in light of the above teachings. Any and all such modifications, variations, and alternatives are intended to fall within the scope of the present invention.

Claims

1. A method for screening large models, characterized in that, include: In response to receiving a large model task, the large model task is classified into scenarios to obtain task scenario types, and the large model task is classified into importance to obtain task importance levels. Based on the task scenario type, task importance level, and large model task, predict the accuracy of each candidate large model's response to the large model task and user satisfaction among multiple candidate large models of different scales. Obtain the task queuing status of each candidate large model, and determine the waiting time of the large model task under each candidate large model based on the task queuing status and the task scenario type; Based on the accuracy of each candidate large model's response to the large model task, user satisfaction, waiting time, and task importance level, a target large model for handling the large model task is determined from among multiple candidate large models.

2. The method according to claim 1, characterized in that, The large model task is classified into scenarios to obtain task scenario types, including: The large model task is input into the scene classification model to obtain the task scene type output by the scene classification model; The scene classification model is based on a classification model and is trained using large model task samples and corresponding scene classification labels; the output category of the scene classification model corresponds to multiple pre-set task scene types.

3. The method according to claim 1, characterized in that, The importance of the large model tasks is classified to obtain task importance levels, including: The large model task is input into the importance classification model to obtain the task importance level output by the importance classification model; The importance classification model is based on a classification model and is trained using large model task samples and corresponding importance level labels; the output category of the importance classification model corresponds to multiple pre-set task importance levels.

4. The method according to claim 1, characterized in that, Based on the task scenario type, task importance level, and large model task, predict the accuracy of each candidate large model's response to the large model task among multiple candidate large models of different scales, as well as user satisfaction, including: After encoding the task scenario type, task importance level, and large model task respectively, the results are input into the prediction model to obtain the accuracy and user satisfaction of each candidate large model's response to the large model task. The prediction model is trained using historical large-scale model tasks and their corresponding task scenario types and task importance levels as training samples, and using the accuracy of user responses and user satisfaction as real labels for the output of the prediction model. The prediction model includes a feature fusion layer, an attention layer, and a fully connected layer.

5. The method according to claim 1, characterized in that, Based on the task queuing status and the task scenario type, the waiting time for the large model task under each candidate large model is determined, including: Based on historical running data, determine the average running time of each candidate large model when processing tasks of each task scenario type; Obtain the current task queuing status of each candidate large model, where the task queuing status includes the number of tasks for each task scenario type; Based on the task scenario type of the large model task, the average running time, and the task queuing status, calculate the waiting time of the large model task under each candidate large model.

6. The method according to claim 1, characterized in that, Based on the accuracy of each candidate large model's response to the large model task, user satisfaction, waiting time, and task importance level, a target large model for handling the large model task is determined from multiple candidate large models, including: For each candidate large model, the complement of the answer accuracy and user satisfaction is calculated, and the complement of the answer accuracy and user satisfaction is weighted and summed to obtain the first evaluation score; Based on the preset waiting time weight, the first evaluation score weight, and the importance weight corresponding to the task importance level, the waiting time and the first evaluation score are weighted and summed to obtain the model cost score; Based on the model cost score corresponding to each candidate large model, a target large model for processing the large model task is determined from among multiple candidate large models.

7. The method according to claim 6, characterized in that, The importance weight corresponding to the task importance level is positively correlated with the task importance level.

8. A large model screening device, characterized in that, include: The task classification module is used to respond to the receipt of a large model task, classify the large model task into scenarios to obtain task scenario types, and classify the large model task into importance levels to obtain task importance levels. The accuracy prediction module is used to predict the accuracy of each candidate large model's response to the large model task, as well as user satisfaction, based on the task scenario type, task importance level, and large model task. The waiting time estimation module is used to obtain the task queuing status of each candidate large model, and determine the waiting time of the large model task under each candidate large model based on the task queuing status and the task scenario type. The large model screening module is used to determine the target large model for processing the large model task from multiple candidate large models based on the accuracy of each candidate large model's response to the large model task, user satisfaction, waiting time, and task importance level.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the large model screening method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the large model screening method according to any one of claims 1-7.

11. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the large model screening method according to any one of claims 1-7.

Images