Model routing method and device

Abstract

The invention discloses a model routing processing method and device, and the method comprises the steps: selecting a target object from a plurality of objects through a model routing module in response to input data; the plurality of objects comprise a plurality of local objects and a plurality of cloud models; the local object is composed of prompt information and a local model; the prompt information is used for guiding the local model to perform reasoning; and based on the target object, processing the input data to obtain a reasoning result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model routing method and apparatus. Background Technology

[0002] In hybrid model routing schemes, a common architecture consists of a local model and multiple cloud models. The routing engine dynamically selects either the local or cloud model to perform inference. While cloud models can enhance the ability of local models to handle complex tasks, this approach suffers from network transmission latency in practical applications, impacting user experience. Summary of the Invention

[0003] The technical solution provided in this application is as follows:

[0004] The first aspect of this application provides a model routing processing method, including:

[0005] In response to input data, a target object is selected from multiple objects through a model routing module; the multiple objects include multiple local objects and multiple cloud models; the local object consists of prompt information and a local model; the prompt information is used to guide the local model to perform inference.

[0006] Based on the target object, the input data is processed to obtain the reasoning result.

[0007] In one possible implementation, the step of selecting a target object from multiple objects in response to input data via a model routing module includes:

[0008] In response to input data, a candidate local object and a candidate cloud model are selected from multiple objects through a model routing module; the candidate local object has the highest prediction index among the multiple local objects; the candidate cloud model has the highest prediction index among the multiple cloud models; the prediction index characterizes the reasoning ability of the local object or the cloud model corresponding to the input data.

[0009] If the difference in the prediction metrics between the candidate local object and the candidate cloud model meets a set difference threshold, the candidate local object is identified as the target object.

[0010] In one possible implementation, the step of selecting a target object from multiple objects in response to input data via a model routing module includes:

[0011] In response to input data, the model routing module determines the prediction index of each local object among multiple objects; the prediction index characterizes the reasoning ability of the local object corresponding to the input data.

[0012] Select one of the local objects whose prediction metric satisfies the set prediction metric threshold and determine it as the target object.

[0013] In one possible implementation, the model routing processing method further includes:

[0014] Obtain the first sample task type; the task of the first sample task type is inferred based on the cloud model;

[0015] Based on the first sample task type and the local model, construct the prompt information;

[0016] Based on the prompt information and the local model, a local object is constructed.

[0017] In one possible implementation, the model routing processing method further includes:

[0018] Obtain the sample input data corresponding to the first sample task type;

[0019] The sample input data is input into the local object, which processes it to obtain the sample inference result.

[0020] Based on the difference between the sample inference result and the benchmark inference result, update the prompt information in the local object;

[0021] The benchmark inference result is obtained by processing the sample input data corresponding to the first sample task type by the cloud model.

[0022] In one possible implementation, constructing the prompt information based on the first sample task type and the local model includes:

[0023] Based on the first sample task type, target parameters are determined; the target parameters represent at least one of the execution objective, constraints on the reasoning process, and input / output format corresponding to the first sample task type.

[0024] Based on the target parameters and the local model, a prompt message is constructed.

[0025] In one possible implementation, obtaining the first sample task type includes:

[0026] Obtain the first historical data of the local model and the multiple cloud models during the historical inference process;

[0027] Based on the first historical data, the task types whose frequency of calling the cloud model meets the set frequency threshold are determined as the first sample task types.

[0028] In one possible implementation, the model routing processing method further includes:

[0029] Under the condition of meeting the set triggering conditions, the second historical data of the local model and the multiple cloud models in the historical reasoning process is obtained;

[0030] Based on the second historical data, it is determined that there is a second sample task type that is different from the first sample task type.

[0031] Based on the second sample task type, construct a second local object.

[0032] In one possible implementation, the selection of a target object from multiple objects via the model routing module in response to input data includes at least one of the following:

[0033] In response to input data, a predicted score for each of the multiple objects is determined by the model routing module; the predicted score includes the object's capability score on a defined task dimension.

[0034] Based on the predicted score of each of the objects, a target object is selected from the plurality of objects;

[0035] In response to input data, the task features corresponding to the input data are determined; the task features characterize the target task attributes corresponding to the input data.

[0036] Determine the similarity between the task features and the task attributes of each of the multiple objects;

[0037] Based on the similarity, a target object is selected from the plurality of objects.

[0038] Another aspect of this application provides a processing apparatus, comprising:

[0039] The model routing module is used to select a target object from multiple objects in response to input data; the multiple objects include multiple local objects and multiple cloud models; the local objects consist of prompt information and local models; the prompt information is used to guide the local models to perform inference.

[0040] The processing module is used to process the input data based on the target object to obtain the reasoning result. Attached Figure Description

[0041] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0042] Figure 1This is a flowchart illustrating a model routing processing method provided in Embodiment 1 of this application;

[0043] Figure 2 This is a flowchart illustrating a model routing processing method provided in Embodiment 2 of this application;

[0044] Figure 3 This is a flowchart illustrating a model routing processing method provided in Embodiment 3 of this application;

[0045] Figure 4 This is a flowchart illustrating a model routing processing method provided in Embodiment 4 of this application;

[0046] Figure 5 This is a flowchart illustrating a model routing processing method provided in Embodiment 5 of this application;

[0047] Figure 6 This is a flowchart illustrating a model routing processing method provided in Embodiment 6 of this application;

[0048] Figure 7 This is a flowchart illustrating a model routing processing method provided in Embodiment 7 of this application;

[0049] Figure 8 This is a schematic diagram of the structure of a model routing device provided in this application. Detailed Implementation

[0050] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0051] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0052] The terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0053] Reference Figure 1This is a flowchart illustrating a model routing processing method provided in Embodiment 1 of this application. Figure 1 As shown, the method may include the following steps:

[0054] Step S101: In response to the input data, a target object is selected from multiple objects through the model routing module; the multiple objects include multiple local objects and multiple cloud models; the local object consists of prompt information and a local model; the prompt information is used to guide the local model to perform inference.

[0055] In this embodiment, the local model in the local object has multi-dimensional capabilities, such as coding, mathematical calculation, language processing, and image recognition. These capabilities work together and are used in a comprehensive manner, enabling the local model to be widely and efficiently adapted to a variety of different application scenarios.

[0056] A local model can include, but is not limited to, an input layer, a core processing layer, and an output layer. The input layer acts as a data receiving port, responsible for receiving different types of data, such as images and text, providing raw materials for subsequent processing. The core processing layer is the heart of the model, containing multiple functional modules that work closely together to complete complex data processing tasks. The output layer is responsible for presenting the processing results in an appropriate format, ensuring that users can intuitively and conveniently obtain the information they need.

[0057] The following sections will use image recognition, language processing, mathematical calculations, and code generation as examples to illustrate several functional modules. For instance, the image recognition module can employ a Convolutional Neural Network (CNN) architecture from deep learning, possessing powerful image feature extraction and recognition capabilities. Through multi-layer convolution and pooling operations, it can automatically learn various features in images, such as edges, textures, and shapes, thereby achieving accurate location and classification of surface defects in products.

[0058] The language processing module, based on Natural Language Processing (NLP) technology, performs semantic understanding and analysis of text. It can identify entities, relationships, sentiment, and other information in the text, extracting key data and knowledge to provide a foundation for subsequent processing.

[0059] The mathematical calculation module can perform various numerical calculations, statistical analyses, model solving, and other operations. It supports multiple data types and calculation precisions to meet the computing needs of different industrial scenarios.

[0060] The code generation module possesses a certain level of code generation and comprehension capabilities, automatically generating corresponding code scripts based on specific task requirements and data formats. Simultaneously, it can parse and optimize existing code, improving execution efficiency and maintainability.

[0061] In this embodiment, the local model is described in conjunction with a specific application scenario. For example, in an industrial manufacturing scenario, when faced with a product surface defect detection task, the input layer of the local model can receive the product surface image data and then pass it to the image recognition module of the core processing layer.

[0062] The image recognition module utilizes a CNN architecture for rapid image scanning and analysis. Convolutional layers extract feature maps from the image, and pooling layers reduce the dimensionality of these feature maps, minimizing computation while preserving important features. Then, fully connected layers classify and regress the extracted features to locate potentially defective regions. For example, when inspecting the surface of a mobile phone casing, it can accurately identify different types of defects such as scratches, dents, and color differences, and mark their locations in the image.

[0063] The image recognition module transmits the located defect area information to the mathematical calculation module. The mathematical calculation module analyzes the pixel data of these areas in real time, accurately measuring the dimensions of the defect, such as length, width, and diameter; it calculates the area of ​​the defect using methods such as pixel counting or geometrically based formulas. Simultaneously, it combines this quantitative information with features such as location and shape in the image to provide a comprehensive basis for assessing the severity of the defect. For example, based on the size and area of ​​the defect, combined with preset scoring criteria, a severity score based on mathematical calculations is given.

[0064] The output layer presents the defect detection results to users in an intuitive way, such as generating a detection report that includes the location, size, shape, and severity score of the defect, or annotating the defect area on the original image, providing accurate decision support for product selection and repair.

[0065] Hints can provide local models with inference direction and focus. For example, hints can guide the local model to focus on specific regions or specific types of lesion features in images, thereby improving the efficiency and accuracy of medical image analysis. Alternatively, hints can prompt the local model to focus on certain key information in medical records, such as main symptoms and severity, enabling the model to quickly and accurately extract useful information from the medical records.

[0066] After receiving input data, the model routing module can select the target object based on multiple factors. For example, it can analyze characteristics such as the type, complexity, and real-time requirements of the input data. For instance, for simple medical data computation tasks with high real-time requirements, if the local model can process the data accurately with guidance, the local object will be prioritized; for particularly complex medical image depth analysis tasks requiring large amounts of data and powerful computing capabilities, a cloud model might be chosen. On the other hand, the current status of the local object and the cloud model can be considered, such as the load of the local model and the availability of the cloud model. By comprehensively considering these factors, the model routing module can intelligently select the most suitable target object to achieve efficient and accurate processing.

[0067] The target object selected from multiple objects by the model routing module could be a local object or a cloud model.

[0068] Step S102: Based on the target object, process the input data to obtain the reasoning result.

[0069] When the target object is a local object, the local model, guided by prompts, fully utilizes its computing power and storage resources to perform a series of inference operations on the input data, including feature extraction, pattern recognition, and classification. Due to the guidance of prompts, the local model can utilize its resources more efficiently and clarify the direction of inference. When handling relatively complex tasks that are within its capabilities and for which the prompts provide clear direction, it can achieve high inference accuracy.

[0070] For example, in an industrial manufacturing scenario, the prompt message could be set to "Focus on areas in the image with abnormal color contrast, linear patterns, and lengths ranging from 0.1 to 0.5 millimeters. These areas may contain minor scratches that meet production standards." This prompt message clearly defines the detection range and feature criteria for the local model, allowing it to clearly understand the image features that need to be focused on.

[0071] After receiving the product surface image data, the input layer of the local model transmits the data to the image recognition module of the core processing layer. The image recognition module performs a preliminary scan of the image based on the prompts, filtering out areas with abnormal color contrast. Because the color distribution on a normal product surface is relatively uniform, areas with abnormal color contrast are likely to contain defects such as scratches. Next, these abnormal areas are further filtered to identify those with straight lines, excluding non-linear color anomalies caused by impurities, stains, etc.

[0072] For the selected linear color anomaly regions, the length is measured using a mathematical calculation module. By counting pixels and combining this with a pre-set conversion ratio between pixels and actual length, the length of each region is accurately calculated.

[0073] Compare the measured length with the 0.1-0.5 mm range specified in the prompt message. If the length is within this range, the area is considered to have a minor scratch that meets the requirements; if the length is outside this range, the area is excluded.

[0074] After the above series of operations, the local model can quickly and accurately identify minute scratches on the product surface that match the characteristics of the prompt information, and determine their location and length in the image. Finally, it generates a detection report containing detailed information such as scratch location and length as the inference result. In this way, the prompt information provides clear inference guidance for the local model, greatly improving detection efficiency and accuracy.

[0075] When the target object is a cloud model, the input data is transmitted to the cloud via the network and processed by the cloud model. Leveraging its powerful computing capabilities and abundant data resources, the cloud model can perform in-depth analysis and processing of particularly complex tasks, yielding accurate inference results.

[0076] In this embodiment, because the local object, guided by the prompts, can handle more relatively complex tasks, the model routing module can reduce unnecessary cloud calls and select the local model for processing when choosing the target object. Guided by the prompts, the local model directly completes data processing and inference operations locally, thereby eliminating network transmission latency while ensuring the accuracy of the inference results. This improvement enables a faster response to input data, providing users with a smoother and more efficient user experience.

[0077] As more and more data processing is done locally, the amount of data transmitted over the network is significantly reduced. Data no longer needs to be frequently transferred between local devices and cloud servers, reducing the risk of malicious interception, theft, or tampering during transmission. This shift in data processing fundamentally enhances data confidentiality and integrity, providing more reliable protection for users' sensitive information, effectively improving system privacy and security, and meeting users' high demands for data security. At the same time, it also saves on costs such as network traffic charges incurred from accessing cloud models.

[0078] Moreover, the use of local objects makes the system less dependent on network conditions to a certain extent, and it can still perform some processing tasks in environments with unstable or no network, thus enhancing network independence.

[0079] As another optional embodiment of this application, refer to Figure 2This is a flowchart illustrating a model routing processing method provided in Embodiment 2 of this application. This embodiment mainly describes one implementation of step S101 above. Figure 2 As shown, step S101 may include, but is not limited to, the following steps:

[0080] Step S1011: In response to the input data, candidate local objects and candidate cloud models are selected from multiple objects through the model routing module; the candidate local object has the highest prediction index among the multiple local objects; the candidate cloud model has the highest prediction index among the multiple cloud models; the prediction index represents the level of reasoning ability of the local object or the cloud model corresponding to the input data.

[0081] In this embodiment, the factors on which the prediction metrics are based can be set as needed and are not limited in this application. For example, the prediction metrics can be set based on the model's historical processing accuracy, processing speed, resource utilization, etc.

[0082] In this embodiment, the task type corresponding to the input data can be identified to determine the target task type. The method of identifying the task type is not limited in this application. For example, it can be determined whether the input data contains key content for a defined task type. For instance, text data containing specific keywords (such as "summation," "average," etc.) can be identified as a mathematical calculation task; data containing image file formats (such as .jpg, .png) and whose file size exceeds a certain threshold can be identified as an image processing task.

[0083] For local objects, the prediction metric can be obtained by statistically analyzing the performance of the local model on past tasks similar to the target task. For example, by statistically analyzing the accuracy, recall, and other metrics of the local model when processing a specific type of image recognition task, a prediction value reflecting its reasoning ability can be calculated.

[0084] For cloud-based models, we can also collect relevant data on their handling of a large number of tasks in the cloud, such as average processing time and error rate, and use this as a basis to determine their predictive metrics.

[0085] In this way, the model routing module can filter from multiple local objects and multiple cloud models to select the candidate local objects and candidate cloud models with the highest inference potential for the current input data. For example, in an image classification task, suppose there are three local objects (local object A, local object B, and local object C) and two cloud models (cloud model X and cloud model Y). The model routing module calculates the prediction metrics for each of these five objects for the current input image. Suppose local object B has the highest accuracy in processing similar images in the past, with a prediction metric of 0.9; cloud model X performs best in processing image classification tasks in the cloud, with a prediction metric of 0.95. Then, local object B will be selected as a candidate local object, and cloud model X will be selected as a candidate cloud model.

[0086] In this embodiment, the model routing module can also determine the predicted score for each of the multiple objects and use this predicted score as the prediction metric. The predicted score can cover the object's ability score on a set task dimension (such as mathematical calculation, text understanding, image processing, etc.). For example, for an object that has both mathematical calculation and text understanding capabilities (e.g., a local object or a cloud model), it can be evaluated from the two dimensions of mathematical calculation and text understanding respectively, and a corresponding ability score can be assigned according to its performance in each dimension. Then, the ability scores of these two dimensions are combined according to certain rules (such as average calculation or weighted calculation) to obtain the predicted score of the object, which is used as the prediction metric.

[0087] Step S1012: If the difference in the prediction metrics between the candidate local object and the candidate cloud model meets the set difference threshold, the candidate local object is determined as the target object.

[0088] When the difference in prediction metrics between a candidate local object and a candidate cloud model meets a set difference threshold, it indicates that the reasoning ability of the candidate local object in the current task is quite close to that of the candidate cloud model. For example, in a preliminary medical image screening scenario, with a difference threshold of 0.05, the prediction metric of the candidate local object (local medical image analysis model) is 0.85, and the prediction metric of the candidate cloud model (a more powerful cloud-based medical image analysis system) is 0.88, resulting in a difference of 0.03, which is less than 0.05. This indicates that the local model already possesses the ability to perform relatively accurate preliminary analysis of this type of image, ensuring that the inference results reach an acceptable quality level and meet the needs of preliminary screening.

[0089] If we always choose a cloud model simply because it is theoretically more capable, without considering the actual capabilities of the local object, it may lead to excessive reliance on cloud resources for tasks that the local object can handle perfectly well. By setting a difference threshold, we can choose the local object when its capabilities are sufficient, ensuring result quality while avoiding unnecessary cloud calls. This prevents processing interruptions or delays caused by network problems or cloud service failures that may result from over-reliance on the cloud model, thus ensuring stable output of inference results.

[0090] The difference threshold can be flexibly set based on the needs and experience of the actual application scenario. For example, in scenarios where processing accuracy requirements are not extremely high, but processing speed and cost control are more important, the difference threshold can be set relatively large, such as 0.2. This means that as long as the predicted metric of the candidate local object differs from that of the candidate cloud model by no more than 0.2, the candidate local object can be selected as the target object. However, in scenarios with extremely strict processing accuracy requirements, such as medical image diagnosis, the difference threshold may be set smaller, such as 0.05, to ensure that the local object is only selected when its inference ability is very close to that of the cloud model, thereby guaranteeing the accuracy of the inference results.

[0091] In this embodiment, continuing with the image classification task described above, we assume a difference threshold of 0.1. We have already identified the candidate local object as local object B (prediction metric 0.9) and the candidate cloud model as cloud model X (prediction metric 0.95). The difference in their prediction metrics is 0.95 - 0.9 = 0.05. Since 0.05 is less than the set difference threshold of 0.1, the model routing module will identify the candidate local object (local object B) as the target object, and the input image will subsequently be processed by local object B.

[0092] In this embodiment, by employing a selection strategy based on differences in prediction metrics and setting difference thresholds, the model routing module can more intelligently and accurately select target objects. While ensuring the quality of inference results, it prioritizes processing local objects as much as possible, thereby reducing unnecessary cloud calls, lowering network transmission latency, and improving overall system performance and user experience.

[0093] As another optional embodiment of this application, refer to Figure 3 This is a flowchart illustrating a routing processing method provided in Embodiment 3 of this application. This embodiment mainly describes one implementation of step S101 above. Figure 3 As shown, step S101 may include, but is not limited to, the following steps:

[0094] Step S1013: In response to the input data, the prediction index of each local object among multiple objects is determined by the model routing module; the prediction index characterizes the reasoning ability of the local object corresponding to the input data.

[0095] From a resource constraint perspective, in some application scenarios, local devices may have limited computing resources, storage resources, or network bandwidth. Frequent calls to cloud models not only consume a large amount of network traffic and increase network transmission costs, but may also lead to excessively long processing times due to network latency. For example, on some mobile or embedded devices, network connections may be unstable and bandwidth limited. If cloud models are considered for every input data processing, it will severely impact user experience and system performance. Therefore, in this embodiment, only the prediction metrics of the local object can be determined first, and the ability of the local object to meet the task requirements can be evaluated first. This can avoid unnecessary cloud calls to a certain extent, reducing resource consumption and costs.

[0096] From a real-time perspective, tasks with extremely high real-time requirements, such as real-time decision-making in autonomous driving and real-time quality inspection in industrial production, need to provide inference results within a very short time. Cloud-based model processing typically involves multiple steps, including data uploading, cloud computing, and result return, inevitably introducing some latency. Local objects, on the other hand, run directly on local devices, enabling rapid response to input data and the provision of inference results. Therefore, in such cases, determining the predictive metrics of local objects first and prioritizing their use can better meet the real-time requirements of the task.

[0097] From a data security perspective, some input data may contain sensitive information, such as users' personal privacy data or companies' trade secrets. Uploading such data to the cloud for processing may pose a risk of data leakage. By determining only the predictive metrics for local objects and processing them directly locally when the local objects can meet the task requirements, data privacy and security can be effectively protected, preventing data from being stolen or tampered with during network transmission.

[0098] In this embodiment, the method for determining the prediction index of each local object can refer to the relevant method of prediction index introduced in Embodiment 2, and will not be repeated here.

[0099] Step S1014: Select one of the local objects whose prediction index meets the set prediction index threshold and determine it as the target object.

[0100] Setting a prediction metric threshold is a quantitative requirement for the reasoning ability of a local object. When the prediction metric of a local object reaches or exceeds this threshold, it is identified as a target object. This ensures that the selected local object has sufficient reasoning ability to provide accurate and reliable inference results, thereby guaranteeing the performance and quality of the entire system. For example, in an image recognition task, setting the prediction metric threshold to 0.8 means that only when the accuracy of the local model reaches or exceeds 80% will it be used as a target object for recognition of the input image. This avoids the problem of excessively high recognition error rates caused by selecting local objects with insufficient reasoning ability.

[0101] When multiple local objects exist, a prediction metric threshold can be set to filter them, allowing the selection of local objects with stable performance and reliable inference capabilities as target objects. This avoids the problem of large fluctuations in inference results due to the selection of unstable local objects, thus improving the stability and reliability of the system. For example, in a multi-model parallel image processing system, there may be multiple local image processing models, and the performance of each model may vary due to factors such as data distribution and model training conditions. By setting a prediction metric threshold, a local model with stable performance and strong inference capabilities can be selected as the target object, ensuring that the system provides stable and reliable inference results under different conditions.

[0102] In this embodiment, in response to input data, the model routing module determines the prediction index of each local object among multiple objects, and selects one of the local objects whose prediction index meets the set prediction index threshold as the target object. This can ensure the accuracy of the inference results while minimizing unnecessary cloud calls, directly reducing network traffic consumption and network transmission costs, reducing the problem of excessive processing time caused by network latency, and improving the user experience.

[0103] As another optional embodiment of this application, refer to Figure 4 This is a flowchart illustrating a model routing processing method provided in Embodiment 4 of this application. Figure 4 As shown, the method may include, but is not limited to, the following steps:

[0104] Step S201: Obtain the first sample task type; the task of the first sample task type is inferred based on the cloud model.

[0105] The cloud-based model generates a large number of task logs during operation. These logs record detailed information about each executed task, including task name, task type, input data characteristics, key parameters of the processing process, and output results.

[0106] In this embodiment, these logs can be comprehensively reviewed and categorized using specialized log analysis tools and techniques, enabling the extraction of different task types. Data mining algorithms can be used to cluster task types, grouping similar task types together; alternatively, initial screening can be performed based on keywords in the task names, followed by further refinement of the classification by combining input data features and key processing parameters. Through these methods, representative task types that demonstrate the advantages of the cloud-based model can be selected as the first sample task types.

[0107] Step S202: Based on the first sample task type and the local model, construct prompt information.

[0108] The first sample task type clearly defines the task direction that the local model needs to learn and process. While the local model may possess multiple potential capabilities, by constructing cue information relevant to the first sample task type, it can be guided to focus its attention and computational resources on this specific task, avoiding wasting resources on irrelevant tasks and improving the model's learning efficiency and inference accuracy. For example, if the first sample task type is "animal recognition" in image classification, the cue information can explicitly tell the local model to focus on animal features in the image and ignore background and other irrelevant information.

[0109] Cloud-based models are typically trained on large datasets and optimized with complex algorithms, possessing rich knowledge and experience in handling tasks similar to the first sample. Local models, however, may lack this knowledge due to limited data or insufficient training time. Hints can transfer key knowledge, rules, and techniques from the cloud model to the local model, helping it quickly master effective methods for handling the task. For example, in text sentiment analysis, the cloud model may understand the correlation between specific words and expressions and sentiment. Hints can convey this knowledge to the local model, enabling it to more accurately determine the sentiment of the text.

[0110] Step S203: Construct a local object based on the prompt information and the local model.

[0111] In this embodiment, the structure and parameters of the local model itself can be kept unchanged, and the prompt information can be associated with the local model to form an organic whole, so that the local model can better handle specific tasks under the guidance of the prompt information.

[0112] The association between prompts and the local model can be achieved in several ways. For example, relevant fields for the prompts can be added to the local model's configuration file, or a dedicated variable can be set up in the local model's runtime environment to store the prompts. Taking a local model for image classification as an example, if the prompts indicate that the model needs to focus on the shape and color features of specific objects in the image, then corresponding fields can be added to the local model's configuration file to record these features.

[0113] Step S204: In response to the input data, select the target object from multiple objects through the model routing module; the multiple objects include multiple local objects and multiple cloud models; the local object consists of prompt information and a local model; the prompt information is used to guide the local model to perform inference.

[0114] Step S205: Based on the target object, process the input data to obtain the reasoning result.

[0115] For a detailed description of steps S204-S205, please refer to the relevant description of steps S101-S102 in Example 1, which will not be repeated here.

[0116] In this embodiment, by acquiring a first sample task type, which is based on a cloud model for inference, and constructing prompt information based on the first sample task type and the local model, the prompt information can transfer key knowledge, rules, and techniques accumulated by the cloud model when processing the first sample task type to the local model. This allows the local object, constructed based on the prompt information and the local model, to quickly master effective methods when handling specific tasks, compensating for its own knowledge deficiencies due to limited data and insufficient training time, thereby approaching the cloud model in terms of inference capabilities. This means that many tasks that originally required cloud models to complete inference can now be handled properly by the local object with its enhanced capabilities, reducing the need to call cloud models and providing users with a smoother and more efficient user experience while ensuring the accuracy of inference results.

[0117] As another optional embodiment of this application, refer to Figure 5 This is a flowchart illustrating a model routing processing method provided in Embodiment 5 of this application. Figure 5 As shown, the method may include, but is not limited to, the following steps:

[0118] Step S301: Obtain the first sample task type; the task of the first sample task type is inferred based on the cloud model.

[0119] Step S302: Based on the first sample task type and the local model, construct prompt information.

[0120] Step S303: Construct a local object based on the prompt information and the local model.

[0121] For a detailed description of steps S301-S303, please refer to the relevant description of steps S201-S203 in Example 4, which will not be repeated here.

[0122] Step S304: Obtain the sample input data corresponding to the first sample task type.

[0123] In this embodiment, input data closely related to the first sample task type can be collected from historical data. This data should be representative and diverse, with the aim of comprehensively covering all possible situations that may occur for this task type.

[0124] For example, suppose the first sample task is "text sentiment analysis," and the analysis focuses on the sentiment (positive, negative, or neutral) of user comments on social media. Then, a large number of user comments from different social media platforms can be collected as sample input data. These comments should cover various topics, such as product reviews (including different categories like electronics, beauty products, and food), movie reviews, and travel experience sharing. Furthermore, the expression of the comments should be diverse, including direct expressions of emotion, such as "This phone is amazing, the camera is fantastic!" (positive sentiment); subtle or complex expressions, such as "Although the movie plot was a bit bland, the actors' performances were still commendable." (relatively neutral, with a slight positive bias); and clearly negative expressions, such as "The service at this restaurant was terrible, the food was served incredibly slowly!" (negative sentiment). Collecting such a rich and diverse sample input data can provide sufficient data support for subsequent updates to the prompts.

[0125] Step S305: Input the sample input data into the local object, which then processes the data to obtain the sample inference result.

[0126] In this step, we continue with the "text sentiment analysis" task as an example. The local object is a model built based on natural language processing technology. Assume the current prompt is "Please analyze whether the following text expresses a positive, negative, or neutral sentiment." When the comment "The special effects of this movie are amazing, and the plot is also very exciting. I highly recommend everyone to watch it!" is input into the local object, it will analyze the keywords and semantics of the text based on the prompt. It will identify words with positive sentiment such as "amazing," "exciting," and "highly recommended," thus determining that the comment expresses a positive sentiment and outputting "positive" as the sample inference result.

[0127] Step S306: Based on the difference between the sample inference result and the benchmark inference result, update the prompt information in the local object.

[0128] The benchmark inference result is obtained by processing the sample input data corresponding to the first sample task type by the cloud model.

[0129] If the difference between the sample inference result and the baseline inference result is large, it indicates that the local object still has obvious shortcomings in processing this type of task, and the prompt information needs to be updated in a targeted manner according to the specific differences.

[0130] The process of updating the tooltip information in the local object may include:

[0131] Step S11: Identify the key factors causing the discrepancy. For example, in text sentiment analysis, if the local object incorrectly identifies some sarcastic comments as positive sentiment, while the baseline result correctly identifies them as negative sentiment, then analysis may reveal that the local object did not understand the sarcastic tone and implied negative sentiment in the text.

[0132] Step S12: Determine the direction for updating the prompt information based on key factors. For the example of the sarcastic comment above, the update direction is to add prompts to the prompt information that identify special expressions such as sarcastic tone and implied emotions.

[0133] In addition to the aforementioned methods that automatically determine the update direction based on analysis, manual intervention can also be introduced to update the prompts. Professionals, leveraging their extensive experience and expertise, can directly review and modify the prompts. Based on their intuitive understanding of the differences between the sample inference results and the baseline inference results, combined with a deep understanding of the local object's task processing process, they can directly add, delete, or modify relevant content in the prompts. This ensures that the prompts more effectively guide the local object to improve its inference capabilities and narrow the gap with the cloud-based model's inference results.

[0134] Step S13: Modify the prompt message according to the update direction. This can be done by adding keywords, adjusting sentence structure, or clarifying the analysis requirements. For example, in the "text sentiment analysis" example, the updated prompt message could include: "Please analyze whether the following text expresses a positive, negative, or neutral sentiment. Pay special attention to whether there are any special expressions such as irony or sarcasm in the text. If so, accurately determine the sentiment tendency based on these expressions."

[0135] By updating the prompts, the accuracy and reliability of local objects in processing complex text are significantly improved. For example, consider the comment, "This movie was so 'wonderful,' it wasted two hours of my time!" Before the update, the local object might only focus on the positive emotion of the word "wonderful," thus incorrectly judging it as a positive sentiment. However, with the updated prompts, the local object notices the ironic expression in the text and, combined with the negative description "wasted two hours of my time," accurately determines that the comment expresses a negative emotion.

[0136] Step S307: In response to the input data, select the target object from multiple objects through the model routing module; the multiple objects include multiple local objects and multiple cloud models; the local object consists of prompt information and a local model; the prompt information is used to guide the local model to perform inference.

[0137] Step S308: Based on the target object, process the input data to obtain the reasoning result.

[0138] For a detailed description of steps S307-S308, please refer to the relevant description of steps S101-S102 in Example 1, which will not be repeated here.

[0139] In this embodiment, by acquiring sample input data corresponding to the first sample task type and inputting the sample input data into the local object, which then processes the data to obtain sample inference results, and updating the prompt information in the local object based on the difference between the sample inference results and the baseline inference results, the inference capability of the local object is continuously optimized, gradually approaching the inference capability of the cloud model. With the enhancement of the local object's capabilities, many tasks that previously required cloud models to complete inference can now be handled effectively by the local object itself. This means that while ensuring the accuracy of the inference results, the reliance on cloud models can be further reduced.

[0140] As another optional embodiment of this application, refer to Figure 6 This is a flowchart illustrating a model routing processing method provided in Embodiment 6 of this application. This embodiment is mainly an implementation of step S202 in Embodiment 4, such as... Figure 6 As shown, the specific steps may include, but are not limited to, the following:

[0141] Step S2021: Based on the first sample task type, determine the target parameters; the target parameters represent at least one of the execution target, the constraints on the reasoning process, and the input / output format corresponding to the first sample task type.

[0142] The execution objective corresponding to the first sample task type clearly defines the expected final result. For example, if the first sample task type is "flower recognition" in image classification, its execution objective is to accurately determine the type of flower in the input image, such as rose, peony, or tulip. Similarly, in a text summarization task, the execution objective is to extract key information from the given text and generate a concise and accurate summary. Clearly defining the execution objective allows the local model to know its direction, focusing on key information during inference and improving the accuracy and relevance of the inference.

[0143] Constraints on the inference process can define the rules, limitations, or capabilities that the local model must follow when processing tasks. Taking question-answering tasks in natural language processing as an example, constraints might include length limits for responses (e.g., responses cannot exceed 50 characters), format requirements (e.g., responses must be in complete sentences), and the source of information for the response (e.g., responses can only be based on given text content and cannot incorporate external knowledge). In image restoration tasks, constraints might involve the scope of the restoration area and the consistency of the restoration style (e.g., maintaining consistency with the overall style of the original image). Clearly defining these constraints helps guide the local model to reason within reasonable limits, avoiding unacceptable results, and also improves the model's inference efficiency and stability.

[0144] Standardizing input and output formats enables better compatibility and interaction between the local model and other parts of the system, ensuring accurate and smooth data transfer and processing across different stages. For example, regarding input formats, image processing tasks may specify the size, resolution, and file format (e.g., JPEG, PNG); text processing tasks may specify the encoding format (e.g., UTF-8), text length range, etc. Regarding output formats, image classification tasks may require output as category labels (e.g., "rose," "peony," etc.); text generation tasks may require output as a text file in a specific format (e.g., TXT format), and the text must contain specific identifiers or paragraph structures.

[0145] Step S2022: Based on the target parameters and the local model, construct the prompt information.

[0146] This step may include, but is not limited to:

[0147] Step S21: Based on the execution target of the first sample task type, construct the initial prompt information.

[0148] If the task is "animal species identification" in image recognition, the prompt message can emphasize "Please accurately determine the species of the animal in the input image, such as cat, dog, bird, etc., to ensure that the identification result is accurate."

[0149] For text sentiment analysis tasks, the objective is to determine the sentiment tendency (positive, negative, or neutral) of the text. The prompt could be something like, "Carefully analyze the text content, accurately determine its expressed sentiment tendency, and give a clear conclusion of whether it is positive, negative, or neutral."

[0150] Step S22: Add constraint hints based on the limiting conditions.

[0151] When there are constraints on the reasoning process, they can be clearly listed in the prompt message. Taking translation tasks in natural language processing as an example, if the constraints require that the translation result must conform to the grammatical rules of the target language and the length must not exceed a certain number of characters, the prompt message can be expressed as "Accurately translate the input text into the target language, ensuring that the translation result conforms to the grammatical rules and the number of characters does not exceed [X] characters."

[0152] In image editing tasks, if the restriction is that editing can only be performed in a specified area and the overall style must be consistent, the prompt message could be "Only edit the specified [specific area] in the image, and be sure to maintain the consistency with the overall style of the original image during the editing process."

[0153] Step S23: Follow the input / output format specifications and prompts for format requirements.

[0154] For example, for image processing tasks, if the input image is required to have a specific size (e.g., 512×512 pixels) and file format (e.g., PNG format), the prompt message could be "The input image size should be 512×512 pixels, and the file format should be PNG. Please ensure that the input meets the requirements."

[0155] The output format should also be specified in detail. For example, if the text generation task requires the output to be in JSON format and contains specific fields (such as "title" and "content"), the prompt message could be "Please present the output results in JSON format, which must include the two fields 'title' and 'content', corresponding to the generated title and body content, respectively."

[0156] The prompts clearly define the role of the local model (e.g., a general local large language model) in the task. For example, in an intelligent customer service system, the local model might act as a customer service representative, responsible for answering user questions and providing relevant information. In an image generation task, the local model might act as an artist, generating images of a specific style according to the user's requirements. Clearly defining the role of the local model helps it better understand the task context and requirements, thereby generating results that better meet expectations.

[0157] Meanwhile, prompts can emphasize the task objective, ensuring that the local model remains focused on the goal throughout the inference process. For example, in image classification tasks, prompts could explicitly state, "Please accurately determine the category of objects in the input image and provide the corresponding category label." In text summarization tasks, prompts could be, "Carefully read the input text, extract key information, and generate a concise and accurate summary." Clearly defining the task objective strengthens the local model's goal orientation, making it more focused and efficient during inference.

[0158] Furthermore, prompts can explain the steps the local model takes to perform the task, guiding it to reason in a logical order. For example, in an image restoration task, the prompts could be described as: "First, identify the damaged areas in the input image; second, generate restoration content based on the overall style of the image and surrounding pixel information; third, fill the damaged areas with the restoration content to ensure the restored image seamlessly blends with the original image." Detailed execution steps help the model better organize its reasoning process and avoid omissions or errors.

[0159] In this embodiment, target parameters are determined based on the first sample task type. The target parameters represent at least one of the execution target, constraints on the inference process, and input / output format corresponding to the first sample task type. Based on the target parameters and the local model, prompt information is constructed, which can clearly define the model role, clarify the task target, and detail the execution steps, allowing the model to clearly position itself, focus on the target, and reason in an orderly manner, thereby improving the accuracy, stability, interpretability, and task completion quality of inference, and enhancing the model's adaptability to different scenarios.

[0160] As another optional embodiment of this application, refer to Figure 7 This is a flowchart illustrating a model routing processing method provided in Embodiment 7 of this application. This embodiment is mainly an implementation of step S201 in Embodiment 4, such as... Figure 7 As shown, the specific steps may include, but are not limited to, the following:

[0161] Step S2011: Obtain the first historical data of the local model and the multiple cloud models during the historical inference process.

[0162] First-hand historical data may include, but is not limited to:

[0163] Task Type: Clearly identify the specific category to which each inference task belongs. Different task types have different requirements for the model's capabilities. For example, in the field of image processing, image classification tasks mainly determine the category of an image, while object detection tasks require locating objects in an image and identifying their category. In intelligent customer service systems, task types may include answering frequently asked questions, providing complex inquiries, and handling complaints. Clearly defining the task type helps analyze the dependence of different task types on local and cloud models.

[0164] Routing Results (Local or Cloud): Records whether each inference task was ultimately completed by the local model or by calling the cloud model. This information directly reflects the routing choices made by the model routing module for different tasks. For example, for some simple questions, the model routing module may choose the local model for a quick answer; while for some complex and highly specialized questions, it may call the cloud model to obtain a more accurate answer. By analyzing the routing results, we can gain a preliminary understanding of which task types are more inclined to use the cloud model.

[0165] Prediction scores of the local model and the cloud model: The prediction score is a quantitative evaluation of the model's inference results. By comparing the prediction scores of the local model and the cloud model, we can understand the difference in inference capabilities between the two models under different task types. If the prediction score of the cloud model is generally higher than that of the local model in a certain task type, it indicates that the cloud model has stronger capabilities in that task type, and the model routing module may be more inclined to call the cloud model.

[0166] The processing results of the cloud model or local model: This is the final answer or solution provided by the cloud model or local model for the inference task. By recording the processing results of the cloud model or local model, we can evaluate the accuracy and reliability of the cloud model or local model under different task types, and further analyze the necessity and rationality of calling the cloud model or local model.

[0167] Step S2012: Based on the first historical data, determine the task type whose frequency of calling the cloud model meets the set frequency threshold as the first sample task type.

[0168] In this embodiment, the number of times each task type calls the cloud model can be counted according to time periods (such as daily, weekly, monthly), and then the average number of calls per day can be calculated as the frequency of calling the cloud model.

[0169] In this embodiment, the frequency threshold can be set according to the needs of the actual application scenario and the system resource status. For example, if the system is sensitive to network transmission costs and the local model has great potential for improvement, the frequency threshold can be set relatively low to more strictly filter out the task types that need to be focused on; if the system has extremely high requirements for the accuracy of the inference results and the cloud model has obvious advantages, the frequency threshold can be appropriately increased.

[0170] Task types that frequently call cloud models at a set frequency threshold often indicate that the local model performs relatively poorly on these tasks and is insufficient to meet real-world needs. By identifying these task types as the first-sample task types, the constructed prompts can focus on these key tasks, guiding the local model to concentrate on learning and improving its inference capabilities on these tasks. For example, in image processing, if the local model frequently calls cloud models in object detection tasks in complex scenes, it indicates a deficiency in the local model for this task. By identifying such tasks as the first-sample task types, the constructed prompts can guide the local model to learn more effective object detection algorithms and feature extraction methods, thereby specifically improving its object detection capabilities in such complex scenes.

[0171] In this embodiment, by acquiring the first historical data of local and multiple cloud models' historical inference, covering information such as task type and routing results, a comprehensive understanding of the model's performance and routing selection under different tasks can be obtained. Based on this, the first sample task type whose cloud model call frequency meets the set threshold can be determined. This can accurately locate tasks where the local model performs poorly and is difficult to meet actual needs. Then, prompt information focusing on key tasks can be constructed to guide the local model to learn and improve in a concentrated manner, effectively enhancing the local model's inference ability on key tasks, improving overall inference efficiency and quality, and at the same time, rationally allocating resources and reducing unnecessary cloud call costs.

[0172] As another optional embodiment of this application, a model routing processing method provided in Embodiment 8 of this application may include, but is not limited to, the following steps:

[0173] Step S401: Obtain the first historical data of the local model and the multiple cloud models during the historical inference process;

[0174] Step S402: Based on the first historical data, determine the task type whose frequency of calling the cloud model meets the set frequency threshold as the first sample task type.

[0175] For a detailed description of steps S401-S402, please refer to the relevant description of steps S2011-S2012 in the above embodiments, which will not be repeated here.

[0176] Step S403: Based on the first sample task type and the local model, construct prompt information.

[0177] Step S404: Construct a local object based on the prompt information and the local model.

[0178] Step S405: In response to the input data, the target object is selected from multiple objects through the model routing module; the multiple objects include multiple local objects and multiple cloud models; the local object consists of prompt information and a local model; the prompt information is used to guide the local model to perform inference;

[0179] Step S406: Based on the target object, process the input data to obtain the reasoning result.

[0180] For a detailed description of steps S403-S406, please refer to the relevant description of steps S202-S205 in Example 4, which will not be repeated here.

[0181] Step S407: Under the condition of meeting the set triggering conditions, obtain the second historical data of the local model and the multiple cloud models in the historical reasoning process;

[0182] Setting trigger conditions allows for flexible and targeted data collection to adapt to different stages and needs of the system. Trigger conditions can be periodic, such as acquiring data at fixed times each week or month, enabling regular evaluation and optimization of model performance and ensuring continuous improvement in system performance. They can also be event-driven, such as triggering data acquisition when user feedback indicates inaccurate inference results or a significant decline in system performance (e.g., slower inference speed, lower accuracy). For example, in an intelligent customer service system, if user satisfaction ratings for customer service responses consistently decrease over a period, it may indicate a problem with the model. In this case, triggering the acquisition of secondary historical data allows for timely analysis of the issue.

[0183] Similar to obtaining the first historical data in step S401, the second historical data also covers various aspects of information from the local model and multiple cloud models during the historical inference process, such as task type and routing results.

[0184] Step S408: Based on the second historical data, determine that there is a second sample task type that is different from the first sample task type.

[0185] In this embodiment, the task types in the second historical data can be compared with the task types in the first sample data to identify those task types that are different from the task types in the first sample data and have certain characteristics and importance as the task types in the second sample data.

[0186] Step S409: Construct a second local object based on the second sample task type.

[0187] The detailed process of step S409 can be found in the relevant description in Example 4, and will not be repeated here.

[0188] When set trigger conditions are met, second historical data from both local and cloud models is retrieved to identify a second sample task type. Based on this, new local objects are constructed, i.e., multiple objects are updated, enriching the local model. This not only specifically enhances the local model's ability to capture and infer features related to new tasks, but also forms a more complete task processing knowledge system through a rich variety of local objects. This allows the local model to have more adaptable solutions when facing various tasks, thereby further reducing the number of calls to the cloud model and effectively reducing network transmission costs and cloud resource consumption. When using the system, users no longer need to wait for long delays caused by frequent calls to the cloud model, and can obtain accurate results feedback faster. The operation process is smoother and more natural, improving the comfort and satisfaction of user interaction with the system.

[0189] In this embodiment, the update process for local objects can be triggered when, but is not limited to, the following two situations:

[0190] On the one hand, if the capabilities of the cloud model are updated—for example, if there are significant changes in the scores across multiple task dimensions in the model evaluation, or if the version of the large cloud model is found to have been upgraded when actually called—this indicates that the cloud model's ability and methods of handling tasks have changed. If local objects are not adjusted accordingly, they may not be able to collaborate efficiently with the cloud model, affecting the overall inference performance.

[0191] On the other hand, when routing data analysis shows a decline in the performance of local objects, such as a decrease in the prediction score of local objects for input data, it means that local objects are not performing as well as before in the current task processing scenario, and may not meet the user's requirements for the accuracy and timeliness of inference results.

[0192] Specifically, a detailed comparative analysis can be conducted between the local object and the cloud model to identify the gaps and shortcomings of the local object in processing tasks. Finally, based on the comparative analysis results, the prompts for the local object are re-optimized. By precisely adjusting the prompts, the local model is guided to better adapt to new task requirements and changes in the cloud model, thereby improving the performance of the local object and ensuring that the system can always provide users with high-quality and efficient inference services, further enhancing the user experience.

[0193] As another optional embodiment of this application, a model routing processing method is provided in Embodiment 9 of this application. This embodiment is mainly an implementation of step S101 in Embodiment 1, and may specifically include, but is not limited to, at least one of the following:

[0194] Step S1015: In response to the input data, determine the predicted score for each of the multiple objects through the model routing module; the predicted score includes the object's capability score on a set task dimension.

[0195] Since a local object consists of a local model and cue information, when calculating the prediction score of a local object, the capabilities of the local model itself and the enhancement effect of the cue information on the model's capabilities can be considered.

[0196] In this embodiment, the basic capabilities of the local model in each set task dimension can be quantified first. For example, for the mathematical calculation dimension, the local model can be asked to perform some simple mathematical calculation tasks, such as basic algebraic operations and statistical calculations, and its basic capability score in this dimension can be evaluated based on the accuracy and speed of its calculation results.

[0197] Furthermore, the analysis examines the impact of prompts on the local model's capabilities across various dimensions. The effect of prompts can be quantified by comparing the performance differences of the local model in handling tasks with and without prompts. For example, comparing changes in metrics such as accuracy and recall of the local model with and without prompts determines the contribution of prompts to the improvement in capabilities across these dimensions.

[0198] In this embodiment, the basic capability score of the local model and the contribution value of the prompt information can be combined to obtain the capability score of the local object on each set task dimension. Specifically, a weighted average method can be used, assigning different weights to the basic capability score and contribution value according to the importance of different dimensions and the significance of the prompt information. For example, for some task dimensions that rely heavily on prompt information, the contribution value of the prompt information can be given a higher weight.

[0199] For cloud-based models, their basic capabilities in various task dimensions can be quantified. For example, in the mathematical calculation dimension, local models can be allowed to perform simple mathematical calculation tasks, such as basic algebraic operations and statistical calculations, and their ability score in this dimension can be evaluated based on the accuracy and speed of their calculation results.

[0200] In this embodiment, the ability scores of the object on each set task dimension can be weighted to obtain a predicted score.

[0201] Alternatively, instead of weighting, the object's ability scores across each defined task dimension can be directly retained. In this case, the predicted score is actually composed of these independent multi-dimensional ability scores. However, this composition is not achieved through weighted aggregation, but rather by retaining the independent information of each dimension to facilitate targeted analysis for specific task requirements.

[0202] Step S1016: Select a target object from the plurality of objects based on the predicted score of each object.

[0203] In this embodiment, corresponding to the implementation of weighting the object's ability scores across various defined task dimensions to obtain a predicted score, the target object can be selected in the following manner:

[0204] Method 1: Compare the predicted scores of each object and select the object with the highest predicted score from among multiple objects as the target object.

[0205] Method 2: Compare the prediction scores of each object, and select the candidate local object and candidate cloud model with the highest prediction scores from multiple objects respectively; if the difference in prediction scores between the candidate local object and the candidate cloud model meets the set prediction score difference threshold, the candidate local object can be determined as the target object. This selection method takes into account that local objects may have advantages such as lower latency and higher data security in some scenarios. When the performance gap between local objects and cloud models is within a certain range, local objects are given priority.

[0206] Corresponding to the implementation method of directly retaining the ability scores of objects on each set task dimension, if the task corresponding to the input data has very high requirements for a certain task dimension, then the object with the highest predicted score on that dimension can be selected as the target object. For example, for a complex mathematical calculation task, the predicted scores of each object on the mathematical calculation dimension are compared, and the object with the highest score is selected, whether it is a local object or a cloud model, to ensure that the task can be processed accurately and efficiently.

[0207] In this embodiment, the model routing module determines the ability scores of each object in the defined task dimensions (such as mathematical calculation, text understanding, etc.), which can accurately quantify the ability level of each object in different tasks. Based on this, the target object is selected from multiple objects, making the selection process more scientific and targeted, ensuring that the ability of the selected target object is highly matched with the task requirements, thereby effectively improving the accuracy, efficiency and quality of task processing and meeting the actual needs of diverse task scenarios.

[0208] Step S1017: In response to the input data, determine the task features corresponding to the input data; the task features characterize the target task attributes corresponding to the input data.

[0209] By analyzing the text content of the input data and identifying the keywords it contains, the task type can be determined. For example, if the input data contains keywords such as "summation" or "average," the task type can be identified as a mathematical calculation task; if it contains keywords such as "image classification" or "object detection," it can be identified as an image processing task.

[0210] In this embodiment, the task type can be determined based on, but is not limited to, the format characteristics of the input data. For example, if the input data is an image file (such as .jpg or .png format), the task type may be related to image processing; if the input data is a text file (such as .txt or .doc format), the task type may be related to text processing.

[0211] For some complex tasks, simply identifying the task type is insufficient; it's also necessary to clarify the task's execution objective. By deeply understanding the semantics of the input data, key information can be extracted to determine the task's objective. For example, for a text about product reviews, it's not enough to simply identify the task type as text sentiment analysis; the execution objective also needs to be clarified as determining whether the review is positive, negative, or neutral, and possibly extracting the key viewpoints and reasons within the review.

[0212] Step S1018: Determine the similarity between the task features and the task attributes of each of the multiple objects.

[0213] The task attributes of a local object can encompass the capabilities of the local model and the task direction guided by the prompts. For example, a local model may have strong feature extraction capabilities in image recognition, enabling it to quickly and accurately identify objects, scenes, and other elements in images; in text processing, it may excel at tasks such as text classification and semantic understanding. Meanwhile, the prompts, as auxiliary information guiding the local model in processing tasks, further clarify the task's direction and focus. For instance, if the prompts guide the local model to focus on specific types of image features, such as images with distinct edges and textures, or guide the model to focus on text with specific themes or sentiments in text processing, then the task attributes of the local object can be comprehensively described as "excelling at recognizing images with specific features (such as images with distinct edges and textures), and focusing on text analysis of [specific themes / sentiments] in text processing."

[0214] The task attributes of cloud-based models can be determined based on their performance in historical tasks, official descriptions, and publicly available evaluation data. Cloud-based models are typically trained and optimized on large-scale data, and have wide applications and excellent performance in different task domains. For example, a cloud-based image processing model may have significant advantages in processing large-scale, high-resolution images, and can quickly and efficiently complete tasks such as image enhancement, restoration, and super-resolution reconstruction. Its task attributes can be described as "efficiently processing large-scale, high-resolution images, possessing powerful image enhancement and restoration capabilities, and performing exceptionally well in image super-resolution reconstruction tasks." A cloud-based text processing model may have leading levels in natural language understanding, machine translation, and text generation. Its task attributes can be described as "high accuracy in natural language understanding tasks, high-quality machine translation, and the ability to generate fluent and accurate text content."

[0215] In this embodiment, a feature matching algorithm can be used to accurately calculate the similarity between task features and object task attributes. First, the task features and object task attributes are represented as vectors. This process can be aided by word embedding techniques, such as pre-trained models like Word2Vec, GloVe, or BERT, to transform textual information into numerical vectors with semantic information. Then, the cosine similarity between these two vectors is calculated.

[0216] Step S1019: Based on the similarity, select the target object from the plurality of objects.

[0217] In this embodiment, the similarity scores of all objects and task features can be directly compared, and the object with the highest similarity score can be selected as the target object.

[0218] Alternatively, a similarity threshold can be pre-set, and only objects with a similarity score exceeding the threshold are eligible to be selected as target objects. If multiple objects exceed the threshold, other factors, such as processing speed and resource consumption, can be further combined for a comprehensive evaluation to select the optimal object.

[0219] In this embodiment, by determining the task features that represent the target task attributes (such as type, execution target, etc.), calculating the similarity between the task features and the task attributes of multiple objects, and selecting the target object based on the similarity, this method can accurately match the task requirements and object characteristics represented by the input data, thereby effectively improving the targeting, accuracy and efficiency of task processing, reducing the risk of task processing errors caused by object mismatch, and better meeting the diverse task requirements in actual business scenarios.

[0220] The model routing processing apparatus provided in this application will be described below. The model routing processing apparatus described below can be referred to in correspondence with the model routing processing method described above.

[0221] Reference Figure 8 The model routing processing device includes a model routing module 100 and a processing module 200.

[0222] The model routing module 100 is used to select a target object from multiple objects in response to input data; the multiple objects include multiple local objects and multiple cloud models; the local objects consist of prompt information and local models; the prompt information is used to guide the local models to perform inference.

[0223] The processing module 200 is used to process the input data based on the target object to obtain the reasoning result.

[0224] Model routing module 100 can be used specifically for:

[0225] In response to input data, a candidate local object and a candidate cloud model are selected from multiple objects through a model routing module; the candidate local object has the highest prediction index among the multiple local objects; the candidate cloud model has the highest prediction index among the multiple cloud models; the prediction index characterizes the reasoning ability of the local object or the cloud model corresponding to the input data.

[0226] If the difference in the prediction metrics between the candidate local object and the candidate cloud model meets a set difference threshold, the candidate local object is identified as the target object.

[0227] Model routing module 100 can be used specifically for:

[0228] In response to input data, the model routing module determines the prediction index of each local object among multiple objects; the prediction index characterizes the reasoning ability of the local object corresponding to the input data.

[0229] Select one of the local objects whose prediction metric satisfies the set prediction metric threshold and determine it as the target object.

[0230] In this embodiment, the model routing processing device may further include:

[0231] Build modules are used for:

[0232] Obtain the first sample task type; the task of the first sample task type is inferred based on the cloud model;

[0233] Based on the first sample task type and the local model, construct the prompt information;

[0234] Based on the prompt information and the local model, a local object is constructed.

[0235] The model routing processing device may further include:

[0236] The update module is used for:

[0237] Obtain the sample input data corresponding to the first sample task type;

[0238] The sample input data is input into the local object, which processes it to obtain the sample inference result.

[0239] Based on the difference between the sample inference result and the benchmark inference result, update the prompt information in the local object;

[0240] The benchmark inference result is obtained by processing the sample input data corresponding to the first sample task type by the cloud model.

[0241] Based on the sample task type and the local model described above, the construction module builds prompt information, which may include:

[0242] Based on the first sample task type, target parameters are determined; the target parameters represent at least one of the execution objective, constraints on the reasoning process, and input / output format corresponding to the first sample task type.

[0243] Based on the target parameters and the local model, a prompt message is constructed.

[0244] The module that obtains the first sample task type can include:

[0245] Obtain the first historical data of the local model and the multiple cloud models during the historical inference process;

[0246] Based on the first historical data, the task types whose frequency of calling the cloud model meets the set frequency threshold are determined as the first sample task types.

[0247] In this embodiment, the building module can also be used for:

[0248] Under the condition of meeting the set triggering conditions, the second historical data of the local model and the multiple cloud models in the historical reasoning process is obtained;

[0249] Based on the second historical data, it is determined that there is a second sample task type that is different from the first sample task type.

[0250] Based on the second sample task type, construct a second local object.

[0251] The model routing module 100 selects a target object from a plurality of objects, which may include at least one of the following:

[0252] In response to input data, a predicted score for each of the multiple objects is determined by the model routing module; the predicted score includes the object's capability score on a defined task dimension.

[0253] Based on the predicted score of each of the objects, a target object is selected from the plurality of objects;

[0254] In response to input data, the task features corresponding to the input data are determined; the task features characterize the target task attributes corresponding to the input data.

[0255] Determine the similarity between the task features and the task attributes of each of the multiple objects;

[0256] Based on the similarity, a target object is selected from the plurality of objects.

[0257] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0258] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0259] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0260] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A model routing processing method, comprising: In response to input data, the target object is selected from multiple objects via the model routing module; The multiple objects include multiple local objects and multiple cloud models; The local object consists of prompt information and a local model; the prompt information is used to guide the local model to perform reasoning. Based on the target object, the input data is processed to obtain the reasoning result.

2. The model routing processing method according to claim 1, wherein selecting a target object from multiple objects through the model routing module in response to input data includes: In response to input data, candidate local objects and candidate cloud models are selected from multiple objects through the model routing module; The candidate local object has the highest prediction index among the multiple local objects; the candidate cloud model has the highest prediction index among the multiple cloud models; the prediction index represents the level of reasoning ability of the local object or the cloud model corresponding to the input data; If the difference in the prediction metrics between the candidate local object and the candidate cloud model meets a set difference threshold, the candidate local object is identified as the target object.

3. The model routing processing method according to claim 1, wherein selecting a target object from multiple objects in response to input data via the model routing module includes: In response to the input data, the model routing module determines the predicted metrics for each local object among multiple objects. The prediction metric characterizes the level of reasoning ability of the local object corresponding to the input data. Select one of the local objects whose prediction metric satisfies the set prediction metric threshold and determine it as the target object.

4. The model routing processing method according to claim 1, further comprising: Obtain the task type of the first sample; The first sample task type is based on cloud-based models for inference; Based on the first sample task type and the local model, construct the prompt information; Based on the prompt information and the local model, a local object is constructed.

5. The model routing processing method according to claim 4, further comprising: Obtain the sample input data corresponding to the first sample task type; The sample input data is input into the local object, which processes it to obtain the sample inference result. Based on the difference between the sample inference result and the benchmark inference result, update the prompt information in the local object; The benchmark inference result is obtained by processing the sample input data corresponding to the first sample task type by the cloud model.

6. The model routing processing method according to claim 4, wherein constructing prompt information based on the first sample task type and the local model includes: Based on the task type of the first sample, determine the target parameters; The target parameter represents at least one of the execution target, constraints on the reasoning process, and input / output format corresponding to the first sample task type; Based on the target parameters and the local model, a prompt message is constructed.

7. The model routing processing method according to claim 4, wherein obtaining the first sample task type includes: Obtain the first historical data of the local model and the multiple cloud models during the historical inference process; Based on the first historical data, the task types whose frequency of calling the cloud model meets the set frequency threshold are determined as the first sample task types.

8. The model routing processing method according to claim 7, further comprising: Under the condition of meeting the set triggering conditions, the second historical data of the local model and the multiple cloud models in the historical reasoning process is obtained; Based on the second historical data, it is determined that there is a second sample task type that is different from the first sample task type. Based on the second sample task type, construct a second local object.

9. The processing method according to claim 1, wherein the step of selecting a target object from multiple objects through a model routing module in response to input data includes at least one of the following: In response to input data, a predicted score for each of the multiple objects is determined by the model routing module; the predicted score includes the object's capability score on a defined task dimension. Based on the predicted score of each of the objects, a target object is selected from the plurality of objects; In response to input data, determine the task characteristics corresponding to the input data; The task features characterize the target task attributes corresponding to the input data; Determine the similarity between the task features and the task attributes of each of the multiple objects; Based on the similarity, a target object is selected from the plurality of objects.

10. A processing apparatus, comprising: The model routing module is used to select a target object from multiple objects in response to input data; The multiple objects include multiple local objects and multiple cloud models; The local object consists of prompt information and a local model; the prompt information is used to guide the local model to perform reasoning. The processing module is used to process the input data based on the target object to obtain the reasoning result.

Images