Model routing methods and devices, electronic devices, and software products

Abstract

This application relates to the field of large language model technology, providing a model routing method and apparatus, electronic device, and program product. The method includes: determining the perplexity of a first micro-language model deployed locally for each token included in a query request; obtaining a first difficulty index based on the perplexity; counting the first number of predefined features in the query request using regular expression matching; obtaining a second difficulty index based on the number of tokens included in the query request and the first number; obtaining a semantic abstraction score corresponding to the query request as a third difficulty index; obtaining a difficulty score value corresponding to the query request based on the first, second, and third difficulty indices; determining a set of target candidate models corresponding to the query request based on the difficulty score value and a preset difficulty threshold; and selecting a candidate model from the set of target candidate models as the target model. This method can improve the rationality of model routing.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and in particular to a model routing method and apparatus, electronic device, and program product. Background Technology

[0002] In intelligent question-and-answer applications based on Large Language Models (LLMs), model routing acts as an intelligent scheduling center, automatically selecting the most suitable model to handle each user's query request based on its specific characteristics. Current model routing strategies often rely on simple rules, such as classifying queries solely by text length: long texts are sent to the large language model deployed in the cloud, while short texts are sent to a lightweight model deployed locally. This approach can lead to inappropriate model selection. Summary of the Invention

[0003] This application provides a model routing method and apparatus, electronic device, and program product, which comprehensively evaluates the difficulty of query requests and performs intelligent routing based on difficulty scores, thereby improving the rationality and reliability of model routing.

[0004] In a first aspect, embodiments of this application provide a model routing method, the method comprising: in response to receiving a user's query request, determining the perplexity of a first micro-language model deployed locally for each token included in the query request, and obtaining a first difficulty index based on the perplexity; counting the first number of predefined features in the query request using regular expression matching; the predefined features include specified professional symbols and / or specified technical keywords; obtaining a second difficulty index based on the number of tokens included in the query request and the first number; obtaining a semantic abstraction score corresponding to the query request as a third difficulty index; obtaining a difficulty score value corresponding to the query request based on the first difficulty index, the second difficulty index, and the third difficulty index; determining a set of target candidate models corresponding to the query request based on the difficulty score value and a preset difficulty threshold; and selecting a candidate model from the set of target candidate models as the target model for processing the query request.

[0005] In one embodiment, obtaining the semantic abstraction score corresponding to a query request includes: inputting the query vector corresponding to the query request into a first micro-language model for one forward propagation to obtain the query semantic vector corresponding to the query request; calculating the norm of the query semantic vector; or, inputting the query semantic vector into a pre-trained feedforward neural network, projecting it through the feedforward neural network to obtain a semantic feature vector, and using the value of a specified dimension in the semantic feature vector as the activation value; the specified dimension is the dimension used to represent the semantic abstraction during the training process of the feedforward neural network; and using the norm or activation value as the semantic abstraction score.

[0006] In one embodiment, where the target candidate model set includes a large language model deployed in the cloud, after determining the target candidate model set corresponding to the query request, the method further includes: in response to a security scanner deployed locally detecting that the query request includes sensitive information, performing desensitization processing on the query request using a second micro-language model deployed locally to obtain multiple candidate rewrite requests corresponding to the query request; and selecting one rewrite request as the target request from the multiple desensitized candidate rewrite requests.

[0007] In one embodiment, selecting a rewrite request as the target request from a plurality of anonymized candidate rewrite requests includes: obtaining the embedding vectors corresponding to each rewrite request in the plurality of anonymized candidate rewrite requests; calculating the semantic similarity between the query vector corresponding to the original query request and the embedding vector corresponding to a rewrite request; determining a sensitive entity set; identifying whether the rewrite request includes sensitive words belonging to the sensitive entity set, and determining a penalty term; obtaining a first probability distribution corresponding to the rewrite request, and obtaining a natural language reference probability distribution corresponding to the original query request, calculating the difference between the first probability distribution and the natural language reference probability distribution using KL divergence to obtain a KL divergence value; obtaining a rewrite score value for each rewrite request in the plurality of candidate rewrite requests based on semantic similarity, penalty term, and KL divergence value; and selecting the rewrite request with the highest rewrite score value as the target request.

[0008] In one embodiment, the method further includes: obtaining the latent factor vector corresponding to the query request or target request; obtaining the capability vectors corresponding to multiple candidate models respectively; determining the bias term; calculating the dot product of the latent factor vector and the capability vector, and using the Sigmoid function to map the sum of the dot product and the bias term to the win rate; the win rate represents the probability that a corresponding candidate model supports responding to the query request or target request; selecting a candidate model from the target candidate model set as the target model for processing the query request, including: selecting a model from the target candidate model set as the target model for processing the query request based at least on the win rate.

[0009] In one embodiment, before selecting a model from multiple candidate models as the target model for processing the query request based at least on the win rate, the method further includes: determining a cost sensitivity factor; calculating the ratio of the cost corresponding to each candidate model among the multiple candidate models to the maximum cost of the multiple candidate models, and obtaining a cost item based on the ratio and the cost sensitivity factor; selecting a model from a set of target candidate models as the target model for processing the query request based at least on the win rate, including: obtaining a decision score corresponding to each model among the multiple candidate models based on the win rate and the cost item; and selecting the model with the highest decision score from the set of target candidate models as the target model for processing the query request.

[0010] In one embodiment, determining the target candidate model set corresponding to the query request based on the difficulty score and a preset difficulty threshold includes: determining the target candidate model set corresponding to the query request as a first candidate model set based on the difficulty score being greater than the preset difficulty threshold and the absolute value of the difference between the difficulty score and the preset difficulty threshold being greater than a preset difference; the first candidate model set includes at least one large language model deployed in the cloud; determining the target candidate model set corresponding to the query request as a second candidate model set based on the difficulty score being less than the preset difficulty threshold and the absolute value of the difference between the difficulty score and the preset difficulty threshold being greater than a preset difference; the second candidate model set includes at least one micro language model deployed locally; determining the target candidate model set corresponding to the query request as a third candidate model set based on the absolute value of the difference between the difficulty score and the preset difficulty threshold being less than or equal to a preset difference; the third candidate model set is the union of the first candidate model set and the second candidate model set.

[0011] This application embodiment also provides a model routing device, which includes: a first determining module, used to determine the perplexity of a first micro-language model deployed locally for each token in a query request, and obtain a first difficulty index based on the perplexity; a second determining module, used to count the first number of predefined features in the query vector through regular expression matching; the predefined features include code snippets and / or mathematical formulas; and obtain a second difficulty index based on the number of tokens included in the query request and the first number; a third determining module, used to obtain a semantic abstraction score corresponding to the query request as a third difficulty index; a comprehensive difficulty scoring module, used to obtain a difficulty score value corresponding to the query request based on the first difficulty index, the second difficulty index, and the third difficulty index; and a routing module, used to determine a set of target candidate models corresponding to the query request based on the difficulty score value and a preset difficulty threshold; and select a candidate model from the set of target candidate models as the target model for processing the query request.

[0012] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the model routing method of the first aspect.

[0013] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the model routing method of the first aspect.

[0014] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the model routing method of the first aspect.

[0015] The model routing method, apparatus, electronic device, and program product provided in this application, upon receiving a user's query request, comprehensively evaluates the difficulty of the query request. First, it determines the perplexity of each token in the query request by a locally deployed micro-language model, obtaining a first difficulty index based on the perplexity. Then, it uses regular expression matching to count the number of predefined features (e.g., code snippets and / or mathematical formulas) in the query vector (a first quantity). Next, based on the number of tokens included in the query request and the first quantity, it obtains a second difficulty index. The second difficulty index evaluates the difficulty of the query request from the perspective of whether it contains predefined features such as code snippets or mathematical formulas. Finally, it obtains the semantic abstraction score corresponding to the query request as a third difficulty index. Based on the first, second, and third difficulty indices, it obtains a difficulty score value corresponding to the query request. Then, based on the difficulty score value and a preset difficulty threshold, it determines a set of target candidate models and selects one candidate model as the target model. The method employs three difficulty metrics to reflect the complexity of query requests from three different dimensions. It goes beyond simply classifying queries based on text length; it comprehensively assesses whether the model has sufficient capacity to handle the query (the first difficulty metric) and the amount of predefined features (such as technical symbols and keywords) contained in the query (the second difficulty metric). This allows for the accurate identification of "short but difficult" tasks (such as mathematical proofs) and "long but simple" tasks (such as copying long texts). It effectively filters out highly complex queries that the local micro-language model cannot handle, avoiding the illusion caused by the local micro-language model forcing an answer. Simultaneously, it intercepts low-difficulty requests locally, saving costs by avoiding uploading to the cloud. This method selects the appropriate model for requests of different difficulties, and the comprehensive difficulty score is interpretable, improving the rationality and reliability of the model's routing strategy. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the model routing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the software module structure of the model routing device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0019] In current applications of large language models, such as intelligent question answering, enterprises and developers face a dilemma of "capability-cost-security." On the one hand, locally deployed models (Local LLMs) offer high security because data remains within the local domain and has low marginal costs, such as electricity / hardware depreciation. However, their capabilities are limited when handling tasks involving long logical reasoning and complex instructions. On the other hand, cloud-based large language models provide callable interfaces (APIs) and offer greater capabilities. However, some cloud-based large language models are expensive due to token-based billing and pose a risk of privacy breaches caused by sending sensitive data to third-party cloud providers.

[0020] In related model routing schemes, a typical routing strategy is based on simple rules (such as text length) or black-box classifiers, such as BERT (Bidirectional Encoder Representations from Transformers). Routing based on simple rules like text length often fails to perceive the semantic difficulty of the task. For example, the short text "Prove the Riemann Hypothesis" may be short but extremely difficult to process, while a long text might be a simple question repeated 10 times, making it easier to process. Routing using black-box classifiers may introduce high inference latency and lacks interpretability, making it impossible to quantify the confidence level of the decision, i.e., reliability cannot be verified.

[0021] Furthermore, some model routing schemes in related technologies directly transmit the user's original query request for query requests that need to be uploaded to the cloud for processing, without semantically anonymizing the request. This could result in the original query request carrying privacy data involving the user's personal information, posing a risk of user privacy data leakage. Additionally, local models are not rewritten using methods such as "dimensionality reduction" (e.g., mind chain decomposition).

[0022] Some related technologies employ relatively passive privacy protection strategies, which, while protecting private data, significantly impair semantic information. For example, some model routing schemes rely primarily on "masking" to achieve privacy protection, such as replacing "Zhang San has diabetes" with "** has **disease**". This destructive modification leads to the loss of key contextual semantics, making effective reasoning impossible. For instance, in medical scenarios, masking specific numerical values ​​and key medical information may result in diagnostic errors.

[0023] In view of this, embodiments of this application propose a model routing method and apparatus, electronic device, and program product. This method adopts an edge-cloud collaborative adaptive differential routing architecture. Its core concept is not a simple binary selection, but rather the introduction of an intermediate layer—a smart gateway—with analysis and rewrite capabilities. The model routing scheme proposed in embodiments of this application includes at least one of the following technical improvements: Information entropy-based difficulty quantification: Using a local miniature language model as a "probe", the perplexity and structural entropy of the input request are calculated to quantify the relative difficulty of the query request response task to the local miniature language model.

[0024] Semantic rewriting from a game theory perspective: Viewing privacy desensitization as a game between utility and privacy, generating intermediate text (rewriting request) that retains logical structure while removing sensitive information by maximizing the payoff function.

[0025] Vectorized matrix factorization routing: The routing problem is modeled as a matrix factorization problem. By calculating the inner product of the query vector and the model capability vector, the win rate of each model is predicted in milliseconds. Combined with the cost function, the optimal decision is made to select the target model that is most suitable for handling user query requests.

[0026] Specifically, the solution in this application proposes three cascaded processing stages: Phase 1, Feature Extraction and Difficulty Assessment: Calculate the semantic perplexity and structural features of the query request to determine whether it needs to be migrated to the cloud. Migrating to the cloud means sending the query request to a large language model in the cloud for processing.

[0027] Phase Two, Privacy-Aware Semantic Rewriting: For query requests that need to be migrated to the cloud and contain sensitive information, semantic rewriting is performed.

[0028] Phase 3, Win Rate Routing Decision Based on Cost-Effectiveness: Based on matrix factorization algorithm, predict the win rate of each level of model, and select the model with the highest cost-effectiveness (lower cost and able to solve the problem) as the target model to respond to the user's question (query request).

[0029] The model routing method proposed in this application embodiment may include at least one of the above three stages.

[0030] like Figure 1 As shown, this application embodiment provides a model routing method, which may include the following process (stage one): Step 101: In response to receiving a user's query request, determine the perplexity of the first micro-language model deployed locally for each token included in the query request, and obtain the first difficulty index based on the perplexity.

[0031] The first micro language model can be one of multiple micro language models deployed locally. A micro language model (SLM) refers to a language model with a relatively small number of parameters, typically ranging from 500 million (0.5B) to 10 billion (10B). For example, the first micro language model could be an SLM with 0.5B parameters. 1B represents 1 billion.

[0032] After receiving a user's query request, a forward pass can be quickly performed using the locally deployed first micro-language model, without generating an answer text, only calculating features.

[0033] The logits, conditional probabilities, or NLL output during a single forward propagation of the first micro-language model are used as the perplexity. During a single forward propagation, the first micro-language model first outputs the raw scores (logits), which are then converted into a conditional probability distribution using the Softmax function, and finally the negative log-likelihood is calculated as the loss function.

[0034] Here, logits refers to the direct output value of the last linear layer when the first micro-language model performs one forward propagation, without processing by the Softmax function. The value in logits represents the model's "raw confidence" for each token and can be any real number (positive, negative, or zero). Logits is usually interpreted as unnormalized log probabilities.

[0035] Conditional probability refers to the probability of the next token predicted by the first micro-language model.

[0036] NLL (Negative Log-Likelihood) is a loss function that measures the difference between the conditional probability distribution predicted by the model and the true label.

[0037] After obtaining the perplexity level, the first difficulty index is obtained based on the perplexity level. Specifically, it can be: performing linear calculation on the perplexity levels corresponding to the multiple tokens included in the query request, and using the logarithm of the calculation result as the first difficulty index. For example, the linear calculation can be to calculate the mean or to multiply the mean by an adjustment coefficient. For example, the mean of the perplexity levels corresponding to the multiple tokens included in the query request is calculated, and the logarithm of the mean is used as the first difficulty index.

[0038] Step 102: Count the first number of predefined features in the query request using regular expression matching.

[0039] Predefined features include specified technical symbols and / or specified technical keywords.

[0040] For example, the specified technical symbols can be code snippets or mathematical formulas. Code snippets refer to any form of programming language content or text with programming language characteristics in the user's query request.

[0041] Specify technical keywords, such as "Riemann Hypothesis" or "Goldbach's Conjecture," which are technically difficult to solve.

[0042] Step 103: Based on the number of tokens included in the query request and the first quantity, obtain the second difficulty index.

[0043] The number of tokens included in the query request, which is the length of the token sequence in the query request, i.e., the total length of the query request. The first quantity is the number of predefined features (technical symbols and / or technical keywords) contained in the query request, i.e., how many code snippets, mathematical formulas, and / or technical keywords indicating the difficulty of the problem are contained in the query request. Generally speaking, the more mathematical formulas or code snippets in the query request, the greater the difficulty of processing the problem (query request).

[0044] Based on the number of tokens included in the query request and the first quantity, a second difficulty index is obtained. Specifically, it can be obtained by directly using the ratio between the first quantity and the number of tokens included in the query request as the second difficulty index, or by performing a linear calculation based on the ratio, such as multiplying it by an adjustment coefficient.

[0045] Step 104: Obtain the semantic abstraction score corresponding to the query request, as the third difficulty indicator.

[0046] Specifically, in some embodiments, the semantic abstraction score can be obtained using either method one or method two as follows: Method 1: Input the query vector corresponding to the query request into the first micro-language model for one forward propagation to obtain the query semantic vector corresponding to the query request; Calculate the norm of the query semantic vector, such as the L2 norm or Euclidean norm, and use the norm as a score for semantic abstraction. For example, calculate the L2 norm of the query semantic vector eq.

[0047] Method 2: The query semantic vector is input into a pre-trained feed-forward network (FNN). The FNN projects the vector to obtain a semantic feature vector, and the values ​​of a specified dimension within this feature vector are used as activation values. These activation values ​​are then used as the semantic abstraction score. For example, an FNN could specifically be an MLP (Multi-Layer Perceptron).

[0048] Here, the specified dimension is the dimension used to represent the semantic abstraction during the training of the feedforward neural network. This specific dimension can be understood as the projection of the abstract semantic degree onto the direction w.

[0049] Specifically, the query vector corresponding to the query request is input into the first micro-language model for a forward propagation to obtain the query semantic vector corresponding to the query request. This can be achieved in the following way: The first micro-language model comprises multiple hidden layers. During a single forward propagation of the first micro-language model, the hidden state output from one of the hidden layers (e.g., the middle or last one) is retrieved. The hidden state represents the internal representation vector of the intermediate semantics of the input information generated by the first micro-language model during each layer of neural network processing. The pooling vector corresponding to the hidden state is then obtained. The pooling vector is the result of compressing a variable-length token-level hidden state sequence into a fixed-length sentence / paragraph-level vector through a specific aggregation operation. This pooling vector is then used as the query semantic vector corresponding to the query request. For example, the query semantic vector `eq` is generated by the first micro-language model from the query vector... q The last hidden state is pooled, for example, by mean pooling.

[0050] Step 105: Based on the first difficulty index, the second difficulty index, and the third difficulty index, obtain the difficulty score value corresponding to the query request.

[0051] For example, based on the first difficulty indicator, the second difficulty indicator, and the third difficulty indicator, the difficulty score corresponding to the query request can be obtained. Specifically, this can be achieved by weighted summation of the first difficulty indicator, the second difficulty indicator, and the third difficulty indicator.

[0052] The weights corresponding to the first, second, and third difficulty indicators can be either dynamically adjustable weight coefficients or fixed default values. If dynamic weight coefficients are used, they can be adaptively adjusted based on the load.

[0053] Step 106: Based on the difficulty score and the preset difficulty threshold, determine the target candidate model set corresponding to the query request; select a candidate model from the target candidate model set as the target model for processing the query request.

[0054] In some embodiments, the set of target candidate models corresponding to a query request is determined based on the difficulty score and a preset difficulty threshold, which can be divided into the following three cases: Scenario 1: If the difficulty score is greater than the preset difficulty threshold, and the absolute value of the difference between the difficulty score and the preset difficulty threshold is greater than the preset difference (i.e., the difficulty score is significantly greater than the preset difficulty threshold), the target candidate model set corresponding to the query request is determined as the first candidate model set.

[0055] The first candidate model set includes at least one large language model deployed in the cloud.

[0056] The preset difference is a value close to 0, for example, the preset difference range is [0, 0.1].

[0057] Scenario 2: If the difficulty score is less than the preset difficulty threshold and the absolute value of the difference between the difficulty score and the preset difficulty threshold is greater than the preset difference (i.e., the difficulty score is significantly less than the preset difficulty threshold), the target candidate model set corresponding to the query request is determined as the second candidate model set.

[0058] The second candidate model set includes at least one locally deployed micro-language model. For example, the first micro-language model mentioned above is one of the models in the second candidate model set.

[0059] Scenario 3: If the absolute value of the difference between the difficulty score and the preset difficulty threshold is less than or equal to the preset difference (i.e., the difficulty score is close to the preset difficulty threshold), the target candidate model set corresponding to the query request is determined as the third candidate model set.

[0060] The third candidate model set is the union of the first candidate model set and the second candidate model set.

[0061] Selecting a candidate model from the set of target candidate models as the target model for processing the query request can be specifically: In some embodiments, the candidate model with the highest decision score from the target candidate model set can be further selected as the target model for processing the query request based on the decision score obtained in stage three. In other embodiments, the candidate model with the lowest cost can be selected from the target candidate model set as the target model for processing the query request.

[0062] Figure 1 The illustrated embodiment reflects the complexity of query requests from multiple dimensions, not just based on text length. It comprehensively assesses whether the model has sufficient capacity to handle the query request (the first difficulty indicator) and how many predefined features the query request contains. These predefined features can be code snippets, mathematical formulas, or other technical symbols and keywords. Technical keywords can be keywords from classic mathematical proofs or other common, high-difficulty keywords in other fields. This allows for accurate identification of "short but difficult" tasks (such as mathematical proofs) and "long but simple" tasks (such as long text copying). It can accurately filter out high-difficulty query requests that the local micro-language model cannot handle, avoiding the illusion caused by the local micro-language model forcing an answer. Simultaneously, it intercepts low-difficulty requests locally, saving costs by avoiding uploading to the cloud and improving the rationality of the model's routing strategy. Furthermore, the complexity assessment scheme of this embodiment is interpretable. Compared to BERT's black-box routing scheme, the reliability of the assessment results can be verified and analyzed, improving the reliability of the model's routing.

[0063] In one specific embodiment, the following comprehensive complexity scoring function can be used in stage one. Difficulty rating: (1) Formula (1) is used to quantize the input query vector. The difficulty of processing local models (such as the first micro-language model).

[0064] in, Indicates the difficulty rating. Indicates the level of confusion. Represents the local model (For example, the first micro-language model) for the second The perplexity of each token.

[0065] This indicates the highest level of difficulty.

[0066] Query vector The total token length. The query vector is the initial vector obtained by mapping the text-formatted query request (used as the input vector for the local model).

[0067] This indicates the number of code snippets counted by regular expression matching. This indicates the number of mathematical formulas counted through regular expression matching. For example, mathematical formulas can be obtained by editing LaTeX or other formula editors. LaTeX is a document preparation system based on the TeX typesetting system.

[0068] This indicates the second difficulty level. Among them, It can be replaced with , This indicates the number of technical keywords in the query request. For example, whether the query request contains technical keywords such as "Riemann Hypothesis".

[0069] This represents the semantic abstraction score, which is the third difficulty indicator.

[0070] These are the dynamic weighting coefficients corresponding to the first, second, and third difficulty indicators, which can be adaptively adjusted according to the system load.

[0071] This is a weighted fusion formula for multi-dimensional features. It doesn't just look at the length, but also at "how much the model doesn't understand the input" (the first term) and "how many technical symbols are contained in the input" (the second term).

[0072] After obtaining the difficulty score, a target candidate model set is determined from multiple candidate model sets based on the comparison between the difficulty score and a preset difficulty threshold. These multiple candidate model sets include a first candidate model set, a second candidate model set, and a third candidate model set. The first candidate model set is a collection of commercially available large language models deployed in the cloud; the second candidate model set is a collection of models deployed locally (including at least one micro-language model); and the third candidate model set is the union of the first and second candidate model sets.

[0073] In Phase 1, a task complexity assessment based on perplexity and feature density (density of predefined features) was implemented. The difficulty score calculated by the above formula (1) was used as one of the criteria for determining whether to migrate to the cloud. This can accurately identify "short and difficult" tasks (such as mathematical proofs) and "long and easy" tasks (such as copying long texts). The logarithmic mean of perplexity (PPL) was introduced, which, compared with the traditional length threshold method, can reflect the semantic level of difficulty. In this way, "high perplexity" requests that the local model cannot handle can be accurately filtered out, avoiding the illusion caused by forced local answers; at the same time, low perplexity (low difficulty) requests are intercepted locally, saving costs.

[0074] In one specific embodiment, in stage one, formula (1) It can be obtained from a single forward propagation of the local micro-language model using the token-level negative log-likelihood (without generating the response text): query vector q Word segmentation yields x1,…,x T And add the start character x0= at the beginning. <bos>(such that t = 1 also has conditional context).

[0075] Use the teacher - forcing input prefix , and the local micro - language model outputs the predicted distribution pθ(·|x<t) of the next token (in implementation, all - position logits can be obtained from the previous forward pass).

[0076] Take the conditional probability of the true token: Define the perplexity of the t - th token as: = , where is obtained by applying softmax to the logits at this position and taking the probability corresponding to the true token is sufficient.

[0077] In stage two, adopt the semantic rewriting (NaPaRe) mechanism based on the maximum utility - privacy ratio.

[0078] When in stage one or , it is determined that the large - language model deployed in the cloud needs to be called. That is, when the set of target candidate models includes the large - language model deployed in the cloud (the determined target candidate set is the first candidate set or the third candidate set), call the local security scanner to detect whether the query request contains sensitive information. If sensitive information is detected, execute the process of stage two, perform semantic rewriting on the query request, generate multiple rewritten candidate sets using the local model, and select the optimal solution as the target request, and upload the target request to the cloud to prevent the unintentional or malicious leakage of sensitive information.

[0079] Among them, sensitive information includes at least one of PII (Personally Identifiable Information, personal identifiable information or personal identity information) and enterprise - sensitive data. For example, PII information may include ID card number, mobile phone number, bank card number, address, etc. Enterprise - sensitive data includes source code, internal API keys, financial data, customer lists, etc.

[0080] Specifically, in some embodiments, in stage two, in response to the local security scanner detecting that the query request includes sensitive information, desensitize the query request through the second micro - language model deployed locally to obtain multiple candidate rewritten requests corresponding to the query request. Then, select one rewritten request as the target request from the multiple desensitized candidate rewritten requests. The target request will replace the original query request and be uploaded to the cloud to prevent the leakage of user privacy data.

[0081] The second micro-language model deployed locally can be the same micro-language model as the first micro-language model mentioned above, or it can be a different model.

[0082] The following rewrite score can be obtained for each candidate rewrite request using a text rewrite selection objective function: (2) in, This represents the embedding vectors corresponding to the original query request and the rewrite request. The query vector... The input is fed into the first miniature language model for forward propagation, and the embedding vector output from one of the hidden layers is used as... The input vector corresponding to the rewrite request will be rewritten. The input is fed into the second miniature language model for forward propagation, and the embedding vector output from one of the hidden layers is used as... . It represents semantic similarity, or semantic fidelity.

[0083] The set of sensitive entities identified in the original query request can also be called the set of sensitive information, including sensitive information (such as names, ID numbers, etc.). express To rewrite the word segmentation in the request.

[0084] Indicator function, if the rewrite request (or rewrite text) still contains sensitive entities. If so, the penalty items will increase sharply.

[0085] : Represents KL divergence, quantization rewrite request probability distribution and natural language reference distribution The differences prevent the generation of rigid machine language.

[0086] The natural language reference probability distribution can be the probability distribution given by an offline-trained reference language model (or statistical language model). It can be understood as the conditional probability distribution given by the reference language model, and can be obtained in the following way: : First, prepare a reference corpus. High-quality, human-written natural language text; can use general corpora (news / wiki / Q&A / dialogue, etc.) or domain-specific corpora consistent with the scenario (such as non-sensitive parts of patent specifications / technical documents / corporate knowledge bases). Uses a tokenizer consistent with the system. Train / fit a reference language model For example, it could be an n-gram, a miniature language model, or a fine-tuned miniature language model, with fixed parameters as a baseline. An n-gram represents a basic language model and text representation method in natural language processing and information retrieval.

[0087] The runtime reference language model The output conditional probability distribution is used as P_ref: for any prefix ,make Then calculate / accumulate KL (or take the average) at each token position as a penalty.

[0088] It should be noted that a simpler empirical distribution (such as unigram / bigram frequency normalization) can also be used as... They represent semantic fidelity ( Privacy penalties ), smoothness ( The weight of ).

[0089] Formula (2) is used to select from the candidate rewrite set. Select the best de-identified text from (including multiple candidate rewrite requests) (Target request) is sent to the cloud.

[0090] Formula (2) is a multi-objective optimization benefit function that requires the rewritten text (i.e. the rewrite request) to be as close as possible to the original text of the query request in the semantic vector space (to ensure that the cloud can understand the logic), while the number of times sensitive entities appear must be 0 (to ensure privacy), and the text fluency must be high.

[0091] The steps in Phase Two above can solve the problem of traditional masking disrupting sentence structure. For example, "Zhang San got diabetes" can be rewritten as "A middle-aged male patient was diagnosed with a metabolic disease," instead of simply "got sick." Or, the original query "Zhang San, 52 years old, blood pressure 180 / 110 mmHg, diagnosed with diabetes for 2 years" can be rewritten as: "A male patient in his early 50s suffers from severe hypertension and long-term metabolic disease."

[0092] This method introduces semantic similarity. divergence with KL (Measure the naturalness of the generated text) as a constraint to ensure that the rewritten text is suitable for large model inference. This achieves the goal of data being usable but not visible, allowing cloud-based business models to obtain sufficient logical context for inference, but preventing access to users' real private data.

[0093] The first and second micro-language models can be the same model or two different local models. For example, in one embodiment, the same locally deployed micro-language model (SLM) can be used to perform both "difficulty assessment" and "rewrite generation" functions.

[0094] If using the same SLM to differentiate between answering questions and generating rewrite candidate sets, control can be achieved through "calling patterns + prompt word templates + output constraints": In Phase 1 (Difficulty Assessment): During the forward propagation process, logits or logprobs are used as perplexity PPL for structural feature statistics, but no decoding or generation is performed (equivalent to the use of max_new_tokens=0). Therefore, the model does not perform the answering task and only outputs features.

[0095] In Phase Two (Semantic Rewriting): The same SLM is invoked in "rewriter" mode, using a special rewriting prompt (clearly stating "rewrite only, do not respond"), and a fixed output format is required (such as returning a list of K rewrites / JSON). Multiple candidate rewrite requests are then generated at once in conjunction with sampling parameters.

[0096] It should be noted that the output of the local SLM in Phase 2 is a "target request that can be deployed to the cloud / inferenced", not the final answer.

[0097] In Phase 3, predictive routing and model hierarchies based on matrix factorization are implemented.

[0098] After determining the query request (the original query request or the rewritten target request) and the target candidate model set, it is necessary to select which specific model to call from the multiple candidate models in the target candidate model set through stage three.

[0099] Specifically, matrix factorization techniques are used to predict the win rate of each model.

[0100] The following formulas (3) and (4) are used for the win rate prediction and cost decision equations. : (3) (4) Formula (3) first predicts the winning rate, and then combines it with formula (4) to obtain the decision score to predict the cost.

[0101] Formula (3) will query the vector Mapped to latent factor vectors , model Mapped to capability vector The dot product after the Sigmoid function Mapping to Model The probability of being able to perfectly solve the problem.

[0102] Query vector The latent feature vectors (latent factor vectors) are generated by the hidden layer of the local router.

[0103] :Model The capability feature vectors are obtained through offline training.

[0104] Model bias term, representing the baseline capability of the model, such as GPT-4. The value was significantly higher than that of Llama-3.

[0105] User-adjustable cost sensitivity factor. The larger the system, the more concerned it is about cost and the more it strives for lower costs. At that time, the system only pursues the strongest capabilities without considering the cost.

[0106] For example, it can be obtained in the following ways : This is the "model-level bias / intercept" for each candidate model m, used to characterize the difference in baseline win rate (overall capability) of that model without considering specific query vectors. It is estimated through offline training on data. Specific acquisition methods (see [link to relevant documentation]). (Same offline training process) Constructing training samples: Collect a batch of historical / annotated query vectors The model m generates the answer and obtains labels through automatic evaluator / human review / user feedback. (Whether it meets the standard).

[0107] Training objective: To achieve maximum likelihood estimation based on cross-entropy, through joint learning. , and : , Gradient descent minimize (L2 regularization can be added); Next, initialization (optional): First, calculate the empirical win rate p of model m on the training set. m Initialize with logarithmic odds The scalar is then updated during training, and the scalar value after training converges is the bias term of model m. .

[0108] Optional, it is also necessary to... Maintenance: When the model version / data distribution changes, periodically retrain with new data or only retrain on new data. Perform online calibration updates.

[0109] In one embodiment, (The capability feature vector of model m) can be trained offline in the form of matrix factorization / implicit feedback recommendation. Essentially, it is a set of trainable parameters (Embedding layer, hidden layer parameters) for each candidate model. The win rate prediction in formula (3) is supervised by historical / benchmark evaluation data.

[0110] Used to obtain The offline training process can be as follows: Construct training samples (problem-model-results), and collect a batch of queries. (From historical routing logs, internal benchmark problem sets, and synthesized problem sets, etc.). For each Let the candidate model set M generate answers one by one, and obtain the effect labels using offline evaluation. The effect labels are binary: (Pass / Fail), for example: if there is a standard answer, compare the correctness; if there is no standard answer, use rules / unit tests / human annotation / review model to score and then threshold.

[0111] Or the effect label is a continuous score: It can be further mapped to binary or directly used for regression or ranking learning.

[0112] For example, the query vector can be obtained in the following manner. : Use the local router's encoder layer Encode the text query as Features such as the perplexity level of stage one and the density of predefined features can also be concatenated. .

[0113] Next, we will learn together. (as well as (And if necessary, also include θ), for example, using formula (3): Minimize cross-entropy (with regularization) as an example: In practical applications, negative sampling / sorting loss can be used, that is, allowing the same The "superior model" scores higher than the "inferior model" to improve training efficiency.

[0114] After training converges, save the corresponding data for each model. and As a model capability profile, only calculation is needed during online routing. It can predict the win rate in milliseconds.

[0115] Optionally, when upgrading a model version or adding a new model, periodically retrain / fine-tune it offline using new logs or a new benchmark set. .

[0116] Formula (4) subtracts the normalized cost term from the win rate to find the extreme value of cost-effectiveness.

[0117] The third stage completes route selection within milliseconds (requiring only one vector dot product operation), without calling expensive large models for "self-evaluation." Compared to the cascading calls of FrugalGPT (which increases latency), the scheme proposed in this application enables parallel prediction; compared to the BERT classifier, this scheme has a very small number of parameters, making it suitable for edge deployment. It achieves dynamic "model hierarchical" designation. For simple problems, the cost item... It will dominate decision-making and tend to favor local models; for complex problems, the win rate... It will lead the decision-making process and automatically upgrade to a business model.

[0118] It should be further explained that, in Phase 1, in one embodiment, the target candidate set can be determined in the following way: a preset difference value of 0 can be set, and the difficulty score value is obtained by formula (1) while the identifier cloud_required is output: cloud_required = ( >Threshold), and output the margin (i.e., the preset difference) of the confidence / distance threshold.

[0119] If cloud_required=1, that is... If Threshold is true, the target candidate model set M = M_cloud, where M_cloud represents the first candidate model set (cloud-based models only / cloud-level models). Local models are directly set to unselectable, for example, setting the decision score Score=-∞ or Cost=+∞ for local models (models in the second candidate model set).

[0120] If cloud_required=0, that is... Threshold is not true, and Threshold, the target candidate model set M = M_local, where M_local represents the second candidate model set (multiple models such as small or medium-sized models deployed locally).

[0121] If cloud_required=0 but Threshold determines whether cloud deployment is an option. Let the target candidate model set M = M_local ∪ M_cloud, but the cost-sensitive factor η prevents cloud deployment from being selected when not necessary.

[0122] It should be noted that for critical samples ( (Approaching the preset difficulty threshold), a three-state / soft constraint can be set, meaning the candidate model set can be divided into three categories: local priority (second candidate model set) / cloud-ready (third candidate model set) / cloud-ready (first candidate model set); only in the cloud-ready and cloud-ready states, the execution phase three is based on The decision score is obtained by combining the cost results with the overall decision score. Then, the model with the highest decision score (highest cost-effectiveness) is selected from the corresponding set as the target model.

[0123] Based on the above exemplary description, it can be seen that in some embodiments, in stage two, the latent factor vector corresponding to the query request or target request can be obtained. ), and obtain the capability vectors corresponding to multiple candidate models respectively ( ), and determine the deviation term (i.e. ); calculate the dot product of the latent factor vector and the capability vector, and use the Sigmoid function to map the sum of the dot product and the bias term to the win rate, which represents the probability that a candidate model supports the response to the query request or the target request.

[0124] In some embodiments, selecting a candidate model from the set of target candidate models as the target model for processing the query request may be based at least on the win rate.

[0125] For example, first determine the cost sensitivity factor; then calculate the cost corresponding to each candidate model among multiple candidate models. And the maximum cost of multiple candidate models The ratio ( Based on ratios and cost sensitivity factors , to obtain the cost item ( Based at least on the win rate, select a model from the target candidate model set as the target model to process the query request. For example, this could be based on the win rate and cost, obtaining the decision scores for each of the multiple candidate models. ); Select the model with the highest decision score from the set of target candidate models as the target model for processing the query request (for example, the argmax function in formula (4) can select the model with the highest decision score from the set M).

[0126] The model routing apparatus provided in the embodiments of this application is described below. The model routing apparatus described below can be referred to in correspondence with the model routing method described above.

[0127] This application provides a model routing device. The corresponding product of this system can be an electronic device used to implement the corresponding function, such as a server or server cluster. Figure 2 As shown, the device may include: The first determining module 201 is used to determine the perplexity of the first micro-language model deployed locally for each token in the query request, and to obtain the first difficulty index based on the perplexity. The second determining module 202 is used to count the first number of predefined features in the query vector by regular expression matching; the predefined features include code snippets and / or mathematical formulas; and to obtain a second difficulty index based on the number of tokens included in the query request and the first number. The third determination module 203 is used to obtain the semantic abstraction score corresponding to the query request, as the third difficulty indicator; The comprehensive difficulty scoring module 204 is used to obtain the difficulty score value corresponding to the query request based on the first difficulty index, the second difficulty index, and the third difficulty index. The routing module 205 is used to determine the set of target candidate models corresponding to the query request based on the difficulty score and the preset difficulty threshold; and to select a candidate model from the set of target candidate models as the target model for processing the query request.

[0128] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call a computer program in the memory 330 to execute the steps of the model routing method, such as: In response to a received user query request, the system determines the perplexity of the first micro-language model deployed locally for each token included in the query request, and obtains a first difficulty index based on the perplexity. It then uses regular expression matching to count the first number of predefined features in the query request; these predefined features include specified technical symbols and / or specified technical keywords. Based on the number of tokens included in the query request and the first number, it obtains a second difficulty index. It acquires the semantic abstraction score corresponding to the query request as a third difficulty index. Based on the first, second, and third difficulty indices, it obtains a difficulty score value corresponding to the query request. Based on the difficulty score value and a preset difficulty threshold, it determines the target candidate model set corresponding to the query request. Finally, it selects a candidate model from the target candidate model set as the target model for processing the query request.

[0129] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the model routing method provided in the above embodiments, such as including: In response to a received user query request, the system determines the perplexity of the first micro-language model deployed locally for each token included in the query request, and obtains a first difficulty index based on the perplexity. It then uses regular expression matching to count the first number of predefined features in the query request; these predefined features include specified technical symbols and / or specified technical keywords. Based on the number of tokens included in the query request and the first number, it obtains a second difficulty index. It acquires the semantic abstraction score corresponding to the query request as a third difficulty index. Based on the first, second, and third difficulty indices, it obtains a difficulty score value corresponding to the query request. Based on the difficulty score value and a preset difficulty threshold, it determines the target candidate model set corresponding to the query request. Finally, it selects a candidate model from the target candidate model set as the target model for processing the query request.

[0131] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to perform the steps of the methods provided in the above embodiments, such as including: In response to a received user query request, the system determines the perplexity of the first micro-language model deployed locally for each token included in the query request, and obtains a first difficulty index based on the perplexity. It then uses regular expression matching to count the first number of predefined features in the query request; these predefined features include specified technical symbols and / or specified technical keywords. Based on the number of tokens included in the query request and the first number, it obtains a second difficulty index. It acquires the semantic abstraction score corresponding to the query request as a third difficulty index. Based on the first, second, and third difficulty indices, it obtains a difficulty score value corresponding to the query request. Based on the difficulty score value and a preset difficulty threshold, it determines the target candidate model set corresponding to the query request. Finally, it selects a candidate model from the target candidate model set as the target model for processing the query request.

[0132] Processor-readable storage media can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.< / bos>

Claims

1. A model routing method, characterized in that, The method includes: In response to receiving a user's query request, the perplexity of a first micro-language model deployed locally for each token included in the query request is determined, and a first difficulty index is obtained based on the perplexity. The first number of predefined features in the query request is counted by regular expression matching; the predefined features include specified professional symbols and / or specified technical keywords. Based on the number of tokens included in the query request and the first number, a second difficulty index is obtained; Obtain the semantic abstraction score corresponding to the query request, as the third difficulty indicator; Based on the first difficulty index, the second difficulty index, and the third difficulty index, the difficulty score value corresponding to the query request is obtained; Based on the difficulty score and the preset difficulty threshold, a set of target candidate models corresponding to the query request is determined; a candidate model is selected from the set of target candidate models as the target model for processing the query request.

2. The method according to claim 1, characterized in that, Obtain the semantic abstraction score corresponding to the query request, including: The query vector corresponding to the query request is input into the first micro-language model for one forward propagation to obtain the query semantic vector corresponding to the query request. Calculate the norm of the query semantic vector; or, The query semantic vector is input into a pre-trained feedforward neural network, and projected through the feedforward neural network to obtain a semantic feature vector. The value of a specified dimension in the semantic feature vector is used as the activation value. The specified dimension is the dimension used to represent the semantic abstraction during the training process of the feedforward neural network. The norm or the activation value is used as the semantic abstraction score.

3. The method according to claim 1, characterized in that, When the target candidate model set includes large language models deployed in the cloud, after determining the target candidate model set corresponding to the query request, the method further includes: In response to a local security scanner detecting that the query request includes sensitive information, the query request is de-identified using a local second micro-language model to obtain multiple candidate rewrite requests corresponding to the query request. From the multiple candidate rewrite requests after de-identification, select one rewrite request as the target request.

4. The method according to claim 3, characterized in that, From the multiple candidate rewrite requests after de-identification, select one rewrite request as the target request, including: Obtain the embedding vectors corresponding to each rewrite request among the multiple candidate rewrite requests after anonymization; calculate the semantic similarity between the query vector corresponding to the original query request and the embedding vector corresponding to a rewrite request; Determine the set of sensitive entities; identify whether the rewrite request includes sensitive words belonging to the set of sensitive entities, and determine the penalty items; Obtain the first probability distribution corresponding to the rewrite request, and obtain the natural language reference probability distribution corresponding to the original query request. Use KL divergence to calculate the difference between the first probability distribution and the natural language reference probability distribution to obtain the KL divergence value. Based on the semantic similarity, the penalty term, and the KL divergence value, a rewrite score is obtained for each of the multiple candidate rewrite requests; the rewrite request with the highest rewrite score is selected as the target request.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Obtain the latent factor vector corresponding to the query request or the target request; Obtain the capability vectors corresponding to multiple candidate models; Identify the deviation term; Calculate the dot product of the latent factor vector and the capability vector, and use the Sigmoid function to map the sum of the dot product and the bias term to a win rate; the win rate represents the probability that a corresponding candidate model supports the response to the query request or the target request. Selecting a candidate model from the set of target candidate models as the target model for processing the query request includes: Based at least on the win rate, select a model from the set of target candidate models as the target model for processing the query request.

6. The method according to claim 5, characterized in that, Before selecting a model from the plurality of candidate models as the target model for processing the query request, based at least on the win rate, the method further includes: Identify cost-sensitive factors; Calculate the ratio of the cost corresponding to each of the multiple candidate models to the maximum cost of the multiple candidate models, and obtain the cost item based on the ratio and the cost sensitivity factor; Based at least on the win rate, selecting a model from the set of target candidate models as the target model for processing the query request includes: Based on the win rate and the cost item, the decision score corresponding to each of the multiple candidate models is obtained; Select the model with the highest decision score from the set of target candidate models as the target model for processing the query request.

7. The method according to claim 1, characterized in that, Based on the difficulty score and a preset difficulty threshold, a set of target candidate models corresponding to the query request is determined, including: Based on the fact that the difficulty score is greater than the preset difficulty threshold and the absolute value of the difference between the difficulty score and the preset difficulty threshold is greater than a preset difference, the target candidate model set corresponding to the query request is determined as the first candidate model set; the first candidate model set includes at least one large language model deployed in the cloud; Based on the fact that the difficulty score is less than the preset difficulty threshold and the absolute value of the difference between the difficulty score and the preset difficulty threshold is greater than a preset difference, the target candidate model set corresponding to the query request is determined as the second candidate model set; the second candidate model set includes at least one micro-language model deployed locally; Based on the fact that the absolute value of the difference between the difficulty score and the preset difficulty threshold is less than or equal to the preset difference, the target candidate model set corresponding to the query request is determined as the third candidate model set; the third candidate model set is the union of the first candidate model set and the second candidate model set.

8. A model routing device, characterized in that, The device includes: The first determining module is used to determine the perplexity of a first micro-language model deployed locally for each token in the query request, and to obtain a first difficulty index based on the perplexity. The second determining module is used to count the first number of predefined features in the query vector by regular expression matching; the predefined features include code snippets and / or mathematical formulas; and to obtain a second difficulty index based on the number of tokens included in the query request and the first number. The third determination module is used to obtain the semantic abstraction score corresponding to the query request, as the third difficulty indicator; The comprehensive difficulty scoring module is used to obtain the difficulty score value corresponding to the query request based on the first difficulty index, the second difficulty index and the third difficulty index; The routing module is used to determine the target candidate model set corresponding to the query request based on the difficulty score and the preset difficulty threshold; and to select a candidate model from the target candidate model set as the target model for processing the query request.

9. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the model routing method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the model routing method according to any one of claims 1 to 7.

Images