A method of determining a model representation and a model route, and an electronic device

Abstract

This application discloses a method and electronic device for determining model representation and model routing. The method includes: determining textual description information for at least one model, wherein the textual description information is used to describe the capability features of the model; constructing orthogonalized semantic vectors for each model based on the textual description information of the at least one model, wherein the orthogonalized semantic vectors of the model are used to characterize the semantics corresponding to the textual description information of the model, and the orthogonalized semantic vectors of different models are orthogonal to each other; and determining a model representation vector for the model based on the orthogonalized semantic vectors of the model, wherein the model representation vector is used to locate the model during the process of model routing for at least one model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for determining model representation and model routing, as well as an electronic device. Background Technology

[0002] To improve the accuracy of information reasoning, platforms such as intelligent question answering typically deploy multiple models. In this case, the platform can route models from among the various options based on the user's input question or dialogue information to determine the most suitable model for handling the specific question. For example, if the question is related to code development, a model skilled in code analysis should be selected. However, because the characteristics of each model cannot be accurately represented, the accuracy of model routing is low, thus affecting the precision of question information processing. Summary of the Invention

[0003] On the one hand, this application provides a method for determining a model representation, including:

[0004] Determine textual description information for at least one model, the textual description information being used to describe the capability characteristics of the model;

[0005] Based on the text description information of at least one model, orthogonalized semantic vectors of each model are constructed respectively. The orthogonalized semantic vectors of the models are used to represent the semantics corresponding to the text description information of the models, and the orthogonalized semantic vectors of different models are orthogonal to each other.

[0006] Based on the orthogonalized semantic vector of the model, the model representation vector of the model is determined, and the model representation vector is used to locate the model during the model routing process for the at least one model.

[0007] In one possible implementation, constructing orthogonalized semantic vectors for each of the at least one model based on its textual description information includes:

[0008] Based on the textual description information of the model, a semantic encoding vector of the model is generated;

[0009] The semantic encoding vectors of at least one model are orthogonalized to obtain orthogonalized semantic vectors for each model.

[0010] In another possible implementation, the step of constructing orthogonalized semantic vectors for each of the at least one model based on the textual description information of the model includes:

[0011] Construct initial orthogonal vectors for each model, ensuring that the initial orthogonal vectors of different models are orthogonal to each other;

[0012] Based on the textual description information of the model, the capability representation features of the model are determined, and the capability representation features are used to characterize the capability magnitude of the model in multiple types of capabilities;

[0013] Based on the capability representation features of the model and the capability types represented by different dimension intervals in the initial orthogonal vector, the initial orthogonal vector of the model is adjusted to obtain the orthogonalized semantic vector of the model. The orthogonalized semantic vector can represent the capability magnitude of the model in different types of capabilities.

[0014] In another possible implementation, determining the model representation vector based on the orthogonalized semantic vector of the model includes at least one of the following:

[0015] The orthogonalized semantic vector of the model is determined as the model representation vector of the model;

[0016] Based on the candidate model representation vector of the model and the semantic vector of the first problem sample, the prediction fitness of the model for the first problem sample is determined. The candidate model representation vector is composed of the orthogonalized semantic vector and the initial bias vector corresponding to the model. Based on the actual fitness of the model for the first problem sample and the prediction fitness, the initial bias vector of the model is adjusted to obtain the bias vector of the model. Based on the bias vector of the model and the orthogonalized semantic vector, the model representation vector of the model is determined.

[0017] In another possible implementation, adjusting the initial bias vector of the model based on the model's actual fit to the first problem sample and the predicted fit includes:

[0018] Based on the actual fit of each model to the first problem sample and the predicted fit, the initial bias vector of the model is adjusted in conjunction with the loss function, which includes a first loss term and a second loss term.

[0019] Wherein, the first loss term is related to the difference between the actual fit of the model to the first problem sample and the predicted fit;

[0020] The second loss term is the L2 regularization term of the initial bias vector.

[0021] In yet another possible implementation, the method for determining the model representation also includes:

[0022] After obtaining the adjusted initial bias vector, norm clipping is performed on the adjusted initial bias vector based on the set maximum allowable norm, and the bias vector obtained by norm clipping is determined as the adjusted bias vector corresponding to the model.

[0023] In another possible implementation, determining the model's prediction fit for the first problem sample based on the candidate model representation vector of the model and the semantic vector of the first problem sample includes:

[0024] Based on the semantic vector of the first problem sample and the candidate model representation vector of the model, the prediction fit of the model for the first problem sample is determined using the routing model.

[0025] The step of adjusting the initial bias vector of the model based on the actual fit and predicted fit of each model to the first problem sample, combined with the loss function, includes:

[0026] Based on the actual fit and predicted fit of each model to the first problem sample, and in conjunction with the loss function, the initial bias vector of the model and the weight parameters of the routing model are adjusted synchronously, wherein the routing model is used to perform model routing for the at least one model.

[0027] In yet another possible implementation, the method for determining the model representation also includes:

[0028] In response to the existence of a newly deployed model, the semantic encoding vector of the new model is determined based on the text description information of the new model;

[0029] Based on the orthogonalized semantic vector of the at least one model, the semantic encoding vector of the newly added model is orthogonalized to obtain the orthogonalized semantic vector of the newly added model.

[0030] Based on the orthogonalized semantic vector of the newly added model, the model representation vector of the newly added model is determined.

[0031] Furthermore, this application also provides a model routing method, including:

[0032] In response to obtaining problem information, determine the semantic vector of the problem information;

[0033] Based on the semantic vector and the model representation vector of each of the at least one model, a target model is determined from the at least one model;

[0034] Based on the semantic vector of the question information, the answer information of the question information is inferred using the target model;

[0035] Wherein, the model representation vector of the model is determined based on the orthogonalized semantic vector of the model, and the orthogonalized semantic vector of the model is constructed based on the textual description information of the model, and is used to represent the vector of the model; the textual description information is used to describe the capability features of the model;

[0036] The orthogonalized semantic vectors of different models are orthogonal.

[0037] In another aspect, this application also provides an electronic device comprising: a processor and a memory;

[0038] The processor is configured to: determine textual description information for at least one model, wherein the textual description information describes the capability features of the model; construct orthogonalized semantic vectors for each model based on the textual description information of the at least one model, wherein the orthogonalized semantic vectors of the models represent the semantics corresponding to the textual description information of the models, and the orthogonalized semantic vectors of different models are orthogonal to each other; and determine model representation vectors for the models based on the orthogonalized semantic vectors of the models, wherein the model representation vectors are used to locate the models during model routing for the at least one model.

[0039] The memory is used to store the computer programs required by the processor to perform operations. Attached Figure Description

[0040] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0041] Figure 1 A flowchart illustrating the method for determining a model representation provided in this application;

[0042] Figure 2 Another flowchart illustrating the method for determining the model representation provided in this application;

[0043] Figure 3 Another flowchart illustrating the method for determining the model representation provided in this application;

[0044] Figure 4 Another flowchart illustrating the method for determining the model representation provided in this application;

[0045] Figure 5 Another flowchart illustrating the method for determining the model representation provided in this application;

[0046] Figure 6 A flowchart illustrating the model routing method provided in this application;

[0047] Figure 7 One component structure of the electronic device provided in this application. Detailed Implementation

[0048] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0049] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0050] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0051] like Figure 1 This is a flowchart illustrating a method for determining a model representation provided in this application. The method of this embodiment can be applied to an electronic device, which can be a device node in a platform that deploys at least one model, or an electronic device outside the platform, without limitation.

[0052] The method in this embodiment may include:

[0053] S101, determine the textual description information for at least one model.

[0054] This textual description is used to describe the model's capabilities. These capabilities characterize the domain scenarios the model is suited for and the magnitude of its capabilities across different types of situations. Domain scenarios can be categorized as data domains, code analysis domains, medical domains, and dialogue domains, among others.

[0055] In one example, the text description information may include at least one of the following: the model's name, the domain label of the applicable domain scenario, the capability description, the model's functional description, and the appropriate problem difficulty level.

[0056] The textual description information can be obtained from the publicly available documentation when the model is released, extracted from the evaluation report of the model evaluation benchmark, or read from the model configuration information of the platform. There are no restrictions on the specific implementation of obtaining the textual description information of the model.

[0057] In this application, the at least one model is a model deployed on a business platform that can be used to process information such as questions, dialogues, or task requests. There are no restrictions on the specific type of model.

[0058] S102, based on the textual description information of at least one model, construct the orthogonal semantic vectors of each model respectively.

[0059] The orthogonalized semantic vectors of this model are used to represent the semantics corresponding to the textual description information of the model, and the orthogonalized semantic vectors of different models are orthogonal to each other.

[0060] The orthogonality between orthogonalized semantic vectors of different models means that the orthogonalized semantic vectors of different models are orthogonal or approximately orthogonal.

[0061] For example, orthogonality of orthogonalized semantic vectors from different models can be achieved if the absolute value of the inner product of the orthogonal semantic vectors of any two models is less than a set threshold. That is, for any two models in at least one model... With model The model orthogonalized semantic vectors With model orthogonalized semantic vectors If it has orthogonality, then the following condition is satisfied: ;

[0062] in, The threshold can be set as needed. For example, It can be .

[0063] The model For at least one model, the first A model, model For at least one model, the first One model, and Not greater than positive integers, and , This represents the total number of models corresponding to at least one model.

[0064] Understandably, the orthogonality of two vectors indicates that they are independent and do not interfere with each other in space, avoiding the situation where two vectors overlap and become indistinguishable. Based on this, since orthogonalized semantic vectors are used to represent the semantics corresponding to the textual description information of the model, the orthogonalized semantic vectors of the model can represent the model's capability characteristics; furthermore, the orthogonality between the orthogonalized semantic vectors of different models ensures that they can be distinguished from each other.

[0065] S103, based on the orthogonalized semantic vector of the model, determines the model representation vector.

[0066] This model representation vector is used to locate (i.e. match) a model during model routing for at least one model.

[0067] Model routing refers to the process of selecting the most suitable model to process user-input question information on a platform that has at least one model deployed (e.g., usually multiple models). In essence, model routing assigns the most appropriate model to each processing task corresponding to each input information.

[0068] In this design, because the orthogonalized semantic vectors of a model can represent its capabilities, the model representation vectors generated based on these vectors can also express semantic information such as the model's capabilities, thus making the model representation vectors interpretable. Furthermore, the orthogonality between the orthogonalized semantic vectors of different models results in low overlap between their model representation vectors, leading to higher discriminative power. This reduces the likelihood of inaccurate model matching during the routing process due to spatial overlap between the model representation vectors of different models.

[0069] In this application, there are multiple possibilities for determining the model representation vector based on the orthogonalized semantic vector of the model, without any specific restrictions.

[0070] As can be seen from the above, in this application, orthogonalized semantic vectors for each model are constructed based on the textual description information of each model. Because the orthogonality between the orthogonalized semantic vectors of different models is ensured, the orthogonalized semantic vectors of the models retain the semantics of the textual description information while also making the orthogonalized semantic vectors of different models spatially independent. Based on this, the model representation vectors determined based on the orthogonalized semantic vectors of the models not only reflect the semantic information such as the capabilities expressed by the corresponding textual description information of the model, but also possess spatial discriminability. This reduces the possibility of inaccurate model routing due to the model representation vectors failing to accurately represent the model's capabilities or lacking discriminability, thus improving the accuracy of model routing. Consequently, it allows for the reasonable selection of model-processed problem information, improving the accuracy of model problem processing.

[0071] In this application, there are several possible implementations for constructing the orthogonalized semantic vectors of the model, which will be discussed in detail below. Figure 2 and Figure 3 This paper explains several possible cases of orthogonalizing semantic vectors for constructing models.

[0072] like Figure 2This illustrates another flowchart of the method for determining a model representation in this application. The method in this embodiment may include:

[0073] S201, determine the textual description information for at least one model.

[0074] Textual descriptions are used to describe the capabilities of the model.

[0075] S202, For each model, generate the semantic encoding vector of the model based on the text description information of the model.

[0076] The semantic encoding vector is used to represent the semantics corresponding to the textual description information of the model. Since the textual description information is used to describe the capabilities of the model, the semantic encoding vector can reflect the capabilities of the model.

[0077] In one example, an encoder can be used to encode the textual description information of the model, and the resulting encoded vector is determined as the model's semantic encoding vector. This encoder can be a pre-trained open-source encoder, such as a large-size embedding encoder, or a specially trained encoder to enhance the understanding of domain keywords and other capability information. The semantic encoding vectors of all models have the same dimension.

[0078] Furthermore, considering that the magnitude of the encoded vectors obtained by the encoder from encoding the text description information (also known as the L2 magnitude, which represents the geometric length of the vector in the high-dimensional space) is different, in order to reduce the interference of the uneven magnitude of the encoded vectors of different models on the stability of subsequent orthogonalization processing, this application can also normalize the encoded vectors of each model and determine the normalized encoded vector corresponding to the model as the semantic encoded vector of that model.

[0079] S203, orthogonalize the semantic encoding vector of at least one model to obtain the orthogonalized semantic vector of each model.

[0080] The orthogonalized semantic vector of a model is used to represent the semantics corresponding to the textual description information of the model, and the orthogonalized semantic vectors of different models are orthogonal to each other.

[0081] The purpose of orthogonalizing the semantic encoding vector of at least one model is to make minimal adjustments to the original semantic encoding vector of the model, so that the adjusted vector retains the original semantic features and satisfies the orthogonality constraint.

[0082] For example, the semantic encoding vectors of at least one model can be orthogonally processed sequentially using the Gram-Schmidt orthogonalization algorithm, according to the order of the semantic encoding vectors of at least one model, to obtain a set of orthogonalized vectors corresponding to the at least one model. The orthogonalized vector corresponding to each model is its orthogonalized semantic vector.

[0083] Of course, QR decomposition or group orthogonalization algorithms can also be used to orthogonalize the semantic encoding vectors of at least one model, without any specific restrictions.

[0084] In this embodiment, the semantic encoding vector of the model is first generated based on the text description information of the model, so that the semantic encoding vector can accurately express the capability characteristics of the model. On this basis, by orthogonalizing the semantic encoding vectors of each model, orthogonalized semantic vectors that satisfy the orthogonality constraint can be constructed while retaining the original semantic features of the semantic encoding vectors of the model.

[0085] S204, based on the orthogonalized semantic vector of the model, determines the model representation vector.

[0086] The model representation vector is used to locate the model during model routing for at least one model.

[0087] exist Figure 2 In this embodiment, if a new model is deployed, in order to ensure that the model representation vector of the new model accurately represents its capability characteristics and is distinguishable from the model representation vectors of other models, this application may employ the following method each time a new model is deployed: Figure 2 In this embodiment, the model representation vectors of the existing model and the newly added model are redefined.

[0088] However, regenerating the orthogonalized semantic vector for each new model would inevitably lead to excessive data processing. Therefore, to reduce data processing volume, in response to the deployment of a new model, the semantic encoding vector of the new model is determined based on its textual description information; the semantic encoding vector of the new model is orthogonalized based on the orthogonalized semantic vector of at least one existing model to obtain its orthogonalized semantic vector; and the model representation vector of the new model is determined based on its orthogonalized semantic vector.

[0089] Specifically, based on the orthogonalized semantic vectors of at least one model, orthogonalizing the semantic encoding vector of the newly added model can ensure that the resulting orthogonalized semantic vector of the newly added model is orthogonal to the orthogonalized semantic vectors of other models. For example, the Gram-Schmidt orthogonalization algorithm can be used to orthogonalize the semantic encoding vector of the newly added model based on the orthogonalized semantic vectors of at least one model, without any specific restrictions.

[0090] It is evident that when a new model is added, it is not necessary to reconstruct the orthogonalized semantic vectors of the existing models. Instead, the semantic encoding vector of the new model can be orthogonalized based on the orthogonalized semantic vectors of the existing models. This ensures that the orthogonalized semantic vector of the new model is orthogonal to the orthogonalized semantic vectors of other models. Based on this, by combining the orthogonalized semantic vector of the new model, a model representation vector that can express the semantic information such as the capability features of the new model and is distinguishable from the vector representations of other models can be generated.

[0091] The following is combined Figure 3 Another possible case for constructing orthogonal semantic vectors in the model is explained. For example... Figure 3 This illustration shows another flowchart of the method for determining a model representation provided in this application. The method in this embodiment may include:

[0092] S301, determine the textual description information for at least one model.

[0093] This textual description is used to describe the model's capabilities.

[0094] S302, construct the initial orthogonal vectors for each model respectively.

[0095] Among them, the initial orthogonal vectors of different models are orthogonal.

[0096] In this application, the initial orthogonal vectors of each model can be randomly generated, as long as they are orthogonal to each other. There are no restrictions on the specific implementation of the initial orthogonal vectors for constructing the model.

[0097] The initial orthogonal vectors of each model have the same dimension.

[0098] S303, for each model, determine the capability representation features of the model based on the model's textual description information.

[0099] This capability representation feature is used to characterize the magnitude of the model's capabilities across multiple types of capabilities.

[0100] For example, a pre-trained feature extraction model can be used to extract or encode features from the textual description information of the model to obtain the capability representation features of the model. This feature extraction model can be an encoder or other types of intelligent models, without restriction.

[0101] S304. For each model, based on the capability representation features of the model and the capability types represented by different dimensional intervals in the initial orthogonal vector, adjust the initial orthogonal vector of the model to obtain the orthogonalized semantic vector of the model.

[0102] In this application, different dimension intervals in the initial orthogonal vector correspond to different capability types. Based on this, and combining the capability representation features of the model to characterize the model's capability magnitude in each type of capability, the values ​​of the dimension intervals in the initial orthogonal vector corresponding to the corresponding capability type can be adjusted so that the values ​​in the dimension intervals corresponding to the corresponding capability type can reflect the model's capability magnitude in that capability type. That is, the adjusted orthogonalized semantic vector can characterize the model's capability magnitude in different types of capabilities.

[0103] For example, suppose the initial orthogonal vector is 300-dimensional, where dimensions 1-60 represent coding ability, dimensions 61-120 represent mathematical ability, dimensions 121-180 represent medical ability, dimensions 181-240 represent general dialogue ability, and dimensions 241-300 represent reasoning ability. The model's ability level or value for each type of ability can correspond to different values ​​in the corresponding dimension range of the initial orthogonal vector. Based on this, assuming the model's textual description indicates strong coding ability, moderate mathematical ability, and weak other abilities, then the dimension range representing coding ability (dimensions 1-60) in the initial orthogonal vector is significantly positively adjusted (or adjusted to a value representing the strong ability level), the dimension range representing mathematical ability is slightly adjusted, and other dimension ranges remain unchanged or are slightly adjusted, thus obtaining an orthogonalized semantic vector that accurately reflects the model's ability distribution.

[0104] Moreover, since different initial orthogonal vectors are orthogonal, and the number of types of abilities that each model is good at is limited, for each model, only a small adjustment is needed in a small number of dimensions of its initial orthogonal vectors based on its ability representation features. This has little impact on the directionality of the initial orthogonal vectors, so that the initial orthogonal vectors of each model after adjustment are still orthogonal, that is, the orthogonalized semantic vectors of each model are orthogonal.

[0105] S305, based on the orthogonalized semantic vector of the model, determines the model representation vector of the model.

[0106] Model representation vectors are used to locate models during model routing for at least one model.

[0107] This step can be referred to in the relevant description of the previous embodiments, and will not be repeated here.

[0108] Understandably, with Figure 2 Similarly, after generating the model representation vectors of each model, if a new model is subsequently deployed, the semantic encoding vector of the new model can be determined based on the text description information of the new model. Based on the orthogonalized semantic vectors of at least one model, the semantic encoding vector of the new model can be orthogonalized. Then, the model representation vector of the new model can be generated based on the orthogonalized semantic vectors of the new model.

[0109] It should be noted that the above Figure 2 and Figure 3 The corresponding implementation example illustrates two ways to construct the orthogonalized semantic vector of the model. In practical applications, semantic encoding vectors of each model can be generated based on the text description information of each model; and initial orthogonal vectors of each model can be generated. Then, an orthogonal transformation matrix is ​​found by solving the optimization problem. The initial orthogonal vector of the model is rotated to a direction that is as aligned as possible with the text encoding vector of the model to obtain the orthogonalized semantic vector of the model.

[0110] Of course, based on the textual description information of each model, orthogonal semantic vectors of the model can also be constructed in other ways without restriction.

[0111] In any of the above embodiments of this application, there are several possibilities for determining the model representation vector based on the orthogonalized semantic vector of the model. The following are some possible examples:

[0112] In the first possible case, the orthogonalized semantic vector of the model is directly determined as the model representation vector of the model.

[0113] In this case, once the orthogonalized semantic vector of the model is determined, the model representation vector of the model can be obtained. The amount of data processing is relatively small and the complexity is low.

[0114] However, the textual description of the model may not accurately depict its true capabilities. Furthermore, to ensure orthogonality between the orthogonalized semantic vectors of different models, there may be a discrepancy between the capability features expressed by the orthogonalized semantic vectors and the true capability features described by the textual description. Therefore, to correct the discrepancy between the model's capability expressed by its orthogonalized semantic vectors and its true capability, in a second possible scenario, this application can further add a bias vector to the orthogonalized semantic vectors of the model, so that the final generated model representation vector can more accurately express the model's capabilities.

[0115] The following is combined Figure 4 The case of determining the model representation vector for the second possible model is explained. For example... Figure 4 This illustration shows another flowchart of the method for determining a model representation provided in this application. The method in this embodiment may include:

[0116] S401, determine textual description information for at least one model.

[0117] Textual descriptions are used to describe the capabilities of the model.

[0118] S402, based on the textual description information of at least one model, construct the orthogonal semantic vectors of each model respectively.

[0119] The orthogonalized semantic vector of the model is used to represent the semantics corresponding to the text description information of the model, and the orthogonalized semantic vectors of different models are orthogonal to each other.

[0120] The above two steps can be referred to in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0121] S403, for each model, based on the candidate model representation vector of the model and the semantic vector of the first problem sample, determine the model's prediction fit for the first problem sample.

[0122] The candidate model representation vector consists of the orthogonalized semantic vector and the initial bias vector corresponding to the model. The candidate model representation vector can be the sum or a weighted sum of the orthogonalized semantic vector and the initial bias vector, where the first weight of the orthogonalized semantic vector and the second weight of the initial bias vector are different. The dimension of the initial bias vector is the same as the dimension of the orthogonalized semantic vector.

[0123] In one example, before step S403, initial bias vectors can be generated for each model. These initial bias vectors can be the same or different for each model; there are no specific restrictions. For example, the initial bias vector of a model can be a vector of all zeros, or a random vector with the same dimension as the orthogonalized semantic vector. Based on this, the orthogonalized semantic vector of the model is added to the initial bias vector to obtain the candidate model representation vector for that model.

[0124] The first question sample is used to optimize the bias vector. For example, the first question sample can be question information used as training samples, such as a question consultation or interactive dialogue.

[0125] In this application, the first problem sample is labeled with the actual fit of different models with the first problem sample. The actual fit of the first problem sample with the model is used to characterize the probability that the model can obtain an accurate answer when it is used to process the first problem sample.

[0126] For example, the actual fitness can be either 0 or 1. A fitness value of 0 indicates that the model is not suitable for handling the first question sample, meaning that the model cannot obtain an accurate answer to the first question sample. An actual fitness value of 1 indicates that the model is suitable for handling the first question sample. The actual fitness can also be a specific value within a set range (such as 0-10 or 0-1), with a higher value indicating a higher probability that the model can obtain an accurate answer to the first question sample.

[0127] The model's predicted fit for the first question sample is predicted based on the candidate model representation vector and the semantic vector of the first question sample. This predictive fit represents the probability that the model can obtain an accurate answer when processing the first question sample. The range and possible outcomes of this predicted fit are similar to those of the actual fit, and will not be elaborated further.

[0128] In one example, the similarity between the candidate representation vector of the model and the semantic vector of the first problem sample can be calculated, and the model's prediction fit for the first problem sample can be determined based on this similarity. For example, the similarity can be used to determine the prediction fit, or the prediction fit can be determined based on the mapping relationship between similarity and fit.

[0129] In another example, the prediction fit of a model for the first problem sample can be determined using a routing model based on the candidate model representation vectors of the model and the semantic vector of the first problem sample. This routing model is used to route models for at least one model; for example, after the routing model outputs the prediction fit based on the semantic vector of the problem information and the model representation vectors of each model, it selects a model suitable for processing the problem information based on the prediction fit of each model.

[0130] In this application, there can be multiple first problem samples. In this case, the prediction fitness of the model to handle each first problem sample can be determined separately. Alternatively, steps S403 to S404 can be executed one first problem sample at a time according to the order of the multiple first problem samples, so as to achieve multiple optimizations of the bias vector.

[0131] S404. Based on the model's actual fit and predicted fit to the first problem sample, adjust the model's initial bias vector to obtain the bias vector for each model.

[0132] In one example, for each model, the degree of deviation of the model's initial bias vector can be determined based on the model's predicted fit and actual fit for the first problem sample. For instance, the smaller the difference between the predicted fit and actual fit for each model, the smaller the deviation of the initial bias vector for each model. If there are multiple first problem samples, the degree of deviation needs to be determined based on the model's predicted fit and actual fit for each first problem sample. Based on this, the initial bias vector can be adjusted according to its degree of deviation. For example, in practical applications, the initial bias vector can be iteratively adjusted multiple times until the accuracy of the model's representation vector meets the requirements.

[0133] In another example, to more accurately and reasonably adjust and determine the initial bias vector, the initial bias vector of the model can be adjusted based on the actual and predicted fit of each model to the first problem sample, combined with the loss function. For example, the loss function could be a mean squared error loss or a cross-entropy loss function.

[0134] In one alternative approach, the loss function includes a first loss term and a second loss term. The first loss term is related to the difference between the model's actual fit and predicted fit for the first problem sample. For example, the first loss term could be the mean squared error between the predicted and actual fits. The second loss term could be an L2 regularization term for the initial bias vector. This second loss term limits the adjustment range of the bias vector (i.e., the initial bias vector) to prevent excessive bias vector shift that could severely affect the directionality of the orthogonalized semantic vector.

[0135] For example, this loss function It can be represented in the following form:

[0136] ;

[0137] in, This is the first loss item.

[0138] in, For the first The predictive fit of each model to the first problem sample; For the first The actual fit of each model to the first problem sample.

[0139] in, For L2 regularization terms, For the first The bias vector of each model (e.g., the initial bias vector to be adjusted or the current bias vector to be adjusted during the iterative adjustment process). This is the L2 regularization coefficient, and its specific value can be set according to actual needs.

[0140] It is understandable that the step of adjusting the initial bias vector based on the loss function can be executed multiple times. For example, the initial bias vector can be adjusted using gradient backpropagation based on the loss function. After each adjustment of the initial bias vector, step S403 can be returned to be executed based on the adjusted initial bias vector until the gradient descent magnitude of the loss function calculated based on the loss function is less than a preset value for a set number of consecutive times, or the number of iterations reaches a set number, etc.

[0141] In the process of iteratively adjusting the initial bias vector, the learning rate strategy of warmup and cosine annealing can be combined to continuously adjust the initial bias vector. The specifics will not be elaborated here.

[0142] Specifically, while ensuring that the adjusted bias vector can correct the gap between the capability features represented by the orthogonalized semantic vector and the model's true capability, this application can reasonably control the adjustment range of the bias vector to reduce its impact on the orthogonality between the orthogonalized semantic vectors of the model. A maximum allowable norm can also be set in this application. Based on this, after adjusting the initial bias vector, norm pruning can be performed on the adjusted initial bias vector based on the set maximum allowable norm to obtain the adjusted bias vector.

[0143] For example, norm clipping of the initial bias vector can be performed as follows: if the L2 magnitude of the adjusted initial bias vector does not exceed the maximum allowable norm, the adjusted initial bias vector remains unchanged; if the L2 magnitude of the adjusted initial bias vector exceeds the maximum allowable norm, the adjusted initial bias vector is scaled proportionally so that the magnitude of the scaled initial bias vector is equal to the maximum allowable norm.

[0144] It is understandable that if the initial bias vector is adjusted through multiple rounds of iterative adjustment, then after each adjustment of the initial bias vector, the adjusted initial bias vector can be subjected to norm clipping based on the maximum allowable norm.

[0145] In one possible implementation, if a routing model is used to route at least one model, the model's predictive fitness for the first problem sample can also be determined using the routing model. Furthermore, to improve the accuracy of the routing model's model matching while ensuring that the model representation vectors of each model more accurately represent the model's capabilities, the weight parameters of the routing model can be adjusted simultaneously with the initial bias vectors of each model, in conjunction with the loss function.

[0146] S405, based on the model's bias vector and orthogonalized semantic vector, determines the model's representation vector.

[0147] In one example, the model representation vector can be the sum of the model's orthogonalized semantic vector and bias vector.

[0148] In another example, the model representation vector is obtained by weighted summation of the orthogonalized semantic vector and the bias vector based on the first weight corresponding to the orthogonalized semantic vector and the second weight corresponding to the bias vector.

[0149] Furthermore, after weighted summing of the orthogonalized semantic vectors and bias vectors of the models to obtain the intermediate representation vectors of the models, the intermediate representation vectors of each model can be normalized to obtain the model representation vectors of each model. For example, the model representation vector of a model... It can be represented as: ;

[0150] in, The orthogonalized semantic vector representing the model; This is the bias vector of the model; The hyperparameter is set to control the correction strength of the bias vector, and its value belongs to (0, 1). It represents normalization.

[0151] In this embodiment, a bias vector is introduced on the basis of the orthogonalized semantic vector of the model. Based on the actual fitness and predicted fitness of each model to the first problem sample, the bias vector of each model is optimized so that the model capability represented by the model representation vector determined by the orthogonalized semantic vector and the bias vector is consistent with the actual capability of the model or makes up for the deviation in the capability information described by the text description information. In this way, the model representation vector can more accurately reflect the true capability of the model.

[0152] It is understandable that after performing model routing for each model using the model representation vector, this application can further fine-tune the bias vector of each model by combining online error problem samples over a period of time. That is, based on the latest collected problem samples, steps S403 and S404 are re-executed to update the bias vector of each model and further update the model representation vector of the model, so as to continuously optimize the model representation vector of each model.

[0153] To facilitate understanding of the scheme in this application, the following is combined with... Figure 5 Let's take one implementation method as an example to illustrate, such as... Figure 5 This illustration shows a flowchart of a method for determining a model representation provided in this application. The method in this embodiment may include:

[0154] S501, determine the textual description information for at least one model.

[0155] The textual description of the model is used to describe the model's capabilities.

[0156] S502 generates semantic encoding vectors for the model based on the textual description information of the model.

[0157] S503, orthogonalize the semantic encoding vector of at least one model to obtain the orthogonalized semantic vector of each model.

[0158] The above steps can be found in the previous introduction, and will not be repeated here.

[0159] For ease of understanding, this embodiment uses one method of generating orthogonalized semantic vectors of the model as an example. Other methods of generating orthogonalized semantic vectors are also applicable to this embodiment.

[0160] S504, determine the initial bias vector for each model respectively.

[0161] For example, the initial bias vector of the model can be a random vector with the same dimension as the orthogonalized semantic vector, or a vector of all zeros.

[0162] S505, for each model, determine the candidate model representation vector based on the orthogonalized semantic vector and the initial bias vector of the model.

[0163] For example, the candidate model representation vector can be obtained by normalizing the weighted sum of the orthogonalized semantic vector of the model and the initial bias vector.

[0164] S506, based on the candidate model representation vector of the model and the semantic vector of the first problem sample, the routing model is used to determine the prediction fit of the model to the first problem sample.

[0165] The routing model is used to perform model routing for at least one model.

[0166] S507, based on the actual and predicted fit of each model to the first problem sample, combined with the loss function, synchronously adjusts the initial bias vector of each model and the weight parameters of the routing model.

[0167] In one example, the backpropagation algorithm, combined with the loss function, is used to determine the initial bias vectors of each model and the weight coefficients of the routing model. For instance, in each iteration, based on the actual and predicted fit of each model to the first problem sample, the gradient corresponding to the loss function is determined. The value of the gradient is used to determine the adjustment direction and step size of the initial bias vectors of each model and the weight parameters of the routing model, and the initial bias vectors of each model and the weight parameters of the routing model are adjusted accordingly. The specific process is not restricted.

[0168] S508, check whether the set learning termination condition has been met. If yes, determine the adjusted initial bias vector corresponding to the model as the model's bias vector and execute step S509; if no, return to execute step S505 based on the adjusted initial bias vector and the routing model.

[0169] The learning termination condition can be set as needed. For example, the learning termination condition may include, but is not limited to, the number of iterations reaching a set number, the function value of the loss function converging, or the gradient of the loss function converging, etc. There are no specific restrictions.

[0170] S509, based on the model's bias vector and orthogonalized semantic vector, determines the model's representation vector.

[0171] This step can be found in the previous related introduction, and will not be repeated here.

[0172] Furthermore, after generating the model representation vectors for each model, if a new model is subsequently deployed, this application can maintain the orthogonalized semantic vectors of each model unchanged. Based on this, the semantic encoding vector of the new model, generated from the text description information of the new model, can be orthogonalized using the previously described method to obtain the orthogonalized semantic vector of the new model, ensuring orthogonality between the orthogonalized semantic vectors of each model and between the new models. Then, steps S504 to S508 are re-executed to redetermine the bias vectors of each model and construct the model representation vectors for each model. This maintains the orthogonalized vectors of each model as the basis vector, while only updating the bias vectors of each model. This reduces the complexity of determining the model representation vectors when a new model is added to the model cluster and requires relatively fewer resources.

[0173] On the other hand, this application also provides a model routing method.

[0174] like Figure 6 This illustration shows a schematic diagram of an implementation process of the model routing method provided in this application. This embodiment can be applied to an electronic device that can route at least one model in a model cluster. For example, the electronic device can be a business node in a platform mentioned above that has at least one model deployed.

[0175] The method in this embodiment may include:

[0176] S601, in response to obtaining problem information, determines the semantic vector of the problem information.

[0177] For example, semantic encoding of the problem information can be performed to obtain the semantic vector of the problem information.

[0178] In this application, depending on the application scenario, the question information can have various possibilities. For example, the question information can be a request for translation of content, a question for consultation, or a dialogue in an interactive conversation, etc., without any specific restrictions.

[0179] S602, Based on the semantic vector and the model representation vector of each of the at least one model, determine the target model from the at least one model.

[0180] The model representation vector is determined based on the orthogonalized semantic vector of the model, which is constructed based on the textual description information of the model and used to represent the model. The textual description information of the model is used to describe the model's capabilities. The orthogonalized semantic vectors of different models are orthogonal to each other.

[0181] For details on the specific implementation of determining the model representation vector, please refer to the previous description of the relevant embodiments of the method for determining the model representation, which will not be repeated here.

[0182] There are multiple possibilities for determining the target model, and no specific restrictions are imposed.

[0183] In one example, the similarity between the semantic vector and the model representation vector of each model can be calculated separately, and the model that meets the similarity requirements (such as the highest similarity or the similarity exceeding a set threshold) can be identified as the target model.

[0184] In another example, this application uses a routing model to determine a target model from at least one model based on the semantic vector and the model representation vectors of at least one model. For example, the semantic vector and the model representation vectors of at least one model are input into the routing model to obtain the fitness of each model determined by the routing model for the problem information, and the model with the highest fitness or whose fitness exceeds a set threshold is determined as the target model.

[0185] The routing model can be trained during the learning process of the bias vector of the model, as described in the previous embodiments; or it can be trained using problem samples alone, without any specific restrictions.

[0186] S603 uses the semantic vector of the question information to infer the answer information of the question information using the target model.

[0187] There are no restrictions on the specific implementation process of inferring the corresponding answer information from the semantic vector of the target model's processing of question information.

[0188] In this embodiment, the model representation vectors of each model are determined based on orthogonalized semantic vectors, ensuring that the model representation vectors of different models are independent and do not interfere with each other in space, possessing discriminability and accurately reflecting the capabilities of each model. Based on this, when question information is obtained, matching the semantic vector of the question information with the model representation vectors allows for precise location of the question to the model with the best matching capabilities. This effectively avoids routing mismatches caused by overlapping model representations or semantic discrepancies, thereby improving the accuracy of model routing and ultimately enhancing the accuracy of the inference of the answer information corresponding to the question information.

[0189] To facilitate a more intuitive understanding of the benefits of the proposed solution, after determining the model representation vectors of each model in the model cluster using the proposed solution, a test dataset was used to test multiple models with different capabilities in different domains within the model cluster.

[0190] The test model cluster includes 12 models, which cover four domains: general dialogue, code, medical, and mathematics, with clear distinctions in ability gradients.

[0191] The selected test dataset includes: 10,000 user requests, 3,000 simple general questions and answers, 4,000 medium-level domain questions, and 3,000 complex professional questions.

[0192] Evaluation metrics include: route matching accuracy, mean of pairwise inner products of vectors, average routing latency per request, and NDCG@3. NDCG@3 refers to the Normalized Discounted Cumulative Gain (NDCG) with a truncation position of 3.

[0193] During the testing process, several methods for constructing model representation vectors were also tested. The final test results are shown in the table below:

[0194]

[0195] Among them, the mean of the vector inner product reflects the degree of overlap of the vectors represented by the model, and the lower the value, the better the discrimination.

[0196] Based on the above results, this solution has the following advantages:

[0197] This scheme can significantly improve the matching accuracy of model routing. Specifically, compared with control scheme 1, the matching accuracy of this scheme is improved by 8.3%; compared with control scheme 2, the matching accuracy of this scheme is improved by 6.6%; and compared with control scheme 3, the matching accuracy of this scheme is improved by 2.4%.

[0198] Furthermore, the overlap between model representation vectors in this solution is significantly reduced. The routing latency of this solution is basically the same as existing solutions, supporting high-concurrency business scenarios.

[0199] In addition, the proposed solution can also fine-tune the bias vectors of each model by combining online error samples over a period of time and adjust the vector representation of the model, which can further improve the routing accuracy to 96.2%. Existing control schemes cannot optimize the representation effect through lightweight iteration.

[0200] This application also provides an electronic device in its embodiments. For example... Figure 7 As shown, it illustrates a schematic diagram of the composition structure of the electronic device, which includes at least a processor 701 and a memory 702;

[0201] The processor 701 is configured to determine textual description information for at least one model, the textual description information being used to describe the capability features of the model; based on the textual description information of the at least one model, constructing orthogonalized semantic vectors for each model, the orthogonalized semantic vectors of the model being used to characterize the semantics corresponding to the textual description information of the model, and the orthogonalized semantic vectors of different models being orthogonal to each other; and based on the orthogonalized semantic vectors of the model, determining a model representation vector for the model, the model representation vector being used to locate the model during model routing for the at least one model.

[0202] The memory 702 is used to store the programs required for the processor to perform operations.

[0203] Of course, the processor can also be used to execute the model routing method described in the above embodiments, which will not be elaborated further.

[0204] It is understood that the electronic device may also include a display unit 703 and an input unit 704.

[0205] Of course, the electronic device can also have more than Figure 7 There are no restrictions on the number of components, whether more or fewer.

[0206] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the methods for determining model representation or model routing provided in this application.

[0207] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the methods for determining model representation or model routing provided in this application.

[0208] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines.

[0209] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence or the part that has been contributed, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0210] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0211] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for determining a model representation, comprising: Determine textual description information for at least one model, the textual description information being used to describe the capability characteristics of the model; Based on the text description information of at least one model, orthogonalized semantic vectors of each model are constructed respectively. The orthogonalized semantic vectors of the models are used to represent the semantics corresponding to the text description information of the models, and the orthogonalized semantic vectors of different models are orthogonal to each other. Based on the orthogonalized semantic vector of the model, the model representation vector of the model is determined, and the model representation vector is used to locate the model during the model routing process for the at least one model.

2. The method for determining model representation according to claim 1, wherein constructing orthogonalized semantic vectors for each of the at least one model based on its textual description information comprises: Based on the textual description information of the model, a semantic encoding vector of the model is generated; The semantic encoding vectors of at least one model are orthogonalized to obtain orthogonalized semantic vectors for each model.

3. The method for determining model representation according to claim 1, wherein constructing orthogonalized semantic vectors for each of the at least one model based on its textual description information comprises: Construct initial orthogonal vectors for each model, ensuring that the initial orthogonal vectors of different models are orthogonal to each other; Based on the textual description information of the model, the capability representation features of the model are determined, and the capability representation features are used to characterize the capability magnitude of the model in multiple types of capabilities; Based on the capability representation features of the model and the capability types represented by different dimension intervals in the initial orthogonal vector, the initial orthogonal vector of the model is adjusted to obtain the orthogonalized semantic vector of the model. The orthogonalized semantic vector can represent the capability magnitude of the model in different types of capabilities.

4. The method for determining a model representation according to claim 1, wherein determining the model representation vector based on the orthogonalized semantic vector of the model includes at least one of the following: The orthogonalized semantic vector of the model is determined as the model representation vector of the model; Based on the candidate model representation vector of the model and the semantic vector of the first problem sample, the prediction fitness of the model for the first problem sample is determined. The candidate model representation vector is composed of the orthogonalized semantic vector and the initial bias vector corresponding to the model. Based on the actual fitness of the model for the first problem sample and the prediction fitness, the initial bias vector of the model is adjusted to obtain the bias vector of the model. Based on the bias vector of the model and the orthogonalized semantic vector, the model representation vector of the model is determined.

5. The method for determining a model representation according to claim 4, wherein adjusting the initial bias vector of the model based on the actual fit of the model to the first problem sample and the predicted fit includes: Based on the actual fit of each model to the first problem sample and the predicted fit, the initial bias vector of the model is adjusted in conjunction with the loss function, which includes a first loss term and a second loss term. Wherein, the first loss term is related to the difference between the actual fit of the model to the first problem sample and the predicted fit; The second loss term is the L2 regularization term of the initial bias vector.

6. The method for determining a model representation according to claim 4 or 5, further comprising: After obtaining the adjusted initial bias vector, norm clipping is performed on the adjusted initial bias vector based on the set maximum allowable norm, and the bias vector obtained by norm clipping is determined as the adjusted bias vector corresponding to the model.

7. The method for determining a model representation according to claim 5, wherein determining the prediction fit of the model to the first problem sample based on the candidate model representation vector of the model and the semantic vector of the first problem sample includes: Based on the semantic vector of the first problem sample and the candidate model representation vector of the model, the prediction fit of the model for the first problem sample is determined using the routing model. The step of adjusting the initial bias vector of the model based on the actual fit and predicted fit of each model to the first problem sample, combined with the loss function, includes: Based on the actual fit and predicted fit of each model to the first problem sample, and in conjunction with the loss function, the initial bias vector of the model and the weight parameters of the routing model are adjusted synchronously, wherein the routing model is used to perform model routing for the at least one model.

8. The method for determining a model representation according to claim 2, further comprising: In response to the existence of a newly deployed model, the semantic encoding vector of the new model is determined based on the text description information of the new model; Based on the orthogonalized semantic vector of the at least one model, the semantic encoding vector of the newly added model is orthogonalized to obtain the orthogonalized semantic vector of the newly added model. Based on the orthogonalized semantic vector of the newly added model, the model representation vector of the newly added model is determined.

9. A model routing method, comprising: In response to obtaining problem information, determine the semantic vector of the problem information; Based on the semantic vector and the model representation vector of each of the at least one model, a target model is determined from the at least one model; Based on the semantic vector of the question information, the answer information of the question information is inferred using the target model; Wherein, the model representation vector of the model is determined based on the orthogonalized semantic vector of the model, and the orthogonalized semantic vector of the model is constructed based on the textual description information of the model, and is used to represent the vector of the model; the textual description information is used to describe the capability features of the model; The orthogonalized semantic vectors of different models are orthogonal.

10. An electronic device comprising: Processor and memory; The processor is configured to: determine textual description information for at least one model, wherein the textual description information describes the capability features of the model; construct orthogonalized semantic vectors for each model based on the textual description information of the at least one model, wherein the orthogonalized semantic vectors of the models represent the semantics corresponding to the textual description information of the models, and the orthogonalized semantic vectors of different models are orthogonal to each other; and determine model representation vectors for the models based on the orthogonalized semantic vectors of the models, wherein the model representation vectors are used to locate the models during model routing for the at least one model. The memory is used to store the computer programs required by the processor to perform operations.

Images