Electronic apparatus for selecting an optimal model based on a query and routing the query accordingly

Abstract

An electronic device for selecting and routing an optimal model according to a query according to the present disclosure comprises: a storage memory in which a plurality of candidate language models are stored, which are trained to generate response data corresponding to query data; and a processor connected to the storage memory; wherein the processor can select a response language model to generate response data for a target query data from among the plurality of candidate language models based on response evaluation data for a plurality of reference response data generated from each of the plurality of candidate language models corresponding to each of the plurality of reference query data.

Description

Technology Field

[0001] The present disclosure relates to an electronic device for selecting and routing an optimal model according to a query, and more specifically, to an electronic device capable of providing a high-quality response to a target query by selecting a response language model to generate a response to a target query from among a plurality of candidate language models based on a quality evaluation of a reference response to a reference query. Background Technology

[0003] In recent years, artificial intelligence technology, particularly in the field of Natural Language Processing (NLP), has seen rapid advancements following the emergence of Transformer-based architectures. Early representative Large Language Models (LLMs), such as GPT-2 and BERT, demonstrated superior performance across various NLP benchmarks by employing a method of pre-training on large-scale general-purpose text data followed by fine-tuning for specific tasks.

[0004] Subsequently, as the scale of language model parameters and the amount of training data were increased exponentially, reinforcement learning techniques (such as Reinforcement Learning with Human Feedback and Direct Preference Optimization) that incorporate human feedback into learning were applied, leading to a dramatic improvement in model inference capabilities and response quality. Recently, research on multimodal language models that process various input forms—including text, images, voice, and video—as well as Retrieval-Augmented Generation (RAG) technology, which searches for external information in real-time and incorporates it into responses, is also being actively pursued.

[0005] In addition, with the expansion of LLM practical applications, there is a significant increase in demand to reduce inference latency and server operating costs. To address this, various model compression techniques, such as model quantization, pruning, and efficient attention mechanisms, are being proposed.

[0006] Meanwhile, research is also being conducted on approaches to improve response quality by utilizing multiple language models simultaneously, moving away from relying on a single LLM. For example, the 'Mixture of Experts' method involves multiple different language models each generating a response for a single input query, and then combining these responses to produce a final response. However, this method has limitations in scalability and practicality in actual service environments because GPU memory usage increases rapidly and response latency also increases, as each model must generate a response for the same query.

[0007] Against this backdrop, a method that selects one of multiple language models based on the difficulty or nature of the query and processes the query using only that model has recently been gaining attention. An example of this is the so-called Hybrid LLM method, which proposes an approach where easy queries are sent to small models and complex queries to large models. However, this method requires prior pairing of small and large models, and the performance degradation relative to all-at-large performance can be more than 10%, indicating that there is still much room for improvement. Prior art literature

[0009] Published Patent Application No. 10-2022-0164559 The problem to be solved

[0010] The present disclosure aims to solve all the problems of the aforementioned prior art.

[0011] In addition, the present disclosure aims to provide an electronic device capable of providing a high-quality response to a target query by selecting a response language model to generate a response to a target query from among a plurality of candidate language models based on a quality evaluation of a reference response to a reference query.

[0012] The purposes of the present disclosure are not limited to those mentioned above, and other purposes and advantages of the present disclosure not mentioned may be understood from the following description and will be more clearly understood from the embodiments of the present disclosure. Furthermore, it will be readily apparent that the purposes and advantages of the present disclosure can be realized by the means and combinations thereof set forth in the claims. means of solving the problem

[0014] An electronic device for selecting and routing an optimal model according to a query according to the present disclosure comprises: a storage memory in which a plurality of candidate language models are stored, which are trained to generate response data corresponding to query data; and a processor connected to the storage memory; wherein the processor can select a response language model to generate response data for a target query data from among the plurality of candidate language models based on response evaluation data for a plurality of reference response data generated from each of the plurality of candidate language models corresponding to each of the plurality of reference query data.

[0015] The processor can input a plurality of reference query data into each of a plurality of candidate language models and receive a plurality of reference response data corresponding to each of a plurality of reference query data from each of a plurality of candidate language models.

[0016] The processor receives the response evaluation data from a response evaluation model trained to evaluate the quality of response data corresponding to the query data, and can match the response evaluation data for each of the multiple candidate language models for each of the multiple reference query data.

[0017] The processor can calculate the similarity with the target query data for each of the plurality of reference query data and select a plurality of similar query data among the plurality of reference query data based on the similarity.

[0018] The processor can check the response evaluation data for each of the plurality of candidate language models matched to each of the plurality of similar query data, and for each of the plurality of similar query data, select the candidate language model in which the response evaluation data satisfies the selection conditions as a provisional language model.

[0019] The processor can select the most frequently selected preliminary language model for each of the plurality of similar query data as the response language model.

[0020] If there are multiple preliminary language models, the processor may select one of the multiple preliminary language models as the response language model based on one or more of the model capacity and usage fee of each of the preliminary language models. Effects of the invention

[0022] According to the present disclosure, by selecting a response language model to generate a response to a target query from among a plurality of candidate language models based on a quality evaluation of a reference response to a reference query, a high-quality response to a target query can be provided. Brief explanation of the drawing

[0024] FIG. 1 is a block diagram conceptually illustrating the components of an electronic device according to one embodiment of the present disclosure. FIG. 2 is a drawing illustrating a storage memory structure of an electronic device according to one embodiment of the present disclosure. FIG. 3 is a block diagram of a processor and a main processor of an electronic device according to one embodiment of the present disclosure. FIG. 4 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure inputting a plurality of reference query data into each of a plurality of candidate language models and receiving reference response data from each of a plurality of candidate language models. FIG. 5 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure receiving response evaluation data from a response evaluation model. FIG. 6 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure selecting similar query data based on the similarity between target query data and a plurality of reference query data. FIG. 7 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure selecting a response language model among a plurality of candidate language models. Specific details for implementing the invention

[0025] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0026] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0027] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0028] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0029] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0030] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0031] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0032] Where it is stated that a component (e.g., Component 1) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., Component 2), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., Component 3).

[0033] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between the certain component and the other component.

[0034] The expression “configured to” as used in this disclosure may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0035] In the embodiment, the 'device' performs at least one function or operation and may be implemented in hardware or software, or in a combination of hardware and software.

[0036] Meanwhile, the various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0037] Hereinafter, embodiments according to the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0038] FIG. 1 is a block diagram conceptually illustrating the components of an electronic device according to one embodiment of the present disclosure, FIG. 2 is a diagram illustrating the storage memory structure of an electronic device according to one embodiment of the present disclosure, and FIG. 3 is a block diagram of a processor and a main processor of an electronic device according to one embodiment of the present disclosure.

[0039] Referring to FIGS. 1 to 3, an electronic device (100) according to one embodiment of the present disclosure may be a device capable of performing an operation to obtain a calculation result by performing a calculation operation based on acquired data.

[0040] The electronic device (100) may be, for example, a computing device, a computational device, a desktop PC, etc.

[0041] Additionally, the electronic device (100) may be a portable computing device and may include at least one of a smartphone, a tablet PC, a laptop PC, a netbook computer, a mobile device, and a wearable device, but is not limited thereto.

[0042] In addition, the electronic device (100) may be a server device, for example, a computer that provides services to clients via a network. The server may be an FTP server, a web server, a database server, or a cloud server, and the server may be built with an operating system such as Linux.

[0043] The electronic device (100) described above is not necessarily implemented as a single device, but may be implemented as multiple different devices that each perform their respective functions and operations.

[0044] The electronic device (100) according to one embodiment of the present disclosure may include a storage memory (110) and a processor (120).

[0045] The storage memory (110) stores various programs or data temporarily or non-temporarily and transmits the stored information to the processor (120) upon the call of the processor (120). Additionally, the storage memory (110) can store various information required for the operation, processing, or control operation of the processor (120) in an electronic format.

[0046] The storage memory (110) may be implemented using a semiconductor storage medium such as ROM and / or RAM. The ROM may include, for example, a conventional ROM, EPROM, EEPROM and / or MASK-ROM. The RAM may include, for example, a DRAM and / or SRAM. The storage memory (110) may be implemented using at least one storage medium capable of storing data permanently or semi-permanently, such as a flash memory device, an SD (Secure Digital) card, a solid state drive (SSD), a hard disk drive (HDD), an optical recording medium such as a magnetic drum, a compact disc (CD), a DVD, or a laser disc, a magnetic tape, a magneto-optical disc and / or a floppy disk.

[0047] For example, the storage memory (110) can store candidate language models (ML) and algorithms.

[0048] The processor (120) controls the overall operation of the electronic device (100). Specifically, the processor (120) is connected to the configuration of the electronic device (100) including a storage memory (110) as described above, and can control the overall operation of the electronic device (100) by executing at least one instruction stored in the storage memory (110) as described above. In particular, the processor (120) can be implemented as a single processor as well as as a plurality of processors.

[0049] The processor (120) may be implemented in various ways. For example, one or more processors may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. One or more processors may control one or any combination of other components of an electronic device and may perform operations or data processing related to communication. One or more processors may execute one or more programs or instructions stored in memory. For example, one or more processors may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory.

[0050] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0051] One or more processors may be implemented as a single-core processor comprising one core, or as one or more multicore processors comprising multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory such as on-chip memory, and a common cache shared by multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0052] For example, the processor (120) may be a GPU (Graphics Processing Unit) equipped with high-speed computation memory (M1).

[0053] When a method according to one embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one of the plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in a multi-core processor, or the first operation and the second operation may be performed by a first core included in a multi-core processor and the third operation may be performed by a second core included in a multi-core processor.

[0054] In embodiments of the present disclosure, the processor (120) may mean a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator or machine learning accelerator, etc., but the embodiments of the present disclosure are not limited thereto.

[0055] The storage memory (110) can store an artificial intelligence model, output data obtained based on input data input to the artificial intelligence model, usage information obtained based on the output data, and reconstruction information for reconstructing the artificial intelligence model obtained based on the usage information.

[0056] The processor (20) can perform operations using the output value of the artificial intelligence model.

[0057] Artificial intelligence models can be constructed by considering the application field of the recognition model, the purpose of learning, or the computing performance of the device. Additionally, the artificial intelligence model may be, for example, a language model based on Transformers.

[0058] An artificial intelligence model can be designed to simulate the structure of the human brain on a computer. The artificial intelligence model may include multiple network nodes with weights that simulate neurons of a human neural network. The multiple network nodes may each form a connection relationship to simulate synaptic activity, in which neurons transmit and receive signals through synapses.

[0059] Machine learning models may include, for example, neural network models or deep learning models developed from neural network models. In deep learning models, multiple network nodes may be located at different depths (or layers) and exchange data according to convolutional connection relationships. For example, models such as Deep Neural Networks (DNN), Recurrent Neural Networks (RNN), and Bidirectional Recurrent Deep Neural Networks (BRDNN) may be used as artificial intelligence models, but are not limited thereto.

[0060] A deep learning model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of these, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0061] A deep learning model may include a learning algorithm. For example, it may include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples mentioned above. An artificial intelligence model may include a plurality of artificial neural network layers.

[0062] A language model is a model trained to output the statistically most appropriate output value (e.g., a vector value corresponding to the answer to the input text) based on input values ​​(e.g., text such as natural language, sentences, and words).

[0063] The language model can be a large language model, such as GPT, Phi, BERT, LaMDA, LLaMA, RoBERTa, etc., but is not limited to these.

[0064] Based on large language models, Natural Language Processing (NLP), which recognizes and interprets input natural language, and Natural Language Generation (NLG), which generates, enhances, and reconstructs words and sentences, can be performed.

[0065] Natural Language Processing (NLP) is a method that computer programs can use to interpret human language.

[0066] Natural language processing models can perform information retrieval operations to search for and identify target data contained in the data, information extraction operations to extract target data contained in the data, operations to classify sentences, documents, etc., operations to classify words, syntactic analysis, sentiment analysis, and operations to determine semantic roles, and are also used in speech recognition models to implement speech recognition and automatic dialogue systems.

[0067] More specifically, large language models can be used for machine translation, text summarization, question-answering systems, dialogue systems, content generation, etc.

[0068] Large language models convert input text data into vector values ​​in a latent space through text embeddings, identify the context of words and phrases with similar meanings as well as other relationships between words, such as parts of speech, based on the vector values ​​in the latent space, and output results for natural language processing, natural language generation, etc., based on the relationships between these vector values.

[0069] Meanwhile, the processor (120) may be a GPU that performs high-speed calculations, and the electronic device (100) may further include a main processor (CPU) which is a CPU.

[0070] The processor (120) is directly connected to the high-speed computation memory (M1) to perform high-speed computation processing, and the main processor (CPU) is connected to the main memory (M2) to perform data storage and offloading processing.

[0071] FIG. 4 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure inputting a plurality of reference query data into each of a plurality of candidate language models and receiving reference response data from each of a plurality of candidate language models; FIG. 5 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure receiving response evaluation data from a response evaluation model; FIG. 6 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure selecting similar query data based on the similarity between target query data and a plurality of reference query data; and FIG. 7 is a diagram illustrating the process of an electronic device according to one embodiment of the present disclosure selecting a response language model among a plurality of candidate language models.

[0072] Referring further to FIGS. 4 to 7, the processor (120) can select a response language model to generate response data for target query data (TQ) among a plurality of candidate language models (DLM), based on response evaluation data (RE) for a plurality of reference response data (RR) generated from each of a plurality of candidate language models (DLM) corresponding to a plurality of reference query data (RQ).

[0073] Specifically, the processor (120) can input a plurality of reference query data (RQ) into each of a plurality of candidate language models (DLM) and receive a plurality of reference response data (RR) corresponding to each of the plurality of reference query data (RQ) from each of the plurality of candidate language models (DLM).

[0074] Here, the candidate language models (DLMs) may be multiple natural language generation models with different structures, sizes, training data, or licensing conditions.

[0075] More specifically, candidate language models (DLMs) are based on different architectures such as GPT-family, BERT-family, LLaMA-family, or other Transformer-based language models, and may consist of models with various parameter sizes such as 7B, 13B, 30B, 65B, and may include open-source models or commercial API-based models.

[0076] Here, multiple reference query data (RQ) may be multiple predefined query statements collected across various domains (e.g., law, medicine, finance, general knowledge, etc.).

[0077] More specifically, reference query data (RQ) may consist of query sentences spanning various topics or domains, such as everyday conversations, technical queries, coding problems, medical questions, legal consultations, and mathematical problems; each query may be expressed in natural language and may be data collected and refined manually or automatically to ensure meaningful representativeness for comparing model response performances.

[0078] For example, if there are 100 candidate language models (DLM) and 500 multiple reference query data (RQ), the processor (120) inputs 500 reference query data (RQ) to each of the 100 candidate language models (DLM) and outputs 500 reference response data (RR) from each of the 100 candidate language models (DLM), thereby outputting a total of 50,000 reference response data (RR).

[0079] Afterward, the processor (120) can receive response evaluation data (RE) from a response evaluation model (EM) trained to evaluate the quality of response data corresponding to the query data.

[0080] Specifically, the processor (120) inputs reference query data (RQ) and reference response data (RR) generated from the reference query data (RQ) into a response evaluation model (EM), and receives response evaluation data (RE) from the response evaluation model (EM).

[0081] In this case, the response evaluation model (EM) is a pre-trained high-performance language model that may be specialized for natural language understanding and evaluation tasks.

[0082] More specifically, the response evaluation model (EM) is based on high-performance commercial or open-source models such as GPT-4, ChatGPT, Claude, and Prometheus, and may be an evaluation model that has been fine-tuned according to separate evaluation guidelines or configured through prompt engineering to quantitatively determine the suitability, accuracy, and completeness between a given query and an answer.

[0083] Here, the response evaluation data (RE) may be data that quantifies the quality of the response output by the candidate language model (DLM).

[0084] More specifically, the response evaluation data (RE) consists of scores ranging from 1 to 5 or from 0 to 5, and may be a quantitative quality indicator where a higher score indicates superior accuracy, consistency, clarity, appropriateness, etc., of the response to the query. Additionally, these scores may be calculated by averaging multiple evaluation results from a single evaluation model or multiple evaluation models, or by a majority vote method.

[0085] In addition, the response evaluation model (EM) can generate and output response evaluation data (RE) based on the following evaluation instructions.

[0087] <Evaluation Instructions>

[0088] You are an evaluator assessing the quality of the response to the given instructions.

[0089] The evaluation score ranges from 1 to 5 points. Please evaluate each response accurately and fairly by referring to the detailed criteria below.

[0090] 0 points - Very poor (Poor): The response does not match the question at all, contains significant errors, or includes mostly irrelevant content.

[0091] 1 point - Insufficient (Fair): The response is somewhat relevant to the question, but it is missing key information or contains significant inaccuracies, and lacks clarity or depth.

[0092] 2 points - Adequate: The response adequately answers the question, but may lack detail or contain minor errors. It may be unclear or partially irrelevant.

[0093] 3 points - Good: Responses are mostly accurate and relevant, and answer questions with sufficient clarity and detail. Some errors or omissions may occur.

[0094] 4 points - Very Good: The response is accurate, highly relevant, and clearly expressed. It answers the question in detail with few errors. It demonstrates a good understanding of the topic.

[0095] 5 points - Excellent: The response perfectly matches the question and contains comprehensive details and insights. It is completely accurate, highly relevant, and clearly written, demonstrating an excellent understanding of the topic.

[0097] Afterwards, the processor (120) can match response evaluation data (RE) for each of the multiple candidate language models (DLM) for each of the multiple reference query data (RQ).

[0098] Next, the processor (120) calculates the similarity with the target query data (TQ) for each of the multiple reference query data (RQ) and can select multiple similar query data (SQ) from among the multiple reference query data (RQ) based on the similarity.

[0099] Here, the target query data (TQ) is a query entered by a user or a query automatically generated by a system, and refers to the target query for which the electronic device (100) intends to generate response data through a finally selected response language model.

[0100] Target query data (TQ) can be in the form of natural language sentences and may include various forms such as questions, commands, and requests regarding a specific domain (e.g., law, medicine, finance, general knowledge, etc.). Additionally, target query data (TQ) may be a single sentence or a query paragraph consisting of multiple sentences.

[0101] To this end, the processor (120) can convert reference query data (RQ) and target query data (TQ) into vector-shaped embedding data using the same sentence embedding model. The embedding model used may be, for example, a pre-trained Sentence-BERT, MiniLM, or other high-performance natural language sentence embedding generation model.

[0102] The processor (120) can calculate the cosine similarity between the embedding vector of the target query data (TQ) and the embedding vector of each reference query data (RQ) using the converted embedding data. Here, the cosine similarity is a numerical value of the directional similarity between two vectors, and the closer the value is to 1, the higher the semantic similarity.

[0103] The processor (120) may select a predetermined number (K) of reference query data (RQ) having the highest similarity to the target query data (TQ) as similar query data (SQ) based on the calculated cosine similarity. At this time, K may be determined according to system settings or empirical criteria.

[0104] Afterwards, the processor (120) checks the response evaluation data (RE) for each of the multiple candidate language models (DLM) matched to each of the multiple similar query data (SQ), and for each of the multiple similar query data (SQ), the candidate language model (DLM) whose response evaluation data (RE) satisfies the selection conditions can be selected as the provisional language model.

[0105] Here, the selection condition is a condition that satisfies the quantitative criteria of the response evaluation data (RE), for example, when the response evaluation score is above a predetermined threshold.

[0106] More specifically, the processor (120) compares the corresponding response evaluation scores for each similar query data (SQ) with those of multiple candidate language models (DLMs), and determines the candidate language model (DLM) as a provisional language model only if the score is greater than or equal to a preset threshold score (e.g., 4.0 points or higher or 80 points or higher).

[0107] In addition, selection criteria may include not only absolute criteria but also relative criteria, and for example, candidate language models (DLMs) that received the top N (e.g., top 3) evaluation scores for the corresponding similar query may be designated as provisional language models.

[0108] Afterwards, the processor (120) can select the most frequently selected preliminary language model for each of the multiple similar query data (SQ) as the response language model.

[0109] At this time, if there are multiple preliminary language models, the processor (120) can select one of the preliminary language models as the response language model based on one or more of the model capacity and usage fee of each preliminary language model.

[0110] Specifically, if there are multiple preliminary language models, the processor (120) may select the preliminary language model with the smallest model capacity as the response language model, or select the preliminary language model with free usage fees or the lowest usage fees as the response language model.

[0111] Here, model capacity can refer to the total number of parameters, memory usage, or computational amount of the candidate language model, and can be classified according to the number of parameters the model possesses, such as 7B, 13B, 30B, 65B, etc. Additionally, model capacity can serve as a key indicator for determining computational performance, response latency, and resource consumption among models with the same architecture.

[0112] Here, the usage fee is a cost incurred in the process of generating response data for a query using a response language model, and may be a usage fee charged per API call unit, a usage fee in the form of a subscription for a certain period, or a cost based on a billing system calculated in proportion to the number of processed tokens. Additionally, the usage fee may be 0 KRW if the model provider offers it for free, and may be determined differently depending on the model's performance, capacity, provision policy, etc.

[0113] Accordingly, the usage fee for the pre-selected language model may be free if the pre-selected language model is provided for free; a one-time usage fee charged for each use if the pre-selected language model has a billing structure where charges are incurred each time it is used; a subscription usage fee charged for subscription use if the pre-selected language model has a subscription billing structure and is currently unsubscribed; or 0 won since there is no additional fee charged because the pre-selected language model has a subscription billing structure and is currently subscribed.

[0114] Meanwhile, in another embodiment, the processor (120) can select the pre-selected language model as the response language model in priority if the usage fee of the pre-selected language model is 0 won because there is no additional fee charged since it is currently subscribed.

[0115] For example, among multiple preliminary language models having the same minimum model capacity, if the remaining preliminary language models excluding one preliminary language model are provided for free and have no usage fees, and one preliminary language model has a subscription billing structure and is currently subscribed so that there is no additional fee charged, then the preliminary language model having a subscription billing structure and is currently subscribed so that there is no additional fee charged can be selected as the response language model.

[0116] Through this, the present invention prevents the occurrence of additional costs and reduces service operating costs by preferentially utilizing the currently subscribed provisional language model, and has the effect of maximizing cost efficiency in the response language model selection process.

[0117] In addition, by prioritizing pre-selected language models that are already subscribed to, changes in configuration or initialization processes caused by model switching can be minimized, thereby enhancing the stability and reliability of response processing.

[0118] Meanwhile, in the case where there are multiple preliminary language models among the preliminary language models that have the lowest model capacity and the lowest usage fee, the processor (120) according to another embodiment may finally select the preliminary language model that has been selected as the response language model many times in the past as the response language model. By assigning priority based on the number of past selections in this way, the system can preferentially utilize a model whose response quality and stability have been verified in the past, and can minimize the deviation in response quality caused by model switching.

[0119] Afterward, the processor (120) can load the response language model and prepare it for use.

[0120] Meanwhile, the processor (120) can pre-load frequently selected candidate language models (DLMs) into the infrastructure in advance to minimize response generation delay in the process of selecting and routing the optimal response language model for the target query in real time.

[0121] Specifically, the processor (120) records the number of times each candidate language model (DLM) is selected as a response language model, selects the candidate language model (DLM) that is included in the top ranking criteria in order of the number of selections as a preloaded model, and can deploy (preload) the preloaded models to the infrastructure in advance.

[0122] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0123] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure. Explanation of the symbols

[0125] 100: Electronic device 110: Storage memory 120: Processor

Claims

Claim 1 An electronic device for selecting and routing an optimal model based on a query, comprising: a storage memory in which a plurality of candidate language models trained to generate response data corresponding to query data are stored; The processor connected to the storage memory; wherein the processor selects a response language model to generate response data for a target query data from among the plurality of candidate language models based on response evaluation data for a plurality of reference response data generated from each of the plurality of candidate language models corresponding to each of the plurality of reference query data, the processor calculates the similarity with the target query data for each of the plurality of reference query data, and selects a predetermined number of reference query data as a plurality of similar query data in order of high similarity among the plurality of reference query data, the processor verifies the response evaluation data for each of the plurality of candidate language models matched to each of the plurality of similar query data, and for each of the plurality of similar query data, selects the candidate language model in which the response evaluation data satisfies the selection condition as a provisional language model, wherein the processor selects the candidate language model that satisfies the selection condition in order of high response evaluation score and has a response evaluation score according to the response evaluation data above a predetermined threshold, and selects the candidate language model that satisfies the selection condition as the provisional language model, and for each of the plurality of similar query data, selects the provisional language model that is selected most frequently as the response language model. An electronic device characterized by, if there are multiple provisional language models selected most frequently, selecting the provisional language model with the lowest model capacity as the response language model or selecting the smallest provisional language model with the lowest usage fee as the response language model, and if there are multiple provisional language models with the lowest model capacity and the lowest usage fee, selecting the provisional language model that has been selected as the response language model many times in the past as the response language model. Claim 2 An electronic device according to claim 1, wherein the processor inputs a plurality of reference query data into each of a plurality of candidate language models and receives a plurality of reference response data corresponding to each of a plurality of reference query data from each of a plurality of candidate language models. Claim 3 An electronic device according to claim 1, wherein the processor receives response evaluation data from a response evaluation model trained to evaluate the quality of response data corresponding to query data, and matches the response evaluation data for each of the plurality of candidate language models for each of the plurality of reference query data. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete

Images