Apparatus and method for generating response using language model

Abstract

This apparatus and method for generating a response may obtain a security document corresponding to a query from a preset database, calculate importance, which is a criterion for determining whether data required through the query is important for security, from a first language model to which at least one of the query or the security document is inputted, determine one of a second language model trained through a plurality of pieces of security information or a third language model trained through a plurality of pieces of public information on the basis of the importance, and generate a response to the query from the determined model.

Description

Response generation device and method using a language model

[0001] The present disclosure relates to a technique for generating a response to a query using a language model.

[0002] Recently, active research has been conducted on language models such as GPT (Generative Pre-trained Transformer). Through these models, it is possible to effectively understand human language and generate responses to input queries.

[0003] One of the language models, LLM, includes Public LLM (Large Language Model), which is learned through general public languages, and sLLM (Smaller Large Language Model), which is learned based only on important information such as company security documents.

[0004] The problem with such language models is that they cannot provide corresponding answers to queries in areas they have not been trained on. In particular, corporate security documents are difficult to expose externally, making it impossible for general language models to learn them. However, companies train their internally developed language models with their security documents to ensure that at least their internal employees can easily access these materials.

[0005] In addition, there is a problem in that it is difficult to find a clear standard for determining whether the data requested through a user's query is critical company information or generally accepted publicly available information.

[0006] The present disclosure aims to provide a technology for generating a response to each query by using language models having different characteristics depending on the importance of the data required through an input query.

[0007] In one aspect, the present embodiments provide a response generating device for generating a response to a query, comprising: obtaining a security document corresponding to a query from a preset database; calculating an importance level that serves as a criterion for determining whether the data required through the query is important in terms of security from a first language model into which at least one of the query and the security document is input; determining one of a second language model learned through a plurality of security information or a third language model learned through a plurality of public information based on the importance level; and generating a response to the query from the determined model.

[0008] In another aspect, the present embodiments provide a method for generating a response to a query, comprising the steps of: calculating an importance level that serves as a criterion for determining the security importance of data required through a query from a first language model into which at least one of a security document corresponding to the query is input, obtained from a pre-set database; determining one of a second language model learned through a plurality of security information or a third language model learned through a plurality of public information based on the importance level; and generating a response to the query from the determined model.

[0009] The present disclosure may provide a technology for generating a response to a query using a language model.

[0010] FIG. 1 is a drawing for explaining the configuration of a device for generating a response to a query according to one embodiment.

[0011] FIG. 2 is a diagram for schematically explaining LLM, which is a type of artificial intelligence according to one embodiment.

[0012] FIG. 3 is a flowchart illustrating the learning process of an artificial intelligence model according to one embodiment.

[0013] FIG. 4 is a flowchart illustrating the process of generating a response to a query input through an LLM according to one embodiment.

[0014] FIG. 5 is a flowchart illustrating the process of calculating the importance of required data through an input query according to one embodiment.

[0015] FIG. 6 is a flowchart illustrating the process of generating a response by inputting a query based on importance according to one embodiment into either sLLM or Public LLM.

[0016] FIG. 7 is a flowchart for schematically explaining the entire process from when a query is input to when a response is generated according to one embodiment.

[0017] FIG. 8 is a flowchart for explaining a method for generating a response to a query according to one embodiment.

[0018] FIG. 9 is a configuration diagram of a computing device including an artificial intelligence model according to one embodiment.

[0019] FIG. 10 is a configuration diagram of a computer system including a client-server that includes an artificial intelligence model according to one embodiment.

[0020] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the embodiments, if it is determined that a detailed description of related known components or functions may obscure the essence of the technical concept, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless otherwise specified.

[0021] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0022] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0023] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0024] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0025] The embodiments are described in detail below with reference to the drawings.

[0026] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the embodiments, if it is determined that a detailed description of related known components or functions may obscure the essence of the technical concept, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless otherwise specified.

[0027] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0028] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0029] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0030] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0031] The embodiments are described in detail below with reference to the drawings.

[0032]

[0033] FIG. 1 is a drawing for explaining the configuration of a device for generating a response to a query according to one embodiment.

[0034] Referring to FIG. 1, the response generating device (100) of the present disclosure includes a security document acquisition unit (110) that acquires a security document corresponding to a query from a preset database.

[0035] The response generating device (100) of the present disclosure can generate a response to a query by inputting the query into either a smaller large language model (sLLM) or a large language model (Pulbic LLM) by considering the importance of the data desired by the user through the input query.

[0036] LLM (Large Language Model) includes sLLM (Smaller Large Language Model), which is trained on information such as personally identifying information (PII) requiring security, financial information, corporate internal policies, and corporate non-public information, depending on the characteristics of the training data, and Pulbic LLM (Large Language Model), which is trained on public information, searchable information, and general question and answer.

[0037] However, it is not easy to distinguish between the aforementioned Personally Identifying Information (PII), financial information, corporate internal policies, corporate non-public information and public information, searchable information, and general Q&A. Therefore, the present disclosure proposes a method of calculating the importance of the required data through an input query and generating a response by inputting the query into either sLLM or Pulbic LLM based on the calculation result.

[0038] For example, the document acquisition unit (110) of the present disclosure can convert a text-type query into a first embedding vector through a preset embedding model and input the first embedding vector into a preset search engine model to acquire a security document with high similarity to the query from a preset database.

[0039] The response generation device (100) of the present disclosure does not immediately input a query entered by a user into the LLM, but can check in advance whether information related to the query is stored in a pre-configured database, and can also input the query into a pre-configured search engine model to retrieve information related to the query. The present disclosure refers to the process of inputting information searched in a pre-configured database into the LLM together with the query before inputting the query into a separate LLM as RAG (Retrieval-Augmented Generation). Through the aforementioned RAG operation, the accuracy of the response output from the LLM can be increased.

[0040] The aforementioned security documents may include documents that are not disclosed, and said security documents may be stored separately in a pre-configured database.

[0041] The response generating device (100) of the present disclosure may perform tagging by setting a tag into a pre-set category based on the content of the input query before querying a security document with high similarity to the query from a pre-set database. For example, the aforementioned pre-set category may include personal information, secret information, and public information. The aforementioned category is merely an example and may be set in various ways as needed.

[0042] As another example, a security level may be set for each of the aforementioned security documents, and the set security level can be used to calculate importance. For instance, if the security level of a security document is high, it may be appropriate to generate a response through sLLM because the accuracy of the response to the query may be lower through Public LLM. Conversely, if the security level of a security document is low, a response can be generated through Public LLM.

[0043] As another example, the aforementioned preset database includes a second embedding vector for each of a plurality of security documents, and the security document corresponding to the query can be determined based on the similarity between the first embedding vector and the second embedding vector.

[0044] As another example, the aforementioned similarity is determined based on a first embedding vector, a second embedding vector, and a preset algorithm, and the preset algorithm may include at least one of a cosine similarity determination algorithm and a Euclidean distance calculation algorithm.

[0045] Embedding is a technique that converts each piece of data into a high-dimensional vector containing numbers while preserving the original meaning of the data. For example, the word 'secondary battery' can be converted into a high-dimensional vector containing numbers such as [0.52, -0.04, 0.16, ... 0.27] through embedding. Furthermore, embedding can convert not only words into vectors but also entire sentences or texts into a single vector. For example, the response generating device of the present disclosure may convert a query entered by a user into a high-dimensional vector containing numbers such as [0.3, -0.04, 0.26, ... 0.45]. The numbers included in the aforementioned high-dimensional vector are merely examples for the convenience of explanation, and the high-dimensional vector can be converted in various ways depending on the embedding model.

[0046] The response generating device (100) of the present disclosure converts an input query into one embedding vector through a preset embedding model and can obtain a secure document through a preset database.

[0047] The security document obtained by the response generating device (100) of the present disclosure does not mean only a document that matches 100% with the query, but also includes a security document that is judged to have a high degree of similarity.

[0048] As described above, the similarity between the query and the security document can be calculated based on at least one of a cosine similarity determination algorithm and a Euclidean distance calculation algorithm.

[0049] The cosine similarity algorithm is a technique that measures the degree of similarity between two data sets using a cosine function by comparing the directionality between two vectors. Similarity can be calculated by Equation 1, where θ is the angle, X and Y are the two vectors to be compared, X o Y is the dot product of the two vectors, and |X| and |Y| are the magnitudes of each vector.

[0050]

[0051] For example, if X is a vector of [2, 1] and Y is a vector of [1, 2], X o Y is 2*1 + 1*2, which is 4, and |X| and |Y| are each √5. Therefore, the first similarity, Cosθ, can be calculated as 0.8.

[0052] Since Cosθ, the first similarity, has only values ​​between -1 and 1, the similarity also has only values ​​between -1 and 1, and it can be determined that the closer Cosθ is to 1, the higher the similarity between the two articles.

[0053] In addition, the aforementioned Euclidean distance calculation algorithm calculates the distance between two points, and the smaller the calculated distance, the higher the first similarity can be determined.

[0054] The Euclidean distance can be calculated based on Equation 2, where d(p,q) is the distance between two points p and q, x1 and y1 are the coordinates of p, and x2 and y2 are the coordinates of q.

[0055]

[0056] For example, assuming p is a vector of [2, 1] and q is [1, 2], we can say that in p, x1 is 2 and y1 is 1, and in q, x2 is 1 and y2 is 2. If we calculate the Euclidean distance based on Equation 2, we get √2. It can be determined that the shorter the Euclidean distance, the higher the similarity.

[0057] Accordingly, the response generating device (100) of the present disclosure can determine a security document that has a high degree of similarity to the input query.

[0058] The response generating device (100) of the present disclosure includes an importance calculation unit (110) that calculates an importance level that serves as a criterion for determining whether the data required through the query is important for security from a first language model in which at least one of a query and a security document is input.

[0059] When the response generating device (100) of the present disclosure obtains a security document for an input query, it determines an LLM to be input by considering the importance of the content included in the security document.

[0060] Before inputting at least one of a query and a security document into a first language model for calculating importance, the response generating device (100) of the present disclosure may perform a preprocessing operation on the input data.

[0061] The aforementioned preprocessing operation is intended to remove unnecessary words from the data contained in the text to retain only the necessary information. The present disclosure proposes a Most Common Word filtering technique that extracts and removes words that frequently appear in the text, such as articles or conjunctions, or a Regular Expression filtering technique that extracts, removes, or transforms words that match a specific pattern.

[0062] For example, when a security document corresponding to an input query is retrieved from a database pre-configured based on the aforementioned similarity, the importance is determined based on the security level of the input query and the security document, and when a security document is not retrieved from the database pre-configured, it can be determined based on keywords included in the query.

[0063] As another example, the response generating device (100) of the present disclosure may determine that security information is required through a query when the importance is above a preset threshold, and determine that only public information is required through a query when the importance is below a preset threshold.

[0064] When a security document corresponding to a query is retrieved by the response generation device (100) of the present disclosure, importance can be calculated through a first language model that is pre-set based on the security level of the query and the security document. A high security level means that the accuracy of the response to the query is higher when using sLLM than when using Public LLM.

[0065] However, the aforementioned security grade is intended only to evaluate the security standards of the contents included in the security document, and as needed, the security document can be divided according to various criteria and input into the first language model along with the query to calculate the importance.

[0066] Additionally, if the response generating device (100) of the present disclosure does not retrieve a security document corresponding to the query, it can determine the importance through a first language model using the content of the query itself. The importance may be calculated by inputting keywords included in the query into the first language model, or the importance may be calculated by referring to tags set for each query through the aforementioned tagging.

[0067] The response generating device (100) of the present disclosure includes a response generating unit (130) that determines one of a second language model learned through a plurality of security information or a third language model learned through a plurality of public information based on importance, and generates a response to a query from the determined model.

[0068] For example, if the response generating device (100) of the present disclosure determines that security information is required through a query, it may route the query, security document, and importance to the second language model through a security Proxy Gateway connected to each of the second language model and the third language model.

[0069] The response generating device (100) of the present disclosure can route the query, security document, and importance to the second language model through a security Proxy Gateway connected to each of the second language model and the third language model when it is determined that the information requested through the query is of high importance and that the information is security information.

[0070] As another example, if the response generating device (100) of the present disclosure determines through a query that only public information is required, it may route the query and importance to a third language model through a security Proxy Gateway, which is one of the network components acting as an intermediary between the client and the server.

[0071] If the response generating device (100) of the present disclosure determines that the importance of the information required through the query is low and that the information will only contain public information, it may route the query, security document, and importance to the third language model through a security Proxy Gateway connected to each of the second language model and the third language model.

[0072] As another example, a security proxy gateway can log a query routed to either the second language model or the third language model, the security document and importance, and a response generated from either the second language model or the third language model.

[0073] The security proxy gateway of the present disclosure can connect or block connections between a client and a server, thereby performing the role of protecting data, recording data, or analyzing network traffic. Accordingly, it can perform the role of a gateway that encrypts and connects sLLM and Public LLM, and simultaneously connects sLLM and Public LLM while logging all queries from users and answers from LLM to ensure security traceability.

[0074] Among the various language models described above, the first language model and the second language model may include at least one sLLM (Smaller Large Language Model), and the third language model may include at least one Public LLM (Large Language Model).

[0075] The present disclosure has the advantage of being able to infer and generate a better response based on the importance of the data desired by the user through a query, and provide it to the user.

[0076] Below, the overall process of generating a response is explained in more detail with reference to the diagram.

[0077] FIG. 2 is a diagram for schematically explaining LLM, which is a type of artificial intelligence according to one embodiment.

[0078] Referring to FIG. 2, artificial intelligence (200) includes machine learning (ML) (210), deep learning (DL) (220), and LLM (240) as sub-concepts.

[0079] Specifically, artificial intelligence (200) is a field that performs repetitive learning in a manner similar to human intelligence and makes judgments based on the results of learning. Artificial intelligence (200) is a broad concept that includes machine learning (210) and deep learning (220), and machine learning (210) is used as a broad concept that includes deep learning (220).

[0080] Machine learning (220) is a field of artificial intelligence (200) that can learn patterns in data and perform decision-making or prediction. It is also a field that develops algorithms and technologies that enable computers to learn based on data, and it is a core technology in various fields such as image processing, image recognition, speech recognition, and internet search, showing excellent performance in prediction and detection.

[0081] The types of machine learning (210) learning methods include supervised learning, unsupervised learning, and reinforcement learning.

[0082] Supervised learning is a learning method in which learning is performed with data input to machine learning (210) and correct answers for that data provided, and unsupervised learning is a learning method in which machine learning learns patterns on its own to find the correct answers, in that data is input to machine learning but there are no correct answers for that data.

[0083] In addition, reinforcement learning is a learning method in which an agent interacts with a given environment and is given a certain reward for actions or judgments made by the agent, and learns in a direction that maximizes the aforementioned reward.

[0084] In addition, machine learning (210) includes deep learning (220), which uses a hierarchical structure to learn patterns of large-scale data using an artificial neural network (ANN) to solve complex problems.

[0085] Deep learning (220) includes Convolutional Neural Networks (CNN) used for image and video processing, Recurrent Neural Networks (RNN) used for processing sequentially input data, Long Short-Term Memory (LSTM) used for processing time series data, and Generative Adversarial Networks (GAN) used for data augmentation.

[0086] In addition, natural language processing (NLP) (230) can be described as a field of artificial intelligence that enables a machine computer to understand, interpret, and generate natural language, which is the language used by humans.

[0087] In particular, the LLM (240), which is one of the artificial intelligence models used through the query generation device of the present disclosure, is a type of artificial intelligence model learned through prompts, which are data input to the model and are a vast amount of text data. The LLM (240) can generate a consistent response to various prompts. Additionally, the LLM (240) can translate language, generate text that meets conditions, or summarize text.

[0088] LLM(240) may include a transformer that finds the relationship between words contained in the input text.

[0089] Learning can be performed through at least one of a Pretraining learning method that learns the patterns and structures of text using a large-scale learning dataset prepared as a learning method of LLM (240) and a Fine-Tuning learning method that performs learning according to the intended use of LLM (240) by labeling a part of the large-scale learning dataset, and can also be performed using a few-shot learning method that performs learning by reflecting examples in the learning data as needed.

[0090] In addition, as mentioned above, LLM (240) includes sLLM and Public LLM, and both sLLM and Pulbic LLM are based on the Transformer architecture.

[0091] sLLM is designed to provide solutions optimized for specific companies or industries, as training data is limited in that learning is performed based on data used in specific companies or industries. Therefore, it is primarily used internally within the company and is not disclosed externally, and access rights are restricted. Consequently, it has the advantage of high performance in specific fields.

[0092] In contrast, Public LLMs may contain somewhat inaccurate information because training is performed based on publicly available data. They have the advantage of being able to respond well to broad or generalized questions and are accessible to everyone. However, they have the disadvantage of being somewhat vulnerable in terms of security.

[0093] The response generating device of the present disclosure can improve the performance of the response by considering the importance of the data desired by the user through the input query, and selecting one of the aforementioned sLLM or Public LLM to input the query.

[0094] FIG. 3 is a flowchart illustrating the learning process of an artificial intelligence model according to one embodiment.

[0095] Referring to Fig. 3, a general learning process of an artificial intelligence model including LLM is illustrated. Although there are various learning methods and processes for artificial intelligence models, LLM is used as an example for the sake of convenience of explanation.

[0096] Specifically, training data to be used for training is collected, and preprocessing is performed on the collected training data (S300).

[0097] Training data for LLM can be collected through web crawling or from databases that provide data related to specific fields. However, the aforementioned methods of collecting training data are merely examples, and data can be collected in various ways as needed.

[0098] Once training data is collected, preprocessing can be performed to input it into an LLM model. For example, if news articles are collected as training data, tokenization can be performed to classify the text of the collected articles into tokens, which are the smallest meaningful units; normalization can be performed to convert the text into a standard form (converting to lowercase, converting to numbers, removing special characters); meaningless words can be removed; or the text can be converted into a vector containing numerical values. The aforementioned preprocessing methods are merely examples and can be configured in various ways as needed.

[0099] Initialization of the artificial intelligence model to be trained is performed (S310).

[0100] Once the AI ​​model is ready for training, the AI ​​model can be initialized. Initializing the AI ​​model involves initializing the weights or loss function used in the model. For example, the aforementioned initialization does not mean setting the weights to zero, but rather setting the weights to a specific value or range.

[0101] When training is performed, a preset loss function is calculated to determine whether the data output from the artificial intelligence model matches reality (S320).

[0102] Through the aforementioned loss function, the performance of the model used in the training process can be evaluated, and weights can also be adjusted.

[0103] For example, the Mean Squared Error, which expresses the difference between the predicted value and the actual value output for an input value input into an artificial intelligence model, can be used as a loss function in the learning process of deep learning.

[0104] As another example, Cross-Entropy Loss can be used as a loss function in the training process of LLM. This loss function calculates the difference between the probability distribution of potential outputs from the LLM for an input text and the probability distribution of the actual answer.

[0105] When training is completed, verification of the artificial intelligence model is performed to check whether the training was performed correctly (S330).

[0106] Validation of LLM is performed by evaluating the performance of the trained model using validation data separately from the training data. Validation methods that may be used include BLEU (Bilingual Evaluation Understudy) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation).

[0107] In addition, the occurrence of overfitting can be determined during the validation process. Overfitting refers to a situation where the model is overly focused on training data, resulting in a decline in performance when non-training data is input.

[0108] The determination of whether overfitting has occurred can be made by whether the loss incurred during the validation process using validation data increases compared to the loss incurred when training is performed using training data.

[0109] Once the verification of the artificial intelligence model is complete, a suitability evaluation of the model is performed, and if it is determined that it is not suitable, the aforementioned training and verification are repeated (S340).

[0110] Once all training and verification of the artificial intelligence model is completed, the model is deployed after a final evaluation is performed (S350).

[0111] FIG. 4 is a flowchart illustrating the process of generating a response to a query input through an LLM according to one embodiment.

[0112] Referring to FIG. 4, the response generating device of the present disclosure can generate a response that is a conclusion by inputting a query, which is text data, into one of a pre-trained sLLM or Public LLM according to the importance of the data required through the query.

[0113] Specifically, when the LLM to be entered for a query is determined, the response generating device of the present disclosure enters at least one of a query, a security document, and an importance level as text into the LLM (S400).

[0114] The response generating device of the present disclosure may translate text data input into the LLM into a specific language, such as English or Korean, or perform preprocessing to remove unnecessary information.

[0115] When text containing a query is input into the LLM, the text is separated into at least one word, and each word is converted into a high-dimensional vector (S410).

[0116] The response generation device of the present disclosure performs tokenization to separate the input text into words when text corresponding to at least one of a query, a security document, and importance is input into an LLM, and can convert each tokenized word back into an embedding vector, which is a high-dimensional vector containing numbers. For example, if there is a word "CAR", it can be converted into an embedding vector [0.3, -0.7,...], which is a high-dimensional vector containing numbers.

[0117] Accordingly, the response generating device of the present disclosure may input query and article information as text in accordance with the input data format of the LLM, or may input them as the original embedding vector.

[0118] When each word is converted into an embedding vector which is a high-dimensional vector, it is input into a transformer to determine the association between words (S420).

[0119] When each embedding vector is input into the transformer, words associated with words included in the input query are selected, and words that can be included in the response are selected based on at least one of the input query, security documents, and importance, and the correct answer probability for each word is calculated.

[0120] Calculate the probability of the correct answer for the words that can be included in the response, and generate an output vector associated with the word with the highest probability (S430).

[0121] When an output vector is generated, the output vector is converted into text to output the conclusion through LLM (S440).

[0122] The converted text is output from the LLM as a response (S450).

[0123] The process of outputting a conclusion through the aforementioned LLM is merely an example, and may be performed by changing some of the order as necessary.

[0124]

[0125] Figure 5 is a flowchart illustrating the process of calculating the importance of required data through an input query according to a specific example.

[0126] Referring to FIG. 5, the response generating device of the present disclosure can determine whether there is a security document related to a query and calculate the importance of the data required through the query through a first language model.

[0127] Specifically, the response generating device of the present disclosure obtains a query entered by a user (500).

[0128] There are no limits on the types or lengths of the query. However, a preprocessing step may be performed before the query is input into the first language model, and the aforementioned preprocessing may involve translating the query into a specific language. The preprocessing may perform tokenization, which classifies the collected text into tokens that are the smallest meaningful units; normalization, which converts the text into a standard form (converting to lowercase, converting to numbers, removing special characters); or remove meaningless words or convert the text into a vector containing numerical values.

[0129] When a query is obtained, the response generating device of the present disclosure queries the secure document through the RAG and database (S510).

[0130] The response generation device of the present disclosure does not immediately input a query entered by a user into the LLM, but can check whether a secure document related to the query is stored in a pre-configured database. It can be queried through a pre-configured search engine or database.

[0131] As described above, the calculation of similarity between a query and a security document can be performed based on conversion to a vector through an embedding model and at least one of a cosine similarity determination algorithm and a Euclidean distance calculation algorithm. When the security document lookup based on similarity is completed, the response generating device of the present disclosure can calculate the importance of the data required through the query through a preset first language model (S520).

[0132] When the aforementioned security document is retrieved, the query and the security level information of the security document are entered together into the first language model, and when the security document is not retrieved, only the query is entered to calculate the importance.

[0133] For example, importance can be output through the first language model as a real value between 0 and 1, and data closer to 1 may be judged to have higher importance. However, the method of judging importance and the value output from the first language model are not limited to this and can be set in various ways as needed.

[0134] FIG. 6 is a flowchart illustrating the process of generating a response by inputting a query based on importance according to one embodiment into either sLLM or Public LLM.

[0135] Referring to FIG. 6, the response generating device of the present disclosure determines whether the data is important based on the importance of the data required through a query and the criteria for determining the type of LLM, and can generate a response from either sLLM or Public LLM based on the result of the determination.

[0136] Specifically, when the importance of the data required through the query is calculated through the first language model described above, the response generating device of the present disclosure determines whether the data required through the query is important based on the importance through a preset threshold (S600).

[0137] The response generating device of the present disclosure can determine that security information is required through a query when the importance is above a preset threshold, and determine that only public information is required through a query when the importance is below a preset threshold.

[0138] When it is determined whether the required data is important through a query, the response generating device of the present disclosure routes the query to either a second language model or a third language model through a secure Proxy Gateway (S610).

[0139] The response generating device of the present disclosure can route a query, a secure document, and an importance to a second language model, sLLM, if the importance is above a preset threshold and the data required through the query is determined to be important.

[0140] Alternatively, the response generating device of the present disclosure may route the query and importance to a third language model, Public LLM, if it is determined that the data required through the query is not important because the importance is below a preset threshold.

[0141] The response generating device of the present disclosure generates a response from one of a second language model or a third language model (S620).

[0142] As described above, the first language model and the second language model include sLLM, and the third language model may include Public LLM. Additionally, the security proxy gateway of the present disclosure may connect or block connections between a client and a server, thereby performing the role of protecting data, recording data, or analyzing network traffic. Accordingly, it can perform the role of a gateway that encrypts and connects sLLM and Public LLM, and simultaneously connects sLLM and Public LLM while logging all queries from users and answers from LLM to ensure security traceability.

[0143] FIG. 7 is a flowchart for schematically explaining the entire process from when a query is input to when a response is generated according to one embodiment.

[0144] Referring to FIG. 7, the response generating device of the present disclosure can input a language model having different characteristics based on the importance of the data requested through the acquired query and generate a response as a result.

[0145] The response generating device of the present disclosure obtains a query entered by a user (S700).

[0146] When a query is obtained, the response generating device of the present disclosure inputs the query into a data importance-based routing model to determine the importance of the data requested through the query (S710).

[0147] The response generating device of the present disclosure can determine the existence of a secure document through the aforementioned RAG and a pre-configured database prior to determining importance through the aforementioned data importance-based routing model, and can determine the data importance by inputting the determination result and the query together into the aforementioned data importance-based routing model.

[0148] The present disclosure refers to a data importance-based routing model as a first language model, and the aforementioned first language model includes sLLM.

[0149] When the data importance is determined, the response generating device of the present disclosure determines whether to route at least one of the query, security document, and importance to sLLM or Public LLM through a security proxy gateway based on the determined importance (S720).

[0150] The response generating device of the present disclosure compares the determined data importance with a preset threshold and, based on the comparison result, determines to route the query, security document, and importance to sLLM if it is determined to be important information, and to route the query and importance to Public LLM if it is determined not to be important information.

[0151] The security proxy gateway of the present disclosure performs the role of a gateway that encrypts and connects sLLM and Public LLM as described above, and at the same time connects sLLM and Public LLM, it can ensure security traceability by logging all queries from users and answers from LLM.

[0152] Specifically, based on the result of comparing the aforementioned data importance with a preset threshold, if the data importance is greater than or equal to the threshold, the response generating device of the present disclosure routes the corresponding query, security document, and importance to sLLM through a security proxy gateway (S730).

[0153] When routed to sLLM, the response generating device of the present disclosure generates a response through sLLM (S740).

[0154] If, based on the comparison result between the data importance and a preset threshold, the data importance is less than the threshold, the response generating device of the present disclosure routes the corresponding query and importance to the Public LLM through a security proxy gateway (S750).

[0155] When routed to a Public LLM, the response generating device of the present disclosure generates a response through the Public LLM (S760).

[0156] FIG. 8 is a flowchart for explaining a method for generating a response to a query according to one embodiment.

[0157] Referring to FIG. 8, the response generation method of the present disclosure includes a security document acquisition step of acquiring a security document corresponding to a query from a preset database (S800).

[0158] The response generation device of the present disclosure can generate a response to a query by inputting the query into either sLLM (smaller Large Language Model) or Pulbic LLM (Large Language Model), taking into account the importance of the data desired by the user through the input query.

[0159] LLM (Large Language Model) includes sLLM (Smaller Large Language Model), which is trained on information such as personally identifying information (PII) requiring security, financial information, corporate internal policies, and corporate non-public information, depending on the characteristics of the training data, and Pulbic LLM (Large Language Model), which is trained on public information, searchable information, and general question and answer.

[0160] However, it is not easy to distinguish between the aforementioned Personally Identifying Information (PII), financial information, corporate internal policies, corporate non-public information and public information, searchable information, and general Q&A. Therefore, the present disclosure proposes a method of calculating the importance of the required data through an input query and generating a response by inputting the query into either sLLM or Pulbic LLM based on the calculation result.

[0161] For example, the response generating device of the present disclosure can convert a text-based query into a first embedding vector through a preset embedding model and input the first embedding vector into a preset search engine model to obtain a security document with high similarity to the query from a preset database.

[0162] The response generation device of the present disclosure does not immediately input a query entered by a user into the LLM, but can check in advance whether information related to the query is stored in a pre-configured database, and can also input the query into a pre-configured search engine model to retrieve information related to the query. The present disclosure refers to the process of inputting information retrieved from a pre-configured database into the LLM together with the query before inputting the query into a separate LLM as RAG (Retrieval-Augmented Generation). Through the aforementioned RAG operation, the accuracy of the response output from the LLM can be improved.

[0163] The aforementioned security documents may include documents that are not disclosed, and said security documents may be stored separately in a pre-configured database.

[0164] The response generating device of the present disclosure may perform tagging by setting tags into pre-set categories based on the content of an input query before querying a security document with high similarity to the query from a pre-set database. For example, the aforementioned pre-set categories may include personal information, secret information, and public information. The aforementioned categories are merely examples and may be set in various ways as needed.

[0165] As another example, a security level may be set for each of the aforementioned security documents, and the set security level can be used to calculate importance. For instance, if the security level of a security document is high, it may be appropriate to generate a response through sLLM because the accuracy of the response to the query may be lower through Public LLM. Conversely, if the security level of a security document is low, a response can be generated through Public LLM.

[0166] As another example, the aforementioned preset database includes a second embedding vector for each of a plurality of security documents, and the security document corresponding to the query can be determined based on the similarity between the first embedding vector and the second embedding vector.

[0167] As another example, the aforementioned similarity is determined based on a first embedding vector, a second embedding vector, and a preset algorithm, and the preset algorithm may include at least one of a cosine similarity determination algorithm and a Euclidean distance calculation algorithm.

[0168] Embedding is a technique that converts each piece of data into a high-dimensional vector containing numbers while preserving the original meaning of the data. Furthermore, embedding can convert not only words into vectors but also entire sentences or texts into a single vector.

[0169] The response generation device of the present disclosure converts an input query into a single embedding vector through a preset embedding model and can obtain a secure document through a preset database.

[0170] The security documents obtained by the response generating device of the present disclosure do not mean only documents that match 100% with the query, but also include security documents that are judged to have a high degree of similarity.

[0171] As described above, the similarity between the query and the security document can be calculated based on at least one of a cosine similarity determination algorithm and a Euclidean distance calculation algorithm.

[0172] The cosine similarity determination algorithm is a technique that measures the degree of similarity between two data by comparing the directionality between two vectors using a cosine function. Similarity can be calculated by the aforementioned mathematical formula 1, where θ is the angle, X and Y are the two vectors to be compared, XY is the dot product of the two vectors, and |X| and |Y| are the magnitudes of each vector.

[0173] Since Cosθ, the first similarity, has only values ​​between -1 and 1, the similarity also has only values ​​between -1 and 1, and it can be determined that the closer Cosθ is to 1, the higher the similarity between the two articles.

[0174] In addition, the aforementioned Euclidean distance calculation algorithm calculates the distance between two points, and the smaller the calculated distance, the higher the first similarity can be determined.

[0175] The Euclidean distance can be calculated based on the aforementioned mathematical formula 2, in which d(p,q) is the distance between two points p and q, x1 and y1 are the coordinates of p, and x2 and y2 are the coordinates of q.

[0176] The shorter the Euclidean distance, the higher the similarity can be judged.

[0177] Accordingly, the response generating device of the present disclosure can determine a security document that has a high similarity to an input query.

[0178] The response generation method of the present disclosure includes a step of calculating importance, which is a criterion for determining whether the data required through a query is important for security, from a first language model into which at least one of a query and a security document is input (S810).

[0179] When the response generating device of the present disclosure obtains a security document for an input query, it determines an LLM to be input by considering the importance of the content included in the security document.

[0180] Before inputting at least one of a query and a security document into a first language model for calculating importance, the response generating device of the present disclosure may perform a preprocessing operation on the input data.

[0181] The aforementioned preprocessing operation is intended to remove unnecessary words from the data contained in the text to retain only the necessary information. The present disclosure proposes a Most Common Word filtering technique that extracts and removes words that frequently appear in the text, such as articles or conjunctions, or a Regular Expression filtering technique that extracts, removes, or transforms words that match a specific pattern.

[0182] For example, when a security document corresponding to an input query is retrieved from a database pre-configured based on the aforementioned similarity, the importance is determined based on the security level of the input query and the security document, and when a security document is not retrieved from the database pre-configured, it can be determined based on keywords included in the query.

[0183] As another example, the response generating device of the present disclosure may determine that security information is required through a query when the importance is above a preset threshold, and determine that only public information is required through a query when the importance is below a preset threshold.

[0184] When a security document corresponding to a query is retrieved, the response generation device of the present disclosure can calculate importance through a first language model that is pre-set based on the security level of the query and the security document. A high security level means that the accuracy of the response to the query is higher when using sLLM than when using Public LLM.

[0185] However, the aforementioned security grade is intended only to evaluate the security standards of the contents included in the security document, and as needed, the security document can be divided according to various criteria and input into the first language model along with the query to calculate the importance.

[0186] In addition, if a security document corresponding to a query is not retrieved, the response generating device of the present disclosure can determine the importance through a first language model using the content of the query itself. The importance may be calculated by inputting keywords included in the query into the first language model, or the importance may be calculated by referring to tags set for each query through the aforementioned tagging.

[0187] The response generation method of the present disclosure includes a response generation step of determining one of a second language model learned through a plurality of security information or a third language model learned through a plurality of public information based on importance, and generating a response to a query from the determined model (S820).

[0188] For example, if the response generating device of the present disclosure determines that security information is required through a query, it may route the query, security document, and importance to the second language model through a security Proxy Gateway connected to each of the second language model and the third language model.

[0189] The response generating device of the present disclosure can route the query, security document, and importance to the second language model through a security Proxy Gateway connected to each of the second language model and the third language model when it is determined that the information requested through the query is of high importance and that the information is security information.

[0190] As another example, if the response generating device of the present disclosure determines through a query that only public information is required, it may route the query and importance to a third language model through a security Proxy Gateway, which is one of the network components acting as an intermediary between the client and the server.

[0191] If the response generating device of the present disclosure determines that the importance of the information required through a query is low and that the information will only contain public information, it may route the query, security document, and importance to the third language model through a security Proxy Gateway connected to each of the second language model and the third language model.

[0192] As another example, a security proxy gateway can log a query routed to either the second language model or the third language model, the security document and importance, and a response generated from either the second language model or the third language model.

[0193] The security proxy gateway of the present disclosure can connect or block connections between a client and a server, thereby performing the role of protecting data, recording data, or analyzing network traffic. Accordingly, it can perform the role of a gateway that encrypts and connects sLLM and Public LLM, and simultaneously connects sLLM and Public LLM while logging all queries from users and answers from LLM to ensure security traceability.

[0194] Among the various language models described above, the first language model and the second language model may include at least one sLLM (Smaller Large Language Model), and the third language model may include at least one Public LLM (Large Language Model).

[0195] Through the operation of the aforementioned configurations, an optimal response to the input query can be provided, and the overall performance of the model can also be improved by using multiple language models, such as LLMs.

[0196] FIG. 9 is a configuration diagram of a computing device including an artificial intelligence model according to one embodiment.

[0197] Referring to FIG. 9, the computing device (900) may include memory (910) and a processor (920), and the memory may include at least one artificial intelligence model (930).

[0198] The memory (910) can store a program for the operation of the processor (920) and can temporarily or permanently store input / output data. The memory (910) may include at least one type of storage medium among RAM, SRAM, ROM, EEPROM, PROM, magnetic memory, magnetic disk, optical disk, hard disk type, multimedia card micro type, flash memory type, card type memory (e.g., SD or XD memory, etc.), volatile memory (e.g., SRAM, DRAM), or non-volatile memory (e.g., NAND Flash).

[0199] In addition, the memory (910) can store various functions and algorithms, and can store various data, applications, software, commands, code, etc.

[0200] The processor (920) can control the overall operation of the response generating device of the present disclosure. The processor (920) can execute one or more programs and may mean a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or a dedicated processor on which methods according to some embodiments of the present disclosure are performed.

[0201] Meanwhile, the computing device (900) of the present disclosure may be a quantum computing device rather than a classic computing device. A quantum computing device performs operations in units of qubits rather than bits. A qubit can have a state in which 0 and 1 are simultaneously superpositioned, and if there are M qubits, 2^M states can be represented simultaneously.

[0202] A quantum computing device can use various types of quantum gates (e.g., Pauli / Rotation / Hadamard / CNOT / SWAP / Toffoli) that receive one or more qubits to perform quantum operations and perform specified operations, and can combine quantum gates to form a quantum circuit with a special function.

[0203] Quantum computing devices can use quantum artificial neural networks (e.g., QCNN, QGRNN) that can perform functions of conventional artificial neural networks (e.g., CNN, RNN) at a faster speed while using fewer parameters.

[0204] Additionally, the memory (910) may store an artificial intelligence model (930) including an LLM that calculates the importance of data required through an input query of the present disclosure or generates a response to an input query. When a task to calculate the importance of data required through an input query or to generate a response to an input query is requested, the processor (920) may execute the artificial intelligence model (930) stored in the memory (910) and output the result.

[0205] For example, the processor (920) may obtain a security document corresponding to a query from a pre-configured database, calculate an importance level that serves as a criterion for determining whether the data required through the query is important for security from a first language model into which at least one of the query and the security document is input, determine one of a second language model learned through multiple security information or a third language model learned through multiple public information based on the importance level, and generate the response to the query from the determined model.

[0206]

[0207] FIG. 10 is a configuration diagram of a computer system including a client-server that includes an artificial intelligence model according to one embodiment.

[0208] Referring to FIG. 10, a computing system according to one embodiment of the present invention may include a computer device (1000) including memory (1030) and a processor (1040), and a server (1010) including memory (1050) and a processor (1060). The computer device (1000) and the server (1010) may be connected via a wired or wireless connection through a network (1020).

[0209] The network (920) connecting the aforementioned computer device (1000) and server (1010) can also be configured as a network of various sizes, such as a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, etc.

[0210] The memory (1030) of the computer device (1000) can store information about the input query.

[0211] The memory (1060) of the server (1010) can store an artificial intelligence model (1070) including an LLM that calculates the importance of the data required through the aforementioned input query or generates a response to the input query.

[0212] The processor (1040) of the computer device (1000) can send a request to the server (1010) to calculate the importance of the required data through a query stored in memory (1030) or to generate a response to the input query.

[0213] The processor (1060) of the server (1010) can calculate the importance of the data required through the input query or generate a response to the input query using an artificial intelligence model (1070) that includes an LLM, and can generate a response to the data required through the input query or generate a response to the input query, and transmit the result to the computer device (1000).

[0214] The foregoing description is merely an illustrative explanation of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations within the scope of the essential characteristics of the technical concept. Furthermore, since these embodiments are intended to explain, not limit, the scope of the technical concept is not limited by these embodiments. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present disclosure.

[0215]

[0216] CROSS-REFERENCE TO RELATED APPLICATION

[0217] This patent application claims priority pursuant to Section 119(a) of the U.S. Patent Act (35 USC § 119(a)) to Korean Patent Application No. 10-2024-0191050 filed on December 19, 2024, all of which are incorporated by reference into this patent application. Furthermore, this patent application claims priority in countries other than the United States for the same reasons as above, all of which are incorporated by reference into this patent application.

[0218]

[0219] CROSS-REFERENCE TO RELATED APPLICATION

[0220] This patent application claims priority pursuant to Section 119(a) of the U.S. Patent Act (35 USC § 119(a)) to Korean Patent Application No. 10-2024-0191050 filed on December 19, 2024, all of which are incorporated by reference into this patent application. Furthermore, this patent application claims priority in countries other than the United States for the same reasons as above, all of which are incorporated by reference into this patent application.

Claims

1. A secure document acquisition unit that acquires a secure document corresponding to a query from a pre-configured database; An importance calculation unit that calculates an importance level serving as a criterion for determining the security importance of the data required through the query from a first language model into which at least one of the above query and the above security document is input; and A response generating device comprising a response generating unit that determines one of a second language model learned through multiple security information or a third language model learned through multiple public information based on the importance, and generates the response to the query from the determined model.

2. In Paragraph 1, The above document acquisition unit is, The above query is converted into a first embedding vector through a pre-configured embedding model, and A response generation device characterized by inputting the first embedding vector into a preset search engine model to obtain the security document with high similarity to the query from the preset database.

3. In Paragraph 2, The above-mentioned pre-configured database is, It includes a second embedding vector for each of a plurality of security documents, and The above document corresponding to the above inquiry is, A response generating device characterized by being determined based on the similarity between the first embedding vector and the second embedding vector.

4. In Paragraph 3, The above similarity is, Determined based on the first embedding vector, the second embedding vector, and a preset algorithm, The above-mentioned preset algorithm is, A response generating device characterized by including at least one of a cosine similarity determination algorithm and a Euclidean distance calculation algorithm.

5. In Paragraph 1, The above-mentioned first language model and the above-mentioned second language model are, It includes at least one sLLM (Smaller Large Language Model), and The above third language model is, A response generation device characterized by including at least one Public LLM (Large Language Model).

6. In Paragraph 1, The above importance is, When the security document is retrieved from the aforementioned preset database, it is determined based on the aforementioned query and the preset security level of the document, and A response generation device characterized by determining based on keywords included in the query when the security document is not retrieved from the above-mentioned preset database.

7. In Paragraph 1, The above response generating unit is, If the above importance is greater than or equal to a preset threshold, it is determined that the above security information is required through the above query, and A response generating device characterized by determining that only the public information is required through the query when the importance is below a preset threshold.

8. In Paragraph 7, The above response generating unit is, A response generating device characterized by routing the query, the security document, and the importance to the first language model through a security proxy gateway connected to each of the second language model and the third language model when it is determined through the above query that the above security information is required.

9. In Paragraph 8, The above response generating unit is, A response generation device characterized by routing the query and the importance to the third language model through the security proxy gateway when it is determined through the above query that only the above public information is required.

10. In Paragraph 9, The above security proxy gateway is, A response generating device characterized by logging the query, the security document, and the importance routed to either the second language model or the third language model, and the response generated from either the second language model or the third language model.

11. A security document acquisition step for obtaining a security document corresponding to a query from a pre-configured database; An importance calculation step for calculating an importance level that serves as a criterion for determining the security importance of the data required through the query from a first language model into which at least one of the above query and the above security document is input; and A response generation method comprising a response generation step of determining one of a second language model learned through multiple security information or a third language model learned through multiple public information based on the importance, and generating the response to the query from the determined model.

12. In Paragraph 11, The above document acquisition step is, The above query is converted into a first embedding vector through a pre-configured embedding model, and A response generation method characterized by inputting the first embedding vector into a preset search engine model to obtain the security document with high similarity to the query from the preset database.

13. In Paragraph 12, The above-mentioned pre-configured database is, It includes a second embedding vector for each of a plurality of security documents, and The above document corresponding to the above inquiry is, A response generation method characterized by being determined based on the similarity between the first embedding vector and the second embedding vector.

14. In Paragraph 13, The above similarity is, Determined based on the first embedding vector, the second embedding vector, and a preset algorithm, The above-mentioned preset algorithm is, A response generation method characterized by including at least one of a cosine similarity determination algorithm and a Euclidean distance calculation algorithm.

15. In Paragraph 11, The above-mentioned first language model and the above-mentioned second language model are, It includes at least one sLLM (Smaller Large Language Model), and The above third language model is, A response generation method characterized by including at least one Public LLM (Large Language Model).

16. In Paragraph 11, The above importance is, When the security document is retrieved from the aforementioned preset database, it is determined based on the aforementioned query and the preset security level of the document, and A method for generating a response characterized by determining based on keywords included in the query when the security document is not retrieved from the above-mentioned preset database.

17. In Paragraph 11, The above response generation step is, If the above importance is greater than or equal to a preset threshold, it is determined that the above security information is required through the above query, and A response generation method characterized by determining that only the public information is required through the query when the importance is below a preset threshold.

18. In Paragraph 17, The above response generation step is, A response generation method characterized by routing the query, the security document, and the importance to the first language model through a security Proxy Gateway connected to each of the second language model and the third language model when it is determined through the above query that the above security information is required.

19. In Paragraph 18, The above response generation step is, A response generation method characterized by routing the query and the importance to the third language model through the security proxy gateway when it is determined through the above query that only the above public information is required.

20. In Paragraph 19, The above security proxy gateway is, A response generation method characterized by logging the query, the security document, and the importance routed to either the second language model or the third language model, and the response generated from either the second language model or the third language model.

Images