Method and Apparatus for Evaluating Output Sensitivity of LLM, Method for Controlling the Output Sensitivity of LLM
By generating summary sentences and calculating similarity distributions, the method enhances the reliability and consistency of large-scale language model evaluations, enabling precise sensitivity adjustments.
Patent Information
- Application Number
- KR1020250121703
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2025-08-29
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2045-08-29
AI Technical Summary
Existing methods for evaluating the output sensitivity of large-scale language models suffer from inconsistencies due to variations in expression and language-specific discrepancies, leading to unreliable evaluation results, particularly in fields requiring accurate decision-making.
A method involving generating summary sentences from multiple outputs of a large-scale language model for the same input, calculating similarity distributions, and adjusting model parameters based on sensitivity evaluation values to enhance reliability and consistency.
Improves the reliability and consistency of output sensitivity evaluations by addressing expression variations and language-specific issues, allowing for tailored sensitivity adjustments.
Smart Images

Figure 112025099152254-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a method and apparatus for evaluating the output sensitivity of a large-scale language model, and a method for adjusting the output sensitivity of a large-scale language model. Background Technology
[0003] Large-scale language models (LLMs) are structures that take a sentence as input and output a sentence, probabilistically selecting words based on the input sentence to produce a new sentence. Due to this characteristic, large-scale language models can generate different output results based on small differences in expression, even if the input sentences have the same or similar meanings. In other words, even if sentences conveying the same meaning are input, the content, style, and narrative method of the sentences output by the large-scale language model may differ.
[0004] In fields where the output of large-scale language models is directly linked to actual decision-making, such as medical consultation, legal advice, and financial advice, the variability of these outputs acts as a significant issue. Therefore, it is important to quantitatively evaluate the output sensitivity of large-scale language models to small changes in input sentences, and various evaluation techniques are being studied for this purpose.
[0005] Methods for evaluating the output sensitivity of large-scale language models include Measuring Unintended Bias, Counterfactual Fairness Analysis, and Text Summarization Metrics. Recently, Distribution-Based Perturbation Analysis (DBPA) has been proposed to improve upon the performance of the aforementioned methods. DBPA is a method that considers the characteristic of large-scale language models generating probabilistically diverse outputs for the same input sentence, constructs an output distribution through iterative sampling, and statistically compares and analyzes it.
[0006] Related prior art includes patent documents Korean Published Patents No. 2025-0103527, No. 2025-0021798, and No. 2024-0066227, and non-patent document "RAUBA, Paulius; WEI, Qiyao; Statistical Hypothesis Testing for Auditing Robustness in Language Models. In Proceedings of the Forty-second International Conference on Machine Learning(ICML 2025), 2025." The problem to be solved
[0008] The present invention is intended to provide a method and apparatus for evaluating the output sensitivity of a large-scale language model with greater reliability.
[0009] In addition, the present invention is intended to provide a method for adjusting the output sensitivity of a large-scale language model. means of solving the problem
[0011] According to one embodiment of the present invention for achieving the above-mentioned purpose, a method for evaluating the output sensitivity of a large-scale language model is provided, comprising the steps of: repeatedly inputting an original sentence and a plurality of modified sentences of the original sentence into a large-scale language model to obtain, from the large-scale language model, a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences; generating a first summary sentence for the first output sentence and a second summary sentence for the second output sentence; generating a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences; and calculating an output sensitivity evaluation value of the large-scale language model from the first and second similarity distributions.
[0012] Additionally, according to another embodiment of the present invention for achieving the above-mentioned purpose, there is provided an output sensitivity evaluation device for a large-scale language model comprising: a memory; and a processor electrically connected to the memory, wherein the processor repeatedly inputs an original sentence and a plurality of modified sentences for the original sentence into a large-scale language model, thereby obtaining a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences from the large-scale language model, generating a first summary sentence for the first output sentence and a second summary sentence for the second output sentence, generating a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences, and calculating an output sensitivity evaluation value of the large-scale language model from the first and second similarity distributions.
[0013] In addition, according to another embodiment of the present invention for achieving the above-mentioned purpose, a method for controlling the output sensitivity of a large-scale language model is provided, comprising the steps of: repeatedly inputting an original sentence and a plurality of modified sentences of the original sentence into a large-scale language model to obtain, from the large-scale language model, a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences; generating a first summary sentence for the first output sentence and a second summary sentence for the second output sentence; generating a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences; calculating an output sensitivity evaluation value of the large-scale language model from the first and second similarity distributions; and adjusting parameters of the large-scale language model according to the output sensitivity evaluation value. Effects of the invention
[0015] According to one embodiment of the present invention, the output sensitivity of a large-scale language model is evaluated through a summary sentence generated from an output sentence of a large-scale language model, thereby improving the reliability of the results of the evaluation of the output sensitivity of a large-scale language model. Brief explanation of the drawing
[0017] FIG. 1 is a diagram illustrating a method for evaluating the output sensitivity of a large-scale language model according to an embodiment of the present invention. Figure 2 is a diagram illustrating the generation of a similarity distribution between summary sentences. FIG. 3 is a diagram illustrating a method for controlling the output sensitivity of a large-scale language model according to an embodiment of the present invention. Specific details for implementing the invention
[0018] The present invention is susceptible to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.
[0020] The aforementioned DBPA method collects output sentences by repeatedly calling multiple outputs generated by a large-scale language model for the original input sentence and modified variant input sentences derived from the original input sentence. Then, the collected output sentences are converted into embedding vectors, and the influence of changes in the input sentence is quantitatively evaluated by comparing the similarity distributions between the embedding vectors.
[0021] However, in the case of the DBPA method, differences in the representation style and length of output sentences are directly reflected in the process of generating embedding vectors, leading to a problem where discrepancies occur in similarity results even when the meaning of the output sentences is identical. These discrepancies arise from a combination of the diversity in the representation and length of output sentences and the influence of embedding models being trained primarily in English. Consequently, variations in the output sensitivity evaluation results for large-scale language models appear depending on the type of embedding model used, which can ultimately lead to a decrease in the consistency and reliability of the output sensitivity evaluation results.
[0022] Accordingly, the present invention proposes a method for evaluating the output sensitivity of a large-scale language model that can simultaneously resolve issues of expression variation in output sentences and language-specific consistency while maintaining the advantages of the existing DBPA method. Furthermore, the present invention proposes a method for improving output sensitivity that can reduce the output sensitivity of a large-scale language model by utilizing the evaluated output sensitivity of the large-scale language model.
[0023] One embodiment of the present invention generates a summary sentence summarizing the output sentence from the output sentence of a large-scale language model to solve the problems of expression variation and language-specific consistency of the output sentence, and calculates an output sensitivity evaluation value of the large-scale language model using this summary sentence. Accordingly, according to one embodiment of the present invention, the reliability of the output sensitivity evaluation result of a large-scale language model can be improved.
[0024] The output sensitivity evaluation method and output sensitivity adjustment method according to one embodiment of the present invention may be performed in a computing device comprising a memory and a processor electrically connected to the memory, and the processor may perform a series of processes for output sensitivity evaluation and output sensitivity adjustment.
[0025] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings.
[0027] FIG. 1 is a diagram illustrating a method for evaluating the output sensitivity of a large-scale language model according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating the generation of a similarity distribution between summary sentences.
[0028] Referring to FIG. 1, a computing device according to an embodiment of the present invention repeatedly inputs an original sentence and a plurality of modified sentences for the original sentence into a large-scale language model, and obtains a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences from the large-scale language model (S110). The large-scale language model receives a prompt containing the original sentence and a prompt containing each of the modified sentences, and can generate and output a sentence corresponding to each of the prompts.
[0029] As an example, a modified sentence may be a sentence in which the attributes of the original sentence are modified or information is omitted. Here, the attributes may be expressions or grammar, etc. For example, if the original sentence is “Tell me what the weather is like this afternoon,” modified sentences such as “Tell me what the weather is like this afternoon,” or “What is the weather like this afternoon?” may be generated. The modified sentence may be obtained through another large-scale language model, and a computing device may obtain the modified sentence by requesting the other large-scale language model to generate a modified sentence that has the same meaning as the original sentence.
[0030] And even if the same sentence is input, various output sentences can be generated due to the characteristics of a large-scale language model, so the computing device repeatedly inputs one original sentence and a modified sentence into the large-scale language model a number of times that is pre-set to obtain various first and second output sentences. For example, if there are 5 modified sentences for one original sentence and the number of times the input is 5, then 5 first output sentences and 5 second output sentences for each of the 5 modified sentences can be obtained.
[0031] At this time, the number of repeated inputs can be pre-set and may vary depending on the embodiment. As an example, the number of repeated inputs may be set proportionally to the length of the original sentence, set by the user, or set according to statistical criteria such as confidence level, effect size, and target.
[0032] The computing device generates a first summary sentence for a first output sentence and a second summary sentence for a second output sentence (S120). In one embodiment, the first and second summary sentences for each of the first and second output sentences can be generated using a pre-trained summary model. At this time, duplicate sentences may be included among the first and second summary sentences, and the duplicate sentences may be removed. In one embodiment, the summary model may be another large-scale language model trained to generate summary sentences for an input sentence.
[0033] Then, the computing device generates a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences (S130). As an example, the computing device may generate embedding vectors for the first and second summary sentences using a pre-trained embedding model, and calculate a first cosine similarity value between the first summary sentences and a second cosine similarity value between the first and second summary sentences using the embedding vectors. The first similarity distribution includes the first cosine similarity value, and the second similarity distribution includes the second cosine similarity value. As an example, the x-axis of the similarity distribution may represent the cosine similarity value, and the y-axis may represent the frequency of the corresponding cosine similarity value.
[0034] As an example, the computing device may generate first and second similarity distributions by defining a first set (210) containing a first summary sentence as an element and a second set (220) containing a second summary sentence as an element. At this time, the second set (220) may be generated for each modified sentence. As in the example described above, when five modified sentences are used, five second sets (220) may be generated, each containing a second summary sentence for each modified sentence. The computing device may select two first summary sentences from the first set (210) and calculate a first cosine similarity value for the selected pair of first summary sentences. The computing device may also calculate a second cosine similarity value between the first summary sentence selected from the first set (210) and the second summary sentence selected from each of the second sets (220). The second similarity distribution can be generated for each second set (220).
[0035] Then, the computing device calculates an output sensitivity evaluation value of a large-scale language model from the first and second similarity distributions (S140). In one embodiment, the computing device may calculate a JSD (Jensen-Shannon Divergence) value and a p-value for the JSD value as the output sensitivity evaluation value. The computing device may calculate the JSD value by calculating the mean distribution for the first and second similarity distributions from the first and second similarity distributions, and by averaging the Kullback-Leibler Divergence (KLD) values calculated between each of the first and second similarity distributions and the mean distribution. The JSD value has a value between 0 and 1. Then, the computing device may calculate a p-value, which is one of the statistical values, from the JSD value.
[0036] A higher JSD value indicates higher sensitivity of the output of the large-scale language model, while a smaller p-value indicates that variations in the input sentence had a significant impact on the output of the large-scale language model.
[0038] FIG. 3 is a diagram illustrating a method for controlling the output sensitivity of a large-scale language model according to an embodiment of the present invention.
[0039] A computing device according to one embodiment of the present invention calculates an output sensitivity evaluation value for a large-scale language model and adjusts the output sensitivity of the large-scale language model using the calculated output sensitivity evaluation value. The output sensitivity evaluation value can be calculated in various ways, and as an example, an output sensitivity evaluation value generated by the output sensitivity evaluation method described in FIG. 1 may be used.
[0040] That is, a computing device according to one embodiment of the present invention, as shown in FIG. 3, repeatedly inputs an original sentence and a plurality of modified sentences for the original sentence into a large-scale language model, thereby obtaining a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences from the large-scale language model (S310), and generates a first summary sentence for the first output sentence and a second summary sentence for the second output sentence (S320). Then, it generates a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences (S330), and calculates an output sensitivity evaluation value of the large-scale language model from the first and second similarity distributions (S340).
[0041] Then, the computing device adjusts parameters for the large-scale language model according to the output sensitivity evaluation value (S350). As an example, the parameters to be adjusted may be the temperature value or top-p value of the large-scale language model.
[0042] When a large-scale language model is used in a field requiring high reliability, the computing device may lower the temperature value or the top-p value to reduce the output sensitivity of the large-scale language model. Conversely, when a large-scale language model is used in a field requiring high creativity, the computing device may raise the temperature value or the top-p value to increase the output sensitivity of the large-scale language model.
[0043] The computing device can adjust the parameters of a large-scale language model so that the output sensitivity evaluation value of the large-scale language model becomes a target value. The target value can be determined based on the reliability or creativity required for the large-scale language model.
[0045] The technical details described above may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiments, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.
[0047] As described above, the present invention has been explained by specific details such as specific components, limited embodiments, and drawings; however, this is provided merely to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments. A person skilled in the art to which the invention pertains can make various modifications and variations from this description. Therefore, the scope of the invention should not be limited to the described embodiments, and all things equivalent to or having equivalent variations to the claims set forth below, as well as the claims themselves, shall be considered to fall within the scope of the concept of the invention.
Claims
Claim 1 A method for evaluating the output sensitivity of a large-scale language model, comprising: repeatedly inputting an original sentence and a plurality of modified sentences of the original sentence into a large-scale language model to obtain, from the large-scale language model, a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences; generating a first summary sentence for the first output sentence and a second summary sentence for the second output sentence; generating a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences; and calculating an output sensitivity evaluation value of the large-scale language model from the first and second similarity distributions, wherein the step of generating the first and second similarity distributions includes: generating an embedding vector for the first and second summary sentences; and using the embedding vector to calculate a cosine similarity value between the first summary sentences and a cosine similarity value between the first and second summary sentences. Claim 2 A method for evaluating the output sensitivity of a large-scale language model, wherein, in claim 1, the modified sentence is a sentence in which the attributes of the original sentence are modified or information is omitted. Claim 3 delete Claim 4 A method for evaluating the output sensitivity of a large-scale language model, wherein the step of calculating the output sensitivity evaluation value comprises: a step of calculating a JSD (Jensen-Shannon Divergence) value from the first and second similarity distributions; and a step of calculating a p-value for the JSD value. Claim 5 A method for evaluating the output sensitivity of a large-scale language model according to claim 1, wherein the step of generating the first and second summary sentences comprises: generating the first and second summary sentences for each of the first and second output sentences using a pre-trained summary model; and removing duplicate sentences among the first and second summary sentences. Claim 6 A device for evaluating the output sensitivity of a large-scale language model, comprising: a memory; and a processor electrically connected to the memory; wherein the processor repeatedly inputs an original sentence and a plurality of modified sentences for the original sentence into a large-scale language model, thereby obtaining a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences from the large-scale language model, generating a first summary sentence for the first output sentence and a second summary sentence for the second output sentence, generating a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences, and calculating an output sensitivity evaluation value of the large-scale language model from the first and second similarity distributions; wherein the processor generates embedding vectors for the first and second summary sentences and calculates a cosine similarity value between the first summary sentences and a cosine similarity value between the first and second summary sentences using the embedding vectors. Claim 7 The method comprises the steps of: repeatedly inputting an original sentence and a plurality of modified sentences of the original sentence into a large-scale language model to obtain, from the large-scale language model, a plurality of first output sentences for the original sentence and a plurality of second output sentences for the modified sentences; generating a first summary sentence for the first output sentence and a second summary sentence for the second output sentence; generating a first similarity distribution between the first summary sentences and a second similarity distribution between the first and second summary sentences; calculating an output sensitivity evaluation value of the large-scale language model from the first and second similarity distributions; and adjusting parameters for the large-scale language model according to the output sensitivity evaluation value, wherein the step of generating the first and second similarity distributions includes generating embedding vectors for the first and second summary sentences. A method for adjusting the output sensitivity of a large-scale language model, comprising the step of calculating a cosine similarity value between the first summary sentences and a cosine similarity value between the first and second summary sentences using the embedding vector. Claim 8 In claim 7, the above parameter is a temperature value or a top-p value, a method for controlling the output sensitivity of a large-scale language model. Claim 9 A method for adjusting the output sensitivity of a large-scale language model, wherein the modified sentence is a sentence in which the attributes of the original sentence are modified or information is omitted. Claim 10 A method for adjusting the output sensitivity of a large-scale language model, wherein the step of generating the first and second summary sentences comprises: generating the first and second summary sentences for each of the first and second output sentences using a pre-trained summary model; and removing duplicate sentences among the first and second summary sentences.
Citation Information
Patent Citations
Method and apparatus for identifying the intent of an input sentence using a lightweight sentence encoder
KR1020230156633A
Method and apparatus for generating summarized document using of sentence similarity relation predictive model, method and apparatus for learning predictive model used to generate summarized document
KR102600839B1
