Electronic device, method, and non-transitory computer-readable storage medium for training language model
A two-stage training process with weight application on score-derived probabilities refines language models to better align with user preferences, addressing inefficiencies and improving output quality.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- NCSOFT CORP
- Filing Date
- 2024-11-12
- Publication Date
- 2026-05-21
AI Technical Summary
Existing language models face challenges in optimizing their responses to align with user preferences, leading to inefficiencies in generating desired outputs.
A method involving a two-stage training process using first and second trained models, where weights are applied to scores derived from probability data to refine the second model, enhancing its alignment with user preferences.
The method improves the language model's ability to generate responses that better match user preferences, reducing overfitting and underfitting, and enhancing overall performance.
Smart Images
Figure KR2024017801_21052026_PF_FP_ABST
Abstract
Description
Electronic device, method, and non-transient computer-readable storage medium for training a language model
[0001] The following descriptions relate to an electronic device, a method, and a non-transient computer-readable storage medium for training a language model.
[0002] A language model can be used for natural language interpretation. For example, the language model may receive natural language-based input (e.g., a user query). For example, the language model may generate an output by performing natural language interpretation in relation to the natural language-based input. For example, the language model may be trained to generate the output containing a response preferred by the user. For example, the language model may be trained through preference optimization.
[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.
[0004] An electronic device is provided. The electronic device may include a memory that stores instructions and includes one or more storage media. The electronic device may include at least one processor that includes a processing circuit. The instructions may cause the electronic device to identify first probability data obtained by applying input data to a first trained model usable for natural language processing when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to identify a first score according to a value within the first probability data when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to identify second probability data obtained by applying the input data to a second trained model obtained based on the first trained model when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to identify a second score according to a value in the second probability data when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to identify a first weight to be applied to the first score according to a value of the first score when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to further train the second trained model based on applying the first score to which the first weight is applied and the second score to which the second weight is applied when executed individually or collectively by the at least one processor.
[0005] A non-transient computer-readable storage medium is provided. The non-transient computer-readable storage medium may store one or more programs. The one or more programs may cause the electronic device to identify first probability data obtained by applying input data to a first trained model usable for natural language processing when executed by the electronic device. The one or more programs may cause the electronic device to identify a first score according to a value in the first probability data when executed by the electronic device. The one or more programs may cause the electronic device to identify second probability data obtained by applying the input data to a second trained model obtained based on the first trained model when executed by the electronic device. The one or more programs may cause the electronic device to identify a second score according to a value in the second probability data when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to identify a first weight to be applied to the first score according to the value of the first score when executed by the electronic device. The above one or more programs may include instructions that cause the electronic device to further train the second trained model based on the first score to which the first weight is applied and the second score to which the second weight is applied when executed by the electronic device.
[0006] A method is provided. The method may be executed within an electronic device. The method may include an operation of identifying first probability data obtained by applying input data to a first trained model usable for natural language processing. The method may include an operation of identifying a first score according to a value in the first probability data. The method may include an operation of identifying second probability data obtained by applying the input data to a second trained model obtained based on the first trained model. The method may include an operation of identifying a second score according to a value in the second probability data. The method may include an operation of identifying a first weight to be applied to the first score according to a value of the first score. The method may include an operation of further training the second trained model based on the first score to which the first weight is applied and the second score to which the second weight is applied.
[0007] Figure 1 is a schematic view of an exemplary electronic device.
[0008] Figure 2 illustrates an example of a training environment for trained models usable for natural language processing.
[0009] Figure 3 is a flowchart illustrating a method to further train a trained model by applying weights to scores obtained from the trained model.
[0010] Figure 4 illustrates an example where weights are identified according to the score obtained from the trained model.
[0011] Figures 5 and 6 illustrate an example in which the value of the loss function is identified according to the score obtained from the trained model.
[0012] Figure 1 is a schematic view of an exemplary electronic device.
[0013] Referring to FIG. 1, the electronic device (101) may include at least one processor (110) and a memory (120).
[0014] At least one processor (110) may include a processing circuit. At least one processor (110) may include a single processor or multiple processors. At least one processor (110) may control the memory (120) and / or one or more components (not shown) of the electronic device (101).
[0015] The memory (120) may store instructions configured to be executed individually and / or collectively by at least one processor (110). The instructions may cause the electronic device (101) to perform operations described with reference to FIGS. 2 through 6. The memory (120) may include one or more storage media.
[0016] Figure 2 illustrates an example of a training environment for trained models usable for natural language processing.
[0017] Referring to FIG. 2, the training environment (200) may include a first trained model (201) and a second trained model (202).
[0018] The first trained model (201) and the second trained model (202) may represent one or more calculations to be performed by at least one processor (110) of FIG. 1. For example, the first trained model (201) and / or the second trained model (202) may include a computational model designed to simulate the neural activity of an organism, a program for performing calculations of said computational model, and / or at least one processor (110) for executing said program.
[0019] The first trained model (201) and the second trained model (202) may be language models usable for natural language processing. For example, the first trained model (201) may correspond to a model that has been trained at least once. The second trained model (202) may be obtained based on the first trained model (201). For example, the second trained model (202) may correspond to a model in which the first trained model (201) has been further trained. For example, an electronic device (101) may provide the second trained model (202) by further training the first trained model (201) through a process (211). The process (211) may include a preference optimization process for training the output data of a language model to increase the probability that it corresponds to a pre-set preferred response and to decrease the probability that the output data corresponds to a pre-set dis-preferred response. However, it is not limited thereto. In FIG. 2, the first trained model (201) and the second trained model (202) may be understood as being distinguished as different language models, or as being a single language model distinguished temporally according to the progress of training.
[0020] The first trained model (201) can obtain first output data (y1) in response to input data (x). The second training model (202) can obtain second output data (y2) in response to input data (x). Each of the input data (x), the first output data (y1), and the second output data (y2) may include at least one token corresponding to a unit that can be output through the first trained model (201) and the second trained model (202). For example, a token may be described as a basic unit in which a language model processes input natural language and generates output natural language. For example, a token may be described as a set of numerical values (e.g., a vector) representing a morpheme and / or a word.
[0021] The electronic device (101) can identify first probability data obtained by applying input data (x) to a first trained model (201). The first probability data can be identified to obtain first output data (y1) output from the first trained model (201) based on the input data (x). For example, the first probability data may represent a value for each token that can be output through the first trained model (201) when the input data (x) is applied to the first trained model (201). For example, the value of the first probability data may represent the probability that the outputtable token can effectively respond to the input data (x) when the input data (x) is applied to the first trained model (201). For example, the electronic device (101) can identify the outputtable token of the first trained model (201), identified as having a maximum value within the first probability data, as the first output data (y1) output from the first trained model (201).
[0022] For example, a first trained model (201) may receive input data (x) corresponding to the natural language 'Who are you'. The input data (x) may include the respective tokens 'Who', 'are', and 'you'. An electronic device (101) may identify the first probability data based on the input data (x) applied to the first trained model (201). The electronic device (101) may identify the outputtable token 'I', identified as having the maximum value within the first probability data, as the first output data (y1). The first trained model (201) may receive input data (x) including the tokens 'Who', 'are', 'you', and 'I'. The electronic device (101) can identify the first probability data based on the applied input data (x) and identify the outputtable token 'am', identified as having the maximum value within the first probability data, as the first output data (y1). As described above, the operation of feeding back the output data to the input data may be stopped if the outputtable token identified as having the maximum value within the probability data corresponds to a pre-set token (e.g., an EoS (End of Sentence) token). The above example is for the purpose of facilitating understanding of the present disclosure and is not limited thereto.
[0023] The electronic device (101) can identify second probability data obtained by applying input data (x) to the second trained model (202). The second probability data can be identified to obtain second output data (y2) output from the second trained model (202) based on the input data (x). For example, the second probability data may represent a value for each token that can be output through the second trained model (202) when the input data (x) is applied to the second trained model (202). For example, the value of the second probability data may represent the probability that the outputtable token can effectively respond to the input data (x) when the input data (x) is applied to the second trained model (202). For example, the electronic device (101) may identify the outputtable token of the second trained model (202), identified as having a maximum value within the second probability data, as the second output data (y2) output from the second trained model (202). The method of operation in which the second trained model (202) outputs the output data may be the same as the method of operation in which the first training model (201) outputs the output data.
[0024] The electronic device (101) can identify a first score obtained from a first trained model (201) and a second score obtained from a second trained model (202) in order to further train a second trained model (202).
[0025] The electronic device (101) can identify the first score based on the first probability data obtained from the first trained model (201) while obtaining the first output data (y1) from the input data (x) using the first trained model (201). For example, the electronic device (101) can identify the first score based on the value within the first probability data corresponding to the first truth data and the second truth data. For example, each of the first truth data and the second truth data may correspond to the input data (x) and may include at least one token. For example, the first truth data may correspond to a preset preferred response that varies according to the input data (x). The second truth data may correspond to a preset dis-preferred response that varies according to the input data (x). However, it is not necessarily limited thereto.
[0026] The electronic device (101) can identify the first score based on a value corresponding to the difference between the first probability according to the first probability data and the first truth data, and the second probability according to the first probability data and the second truth data. For example, the electronic device (101) can identify the value corresponding to the difference between the first average probability obtained from the first probability and the second average probability obtained from the second probability as the value of the first score. For example, the average probability can be calculated to compensate for or normalize the effect of the number of tokens on the first probability and / or the second probability.
[0027]
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
[0035]
[0036] For example, since each of the first probability and the second probability is expressed as the product of the values corresponding to the truth data within the probability data, the numerical range being compared may vary significantly depending on the number of tokens in the truth data. The electronic device (101) can mitigate the variation in the numerical range being compared even if the number of tokens in the first truth data and the second truth data are different by identifying the value of the first score using the first average probability and the second average probability.
[0037] The electronic device (101) can identify the second score based on the second probability data obtained from the second trained model (202) while obtaining the second output data (y2) from the input data (x) using the second trained model (202). For example, the electronic device (101) can identify the second score based on the values within the second probability data corresponding to the first truth data and the second truth data.
[0038] The electronic device (101) can identify the second score based on a value corresponding to the difference between the third probability according to the second probability data and the first truth data, and the fourth probability according to the second probability data and the second truth data. For example, the electronic device (101) can identify the value corresponding to the difference between the third average probability obtained from the third probability and the fourth average probability obtained from the fourth probability as the value of the second score.
[0039]
[0040]
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
[0047] The electronic device (101) can further train a second trained model (202) based on applying a first weight to the first score and applying a second weight to the second score.
[0048] The electronic device (101) can identify the first weight to be applied to the first score according to the value of the first score. The electronic device (101) can identify the first weight having a higher value as the value of the first score increases. For example, the electronic device (101) can identify the first weight as a second value according to the value of the first score which is a first value. The electronic device (101) can identify the first weight as a fourth value which is higher than the second value according to the value of the first score which is a third value which is higher than the first value.
[0049] For example, the electronic device (101) can control the influence of the first trained model (201) on the training process of the second trained model (202) by adjusting the value of the first weight. For example, if the value of the first weight is relatively high, the influence of the first trained model (201) on the training process of the second trained model (202) may be relatively large. For another example, if the value of the first weight is relatively low, the influence of the first trained model (201) on the training process of the second trained model (202) may be relatively small.
[0050]
[0051]
[0052]
[0053]
[0054]
[0055]
[0056]
[0057]
[0058] The electronic device (101) can identify the value of the loss function based on the difference between the first score to which the first weight is applied and the second score to which the second weight is applied. For example, the electronic device (101) can identify a seventh value by applying the first weight to the first score. The electronic device (101) can identify an eighth value by applying the second weight to the second score. The electronic device (101) can identify the value corresponding to the difference between the seventh value and the eighth value as the value of the loss function. The electronic device (101) can further train the second trained model (202) based on the identified value of the loss function.
[0059]
[0060]
[0061]
[0062]
[0063]
[0064]
[0065]
[0066]
[0067]
[0068]
[0069]
[0070] Referring to Table 1 below, a table showing the evaluation values of a trained model to which the training method according to the present disclosure is applied is shown.
[0071] 'ORPO (Odds Ratio Preference Optimization)', 'KTO (Kahneman-Tversky Optimization)', 'SimPO (Simple Preference Optimization)', 'DPO (Direct Preference Optimization)', and 'MIPO (Modulated Intervention Preference Optimization)' each represent different training methods, and 'MIPO' may represent a training method according to the present disclosure.
[0072] 'Model 1', 'Model 2', and 'Model 3' each may represent different language models to which the above training methods are applied. 'Set 1' and 'Set 2' each may represent different data sets (e.g., the input data, the first truth data, and the second truth data).
[0073] 'WR (win rate)' may refer to an evaluation value regarding the extent to which the language model to which the training methods are applied outputs the second truth data without outputting the first truth data when compared with a reference language model (e.g., Chat GPT). 'LC (length controlled win rate)' may refer to a 'WR (win rate)' that takes into account the number of tokens of the first truth data and the second truth data when compared with the reference language model. 'Avg.' may correspond to an evaluation value identified by a specific language model (e.g., Chat GPT) by evaluating whether the language model to which the training methods are applied outputs valid data in response to a pre-set dataset.
[0074] Referring to Table 1 below, the training method according to the present disclosure (MIPO) explains that, when compared to other training methods (ORPO, KTO, SimPO, DPO), the training method according to the present disclosure has the most evaluation items identified as having the highest evaluation values. When compared to the training method (DPO), the training method according to the present disclosure (MIPO) has improved evaluation values in all evaluation items.
[0075]
[0076] Figure 3 is a flowchart illustrating a method to further train a trained model by applying weights to scores obtained from the trained model.
[0077] Referring to FIG. 3, in operation 301, the electronic device (101) can identify the first probability data obtained by applying input data (x) to the first trained model (201). For example, as described above, the first probability data can represent values for each token that can be output through the first trained model (201) when input data (x) is applied to the first trained model (201).
[0078]
[0079] In operation 303, the electronic device (101) can identify the second probability data obtained by applying input data (x) to the second trained model (202). For example, as described above, the second probability data may represent values for each token that can be output through the second trained model (202) when input data (x) is applied to the second trained model (202).
[0080]
[0081]
[0082]
[0083] Figure 4 illustrates an example where weights are identified according to the score obtained from the trained model.
[0084]
[0085]
[0086]
[0087]
[0088] Figures 5 and 6 illustrate an example in which the value of the loss function is identified according to the score obtained from the trained model.
[0089]
[0090] The graph (501) is illustrated to aid in understanding the graph (502) and may indicate that the output value (Loss) of the loss function corresponds to a constant.
[0091]
[0092]
[0093]
[0094]
[0095] According to one embodiment, an electronic device can train a language model. For example, the electronic device may require a method of training by taking into account the existing training state of the language model.
[0096] The technical problems to be solved in this document are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this invention belongs from the description below.
[0097] An electronic device (e.g., electronic device (101)) as described above may include a memory (e.g., memory (120)) that stores instructions and includes one or more storage media; and at least one processor (e.g., at least one processor (110)) that includes a processing circuit. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to: identify first probability data obtained by applying input data to a usable first trained model, identify a first score according to a value in the first probability data, identify second probability data obtained by applying the input data to a second trained model obtained based on the first trained model, identify a second score according to a value in the second probability data, identify a first weight to be applied to the first score according to a value in the first score, and further train the second trained model based on the first score to which the first weight is applied and the second score to which the second weight is applied.
[0098] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may cause the second trained model to be further trained based on applying the first weight to the first score and applying the second weight to the second score, which is identified as a second value according to the value of the first score which is a first value, and to further train the second trained model based on applying the first weight to the first score and applying the second weight to the second score, which is identified as a fourth value higher than the second value according to the value of the first score which is a third value higher than the first value.
[0099] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify the value of the first weight that converges to a fifth value as the value of the first score decreases, and identify the value of the first weight that converges to a sixth value as the value of the first score increases. The value of the second weight may be the same as the sixth value.
[0100] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify the value of the first weight as a fifth value according to the value of the first score lower than the threshold value, and identify the value of the first weight as a sixth value according to the value of the first score higher than the threshold value.
[0101] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to: identify a seventh value by applying the first weight to the first score, identify an eighth value by applying the second weight to the second score, identify a value corresponding to the difference between the seventh value and the eighth value as the value of the loss function, and further train the second trained model based on the value of the identified loss function.
[0102] The above instructions may cause the electronic device to be executed individually or collectively by the at least one processor: when the input data is applied to the first trained model, the first probability data representing the value for each token that can be output through the first trained model is identified, and the token that can be output identified as having a maximum value within the first probability data is identified as output data output from the first trained model; and when the input data is applied to the second trained model, the second probability data representing the value for each token that can be output through the second trained model is identified, and the token that can be output identified as having a maximum value within the second probability data is identified as output data output from the second trained model.
[0103] The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to identify a value corresponding to a first truth data within the acquired first probability data whenever the first probability data is acquired in the first trained model, and to identify the value identified through an operation on the values corresponding to the first truth data within the acquired first probability data as a first probability, and whenever the first probability data is acquired in the first trained model, to identify a value corresponding to a second truth data within the acquired first probability data, and to identify the value identified through an operation on the values corresponding to the second truth data within the acquired first probability data as a second probability, and to identify the first score based on a value corresponding to the difference between the first probability and the second probability.
[0104] Each of the above first truth data and the above second truth data includes at least one token, and when the instructions are executed individually or collectively by the above at least one processor: the electronic device may be configured to identify a value corresponding to the value obtained by dividing the first probability by the number of tokens included in the first truth data as a first average probability, identify a value corresponding to the value obtained by dividing the second probability by the number of tokens included in the second truth data as a second average probability, and identify a value corresponding to the difference between the first average probability and the second average probability as the value of the first score.
[0105] The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to identify a value corresponding to the first truth data within the acquired second probability data whenever the second probability data is acquired in the second trained model, and to identify the value identified through the operation on the values corresponding to the first truth data within the acquired second probability data as a third probability, and whenever the second probability data is acquired in the second trained model, to identify a value corresponding to the second truth data within the acquired second probability data, and to identify the value identified through the operation on the values corresponding to the second truth data within the acquired second probability data as a fourth probability, and to identify the second score based on a value corresponding to the difference between the third probability and the fourth probability.
[0106] When the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify a value corresponding to the value obtained by dividing the third probability by the number of tokens included in the first truth data as the third average probability, identify a value corresponding to the value obtained by dividing the fourth probability by the number of tokens included in the second truth data as the fourth average probability, and identify a value corresponding to the difference between the third average probability and the fourth average probability as the value of the second score.
[0107] A non-transient computer-readable storage medium as described above may store one or more programs. The one or more programs may include instructions that, when executed by an electronic device (e.g., electronic device (101)), identify first probability data obtained by applying input data to a first trained model usable for natural language processing, identify a first score according to a value in the first probability data, identify second probability data obtained by applying the input data to a second trained model obtained based on the first trained model, identify a second score according to a value in the second probability data, identify a first weight to be applied to the first score according to a value in the first score, and cause the electronic device to further train the second trained model based on the first score with the first weight and the second score with the second weight.
[0108] The above one or more programs may include instructions that cause the electronic device to further train the second trained model based on applying the first weight to the first score and the second weight to the second score, which is identified as a second value according to the value of the first score which is a first value, when executed by the electronic device, and further train the second trained model based on applying the first weight to the first score and the second weight to the second score, which is identified as a fourth value higher than the second value according to the value of the first score which is a third value higher than the first value.
[0109] The above one or more programs may include instructions that cause the electronic device to identify the value of the first weight that converges to a fifth value as the value of the first score decreases when executed by the electronic device, and to identify the value of the first weight that converges to a sixth value as the value of the first score increases.
[0110] The above one or more programs may include instructions that cause the electronic device to identify the value of the first weight as a fifth value according to the value of the first score lower than the threshold value when executed by the electronic device, and to identify the value of the first weight as a sixth value according to the value of the first score higher than the threshold value.
[0111] The above one or more programs may include instructions that cause the electronic device to identify, whenever the first probability data is obtained from the first trained model when executed by the electronic device, a value corresponding to the first truth data within the obtained first probability data, and a value identified through an operation on the values corresponding to the first truth data within the obtained first probability data as a first probability, and whenever the first probability data is obtained from the first trained model, a value corresponding to the second truth data within the obtained first probability data, and a value identified through an operation on the values corresponding to the second truth data within the obtained first probability data as a second probability, and to identify the first score based on a value corresponding to the difference between the first probability and the second probability.
[0112] Each of the first truth data and the second truth data comprises at least one token, and the one or more programs may include instructions that cause the electronic device to identify a value corresponding to the value obtained by dividing the first probability by the number of tokens included in the first truth data as a first average probability, identify a value corresponding to the value obtained by dividing the second probability by the number of tokens included in the second truth data as a second average probability, and identify a value corresponding to the difference between the first average probability and the second average probability as the value of the first score.
[0113] The above one or more programs may include instructions that cause the electronic device to identify, whenever the second probability data is acquired in the second trained model, a value corresponding to the first truth data within the acquired second probability data, and a value identified through the operation on the values corresponding to the first truth data within the acquired second probability data as a third probability, and whenever the second probability data is acquired in the second trained model, a value corresponding to the second truth data within the acquired second probability data, and a value identified through the operation on the values corresponding to the second truth data within the acquired second probability data as a fourth probability, and to identify the second score based on a value corresponding to the difference between the third probability and the fourth probability.
[0114] The above one or more programs may include instructions that cause the electronic device, when executed by the electronic device, to identify a value corresponding to the value obtained by dividing the third probability by the number of tokens included in the first truth data as the third average probability, identify a value corresponding to the value obtained by dividing the fourth probability by the number of tokens included in the second truth data as the fourth average probability, and identify a value corresponding to the difference between the third average probability and the fourth average probability as the value of the second score.
[0115] A method executed within an electronic device (e.g., electronic device (101)) as described above may include: identifying first probability data obtained by applying input data to a first trained model usable for natural language processing; identifying a first score according to a value in the first probability data; identifying second probability data obtained by applying the input data to a second trained model obtained based on the first trained model; identifying a second score according to a value in the second probability data; identifying a first weight to be applied to the first score according to a value in the first score; and further training the second trained model based on the first score to which the first weight is applied and the second score to which the second weight is applied.
[0116] According to one embodiment, an electronic device can train a language model. For example, the electronic device can train the language model by taking into account the existing training state of the language model. For example, by training the language model by taking into account the existing training state of the language model, the electronic device can mitigate overfitting and / or underfitting of additional training performed on the language model.
[0117] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.
[0118] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.
[0119] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.
[0120] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0121] Various embodiments of this document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., internal memory or external memory) readable by a machine (e.g., an electronic device). For example, a processor (e.g., a processor) of the machine (e.g., an electronic device) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.
[0122] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.
[0123] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
Claims
1. In an electronic device, Memory that stores instructions and includes one or more storage media; and It includes at least one processor comprising a processing circuit, and When the above instructions are executed individually or collectively by the at least one processor: Identify first probability data obtained by applying input data to a first trained model usable for natural language processing, and Identifying a first score according to a value within the first probability data above, and Identifying second probability data obtained by applying the input data to the second trained model obtained based on the first trained model, and Identifying a second score based on the value within the second probability data above, and Identifying a first weight to be applied to the first score according to the value of the first score, and Based on the first score with the first weight applied and the second score with the second weight applied, to further train the second trained model, The above electronic device, causing, Electronic device.
2. In claim 1, when the instructions are executed individually or collectively by at least one processor: Based on applying the first weight to the first score and applying the second weight to the second score, which are identified as the second value according to the value of the first score which is the first value, the second trained model is further trained, and To further train the second trained model based on applying the first weight to the first score and applying the second weight to the second score, which is identified as a fourth value higher than the second value according to the value of the first score, which is a third value higher than the first value. The above electronic device, causing, Electronic device.
3. In claim 1, when the instructions are executed individually or collectively by at least one processor: Identify the value of the first weight that converges to the fifth value as the value of the first score decreases, and To identify the value of the first weight that converges to the sixth value as the value of the first score increases, The above electronic device, causing, Electronic device.
4. In claim 3, the value of the second weight is, Identical to the 6th value above, Electronic device.
5. In claim 1, when the instructions are executed individually or collectively by at least one processor: Based on the value of the first score that is lower than the threshold value, the value of the first weight is identified as the fifth value, and According to the value of the first score that is higher than the threshold value, the value of the first weight is identified as the sixth value. The above electronic device, causing, Electronic device.
6. In claim 1, when the instructions are executed individually or collectively by at least one processor: Identifying the seventh value by applying the first weight to the first score, and Identify the 8th value by applying the 2nd weight to the 2nd score, and The value corresponding to the difference between the above 7th value and the above 8th value is identified as the value of the loss function, and Based on the value of the identified loss function, to further train the second trained model, The above electronic device, causing, Electronic device.
7. In claim 1, when the instructions are executed individually or collectively by at least one processor: When the input data is applied to the first trained model, the first probability data representing the value for each token that can be output through the first trained model is identified, and Identifying the outputtable token identified as having a maximum value within the first probability data as output data output from the first trained model, and When the input data is applied to the second trained model, the second probability data representing the value for each token that can be output through the second trained model is identified, and To identify the outputtable token identified as having a maximum value within the second probability data as output data output from the second trained model, The above electronic device, causing, Electronic device.
8. In claim 1, when the instructions are executed individually or collectively by at least one processor: Whenever the first probability data is obtained from the first trained model, a value corresponding to the first truth data is identified within the obtained first probability data, and Identifying a value identified through an operation on the values corresponding to the first truth data within the first probability data obtained above as the first probability, and Whenever the first probability data is obtained from the first trained model, a value corresponding to the second truth data is identified within the obtained first probability data, and Identifying the value identified through the operation on the values corresponding to the second truth data within the first probability data obtained above as the second probability, and To identify the first score based on a value corresponding to the difference between the first probability and the second probability, The above electronic device, causing, Electronic device.
9. In claim 8, each of the first truth data and the second truth data is, Includes at least one token, When the above instructions are executed individually or collectively by the at least one processor: The value corresponding to the value obtained by dividing the above first probability by the number of tokens included in the above first truth data is identified as the first average probability, and The value corresponding to the value obtained by dividing the above second probability by the number of tokens included in the above second truth data is identified as the second average probability, and To identify the value corresponding to the difference between the first average probability and the second average probability as the value of the first score, The above electronic device, causing, Electronic device.
10. In claim 8, when the instructions are executed individually or collectively by at least one processor: Whenever the second probability data is obtained from the second trained model, a value corresponding to the first truth data is identified within the obtained second probability data, and Identifying the value identified through the operation on the values corresponding to the first truth data within the second probability data obtained above as the third probability, and Whenever the second probability data is obtained from the second trained model, a value corresponding to the second truth data is identified within the obtained second probability data, and Identifying the value identified through the operation on the values corresponding to the second truth data within the second probability data obtained above as the fourth probability, and To identify the second score based on the value corresponding to the difference between the third probability and the fourth probability, The above electronic device, causing, Electronic device.
11. In claim 10, when the instructions are executed individually or collectively by at least one processor: The value corresponding to the value obtained by dividing the above third probability by the number of tokens included in the above first truth data is identified as the third average probability, and The value corresponding to the value obtained by dividing the above fourth probability by the number of tokens included in the above second truth data is identified as the fourth average probability, and To identify the value corresponding to the difference between the third average probability and the fourth average probability as the value of the second score, The above electronic device, causing, Electronic device.
12. In a non-transient computer-readable storage medium storing one or more programs, said one or more programs are, When executed by an electronic device, Identify first probability data obtained by applying input data to a first trained model usable for natural language processing, and Identifying a first score according to a value within the first probability data above, and Identifying second probability data obtained by applying the input data to the second trained model obtained based on the first trained model, and Identifying a second score based on the value within the second probability data above, and Identifying a first weight to be applied to the first score according to the value of the first score, and Based on the first score with the first weight applied and the second score with the second weight applied, to further train the second trained model, Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
13. In claim 12, the one or more programs, when executed by the electronic device, Based on applying the first weight to the first score and applying the second weight to the second score, which are identified as the second value according to the value of the first score which is the first value, the second trained model is further trained, and To further train the second trained model based on applying the first weight to the first score and applying the second weight to the second score, which is identified as a fourth value higher than the second value according to the value of the first score, which is a third value higher than the first value. Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
14. In claim 12, the one or more programs, when executed by the electronic device, Identify the value of the first weight that converges to the fifth value as the value of the first score decreases, and To identify the value of the first weight that converges to the sixth value as the value of the first score increases, Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
15. In claim 12, the one or more programs, when executed by the electronic device, Based on the value of the first score that is lower than the threshold value, the value of the first weight is identified as the fifth value, and According to the value of the first score that is higher than the threshold value, the value of the first weight is identified as the sixth value. Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
16. In Claim 12, When the above one or more programs are executed by the electronic device, Whenever the first probability data is obtained from the first trained model, a value corresponding to the first truth data is identified within the obtained first probability data, and Identifying a value identified through an operation on the values corresponding to the first truth data within the first probability data obtained above as the first probability, and Whenever the first probability data is obtained from the first trained model, a value corresponding to the second truth data is identified within the obtained first probability data, and Identifying the value identified through the operation on the values corresponding to the second truth data within the first probability data obtained above as the second probability, and To identify the first score based on a value corresponding to the difference between the first probability and the second probability, Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
17. In claim 16, each of the first truth data and the second truth data is, Includes at least one token, When the above one or more programs are executed by the electronic device, The value corresponding to the value obtained by dividing the above first probability by the number of tokens included in the above first truth data is identified as the first average probability, and The value corresponding to the value obtained by dividing the above second probability by the number of tokens included in the above second truth data is identified as the second average probability, and To identify the value corresponding to the difference between the first average probability and the second average probability as the value of the first score, Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
18. In claim 16, the one or more programs, when executed by the electronic device, Whenever the second probability data is obtained from the second trained model, a value corresponding to the first truth data is identified within the obtained second probability data, and Identifying the value identified through the operation on the values corresponding to the first truth data within the second probability data obtained above as the third probability, and Whenever the second probability data is obtained from the second trained model, a value corresponding to the second truth data is identified within the obtained second probability data, and Identifying the value identified through the operation on the values corresponding to the second truth data within the second probability data obtained above as the fourth probability, and To identify the second score based on the value corresponding to the difference between the third probability and the fourth probability, Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
19. In claim 18, the one or more programs, when executed by the electronic device, The value corresponding to the value obtained by dividing the above third probability by the number of tokens included in the above first truth data is identified as the third average probability, and The value corresponding to the value obtained by dividing the above fourth probability by the number of tokens included in the above second truth data is identified as the fourth average probability, and To identify the value corresponding to the difference between the third average probability and the fourth average probability as the value of the second score, Instructions including those that cause the above electronic device Non-transient computer-readable storage media.
20. In a method executed within an electronic device, An operation of identifying first probability data obtained by applying input data to a first trained model usable for natural language processing, The operation of identifying a first score according to a value within the first probability data above, The operation of identifying second probability data obtained by applying the input data to the second trained model obtained based on the first trained model, The operation of identifying a second score according to a value within the second probability data above, An operation to identify a first weight to be applied to the first score according to the value of the first score, A method comprising further training the second trained model based on the first score to which the first weight is applied and the second score to which the second weight is applied, method.