Large language model-oriented lexical element management method, electronic equipment and medium

Abstract

The invention discloses a lexical element management method for a large language model, electronic equipment and a medium, and the method comprises the steps: calculating the loss corresponding to each lexical element in an input sequence in a large language model training stage, and obtaining a first loss sequence; smoothing the first loss sequence to obtain a second loss sequence; sorting loss values in the second loss sequence, and screening current candidate abnormal lexical elements; combining the current candidate abnormal lexical elements with a global abnormal lexical element pool corresponding to the previous training round, and cutting according to a pool size threshold value to obtain a global abnormal lexical element pool corresponding to the current training round; in the reasoning stage of the large language model, judging whether the candidate lexical elements are abnormal lexical elements or not according to the global abnormal lexical element pool; when an abnormal lexical element is detected, generating a plurality of candidate output sequences by taking the abnormal lexical element as a node, and calculating the average uncertainty of each candidate output sequence; and selecting the path with the minimum average uncertainty so as to obtain an output sequence.

Description

Technical Field

[0001] This invention belongs to the field of large language model technology, and particularly relates to a lexical management method, electronic device, and medium for large language models. Background Technology

[0002] In recent years, large-scale language models (LLMs) have become the foundational architecture of the natural language processing field, forming a complete technical paradigm of "pre-training - supervised fine-tuning (SFT) - instruction alignment - inference generation". However, with the continuous growth of model size and context length, the management and optimization of tokens has become increasingly prominent, becoming a key bottleneck restricting model training efficiency and generation quality.

[0003] During the fine-tuning or alignment training phase, large language models perform word-by-word prediction on the input sequence, and their word-level loss exhibits a significant long-tail distribution. Specifically, although high-perplexity or high-loss words account for a relatively small proportion of the overall distribution, they cause significant perturbations to gradient updates, easily leading to training instability, model parameter shifts, and catastrophic "forgetting." Existing fine-tuning methods mostly employ static strategies such as masking, discarding, or adding noise to handle high-loss words, but these methods have fundamental flaws: on the one hand, they simply and crudely discard words that may result in the loss of key semantic information; on the other hand, they lack a dynamic evaluation mechanism for word importance, making it impossible to adaptively adjust optimization strategies at different training stages. Although Parameter Efficient Fine-Tuning (PEFT) techniques such as LoRA can alleviate some of the problems, research has confirmed that they still cannot avoid catastrophic forgetting, and fine-tuning performance is strongly negatively correlated with the amount of forgetting.

[0004] In the inference generation stage, large language models employ probabilistic sampling (such as top-k, top-p) or greedy / beam search for sequence decoding. However, lexical-level prediction uncertainty—manifested as high-entropy or high-variance distribution—is often directly related to erroneous outputs. Research indicates that lexical-level uncertainty estimation can effectively identify weak points in the generation process, providing a theoretical basis for dynamic intervention. Summary of the Invention

[0005] To address the shortcomings of existing technologies, embodiments of the present invention provide a lexical management method, electronic device, and medium for large language models.

[0006] In a first aspect, embodiments of the present invention provide a lexical management method for large language models, the method comprising the following steps:

[0007] During the training phase of the large language model, the loss corresponding to each word in the input sequence is calculated to obtain the first loss sequence; the first loss sequence is smoothed to obtain the second loss sequence; the loss values ​​in the second loss sequence are sorted to filter the current candidate abnormal words; the current candidate abnormal words are merged with the global abnormal word pool corresponding to the previous training round and pruned according to the pool size threshold to obtain the global abnormal word pool corresponding to the current training round.

[0008] During the inference phase of the large language model, the candidate lexicon is determined to be an abnormal lexicon based on the global abnormal lexicon pool. When an abnormal lexicon is detected, several candidate output sequences are generated with the abnormal lexicon as the node, and the average uncertainty of each candidate output sequence is calculated. The path with the minimum average uncertainty is selected to obtain the output sequence.

[0009] In a second aspect, embodiments of the present invention provide an electronic device, comprising:

[0010] At least one processor; and

[0011] A memory communicatively connected to the at least one processor; wherein,

[0012] The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described lexical management method for large language models.

[0013] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described lexical management method for large language models.

[0014] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-described lexical management method for large language models.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] This invention provides a lexical management method for large language models. During the training phase of the large language model, anomalous lexical units are identified, and a global anomalous lexical unit pool is constructed. During the inference phase of the large language model, the global anomalous lexical unit pool is used to determine whether candidate lexical units are anomalous, and the output sequence is selected through probabilistic uncertainty. This invention avoids the information loss caused by simply discarding high-loss lexical units and overcomes the problem that high-uncertainty lexical units in the inference phase can easily lead to hallucinations or erroneous outputs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a lexical management method for large language models provided in an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of a lexical management method for large language models provided in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram illustrating the abnormal word update process during training, as provided in an embodiment of the present invention.

[0021] Figure 4 A schematic diagram of one-dimensional Gaussian convolution smoothing provided for an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram illustrating branch generation and aggregation during the reasoning process provided in an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram illustrating the updating of the abnormal word pool during the inference process provided in an embodiment of the present invention;

[0024] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0027] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a lexical management method for large language models, the method comprising:

[0028] Step S1: During the large language model training phase, calculate the loss corresponding to each word in the input sequence to obtain the first loss sequence; smooth the first loss sequence to obtain the second loss sequence; sort the loss values ​​in the second loss sequence and select the top r% of words as the current candidate abnormal words; merge the current candidate abnormal words with the global abnormal word pool corresponding to the previous training round and prune them according to the pool size threshold to obtain the global abnormal word pool corresponding to the current training round t.

[0029] Specifically, step S1 includes the following sub-steps:

[0030] Step S101: Calculate the first loss value corresponding to each word in the input sequence to obtain the first loss sequence. .

[0031] The first loss value is either cross-entropy loss or negative log-likelihood value.

[0032] Step S102: Smooth the first loss sequence to obtain the second loss sequence. , .

[0033] Specifically, such as Figure 4 As shown, a one-dimensional convolutional smoothing filter is applied to the first loss sequence to suppress noise fluctuations between adjacent words and preserve local structural features, thereby obtaining the smoothed second loss sequence. Preferably, in this example, the convolutional smoothing uses a Gaussian kernel, and its specific calculation process is as follows:

[0034]

[0035]

[0036] In the formula, Let represent the Gaussian convolution kernel, z represent the normalization factor, and N represent the total number of terms in the input sequence. Indicates the index of the current term. Represents the i-th word element The corresponding first loss value, This represents the second loss value corresponding to the current word n. Indicates an indicator function, This represents the half-width value of a one-dimensional convolution window.

[0037] Step S103: Sort the second loss values ​​in the second loss sequence and select the top r% of words as the current candidate abnormal words. The expression is as follows:

[0038]

[0039] In the formula, Let represent the second loss sequence, and k represent the parameters of the topk function.

[0040] Step S104, as follows Figure 3 As shown, the current candidate abnormal word units Global anomaly lexicon pool corresponding to the previous training round Merge the pools and prune them according to the pool size threshold β to obtain the global abnormal word pool corresponding to the current training round t. The expression is as follows:

[0041]

[0042] In the formula, This is the global abnormal word pool corresponding to the current training round t. This is the global abnormal word pool corresponding to the previous training round t-1, and k represents the parameter of the topk function.

[0043] Furthermore, step S1 also includes:

[0044] Step S105: When the current training round t exceeds the warm-up step S, i.e. For the global abnormal word pool corresponding to the current training round t The abnormal terms in the loss function are rescaled using loss weights, and the rescaled loss values ​​are used for backpropagation. The expression is as follows:

[0045]

[0046] In the formula, This is the loss value after rescaling. The first loss value, This is the second loss value after smoothing. This represents the global anomaly pool for the current training round t, where N represents the total number of tokens in the input sequence. This is the uncertainty adjustment coefficient.

[0047] It should be noted that in the training of large language models, traditional methods for handling high-loss lexical units (outliers) often involve direct masking or discarding, which leads to the loss of crucial information. This invention innovatively employs a strategy of smoothing the first loss sequence to obtain a second loss sequence set and rescaling the loss weights to solve the training instability problem caused by long-tailed high-loss lexical units, while avoiding information loss. Specifically, by smoothing the first loss sequence to obtain the second loss sequence, random noise fluctuations between adjacent lexical units are eliminated, and local structural features are used to prevent misjudgments caused by single-point numerical abrupt changes, thus improving the accuracy of anomaly detection. Furthermore, rescaling the loss weights can dynamically suppress the excessive gradient influence of high-loss lexical units while preserving their semantic information.

[0048] Step S2: In the inference stage of the large language model, the candidate word is determined to be an abnormal word based on the global abnormal word pool. When an abnormal word is detected, several candidate output sequences are generated with the abnormal word as a node, and the average uncertainty of each candidate output sequence is calculated. The path with the minimum average uncertainty is selected to obtain the output sequence.

[0049] Specifically, step S2 includes the following sub-steps:

[0050] Step S201: Calculate the negative log probability of the candidate word. If the negative log probability of the candidate word is higher than the lower bound of the loss of the global abnormal word pool, then the candidate word is determined to be an abnormal word.

[0051] The expression for the negative logarithmic probability of candidate lexical units is as follows:

[0052]

[0053] In the formula, Let represent the negative logarithmic probability of the i-th word. Represents the i-th word element. This represents the preceding sequence of the i-th word.

[0054] The process of determining the lower bound of the loss for the global outlier word pool includes: obtaining the first loss value for all words belonging to the global outlier word pool corresponding to the training round, thus obtaining the loss distribution; and using the q-th quantile of the loss distribution as the lower bound of the loss for the global outlier word pool. The expression is as follows:

[0055]

[0056] In the formula, The first term represents the loss distribution. Quantiles This represents the lower bound of the loss for the global outlier lexicon pool. This represents the first loss value corresponding to the i-th word. This represents the global abnormal word pool corresponding to training round t.

[0057] Step S202: When an abnormal word is detected, several branches are generated with the abnormal word as a node. Each branch is a candidate output sequence, and so on to obtain the inference tree. The average uncertainty of each candidate output sequence is calculated to obtain the average uncertainty of each path in the inference tree. The path with the smallest average uncertainty is selected to obtain the output sequence.

[0058] Assuming the current branch In the Word elements were generated at each node. Then the uncertainty of this lexical element Defined as

[0059]

[0060] Assuming the current branch From node Initially, the expression for the average uncertainty of this branch is as follows:

[0061]

[0062] In the formula, This represents the b-th branch of the j-th node. average uncertainty, Let represent the length of the subsequent sequence of the b-th branch of the j-th node. Let represent the i-th word in the b-th branch of the j-th node, and k represent the position of the j-th node in the preceding sequence.

[0063] After selecting the branch with the lowest average uncertainty, continue searching upwards to find the optimal branch. In other words, when choosing the current path, prioritize paths that offer less uncertainty, thus ensuring the stability and high quality of the final output sequence.

[0064]

[0065] In the formula, This represents the optimal subsequent branch of the j-th node.

[0066] Once all branches of the inference tree have been generated, the optimal path is selected based on the average uncertainty of each branch. The final inference result is the optimal path selected from multiple generated paths, ensuring that the generated output sequence has high stability and low uncertainty. Final output sequence Composed of all tokens generated by the optimal path:

[0067]

[0068] For example, such as Figure 5 As shown, Figure 5 The generated inference tree is shown. Starting from the bottom-level node, taking the first node (j=1) as an example, a candidate output sequence with an average uncertainty of 2.1 is selected, and then traced back to the parent node of the first node. The average uncertainty of this candidate path is 5.2 + 2.1 = 7.3. Taking the second node (j=2) as an example, a candidate output sequence with an average uncertainty of 0.9 is selected, and then traced back to the parent node of the second node. The average uncertainty of this candidate path is 2.3 + 0.9 = 3.2. The path with an average uncertainty of 3.21 is selected as the final output sequence.

[0069] It should be noted that, to address the issue of high-uncertainty terms in the inference stage easily leading to hallucinations or erroneous outputs, this invention introduces a dynamic branching parallel inference mechanism. Multi-branch parallel generation is triggered only when abnormal terms are detected, ensuring efficient use of computational resources and avoiding the model getting trapped in local optima or erroneous paths by exploring multiple possibilities. Furthermore, by selecting the optimal path through an aggregation strategy, the final generated output sequence outperforms traditional greedy search or single-sample results in terms of logical consistency and correctness.

[0070] Furthermore, the lexical management method for large language models provided by this invention also includes:

[0071] Step S3, as follows Figure 6 As shown, the quality score of the historical output sequence generated by the large language model inference is obtained, and the global abnormal word pool corresponding to the next training round t+1 is dynamically adjusted based on the quality score of the historical output sequence during the inference stage. The expression is as follows:

[0072]

[0073] in,

[0074]

[0075]

[0076] In the formula, Represents the i-th word element. This represents the set of anomalous words in the historical output sequence generated in the current training round t. A set of anomalous terms representing the historical output sequence Quality rating This represents the historical average score. express The upper realm, This represents the set of words to be added. This represents the set of terms to be removed. Represents the i-th word element The corresponding first loss value, This represents the lower bound of the loss corresponding to the abnormal tokens in the current training round t. This represents the global abnormal word pool corresponding to the current training round t.

[0077] Update the lower bound of the loss for anomalous words in the next training epoch t+1, as shown in the following expression:

[0078]

[0079] In the formula, The first term represents the loss distribution. Quantiles are used to smoothly adjust boundaries.

[0080] It should be noted that this invention is not merely a one-way generation process, but also incorporates a user rating and feedback mechanism. This invention can dynamically adjust the lower bound and members of the global abnormal word pool based on user evaluations of the generated quality. This means that after deployment, the large language model can continuously update its understanding of abnormal words based on feedback from real-world application scenarios, forming a positive iterative closed loop.

[0081] Furthermore, the method proposed in this invention does not depend on specific large model architectures (such as Transformer, LLAMA, GPT, etc.), and the increase in computational cost is controllable. Simultaneously, the method of this invention can be used in conjunction with other fine-tuning optimization techniques (such as NEFTUNE) to provide additional performance gains without altering the original workflow, facilitating rapid deployment and application within existing large model training and inference frameworks.

[0082] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the lexical management method for large language models as described above. Figure 7 The diagram shown is a hardware structure diagram of any device with data processing capabilities for the lexical management method for large language models provided in this embodiment of the invention, except... Figure 7 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0083] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the lexical management method for large language models as described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0084] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A token management method for large language models, characterized by, The method comprises: In the large language model training stage, a loss corresponding to each token in the input sequence is calculated to obtain a first loss sequence; the first loss sequence is smoothed to obtain a second loss sequence; the loss values in the second loss sequence are sorted to screen a current candidate abnormal token; the current candidate abnormal token is merged with a global abnormal token pool corresponding to a previous training round, and is cropped according to a pool size threshold to obtain a global abnormal token pool corresponding to a current training round; In the inference stage of the large language model, whether a candidate token is an abnormal token is determined according to the global abnormal token pool; when an abnormal token is detected, a plurality of candidate output sequences are generated with the abnormal token as a node, and the average uncertainty of each candidate output sequence is calculated; the path with the minimum average uncertainty is selected to obtain an output sequence.

2. The method of claim 1, wherein, The process of smoothing the first loss sequence to obtain the second loss sequence comprises: For each token in the first loss sequence, the following is performed: A Gaussian convolution kernel is defined, which is subject to a standard normal distribution; The positions of all domain tokens within a one-dimensional convolution window half-width range centered on the current token are determined, the offset of each domain token relative to the current token is obtained, and the value of the Gaussian convolution kernel at each offset is accumulated to obtain a normalization factor; The second loss value of the current token is calculated according to the first loss value of each domain token within the one-dimensional convolution window half-width range and the normalization factor; By analogy, the second loss value corresponding to each token is obtained, and thus the second loss sequence is obtained.

3. The method of claim 1, wherein, When the current training round exceeds the warm-up step, the loss weight of the abnormal token in the global abnormal token pool corresponding to the current training round is rescaled, and the rescaled loss value is used for back propagation; Wherein, each token in the input sequence is traversed, and the following loss weight rescaling operation is performed for the current token: The first loss value corresponding to the current token and the smoothed second loss value are obtained; The second loss value corresponding to the current token is compared with the minimum loss threshold, if the second loss value is greater than or equal to the minimum loss threshold, the inverse of the product of the preset uncertainty adjustment coefficient and the second loss value is calculated, and the inverse is taken as a weighting coefficient to multiply the first loss value corresponding to the current token to obtain a modified loss term of the current token; if the second loss value is less than the minimum loss threshold, the first loss value corresponding to the current token is determined as the modified loss term of the current token; wherein the minimum loss threshold is determined according to the global abnormal token pool corresponding to the current training round; The modified loss terms of all tokens in the input sequence are accumulated, and the accumulated result is divided by the total number of tokens in the input sequence to obtain the rescaled loss value.

4. The method of claim 1, wherein, The process of determining whether a candidate token is an abnormal token according to the global abnormal token pool comprises: The negative log probability of the candidate token is calculated; When the negative log probability of a candidate word is higher than the lower bound of the loss of the global abnormal word pool, the candidate word is an abnormal word. The process of determining the lower bound of the loss of the global abnormal word pool includes: obtaining the first loss value corresponding to all words in the global abnormal word pool belonging to the training round, thereby obtaining the loss distribution; and taking the qth quantile of the loss distribution as the lower bound of the loss of the global abnormal word pool.

5. The method of claim 1, wherein, The process of calculating the average uncertainty of each candidate output sequence includes: Determine the subsequent sequences contained in the candidate sequence from the current node to the end of the candidate sequence; The uncertainty value of the subsequent sequence is obtained by calculating and accumulating the uncertainty value corresponding to each word in the subsequent sequence based on the negative log probability. The difference between the total length of the candidate sequence and the position of the current node is used as the length of the subsequent sequence; Divide the uncertainty value of the subsequent sequence by the length of the subsequent sequence to obtain the average uncertainty of the candidate output sequence.

6. The method of claim 1, wherein, The method further includes: Obtain the quality score of the historical output sequence generated by the large language model inference; adjust the global abnormal word pool corresponding to the next training round accordingly.

7. The method of claim 6, wherein, The process of adjusting the global outlier lexical pool for the next training round based on the quality score of the historical output sequence generated by the large language model inference includes: Determine the set of words to be added; including: obtaining the output sequence generated in the current training round and its quality score, and comparing the quality score with the historical average score; if the quality score is less than the historical average score, then select all words in the output sequence whose first loss value is greater than or equal to the lower bound of the current global abnormal word pool, and construct the set of words to be added from the selected words; Determine the set of words to be removed; including: if the quality score of the output sequence is not less than the historical average score, then select all words in the output sequence that already exist in the current global abnormal word pool, and construct the set of words to be removed from the selected words; The set of words to be added is merged into the current global abnormal word pool, and the words with the minimum loss in the set of words to be removed are removed from the merged word pool, thereby obtaining the global abnormal word pool corresponding to the next training round.

8. An electronic device, comprising: include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the lexical management method for large language models as described in any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the lexical management method for large language models as described in any one of claims 1-7.

10. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the lexical management method for large language models as described in any one of claims 1-7.

Images