A method for managing word units for large language models, electronic equipment, and medium
By constructing a global abnormal lexical pool and a dynamic branching parallel inference mechanism, the lexical management problem in the training and inference stages of large language models is solved, improving the training stability and output accuracy of the model, and making it suitable for the training and inference stages of large language models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2026-02-28
- Publication Date
- 2026-06-05
AI Technical Summary
Existing large language models suffer from word-level management and optimization problems during the training and inference phases, leading to training instability, information loss, and erroneous output. Existing methods cannot effectively identify and handle high-loss words.
By calculating the loss value of the input sequence, smoothing and sorting are performed, a global abnormal word pool is constructed, abnormal words are identified and pruned during the training phase, and candidate words are judged based on the abnormal word pool during the inference phase to generate the optimal output sequence. The output is optimized by adopting a dynamic branching parallel inference mechanism and a user feedback mechanism.
It effectively avoids information loss and high uncertainty output caused by high-loss lexical units, improves the stability of model training and the accuracy of generation, and dynamically adjusts the strategy to adapt to the needs of different training stages.
Smart Images
Figure CN121743496B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of large language model technology, and particularly relates to a lexical management method, electronic device, and medium for large language models. Background Technology
[0002] In recent years, large-scale language models (LLMs) have become the foundational architecture of the natural language processing field, forming a complete technical paradigm of "pre-training - supervised fine-tuning (SFT) - instruction alignment - inference generation". However, with the continuous growth of model size and context length, the management and optimization of tokens has become increasingly prominent, becoming a key bottleneck restricting model training efficiency and generation quality.
[0003] During the fine-tuning or alignment training phase, large language models perform word-by-word prediction on the input sequence, and their word-level loss exhibits a significant long-tail distribution. Specifically, although high-perplexity or high-loss words account for a relatively small proportion of the overall distribution, they cause significant perturbations to gradient updates, easily leading to training instability, model parameter shifts, and catastrophic "forgetting." Existing fine-tuning methods mostly employ static strategies such as masking, discarding, or adding noise to handle high-loss words, but these methods have fundamental flaws: on the one hand, they simply and crudely discard words that may result in the loss of key semantic information; on the other hand, they lack a dynamic evaluation mechanism for word importance, making it impossible to adaptively adjust optimization strategies at different training stages. Although Parameter Efficient Fine-Tuning (PEFT) techniques such as LoRA can alleviate some of the problems, research has confirmed that they still cannot avoid catastrophic forgetting, and fine-tuning performance is strongly negatively correlated with the amount of forgetting.
[0004] In the inference generation stage, large language models employ probabilistic sampling (such as top-k, top-p) or greedy / beam search for sequence decoding. However, lexical-level prediction uncertainty—manifested as high-entropy or high-variance distribution—is often directly related to erroneous outputs. Research indicates that lexical-level uncertainty estimation can effectively identify weak points in the generation process, providing a theoretical basis for dynamic intervention. Summary of the Invention
[0005] To address the shortcomings of existing technologies, embodiments of the present invention provide a lexical management method, electronic device, and medium for large language models.
[0006] In a first aspect, embodiments of the present invention provide a lexical management method for large language models, the method comprising the following steps:
[0007] During the training phase of the large language model, the loss corresponding to each word in the input sequence is calculated to obtain the first loss sequence; the first loss sequence is smoothed to obtain the second loss sequence; the loss values in the second loss sequence are sorted to filter the current candidate abnormal words; the current candidate abnormal words are merged with the global abnormal word pool corresponding to the previous training round and pruned according to the pool size threshold to obtain the global abnormal word pool corresponding to the current training round.
[0008] During the inference phase of the large language model, the candidate lexicon is determined to be an abnormal lexicon based on the global abnormal lexicon pool. When an abnormal lexicon is detected, several candidate output sequences are generated with the abnormal lexicon as the node, and the average uncertainty of each candidate output sequence is calculated. The path with the minimum average uncertainty is selected to obtain the output sequence.
[0009] In a second aspect, embodiments of the present invention provide an electronic device, comprising:
[0010] At least one processor; and
[0011] A memory communicatively connected to the at least one processor; wherein,
[0012] The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described lexical management method for large language models.
[0013] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described lexical management method for large language models.
[0014] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-described lexical management method for large language models.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0016] This invention provides a lexical management method for large language models. During the training phase of the large language model, anomalous lexical units are identified, and a global anomalous lexical unit pool is constructed. During the inference phase of the large language model, the global anomalous lexical unit pool is used to determine whether candidate lexical units are anomalous, and the output sequence is selected through probabilistic uncertainty. This invention avoids the information loss caused by simply discarding high-loss lexical units and overcomes the problem that high-uncertainty lexical units in the inference phase can easily lead to hallucinations or erroneous outputs. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 A flowchart of a lexical management method for large language models provided in an embodiment of the present invention;
[0019] Figure 2 This is a schematic diagram of a lexical management method for large language models provided in an embodiment of the present invention;
[0020] Figure 3 This is a schematic diagram illustrating the abnormal word update process during training, as provided in an embodiment of the present invention.
[0021] Figure 4 A schematic diagram of one-dimensional Gaussian convolution smoothing provided for an embodiment of the present invention;
[0022] Figure 5 This is a schematic diagram illustrating branch generation and aggregation during the reasoning process provided in an embodiment of the present invention.
[0023] Figure 6 This is a schematic diagram illustrating the updating of the abnormal word pool during the inference process provided in an embodiment of the present invention;
[0024] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.
[0027] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a lexical management method for large language models, the method comprising:
[0028] Step S1: During the large language model training phase, calculate the loss corresponding to each word in the input sequence to obtain the first loss sequence; smooth the first loss sequence to obtain the second loss sequence; sort the loss values in the second loss sequence and select the top r% of words as the current candidate abnormal words; merge the current candidate abnormal words with the global abnormal word pool corresponding to the previous training round and prune them according to the pool size threshold to obtain the global abnormal word pool corresponding to the current training round t.
[0029] Specifically, step S1 includes the following sub-steps:
[0030] Step S101: Calculate the first loss value corresponding to each word in the input sequence to obtain the first loss sequence. .
[0031] The first loss value is either cross-entropy loss or negative log-likelihood value.
[0032] Step S102: Smooth the first loss sequence to obtain the second loss sequence. , .
[0033] Specifically, such as Figure 4 As shown, a one-dimensional convolutional smoothing filter is applied to the first loss sequence to suppress noise fluctuations between adjacent words and preserve local structural features, thereby obtaining the smoothed second loss sequence. Preferably, in this example, the convolutional smoothing uses a Gaussian kernel, and its specific calculation process is as follows:
[0034]
[0035]
[0036] In the formula, Let represent the Gaussian convolution kernel, z represent the normalization factor, and N represent the total number of terms in the input sequence. Indicates the index of the current term. Represents the i-th word element The corresponding first loss value, This represents the second loss value corresponding to the current word element n. Indicates an indicator function, This represents the half-width value of a one-dimensional convolution window.
[0037] Step S103: Sort the second loss values in the second loss sequence and select the top r% of words as the current candidate abnormal words. The expression is as follows:
[0038]
[0039] In the formula, Let represent the second loss sequence, and k represent the parameters of the topk function.
[0040] Step S104, as follows Figure 3 As shown, the current candidate abnormal word units Global anomaly lexicon pool corresponding to the previous training round Merge the pools and prune them according to the pool size threshold β to obtain the global abnormal word pool corresponding to the current training round t. The expression is as follows:
[0041]
[0042] In the formula, This is the global abnormal word pool corresponding to the current training round t. This is the global abnormal word pool corresponding to the previous training round t-1, and k represents the parameter of the topk function.
[0043] Furthermore, step S1 also includes:
[0044] Step S105: When the current training round t exceeds the warm-up step S, i.e. For the global abnormal word pool corresponding to the current training round t The abnormal terms in the loss function are rescaled using loss weights, and the rescaled loss values are used for backpropagation. The expression is as follows:
[0045]
[0046] In the formula, This is the loss value after rescaling. The first loss value, This is the second loss value after smoothing. This represents the global outlier word pool corresponding to the current training round t, where N represents the total number of words in the input sequence. This is the uncertainty adjustment coefficient.
[0047] It should be noted that in the training of large language models, traditional methods for handling high-loss lexical units (outliers) often involve direct masking or discarding, which leads to the loss of crucial information. This invention innovatively employs a strategy of smoothing the first loss sequence to obtain a second loss sequence set and rescaling the loss weights to solve the training instability problem caused by long-tailed high-loss lexical units, while avoiding information loss. Specifically, by smoothing the first loss sequence to obtain the second loss sequence, random noise fluctuations between adjacent lexical units are eliminated, and local structural features are used to prevent misjudgments caused by single-point numerical abrupt changes, thus improving the accuracy of anomaly detection. Furthermore, rescaling the loss weights can dynamically suppress the excessive gradient influence of high-loss lexical units while preserving their semantic information.
[0048] Step S2: In the inference stage of the large language model, the candidate word is determined to be an abnormal word based on the global abnormal word pool. When an abnormal word is detected, several candidate output sequences are generated with the abnormal word as a node, and the average uncertainty of each candidate output sequence is calculated. The path with the minimum average uncertainty is selected to obtain the output sequence.
[0049] Specifically, step S2 includes the following sub-steps:
[0050] Step S201: Calculate the negative log probability of the candidate word. If the negative log probability of the candidate word is higher than the lower bound of the loss of the global abnormal word pool, then the candidate word is determined to be an abnormal word.
[0051] The expression for the negative logarithmic probability of candidate lexical units is as follows:
[0052]
[0053] In the formula, Let represent the negative logarithmic probability of the i-th word. Represents the i-th word element. This represents the preceding sequence of the i-th word.
[0054] The process of determining the lower bound of the loss for the global outlier word pool includes: obtaining the first loss value for all words belonging to the global outlier word pool corresponding to the training round, thus obtaining the loss distribution; and using the q-th quantile of the loss distribution as the lower bound of the loss for the global outlier word pool. The expression is as follows:
[0055]
[0056] In the formula, The first term represents the loss distribution. Quantiles This represents the lower bound of the loss for the global outlier lexicon pool. This represents the first loss value corresponding to the i-th word. This represents the global abnormal word pool corresponding to training round t.
[0057] Step S202: When an abnormal word is detected, several branches are generated with the abnormal word as a node. Each branch is a candidate output sequence, and so on to obtain the inference tree. The average uncertainty of each candidate output sequence is calculated to obtain the average uncertainty of each path in the inference tree. The path with the smallest average uncertainty is selected to obtain the output sequence.
[0058] Assuming the current branch In the Word elements were generated at each node. Then the uncertainty of this lexical element Defined as
[0059]
[0060] Assuming the current branch From node Initially, the expression for the average uncertainty of this branch is as follows:
[0061]
[0062] In the formula, This represents the b-th branch of the j-th node. average uncertainty, Let represent the length of the subsequent sequence of the b-th branch of the j-th node. Let represent the i-th word in the b-th branch of the j-th node, and k represent the position of the j-th node in the preceding sequence.
[0063] After selecting the branch with the lowest average uncertainty, continue searching upwards to find the optimal branch. In other words, when choosing the current path, prioritize paths that offer less uncertainty, thus ensuring the stability and high quality of the final output sequence.
[0064]
[0065] In the formula, This represents the optimal subsequent branch of the j-th node.
[0066] Once all branches of the inference tree have been generated, the optimal path is selected based on the average uncertainty of each branch. The final inference result is the optimal path selected from multiple generated paths, ensuring that the generated output sequence has high stability and low uncertainty. Final output sequence Composed of all tokens generated by the optimal path:
[0067]
[0068] For example, such as Figure 5 As shown, Figure 5 The generated inference tree is shown. Starting from the bottom-level node, taking the first node (j=1) as an example, a candidate output sequence with an average uncertainty of 2.1 is selected, and then traced back to the parent node of the first node. The average uncertainty of this candidate path is 5.2 + 2.1 = 7.3. Taking the second node (j=2) as an example, a candidate output sequence with an average uncertainty of 0.9 is selected, and then traced back to the parent node of the second node. The average uncertainty of this candidate path is 2.3 + 0.9 = 3.2. The path with an average uncertainty of 3.21 is selected as the final output sequence.
[0069] It should be noted that, to address the issue of high-uncertainty terms in the inference stage easily leading to hallucinations or erroneous outputs, this invention introduces a dynamic branching parallel inference mechanism. Multi-branch parallel generation is triggered only when abnormal terms are detected, ensuring efficient use of computational resources and avoiding the model getting trapped in local optima or erroneous paths by exploring multiple possibilities. Furthermore, by selecting the optimal path through an aggregation strategy, the final generated output sequence outperforms traditional greedy search or single-sample results in terms of logical consistency and correctness.
[0070] Furthermore, the lexical management method for large language models provided by this invention also includes:
[0071] Step S3, as follows Figure 6 As shown, the quality score of the historical output sequence generated by the large language model inference is obtained, and the global abnormal word pool corresponding to the next training round t+1 is dynamically adjusted based on the quality score of the historical output sequence during the inference stage. The expression is as follows:
[0072]
[0073] in,
[0074]
[0075]
[0076] In the formula, Represents the i-th word element. This represents the set of anomalous words in the historical output sequence generated in the current training round t. A set of anomalous terms representing the historical output sequence Quality rating This represents the historical average score. express The upper realm, This represents the set of lexical units to be added. This represents the set of terms to be removed. Represents the i-th word element The corresponding first loss value, This represents the lower bound of the loss corresponding to the abnormal tokens in the current training round t. This represents the global abnormal word pool corresponding to the current training round t.
[0077] Update the lower bound of the loss for anomalous words in the next training epoch t+1, as shown in the following expression:
[0078]
[0079] In the formula, The first term represents the loss distribution. Quantiles are used to smoothly adjust boundaries.
[0080] It should be noted that this invention is not merely a one-way generation process, but also incorporates a user rating and feedback mechanism. This invention can dynamically adjust the lower bound and members of the global abnormal word pool based on user evaluations of the generated quality. This means that after deployment, the large language model can continuously update its understanding of abnormal words based on feedback from real-world application scenarios, forming a positive iterative closed loop.
[0081] Furthermore, the method proposed in this invention does not depend on specific large model architectures (such as Transformer, LLAMA, GPT, etc.), and the increase in computational cost is controllable. Simultaneously, the method of this invention can be used in conjunction with other fine-tuning optimization techniques (such as NEFTUNE) to provide additional performance gains without altering the original workflow, facilitating rapid deployment and application within existing large model training and inference frameworks.
[0082] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the lexical management method for large language models as described above. Figure 7 The diagram shown is a hardware structure diagram of any device with data processing capabilities for the lexical management method for large language models provided in this embodiment of the invention, except... Figure 7 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.
[0083] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the lexical management method for large language models as described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.
[0084] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.
Claims
1. A lexical management method for large language models, characterized in that, The method includes: During the training phase of the large language model, the loss corresponding to each word in the input sequence is calculated to obtain the first loss sequence; the first loss sequence is smoothed to obtain the second loss sequence; the loss values in the second loss sequence are sorted to filter the current candidate abnormal words; the current candidate abnormal words are merged with the global abnormal word pool corresponding to the previous training round and pruned according to the pool size threshold to obtain the global abnormal word pool corresponding to the current training round. During the inference phase of the large language model, the global abnormal word pool is used to determine whether the candidate word is an abnormal word. When an abnormal word is detected, several candidate output sequences are generated with the abnormal word as the node, and the average uncertainty of each candidate output sequence is calculated. The path with the minimum average uncertainty is selected to obtain the output sequence. The process of calculating the average uncertainty of each candidate output sequence includes: Determine the subsequent sequences contained in the candidate sequence from the current node to the end of the candidate sequence; The uncertainty value of the subsequent sequence is obtained by calculating and accumulating the uncertainty value corresponding to each word in the subsequent sequence based on the negative log probability. The difference between the total length of the candidate sequence and the position of the current node is used as the length of the subsequent sequence; Divide the uncertainty value of the subsequent sequence by the length of the subsequent sequence to obtain the average uncertainty of the candidate output sequence.
2. The lexical management method for large language models according to claim 1, characterized in that, The process of smoothing the first loss sequence to obtain the second loss sequence includes: Perform the following for each term in the first loss sequence: Define a Gaussian convolution kernel that follows a standard normal distribution; Determine the positions of all neighborhood words within half the width of a one-dimensional convolution window centered on the current word word, obtain the offset of each neighborhood word word relative to the current word word, and sum the values of the Gaussian convolution kernel at each offset to obtain the normalization factor. Calculate the second loss value of the current word based on the first loss value and normalization factor corresponding to each domain word within the half-width range of the one-dimensional convolution window; By analogy, the second loss value corresponding to each word is obtained, thus obtaining the second loss sequence.
3. The lexical management method for large language models according to claim 1, characterized in that, When the current training round exceeds the warm start step, the loss weights of the abnormal words in the global abnormal word pool corresponding to the current training round are rescaled, and the rescaled loss values are used for backpropagation. Specifically, for each word in the input sequence, the following loss weight rescaling operation is performed: Obtain the first loss value and the smoothed second loss value corresponding to the current word element; The second loss value corresponding to the current word element is compared with the minimum loss threshold. If the second loss value is greater than or equal to the minimum loss threshold, the reciprocal of the product of the second loss value and the preset uncertainty adjustment coefficient is calculated, and this reciprocal is used as a weighting coefficient to multiply the first loss value corresponding to the current word element to obtain the corrected loss term for the current word element. If the second loss value is less than the minimum loss threshold, the first loss value corresponding to the current word element is determined as the corrected loss term for the current word element. The minimum loss threshold is determined based on the global abnormal word element pool corresponding to the current training round. The corrected loss terms for all words in the input sequence are summed, and the sum is divided by the total number of words in the input sequence to obtain the rescaled loss value.
4. The lexical management method for large language models according to claim 1, characterized in that, The process of determining whether a candidate word is an anomalous word based on the global anomalous word pool includes: Calculate the negative log probability of candidate lexical units; When the negative log probability of a candidate word is higher than the lower bound of the loss of the global abnormal word pool, the candidate word is an abnormal word. The process of determining the lower bound of the loss of the global abnormal word pool includes: obtaining the first loss value corresponding to all words in the global abnormal word pool belonging to the training round, thereby obtaining the loss distribution; and taking the qth quantile of the loss distribution as the lower bound of the loss of the global abnormal word pool.
5. The lexical management method for large language models according to claim 1, characterized in that, The method further includes: Obtain the quality score of the historical output sequence generated by the large language model inference; adjust the global abnormal word pool corresponding to the next training round accordingly.
6. The lexical management method for large language models according to claim 1, characterized in that, The process of adjusting the global outlier lexical pool for the next training round based on the quality score of the historical output sequence generated by the large language model inference includes: Determine the set of words to be added; including: obtaining the output sequence generated in the current training round and its quality score, and comparing the quality score with the historical average score; if the quality score is less than the historical average score, then select all words in the output sequence whose first loss value is greater than or equal to the lower bound of the current global abnormal word pool, and construct the set of words to be added from the selected words; Determine the set of words to be removed; including: if the quality score of the output sequence is not less than the historical average score, then select all words in the output sequence that already exist in the current global abnormal word pool, and construct the set of words to be removed from the selected words; The set of words to be added is merged into the current global abnormal word pool, and the words with the minimum loss in the set of words to be removed are removed from the merged word pool, thereby obtaining the global abnormal word pool corresponding to the next training round.
7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the lexical management method for large language models as described in any one of claims 1-6.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the lexical management method for large language models as described in any one of claims 1-6.
9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the lexical management method for large language models as described in any one of claims 1-6.