Large model text generation method based on semantic adaptation and dynamic hierarchical comparison

Abstract

The application discloses a large model text generation method based on semantic adaptation and dynamic hierarchical comparison, and belongs to the technical field of artificial intelligence and natural language processing, which can at least partially solve the problems of lack of semantic perception ability, fixed hierarchical selection strategy and waste of computing resources in the prior art contrast decoding technology, and introduces a semantic routing module in the decoding process.The method obtains the hidden state of different levels of the model and the final layer output distribution when the model reasons and generates each word element; the semantic routing module is used for semantic attribute determination and uncertainty calculation on the current generation state; according to the determination result, a strategy is dynamically selected from a preset decoding strategy set to calculate the final output value; and the final output value is normalized and combined with a sampling algorithm to generate a word element.The application realizes dynamic balance of factuality and fluency, suppresses hallucinations and reduces reasoning delay.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing technology, specifically to a large-model text generation method based on semantic adaptation and dynamic hierarchical comparison. Background Technology

[0002] Large language models face the illusion problem during generation, generating content that appears reasonable but contains factual errors. Existing contrastive decoding techniques reinforce factual accuracy by comparing the output distributions of high-level and low-level models, but still suffer from the following significant drawbacks: First, it lacks semantic awareness. Existing technologies typically treat all generated tokens equally, failing to distinguish between entity words and functional words. Entity words refer to words that carry factual information, such as nouns, proper nouns, numbers, and technical terms. Functional words refer to words that do not carry factual information, such as pronouns, particles, conjunctions, prepositions, punctuation marks, and general verbs. Forcibly comparing functional words across different layers often disrupts the grammatical structure of sentences, leading to a decrease in the fluency of the generated text.

[0003] Second, the hierarchical selection strategy is rigid. Existing technologies mainly rely on Jensen-Shannon divergence for statistical hierarchical selection. In scenarios involving factual accuracy requirements, statistical differences alone are insufficient to locate specific hierarchies containing erroneous factual memories, leading to unstable hallucination suppression effects.

[0004] Third, there is a waste of computational resources and inference latency. Performing complex full-level comparison calculations for each token significantly increases the computational load and latency of inference, making it difficult to meet the real-time requirements of industrial applications.

[0005] Therefore, how to design a text generation method that can perceive semantic attributes, dynamically select decoding strategies, and simultaneously take into account factual accuracy and fluency while possessing advantages in reasoning efficiency is a technical problem that urgently needs to be solved in the current field of large language model applications. Summary of the Invention

[0006] The present invention aims to at least solve one of the technical problems existing in the prior art, and to provide a large model text generation method based on semantic adaptation and dynamic hierarchical comparison.

[0007] To achieve the above objectives, this invention provides a large-model text generation method based on semantic adaptation and dynamic hierarchical comparison, comprising the following steps: In the model inference generation When there are 1 word units, the output states of different levels in the large language model are obtained. The output states include the hidden state vectors of each layer, the hidden state of the final layer, and the vocabulary probability distribution of the final layer. Using a pre-trained semantic routing module, the semantic category of the current word to be generated is predicted by a classifier based on the hidden state of the final layer, and the information entropy is calculated based on the vocabulary probability distribution. Based on the semantic category and the information entropy, a decoding strategy is dynamically selected from a preset decoding strategy set according to a selection rule to calculate the final output value; the decoding strategy set includes at least a strong fact correction mode, a fluency protection mode, and a regular monitoring mode; wherein, the selection rule is as follows: when the semantic category is an entity class and the information entropy is greater than a preset entropy threshold, the strong fact correction mode is triggered; when the semantic category is a function class, the fluency protection mode is triggered; when the semantic category is a normal content class and the information entropy is in a preset intermediate range, the regular monitoring mode is triggered. The output value is normalized and combined with a sampling algorithm to generate the first... Each word element.

[0008] Furthermore, the semantic routing module is based on the hidden state of the final layer. The semantic category of the current word to be generated is predicted by the classifier. ; The semantic category Includes entity classes, functional classes, and general content classes; The entity class includes nouns, proper nouns, numbers, and technical terms; The functional class includes function words, punctuation marks, and general verbs; the general content class includes other words that do not belong to the entity class or the functional class.

[0009] Furthermore, the information entropy The calculation formula is: ; in, This represents the information entropy. Indicates a vocabulary list, This represents the lexical probabilities in the final layer. The probability value; The information entropy Compared with the preset entropy threshold, when the information entropy When the value exceeds the preset entropy threshold, the current generation state is determined to be in a state of high uncertainty.

[0010] Furthermore, the strong fact-correction mode is executed as follows: Calculate the vocabulary probability distribution of the final layer. Probability distribution of each preset candidate layer vocabulary The candidate layer with the largest KL divergence or Jensen-Shannon divergence value is selected as the negative layer. ; The final output value is calculated using the weighted comparison formula: ; in, This represents the final output value. This represents the output value of the final layer. Indicates the negative layer The output value, This indicates the preset enhancement coefficient.

[0011] Furthermore, the candidate layer is selected from the range of one-half of the total number of layers in the large language model to the level between the total number of layers minus two.

[0012] Furthermore, the smoothness protection mode is executed as follows: Skip the inter-layer comparison calculation step and directly set the final output value equal to the final layer's output value, that is: ; in, This represents the final output value. This represents the output value of the final layer.

[0013] Furthermore, the execution method of the conventional monitoring mode is as follows: Calculate the Jensen-Shannon divergence between the output distribution of each early layer and the final layer; The early layer with the largest Jensen-Shannon divergence value is selected as the candidate layer; the final layer output and the candidate layer output are dynamically weighted and fused using a preset correction coefficient to obtain the final output value; The correction factor is less than the enhancement factor in the strong fact correction mode.

[0014] Furthermore, the classifier is a linear probe model built based on a fully connected layer, and the input dimension of the classifier is consistent with the hidden layer dimension of the large language model; The classifier was fine-tuned and trained on labeled data in a preset domain, which includes the domain of power parameters, the domain of industrial control instructions, or the domain of cybersecurity terminology.

[0015] Furthermore, the normalization process and the sampling algorithm include: The output value The input is normalized using a normalized exponential function to obtain the final sampling probability distribution. ; The top-k sampling strategy or the top-p sampling strategy is used to select the first sample from the final sampling probability distribution. Each word element; The generated first The word units are concatenated into the input sequence and then processed. A cycle for generating each word element.

[0016] Furthermore, obtaining the output states at different levels in the large language model also includes: The current context input sequence is fed into the large language model, and the hidden state vector of each layer of the large language model is extracted. Obtain the hidden state of the final layer The hidden states of the final layer are projected onto the vocabulary space through the language model head to obtain the vocabulary probability distribution of the final layer. and its corresponding output value vector ; For each candidate layer, the hidden state of each candidate layer is projected onto the vocabulary space through the language model head to obtain the output value vector corresponding to each candidate layer.

[0017] The beneficial effects of this invention are as follows: This invention introduces a semantic routing module, which can distinguish between entity words and function words. While ensuring the accuracy of entity words, it avoids excessive intervention in function words, achieving a dynamic balance between factuality and fluency. This effectively solves the problem of decreased sentence fluency caused by existing comparison decoding technology.

[0018] This invention directly skips the inter-layer comparison calculation step for functional words and high-confidence tokens, which greatly reduces the amount of matrix operations, lowers the average inference latency, and improves inference efficiency.

[0019] The semantic routing module of this invention can be fine-tuned for specific vertical domains, making it more sensitive to proper nouns in specific domains, achieving customized illusion suppression, and possessing domain adaptation capabilities.

[0020] This invention forms a gradient-based illusion suppression mechanism by setting three decoding strategies of different intensities and dynamically selecting them. This mechanism can adopt the optimal strategy in different semantic scenarios, thereby improving the overall generation quality. Attached Figure Description

[0021] Figure 1 This is a flowchart of a large-model text generation method based on semantic adaptation and dynamic hierarchical comparison, according to an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the system architecture and the connection relationship between the semantic routing module in an embodiment of the present invention. Figure 3 This is a schematic diagram of the logical decision tree for dynamic policy routing in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the strategy triggering and correction effects in a text generation scenario within the power industry, according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates this application. It should be understood that the specific embodiments described in this specification are merely for explaining this application and are not intended to limit it.

[0023] The large-model text generation method of this invention, based on semantic adaptation and dynamic hierarchical comparison, can run on a server with high-performance computing units and is an improvement on a large language model based on the Transformer architecture. This invention introduces a lightweight semantic routing module in the model decoding stage. This module dynamically determines the decoding strategy based on the semantic attributes of the current context and model uncertainty, achieving a balance between factual accuracy and fluency.

[0024] See Figure 2 The system architecture of this invention includes two core components: a Transformer model backbone and a semantic routing module. The Transformer model backbone contains multiple Transformer layers from the embedding layer to the final layer, with each layer outputting a corresponding hidden state vector. The semantic routing module receives the hidden state of the final layer, outputs a semantic category signal through a linear probe classifier, and simultaneously outputs an uncertainty signal through an entropy calculator. The dynamic contrast calculation unit performs corresponding contrast operations or bypass operations based on the semantic category signal and the uncertainty signal, combined with the output values ​​of the final layer and the candidate negative layer, ultimately outputting the output value used for sampling.

[0025] Example 1 This embodiment uses a large language model based on the Transformer architecture to describe in detail the overall process of the text generation method of this invention. The total number of layers in the large language model... The settings are: 32, hidden layer dimension is set to 4096, vocabulary size is set to... The value is set to 32000. In this embodiment, the classifier of the semantic routing module is pre-trained on the named entity recognition dataset, and the labeled categories are entity class, function class, and general content class.

[0026] See Figure 1 The text generation method in this embodiment includes the following steps: Step S1: Multi-level state acquisition.

[0027] In the model inference generation At the moment of each token, the system acquires state information from different levels of the Transformer architecture in parallel.

[0028] The first step is input processing. The current context input sequence is fed into the large language model. After passing through word embedding layers and positional encoding, the input sequence forms the initial hidden state representation.

[0029] The second step is hidden state extraction. The large language model executes self-attention mechanisms and feedforward neural network operations layer by layer, outputting the hidden state vector of each layer. Let the total number of layers in the large language model be... Extract the hidden state vector of each layer i, denoted as . ,in The value range is 0 to , Indicates the sequence number of the currently generated token.

[0030] The third step is distribution projection. This obtains the hidden state of the final layer. The hidden states of the final layer are projected onto the vocabulary space through the language model head to obtain the vocabulary probability distribution of the final layer. and its corresponding output value vector The language model head is a linear projection layer that maps the hidden layer dimensions to the vocabulary dimension. In this embodiment, the language model head maps the 4096-dimensional hidden states to a 32000-dimensional vocabulary space vector.

[0031] For early candidate layers, the number of layers selected is... to The layers between them are used as a candidate layer set. In this embodiment, The set is 32, and the candidate layer set consists of layers 16 to 30. For each layer in the candidate layer set, the hidden state of that layer is projected onto the vocabulary space through the language model head to obtain the corresponding output value vector.

[0032] Step S2: Semantic attributes and uncertainty determination.

[0033] This step is the core of the invention. The system introduces a semantic routing module, which includes two sub-modules: a classifier and an entropy calculator.

[0034] The classifier is constructed and trained as follows. The classifier is a fully connected layer with an input dimension consistent with the hidden layer dimension of the large language model; in this embodiment, the input dimension is 4096. The classifier's output dimension is 3, corresponding to three semantic categories: entity class, function class, and general content class. The entity class includes nouns, proper nouns, numbers, and technical terms. The function class includes pronouns, conjunctions, prepositions, punctuation, and common high-frequency verbs. The general content class includes all other words not belonging to the entity or function classes.

[0035] The classifier's training data was annotated using a named entity recognition dataset. The annotation process categorized each word into an entity class, function class, or general content class. The training process employed a cross-entropy loss function to optimize the classifier parameters, with a learning rate of 0.001, 50 training epochs, and a batch size of 64.

[0036] In generating the first When there are 1 word elements, the final hidden state will be... Input a classifier, and the classifier outputs the predicted semantic category of the word to be generated. .

[0037] The entropy calculator calculates values ​​as follows: based on the probability distribution of the final layer vocabulary. Calculate the Shannon information entropy. The formula for calculating information entropy is: ; in, Represents information entropy. Indicates a vocabulary list, This represents the lexical probabilities in the final layer of the vocabulary. The probability value. In this embodiment, a preset entropy threshold is set. It is 2.5. When Greater than When this occurs, it indicates that the model lacks confidence in the current prediction, posing a risk of illusion, and is classified as a state of high uncertainty. When the value is less than or equal to 1.0, it is considered a low uncertainty state. When the value is greater than 1.0 and less than or equal to 2.5, it is determined to be a state of moderate uncertainty.

[0038] Step S3: Dynamic policy routing.

[0039] See Figure 3 Based on the semantic category output in step S2 and entropy The system automatically selects from three preset decoding strategies.

[0040] Strategy A is a strong fact-correction mode. The trigger condition is semantic category. Equal to entity class and entropy value Greater than the preset entropy threshold The execution logic is as follows: The first step is to calculate the probability distribution of the final-layer vocabulary. Probability distribution of vocabulary for each candidate layer The KL divergence between them. The formula for calculating the KL divergence is: ; in, Denotes KL divergence, This represents the lexical units in the probability distribution of the final-level vocabulary. The probability value, Represents the lexical units in the probability distribution of the i-th level vocabulary. The probability value.

[0041] The second step is to select the candidate layer with the largest KL divergence value. This is the negative layer. This layer represents the level where erroneous factual memories or noisy information that has not yet been fully corrected within the model.

[0042] The third step is to calculate the final output value using the strong contrast decoding formula. The formula is: ; in, This represents the final output value. This represents the output value of the final layer. Representing the negative layer The output value, This represents a preset enhancement coefficient. In this embodiment, the enhancement coefficient... Set to 2.0. The range of the enhancement factor is: It is used to amplify the difference between correct facts and false illusions.

[0043] Strategy B is a fluency protection mode. The trigger condition is semantic category. This is equivalent to a functional class. The execution logic is as follows: The inter-layer comparison calculation is skipped directly. The final output value is set equal to the final layer's output value, i.e.: ; in, This represents the final output value. This represents the output value of the final layer. This strategy avoids unnecessary comparative subtraction operations on function words, punctuation, and general verbs, preventing grammatical structure collapse due to over-correction, while also reducing inference computational power consumption.

[0044] Strategy C is the standard monitoring mode. The trigger condition is a scenario other than the two cases mentioned above, i.e., semantic category. Equal to a normal content class and entropy value It is currently in the preset middle range. The execution logic is as follows: The first step is to calculate the Jensen-Shannon divergence between the output distributions of each early layer and the final layer. The formula for calculating the Jensen-Shannon divergence is: ; in, The mean distributions of the two distributions are given. This represents the KL divergence.

[0045] The second step is to select the early layer with the largest Jensen-Shannon divergence value as the candidate layer.

[0046] The third step involves dynamically weighting and fusing the final layer output and the candidate layer output using a preset correction coefficient to obtain the final output value. The correction coefficient is set to 0.5, which is less than the enhancement coefficient in the strong fact correction mode, thus providing a gentle correction.

[0047] Step S4: Next word sampling.

[0048] The first step is to process the final output value obtained in step S3. The input is normalized using a normalized exponential function to obtain the final sampling probability distribution. The formula for calculating the normalized exponential function is: ; in, Indicates word elements The final sampling probability, Indicates word elements The corresponding final output value.

[0049] The second step involves selecting the next lexical unit from the final sampling probability distribution using either a Top-k sampling strategy or a Top-p sampling strategy. In the Top-k sampling strategy, k is set to 50, and random sampling is performed from the top 50 lexical units with the highest probabilities according to the probability distribution. In the Top-p sampling strategy, p is set to 0.9, and random sampling is performed from the set of lexical units with a cumulative probability of 0.9.

[0050] The third step is to concatenate the generated word units into the input sequence, and then proceed to the next step. The generation cycle of each word element is repeated. Steps S1 to S4 are repeated until an end marker is generated or the preset maximum generation length is reached.

[0051] Example 2 This embodiment uses knowledge-based question answering in the power industry as an application scenario to describe in detail the specific application process of this invention in a specific vertical field. In this embodiment, the total number of layers of the large language model... Set to 32, the hidden layer dimension is set to 4096, and the vocabulary size is set to 32000. The classifier of the semantic routing module is fine-tuned and trained on the power system professional term dataset, and the labeled categories include entity class words and functional class words such as power equipment names, technical parameters, control instructions, etc.

[0052] Scenario description: The user inputs a question to the large language model, asking to describe the main functions of the DCS system of a certain power plant. The answer generated by the model contains that the core controller model of the DCS system is Supmax-800, which has functions of data acquisition, logic control and alarm management.

[0053] The text generation method of this embodiment includes the following steps: Step S1: Multilayer state acquisition.

[0054] Send the context sequence input by the user into the large language model. The large language model processes the input sequence layer by layer, extracts the hidden state vectors of each layer, and obtains the hidden state of the final layer , projects it to the vocabulary space through the language model head, and obtains the final layer vocabulary probability distribution and the output value vector . At the same time, project the hidden states of the 16th layer to the 30th layer in the candidate layer set to the vocabulary space through the language model head respectively, and obtain the output value vectors of each candidate layer.

[0055] Step S2: Semantic attribute and uncertainty determination.

[0056] The semantic determination results of different tokens in the generation process are described below respectively.

[0057] When generating the token "this", input the hidden state of the final layer into the classifier, and the classifier outputs that the semantic category is functional. This token is a pronoun and belongs to functional class words. At the same time, calculate the information entropy of the final layer vocabulary probability distribution, and the entropy value is 0.3, which is much lower than the preset entropy threshold of 2.5, and the model has a high confidence in the prediction of this token.

[0058] When generating the token "DCS system", the classifier outputs that the semantic category is entity class. This token is a proper noun and belongs to the equipment name in the power field. Calculate the information entropy, and the entropy value is 0.6, which is lower than the preset entropy threshold of 2.5, and the model has a relatively high confidence in the prediction of this token.

[0059] When generating the token "of", the classifier outputs that the semantic category is functional. This token is a structural particle and belongs to functional class words. Calculate the information entropy, and the entropy value is 0.1, and the model has an extremely high confidence in the prediction of this token.

[0060] When generating the token "core controller", the classifier outputs a semantic category of general content. This token is a combination of common nouns. Calculate the information entropy, and the entropy value is 1.5, which is within the preset intermediate range.

[0061] When generating the token "model number", the classifier outputs a semantic category of general content. Calculate the information entropy, and the entropy value is 1.2, which is within the preset intermediate range.

[0062] When generating the token "is", the classifier outputs a semantic category of function. This token is a general verb and belongs to the function vocabulary. Calculate the information entropy, and the entropy value is 0.2.

[0063] When generating the specific model number "Supmax-800", the classifier outputs a semantic category of entity. This token is the specific model parameter of an electrical device and belongs to the entity vocabulary. Calculate the information entropy, and the entropy value is 3.2, which is greater than the preset entropy threshold of 2.5. The model hesitates among multiple models and is in a state of high uncertainty.

[0064] Step S3: Dynamic policy routing.

[0065] Refer to Figure 4 , and the policy routing results corresponding to each token are described separately below.

[0066] When generating the token "the", the semantic category is function, triggering Policy B. The system skips the inter-layer comparison calculation and directly uses the output of the final layer as the final output value.

[0067] When generating the token "DCS system", the semantic category is entity but the entropy value is 0.6, which is lower than the preset entropy threshold of 2.5 and does not meet the trigger condition of Policy A. The system enters the regular monitoring mode of Policy C, calculates the Jensen-Shannon divergence between each early layer and the final layer, selects the layer with the largest divergence value as the candidate layer, and performs gentle correction using a correction coefficient of 0.5.

[0068] When generating the token "of", the semantic category is function, triggering Policy B. The system directly uses the output of the final layer.

[0069] When generating the token "core controller", the semantic category is general content and the entropy value is 1.5, which is within the preset intermediate range, triggering Policy C. The system performs adaptive dynamic layer selection, selects the early layer with the largest Jensen-Shannon divergence for gentle correction.

[0070] When generating the token "model number", the semantic category is general content and the entropy value is 1.2, triggering Strategy C. The system performs a gentle correction.

[0071] When generating the token "is", the semantic category is functional, triggering Strategy B. The system directly uses the output of the final layer.

[0072] When generating the specific model number "Supmax-800", the semantic category is entity and the entropy value is 3.2, which is greater than the preset entropy threshold of 2.5, triggering Strategy A. The system executes the strong fact correction mode. Calculate the KL divergence between the probability distribution of the final layer token list and the probability distributions of each candidate layer token lists. It is found that the KL divergence value of the 16th layer is the largest, and the 16th layer is selected as the negative layer. The output distribution of the 16th layer tends to output the wrong model number "Supmax-600", which is an old version model number. Apply the strong contrast decoding formula, and the enhancement coefficient is set to 2.0: ; In the calculation results, the output value corresponding to the wrong option "Supmax-600" is significantly suppressed, and the output value corresponding to the correct option "Supmax-800" is highlighted.

[0073] Step S4: Sampling the next word.

[0074] Input the final output value into the softmax function for normalization. Adopt the Top-k sampling strategy, and the k value is set to 50. Sample from the top 50 tokens with the highest probabilities. Since the probability value of "Supmax-800" is significantly higher than other candidate tokens after the strong fact correction, the system accurately generates "Supmax-800". Concatenate the generated token to the input sequence and continue to generate subsequent tokens.

[0075] Through the above implementation manners, the present invention can specifically address the problem of confusion of common proper nouns in the power industry, and significantly reduce the hallucination rate of equipment parameters and equipment names without sacrificing the overall generation speed.

[0076] Embodiment III This embodiment takes the generation of safety instructions in the industrial control field as the application scenario, and details the specific application process of the present invention in the industrial control instruction recognition scenario. In this embodiment, the total number of layers of the large language model is set to 40, the hidden layer dimension is set to 5120, and the token list size The value is set to 50,000. The classifier of the semantic routing module was fine-tuned and trained on the industrial control instruction dataset. The labeled categories include entity words such as PLC control instruction names, DCS system parameters, and communication protocol identifiers, as well as functional words.

[0077] Scenario Description: A user inputs a question into a large language model: Please explain the reading process of function code 03 in the Modbus TCP protocol. The model's generated answer includes technical parameters such as the function code number, register address, and data length.

[0078] The text generation method in this embodiment includes the following steps.

[0079] Step S1: Multi-level state acquisition.

[0080] The user-input context sequence is fed into the large language model. The large language model processes the input sequence layer by layer, extracting the hidden state vector of each layer. The hidden state of the final layer is then obtained. The probability distribution of the final vocabulary is obtained by projecting the language model head onto the vocabulary space. and output value vector Meanwhile, the hidden states of layers 20 to 38 in the candidate layer set are projected onto the vocabulary space through the language model head to obtain the output value vector of each candidate layer.

[0081] Step S2: Semantic attributes and uncertainty determination.

[0082] The following describes the semantic determination results of different lexical units during the generation process.

[0083] When generating the lexical "Modbus", the classifier outputs the semantic category as entity class. This lexical is a proprietary name for an industrial communication protocol. Calculate the information entropy, and the entropy value. The value is 0.4, which is lower than the preset entropy threshold of 2.5, indicating that the model has a high confidence level for this term.

[0084] When generating the word "TCP", the classifier outputs the semantic category as entity class. The information entropy is calculated, and the entropy value is... The confidence level is 0.5, indicating a high level of confidence in the model.

[0085] When generating the lexical unit "function code", the classifier outputs the semantic category as entity class. The information entropy is calculated, and the entropy value is... It is 0.8.

[0086] When the lexical "03" is generated, the classifier outputs the semantic category as entity class. This lexical is a specific function code number. The information entropy is calculated, and the entropy value is... The value is 2.8, which is greater than the preset entropy threshold of 2.5, indicating that the model hesitates between multiple function code numbers.

[0087] When generating the token "for", the classifier outputs a semantic category of functional class. Calculate the information entropy, and the entropy value is 0.2.

[0088] When generating the token "read hold register", the classifier outputs a semantic category of entity class. Calculate the information entropy, and the entropy value is 2.9, which is greater than the preset entropy threshold of 2.5. There is a risk that the model confuses "hold register" with "input register".

[0089] Step S3: Dynamic policy routing.

[0090] When generating the token "Modbus", the semantic category is entity class but the entropy value is 0.4, which is lower than the preset entropy threshold of 2.5, triggering the normal monitoring mode of Policy C. The system calculates the Jensen-Shannon divergence between each early layer and the final layer, selects the layer with the largest divergence value as the candidate layer, and performs gentle correction with a correction coefficient of 0.5.

[0091] When generating the token "TCP", Policy C is also triggered and gentle correction is performed.

[0092] When generating the token "function code", Policy C is triggered and gentle correction is performed.

[0093] When generating the token "03", the semantic category is entity class and the entropy value is 2.8, which is greater than the preset entropy threshold of 2.5, triggering the strong fact correction mode of Policy A. The system calculates the KL divergence between the final layer word list probability distribution and the word list probability distributions of each candidate layer. It is found that the KL divergence value of the 22nd layer is the largest, and the 22nd layer is selected as the negative layer. The output distribution of the 22nd layer tends to output other function code numbers such as "01" or "04". Apply the strong contrast decoding formula, and the enhancement coefficient is set to 2.0: ; In the calculation results, the output value of the incorrect function code number is suppressed, and the output value of the correct function code "03" is highlighted.

[0094] When generating the token "for", the semantic category is functional class, triggering Policy B. The system directly uses the output of the final layer.

[0095] When generating the token "read hold register", the semantic category is entity class and the entropy value is 2.9, which is greater than the preset entropy threshold of 2.5, triggering Policy A. The system executes the strong fact correction mode, selects the candidate layer with the largest KL divergence as the negative layer, and applies the strong contrast decoding formula. In the calculation results, the output value of "input register" is suppressed, and the output value of "hold register" is highlighted.

[0096] Step S4: Next word sampling.

[0097] The final output value is normalized using a normalized exponential function. A Top-p sampling strategy is employed, with p set to 0.9, sampling is performed from the set of lexical terms with a cumulative probability of 0.9. Due to strong fact correction, the probability of correct lexical terms is significantly increased, and the system accurately generates the target content. The generated lexical terms are then concatenated into the input sequence to continue generating subsequent lexical terms.

[0098] Through the above implementation methods, the present invention can address the common problems of protocol parameter confusion and instruction numbering errors in the field of industrial control, ensuring the accuracy of technical parameters in the generated content.

[0099] Example 4 This embodiment details the fine-tuning training process of the classifier in the semantic routing module. This embodiment uses the power system domain as the target domain, and the classifier is fine-tuned on a power system terminology dataset.

[0100] The classifier's network structure is a single fully connected layer. The input dimension is consistent with the hidden layer dimension of the large language model, which is set to 4096 in this embodiment. The output dimension is 3, corresponding to the three semantic categories: entity class, function class, and general content class. The classifier has approximately 12288 parameters. Compared to the billions of parameters in the large language model, the classifier is a lightweight component and will not significantly increase inference latency.

[0101] The training data was constructed as follows: Text corpora totaling 100,000 sentences were collected from power system technical documents, equipment manuals, and operation and maintenance reports. Each sentence's lexical units were manually labeled, categorized into entity, function, and general content types. Entity-type labeled lexical units included power equipment names, model parameters, voltage levels, power values, and protection device codes. Function-type labeled lexical units included auxiliary words, prepositions, conjunctions, punctuation marks, and general verbs. General content-type labeled lexical units included all other words not belonging to the first two categories. After labeling, the dataset was divided into training, validation, and test sets in an 8:1:1 ratio.

[0102] The training process is as follows: The parameters of the large language model are kept fixed, and only the parameters of the classifier are trained. During training, the final hidden state at each word's corresponding position is used as the input to the classifier, and the manually labeled semantic category is used as the target output. The cross-entropy loss function is used, and the calculation formula is: ; in, Indicates the loss value. In this embodiment, the number of semantic categories is represented. Equals 3, This represents the one-hot encoded value of the target category. This represents the predicted probability of the c-th class output by the classifier.

[0103] The optimizer used is the Adam optimizer, with a learning rate set to 0.001. The number of training epochs is set to 50, and the batch size is set to 64. An early stopping strategy is employed during training: training is terminated and the optimal model parameters are saved when the classification accuracy on the validation set no longer improves for 10 consecutive epochs.

[0104] After training, the classifier's performance was evaluated on the test set. The classifier achieved a classification accuracy of 95.3% on the power system domain test set. The recall rate was 93.8% for entity classes, 97.2% for function classes, and 94.1% for general content classes.

[0105] The inference latency test results for the classifier are as follows. In a single forward propagation, the classifier's inference time is approximately 0.02 milliseconds, compared to approximately 2 milliseconds for a single-layer Transformer in a large language model. The additional latency introduced by the classifier accounts for only 1%, which meets the requirements for real-time inference.

[0106] In transfer applications across different domains, the classifier can achieve domain adaptation by changing the training dataset. When the training data is changed to an industrial control instruction dataset, the classifier achieves a classification accuracy of 94.7% on the industrial control domain test set. When the training data is changed to a cybersecurity terminology dataset, the classifier achieves a classification accuracy of 93.9% on the cybersecurity domain test set. These results demonstrate that the classifier possesses good domain transfer capabilities.

[0107] In summary, the embodiments disclosed herein have at least the following technical effects: This invention introduces a semantic routing module, which can distinguish between entity-based words and function-based words. While ensuring the accuracy of entity-based words, it avoids excessive intervention in function-based words, thus achieving a dynamic balance between factual accuracy and fluency.

[0108] This invention directly skips the inter-layer comparison calculation step for functional words and high-confidence words, which greatly reduces the amount of matrix operations, lowers the average inference latency, and improves inference efficiency.

[0109] The semantic routing module of this invention can be fine-tuned for specific vertical domains, making it more sensitive to proper nouns in those domains and achieving customized illusion suppression.

[0110] This invention forms a gradient-based illusion suppression mechanism by setting three decoding strategies of different intensities and dynamically selecting them. This mechanism can adopt the optimal strategy in different semantic scenarios, thereby improving the overall generation quality.

[0111] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A large-model text generation method based on semantic adaptation and dynamic hierarchical comparison, characterized in that, Includes the following steps: In the model inference generation When there are 1 word units, the output states of different levels in the large language model are obtained. The output states include the hidden state vectors of each layer, the hidden state of the final layer, and the vocabulary probability distribution of the final layer. Using a pre-trained semantic routing module, the semantic category of the current word to be generated is predicted by a classifier based on the hidden state of the final layer, and the information entropy is calculated based on the vocabulary probability distribution. Based on the semantic category and the information entropy, a decoding strategy is dynamically selected from the preset decoding strategy set according to the selection rules to calculate the final output value; The decoding strategy set includes at least a strong fact correction mode, a fluency protection mode, and a regular monitoring mode; wherein, the selection rule is as follows: when the semantic category is an entity class and the information entropy is greater than a preset entropy threshold, the strong fact correction mode is triggered; when the semantic category is a function class, the fluency protection mode is triggered; and when the semantic category is a normal content class and the information entropy is in a preset middle range, the regular monitoring mode is triggered. The output value is normalized and combined with a sampling algorithm to generate the first... Each word element.

2. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 1, characterized in that, The semantic routing module is based on the hidden state of the final layer. The semantic category of the current word to be generated is predicted by the classifier. ; The semantic category Includes entity classes, functional classes, and general content classes; The entity class includes nouns, proper nouns, numbers, and technical terms; The functional class includes function words, punctuation marks, and general verbs; the general content class includes other words that do not belong to the entity class or the functional class.

3. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 2, characterized in that, The classifier is a linear probe model built on a fully connected layer, and the input dimension of the classifier is consistent with the hidden layer dimension of the large language model. The classifier was fine-tuned and trained on labeled data in a preset domain, which includes the domain of power parameters, the domain of industrial control instructions, or the domain of cybersecurity terminology.

4. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 1, characterized in that, The information entropy The calculation formula is: ； in, This represents the information entropy. Indicates a vocabulary list, This represents the lexical probabilities in the final layer. The probability value; The information entropy Compared with the preset entropy threshold, when the information entropy When the value exceeds the preset entropy threshold, the current generation state is determined to be in a state of high uncertainty.

5. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 1, characterized in that, The strong fact correction mode is executed as follows: Calculate the vocabulary probability distribution of the final layer. Probability distribution of each preset candidate layer vocabulary The candidate layer with the largest KL divergence or Jensen-Shannon divergence value is selected as the negative layer. ; The final output value is calculated using the weighted comparison formula: ； in, This represents the final output value. This represents the output value of the final layer. Indicates the negative layer The output value, This indicates the preset enhancement coefficient.

6. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 5, characterized in that, The candidate layer is selected from the range of one-half of the total number of layers in the large language model to the level between the total number of layers minus two.

7. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 1, characterized in that, The smoothness protection mode is executed as follows: Skip the inter-layer comparison calculation step and directly set the final output value equal to the final layer's output value, that is: ； in, This represents the final output value. This represents the output value of the final layer.

8. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 5, characterized in that, The execution method of the conventional monitoring mode is as follows: Calculate the Jensen-Shannon divergence between the output distribution of each early layer and the final layer; The early layer with the largest Jensen-Shannon divergence value is selected as the candidate layer; the final layer output and the candidate layer output are dynamically weighted and fused using a preset correction coefficient to obtain the final output value; The correction factor is less than the enhancement factor in the strong fact correction mode.

9. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to claim 1, characterized in that, The normalization process and the sampling algorithm include: The output value The input is normalized using a normalized exponential function to obtain the final sampling probability distribution. ; The top-k sampling strategy or the top-p sampling strategy is used to select the first sample from the final sampling probability distribution. Each word element; The generated first The word units are concatenated into the input sequence and then processed. A cycle for generating each word element.

10. The large-model text generation method based on semantic adaptation and dynamic hierarchical comparison according to any one of claims 1 to 9, characterized in that, Obtaining the output states of different levels in the large language model also includes: The current context input sequence is fed into the large language model, and the hidden state vector of each layer of the large language model is extracted. Obtain the hidden state of the final layer The hidden states of the final layer are projected onto the vocabulary space through the language model head to obtain the vocabulary probability distribution of the final layer. and its corresponding output value vector ; For each candidate layer, the hidden state of each candidate layer is projected onto the vocabulary space through the language model head to obtain the output value vector corresponding to each candidate layer.

Images