Text generation method, apparatus, device, medium, and program

Abstract

Embodiments of the present application provide a text generation method, device, equipment, medium and program. The method performs structural analysis on an input instruction of an LLM to obtain a plurality of sub-instructions included in the input instruction. In the generation process of each token, an attention matrix of an hth attention head of an lth layer of the LLM is obtained. The key sub-instruction in the current generation stage is dynamically identified from the plurality of sub-instructions according to the attention matrix. The key sub-instruction is the sub-instruction that contributes most to the prediction of the next token. The attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix are shielded to obtain a guided attention matrix. Since the irrelevant sub-instructions of the key sub-instruction are shielded in the guided attention matrix, the LLM can focus on the key sub-instruction in the current generation stage when generating the next token, avoiding the interference of irrelevant information, and improving the quality of the output result of the LLM.

Description

Technical Field

[0001] This application relates to the field of Natural Language Processing (NLP), and more particularly to a text generation method, apparatus, device, medium, and program. Background Technology

[0002] Large Language Models (LLMs) are models based on deep learning and NLP techniques, aiming to understand and generate human language. LLMs have achieved significant results in various NLP tasks, such as machine translation, text summarization, question answering systems, and sentiment analysis.

[0003] In real-world scenarios, some large language models involve the execution of complex instructions. Complex instructions typically contain multiple sub-instructions, each potentially with different semantic and format constraints, which must be satisfied simultaneously. Currently, large language models have poor adherence to complex instructions, resulting in low-quality outputs. Summary of the Invention

[0004] This application provides a text generation method, apparatus, device, medium, and program that improves the quality of the output results of a large language model.

[0005] In a first aspect, embodiments of this application provide a text generation method, the method comprising: obtaining the attention matrix of the h-th attention head of the l-th layer of a large language model, wherein the attention matrix is ​​obtained by the large language model processing an input instruction, the large language model comprising L layers of processing units, each layer of processing units comprising a multi-head attention module, the multi-head attention module comprising H attention heads, where L is a positive integer, H is an integer greater than or equal to 2, l takes at least one value from 1 to L, and h takes at least one value from 1 to H; performing structural analysis on the input instruction to obtain multiple sub-instructions included in the input instruction; determining a key sub-instruction from the multiple sub-instructions included in the input instruction according to the attention matrix, the key sub-instruction being the sub-instruction that contributes the most to predicting the next label; masking the attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix to obtain a guided attention matrix; and generating an output result corresponding to the input instruction by the large language model based on the guided attention matrix.

[0006] In some exemplary embodiments, determining a key sub-instruction from a plurality of sub-instructions included in the input instruction according to the attention matrix includes: calculating an attention score for each sub-instruction's contribution to predicting the next label based on the attention matrix, wherein the attention score is the contribution value of the sub-instruction to predicting the next label; and determining the key sub-instruction from the plurality of sub-instructions based on the contribution values ​​of the plurality of sub-instructions to predicting the next label.

[0007] In some exemplary embodiments, calculating the attention score for each sub-instruction pair's contribution to predicting the next label based on the attention matrix includes: determining mutually exclusive sub-instruction pairs in the input instruction based on the structure type of the input instruction and the plurality of sub-instructions included in the input instruction, wherein each mutually exclusive sub-instruction pair consists of two mutually exclusive sub-instructions; masking the attention scores corresponding to interfering labels between the mutually exclusive sub-instruction pairs in the attention matrix to obtain a modified attention matrix; and calculating the attention score for each sub-instruction pair's contribution to predicting the next label based on the modified attention matrix.

[0008] In some exemplary embodiments, the contribution value of the sub-instruction to predicting the next label is the average attention score of the contribution of the label set of the sub-instruction to predicting the next label; the step of calculating the attention score of each sub-instruction to predicting the next label according to the modified attention matrix includes: for each sub-instruction, calculating the average of the attention scores corresponding to all labels between the k-th label of the sub-instruction and the next label in the modified attention matrix to obtain the average attention score of the contribution of the k-th label to predicting the next label, where k takes the value 1, 2, ..., N, and N is the number of labels included in the label set of the sub-instruction; calculating the average of the average attention scores of the N labels included in the label set of the sub-instruction to predicting the next label to obtain the average attention score of the contribution of the sub-instruction to predicting the next label.

[0009] In some exemplary embodiments, the contribution value of the sub-instruction to predicting the next label is the average attention score of the contribution of the label set of the sub-instruction to predicting the next label; the step of calculating the attention score of each sub-instruction to predicting the next label according to the attention matrix includes: for each sub-instruction, calculating the average of the attention scores corresponding to all labels between the k-th label of the sub-instruction and the next label in the attention matrix to obtain the average attention score of the contribution of the k-th label to predicting the next label, where k takes the value 1, 2, ..., N, and N is the number of labels included in the label set of the sub-instruction; calculating the average of the average attention scores of the N labels included in the label set of the sub-instruction to predicting the next label to obtain the average attention score of the contribution of the sub-instruction to predicting the next label.

[0010] In some exemplary embodiments, determining the key sub-instruction from the plurality of sub-instructions based on their contribution values ​​includes: calculating the information entropy of each sub-instruction based on its contribution value to predicting the next marker; and determining the key sub-instruction from the plurality of sub-instructions based on its contribution value and information entropy to predicting the next marker.

[0011] In some exemplary embodiments, determining the key sub-instruction from the plurality of sub-instructions based on their contribution values ​​and information entropy to predicting the next marker includes: when the information entropy of each sub-instruction is less than a confidence threshold, determining the sub-instruction with the largest contribution value to predicting the next marker as the key sub-instruction from the plurality of sub-instructions; when the information entropy of at least one sub-instruction is not less than the confidence threshold, filtering out sub-instructions with information entropy not less than the confidence threshold, and determining the sub-instruction with the largest contribution value to predicting the next marker as the key sub-instruction from the remaining sub-instructions.

[0012] In some exemplary embodiments, the confidence threshold is related to the number of sub-instructions included in the input instruction; the more sub-instructions included in the input instruction, the smaller the confidence threshold.

[0013] In some exemplary embodiments, the step of masking the attention scores between the mutually exclusive sub-instruction pairs in the attention matrix to obtain a modified attention matrix includes: determining a first bias matrix based on the mutually exclusive sub-instruction pairs and the attention matrix, wherein the first bias matrix has the same size as the attention matrix, the attention scores between the mutually exclusive sub-instruction pairs in the first bias matrix are preset negative masking factors, the remaining attention scores in the first bias matrix are 0, and the masking factor is greater than 0; and adding the first bias matrix and the attention matrix to obtain the modified attention matrix.

[0014] In some exemplary embodiments, masking the attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix to obtain a guided attention matrix includes: determining irrelevant sub-instructions of the key sub-instruction based on the contribution value of each sub-instruction to the prediction of the next label; determining a second bias matrix based on the irrelevant sub-instructions of the key sub-instruction and the attention matrix, wherein the second bias matrix has the same size as the attention matrix, the attention score corresponding to the label of the irrelevant sub-instruction of the key sub-instruction in the second bias matrix is ​​a preset negative masking factor, the remaining attention score in the second bias matrix is ​​0, and the masking factor is greater than 0; and adding the second bias matrix and the attention matrix to obtain the guided attention matrix.

[0015] In some exemplary embodiments, the step of performing structural analysis on the input instruction to obtain a plurality of sub-instructions included in the input instruction includes: inputting the input instruction into a structural analysis and recognition model, and the structural analysis and recognition model outputting the plurality of sub-instructions included in the input instruction.

[0016] In some exemplary embodiments, the following steps are performed by the structure analysis and recognition module: performing structure analysis on the input instruction to obtain multiple sub-instructions included in the input instruction; the following steps are performed by the attention guidance module: obtaining the attention matrix of the h-th attention head of the l-th layer in the large language model; determining a key sub-instruction from the multiple sub-instructions included in the input instruction according to the attention matrix; masking the attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix to obtain the guided attention matrix, and sending the guided attention matrix to the large language model; wherein, the attention guidance module and the structure analysis and recognition module interact with the large language model in the form of plug-ins.

[0017] Secondly, embodiments of this application provide a text generation apparatus, the apparatus comprising: an acquisition module, configured to acquire the attention matrix of the h-th attention head of the l-th layer of a large language model, wherein the attention matrix is ​​obtained by the large language model processing an input instruction, the large language model comprising L layers of processing units, each layer of processing units comprising a multi-head attention module, the multi-head attention module comprising H attention heads, where L is a positive integer, H is an integer greater than or equal to 2, l takes at least one value from 1 to L, and h takes at least one value from 1 to H; a structure analysis and recognition module, configured to perform structure analysis on the input instruction to obtain multiple sub-instructions included in the input instruction; an attention guidance module, further configured to determine a key sub-instruction from the multiple sub-instructions included in the input instruction according to the attention matrix, the key sub-instruction being the sub-instruction that contributes the most to predicting the next label; and to mask the attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix to obtain a guided attention matrix; and the large language model, configured to generate an output result corresponding to the input instruction according to the guided attention matrix.

[0018] Thirdly, embodiments of this application provide an electronic device, the electronic device comprising: a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the method as described in the first aspect above.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to perform the method described in the first aspect above.

[0020] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0021] The technical solution provided in this application analyzes the structure of the input instructions of the LLM to obtain multiple sub-instructions included in the input instructions. During the generation of each tag, the attention matrix of the h-th attention head of the l-th layer of the LLM is obtained. Based on the attention matrix, the key sub-instructions in the current generation stage are dynamically identified from the multiple sub-instructions. The key sub-instructions are those that contribute the most to predicting the next tag. The attention scores of irrelevant sub-instructions of the key sub-instructions in the attention matrix are masked to obtain the guided attention matrix, which is then sent to the LLM for subsequent processing. Since the irrelevant sub-instructions of the key sub-instructions are masked in the guided attention matrix, the LLM can focus on the key sub-instructions in the current generation stage when generating the next tag, avoiding interference from irrelevant information, and generating more coherent, accurate, and input instruction-compliant outputs, thereby improving the quality of the LLM's output results. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A schematic diagram of three types of complex instructions;

[0024] Figure 2 This is a schematic diagram of the structure of a large language model applicable to the embodiments of this application;

[0025] Figure 3 A schematic diagram of the structured attention guidance architecture provided in an embodiment of this application;

[0026] Figure 4 A flowchart of a text generation method is provided for Embodiment 1 of this application;

[0027] Figure 5 This is a schematic diagram showing the existing output of the large language model and the guided output obtained after using the method of this application;

[0028] Figure 6 This is a flowchart of one implementation of step S104 in Embodiment 1;

[0029] Figure 7 This is a schematic diagram illustrating the changes in the attention matrix during the guidance process;

[0030] Figure 8 This is a schematic diagram of the structure of the text generation device provided in Embodiment 3 of this application;

[0031] Figure 9 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0034] To facilitate understanding of the embodiments of this application, before describing the various embodiments of this application, some concepts involved in all embodiments of this application will be appropriately explained.

[0035] Large Language Models (LLMs) are models based on deep learning and Natural Language Processing (NLP) techniques, aiming to understand and generate human language. LLMs typically consist of billions or even tens of billions of parameters and can learn language patterns, semantic information, and knowledge from massive amounts of text data. LLMs have achieved significant results in various NLP tasks, including but not limited to machine translation, text summarization, question answering systems, and sentiment analysis.

[0036] Autoregressive Language Models (ALMs) are probabilistic natural language processing models that aim to predict the next word based on a known sequence of words. In an ALM, the occurrence of each word depends on the preceding word sequence, making it a conditional probability model. The training process for ALMs typically involves Maximum Likelihood Estimation (MLE) or other optimization methods to learn the model parameters, thereby maximizing the overall probability on the training set. In practical applications, some LLMs can employ autoregressive generation mechanisms to generate text.

[0037] Attention mechanism, a key technique in LLM (Linguistic Programming), enhances the model's focus on important information by dynamically assigning weights to different parts of the input sequence. Specifically, it allows the model to consider other related words when processing a particular word, thereby capturing long-distance dependencies and contextual information. This mechanism not only improves the model's expressive power but also significantly enhances its performance in natural language processing tasks, enabling the model to better understand and generate language.

[0038] Attention steering, or attention steering, is a technique in LLM (Limited Language Management) used to enhance model generation capabilities and control output content. By guiding (or adjusting) attention weights, the model's focus can be better controlled during text generation, making it more aligned with specific needs or contexts. Specifically, attention steering adjusts the model's internal attention distribution, causing it to focus more on certain words, phrases, or topics during generation. This mechanism can be implemented in various ways, such as by providing additional contextual information, introducing specific cues, or using supervision signals from training to optimize attention allocation. The advantage of attention steering is that it improves the relevance and consistency of generated text, making the model more flexible and effective in specific tasks or application scenarios. This approach has broad application potential in dialogue systems, text generation, and information retrieval.

[0039] Instruction following, in LLM (Low-Level Modeling), refers to the model's ability to understand and execute user-provided instructions or requests. This ability enables the model to generate relevant text, answer questions, or complete specific tasks based on the user's specific requirements. The key to instruction following is the model's ability to parse the intent of the instruction and generate the expected output. This typically involves understanding natural language, grasping context, and adapting to the task. To improve instruction following performance, researchers often use a large number of instruction-response pairs during model training to help the model learn how to better understand and execute various types of instructions. The ability to follow instructions makes LLM more flexible and practical in real-world applications, meeting user needs in various scenarios such as dialogue, information retrieval, and text generation.

[0040] Complex instructions, or complex instructions, refer to task instructions that involve multiple constraints during execution. These instructions typically contain multiple sub-instructions, each with different semantic and format constraints that must be satisfied simultaneously. As the complexity of real-world tasks increases, research on complex instructions becomes increasingly important. Complex instructions are more than just simple task descriptions; they require the integration of various constraints, such as semantic and format constraints. In general, research on complex instructions aims to improve the ability of LLM (Limited Modeling Model) to understand and execute complex tasks, especially when dealing with multi-level, multi-constraint instructions. In-depth research on complex instructions can better enhance the instruction compliance ability of models, making them more effective in practical applications.

[0041] Complex instructions include three common structure types: chain structure, bench structure, and parallel structure. (See reference...) Figure 1 As shown, Figure 1 This is a schematic diagram of three types of complex instructions.

[0042] Complex instructions with a chained structure require the sequential execution of multiple sub-instructions (or subtasks). Without loss of generality, a complex instruction with a chained structure consists of m sub-instructions {T1, T2, ..., T...}. m}, m subinstructions need to be executed sequentially, subinstruction T k+1 The generation depends on the generation of the first k sub-instructions.

[0043] For example, a complex instruction with a chain structure is: Please introduce “Mona Lisa” briefly. Firstly, introduce the year of creation, then, describe the background of the work's creation, and finally, summarize the impact of the work. This complex instruction includes three sub-instructions, where each of the three keywords, Firstly, then, and finally, is followed by a sub-instruction, and the three sub-instructions are executed sequentially.

[0044] Branching structures, also known as selection structures, involve complex instructions that select a specific branch based on given conditions and execute sub-instructions associated with that selected branch. Without loss of generality, a complex instruction in a branching structure includes m distinct branches, each corresponding to a sub-instruction {T1, T2, ..., T...}. m} and their respective expected outputs {Y1,Y2,…,Y} m Define a selection function S with a value range of [1, m]. Taking the selection condition Cond as input, the selection function S determines the sub-instruction to be executed. Finally, the expected result of the complex instruction is Y. {S(Cond)} .

[0045] For example, a complex instruction for a branch structure is:

[0046] Please introduce the following painting:

[0047] -if the work contains any animal, the description should be inEnglish;

[0048] -otherwise, the description should be in Chinese;

[0049] Painting: "Mona Lisa"

[0050] This complex instruction consists of two branches. The keywords "if the work contains any animal" and "otherwise" each correspond to one branch, and the content following these two keywords is the sub-instruction.

[0051] Complex instructions in a parallel architecture require the independent execution of multiple sub-instructions. Without loss of generality, a complex instruction in a parallel architecture consists of m sub-instructions {T1, T2, ..., T...}. m Each sub-instruction executes independently and produces its own expected output {Y1,Y2,…,Y}. m The expected result of generating complex instructions is a parallel combination of the individual outputs of these sub-instructions.

[0052] For example, a complex instruction set for a parallel architecture would be:

[0053] Please introduce the following painting:

[0054] Painting 1: "Mona Lisa"

[0055] Painting2: “The Starry Night”

[0056] This complex instruction consists of two sub-instructions, with each of the keywords “Painting1:” and “Painting2:” followed by a sub-instruction.

[0057] Optionally, the complexity of a complex instruction can be defined. Complexity can be defined by the number of sub-instructions included in the complex instruction; the more sub-instructions a complex instruction includes, the higher its complexity. As complexity increases, the challenge of the model conforming to relevant requirements also intensifies proportionally. It is worth noting that each sub-instruction may contain multiple constraints that must be satisfied simultaneously.

[0058] Figure 2 This is a schematic diagram of the structure of a large language model applicable to embodiments of this application, such as... Figure 2 As shown, the large language model includes L layers of processing units, each of which includes a multi-head attention (MHA) module and a fully connected feed-forward network (FFN).

[0059] Multi-head attention modules are used to process the input sequence in parallel using multiple attention heads. Each attention head maps the input sequence to a subspace and independently computes attention within its own subspace. Multiple attention heads can capture the features of the input sequence in multiple different subspaces. For example, one attention head focuses on the syntactic structure of the input sequence, while another focuses on the semantic relationships. By using multiple attention heads, the input sequence can be understood more comprehensively from multiple perspectives.

[0060] Each attention head employs a self-attention mechanism to assign different weights to each element (such as a word) in the input sequence, determining which other elements in the sequence require priority when processing a particular element. Each attention head has three key vectors: a query vector (Q), a key vector (K), and a value vector (V). The query vector, also called the query matrix, is used to evaluate the relevance of the current element to other elements; the key vector, also called the key matrix, is used to match the query vector to determine relevance; and the value vector, also called the value matrix, is the actual representation of the elements.

[0061] Elements in the input sequence are also called tokens. Tokens are the basic units of text processing. In large language models, input instructions are broken down into multiple tokens according to certain rules. Tokens can be words, phrases, punctuation marks, single Chinese characters, etc. Through a multi-head attention module, the model can capture the relationships between different tokens.

[0062] For example, the output vector of each head can be represented as:

[0063]

[0064] Where Q represents the query vector, K represents the key vector, V represents the value vector, and d k It is the dimension of attention head, QK T This represents the transpose (or dot product) of the query vector Q and the key vector K, where softmax() is the activation function.

[0065] QK T The result of the calculation is an attention matrix. Each element of the attention matrix represents the degree of association between the query vector and the key vector. Rows in the matrix correspond to different query vectors, and columns correspond to different key vectors. The larger the element value, the stronger the association between the query vector and the key vector at that position. In subsequent calculations, the corresponding value vector V will be assigned a higher weight, and more attention will be paid to the information at that position. The attention matrix is ​​normalized using softmax() to obtain attention weights, which are then multiplied by the value vector V to obtain a new representation.

[0066] Multiple attention heads will have multiple different sets of query vectors, key vectors, and value vectors. Each attention head independently calculates its own output vector (also called the output matrix). The output vectors of multiple attention heads are concatenated to obtain the final output result of the multi-head attention module.

[0067] Feedforward networks typically consist of multiple fully connected layers containing non-linear activation functions (such as ReLU). Their main function is to further transform and non-linearly map the feature representations output by the multi-head attention module. Linear transformations can combine and weight features, while non-linear activation functions introduce non-linear factors, enabling the model to learn more complex functional relationships.

[0068] In large language models, multi-head attention modules and feedforward networks work together to understand and process input sequences. The multi-head attention module is responsible for capturing dependencies in the sequence and extracting multi-perspective features, while the feedforward network further transforms, integrates, and refines these features, ultimately outputting high-quality results.

[0069] Understandable Figure 2 The diagram shown is a simplified representation of a large language model structure; different large language models may include more modules. Taking the Transformer architecture as an example, a Transformer includes multiple encoders and multiple decoders. The input sequence undergoes feature extraction through multiple encoders, and the output of the last encoder serves as the input to each decoder. The multiple encoders are structurally identical, each comprising two modules: a multi-head attention module and a feedforward network. Each decoder also includes a multi-head attention module and a feedforward network. The decoder further includes an encoder-decoder attention module, located between the multi-head attention module and the feedforward network.

[0070] In existing technologies, large language models have poor ability to follow complex instructions, resulting in low quality of the output results of complex instructions generated by large language models, which do not meet user expectations. For example, the output results of the generated complex instructions are confusing or ambiguous.

[0071] In existing technologies, to improve the alignment ability of large language models in practical tasks, the following methods are mainly used to optimize large language models:

[0072] (1) Fine-tuning the model using pre-formed complex instructions can enable the model to better follow complex instructions. However, fine-tuning a large language model requires a large and diverse dataset of complex instructions, and obtaining this data is a huge challenge. In addition, the fine-tuning process is time-consuming and computationally resource-intensive, which limits its feasibility in practical applications.

[0073] (2) The agent-based method continuously feeds back information during the model inference process and iteratively optimizes the model output based on the feedback information. However, this method requires high-quality feedback information, which is difficult to guarantee in real-world scenarios. Furthermore, this method usually requires multiple inference iterations, resulting in low generation efficiency and making it difficult to meet the real-time requirements of scenarios.

[0074] (3) Before inputting complex instructions into the large language model, the complex instructions are preprocessed to decompose them into sub-instructions or generate high-level plans. The large language model generates output results based on the preprocessed sub-instructions or high-level plans. However, preprocessing adds an extra burden. In addition, when processing complex instructions, LLM allocates attention to the entire instruction, allowing irrelevant information to interfere with the generation process, resulting in chaotic or ambiguous model output.

[0075] Therefore, this application provides a text generation method that does not require fine-tuning of the LLM or modification of the LLM model parameters. It can improve the model's ability to follow complex instructions without changing the original structure of the model, thereby improving the quality of the output results of complex instructions generated by the LLM.

[0076] Figure 3 This is a schematic diagram of the Structured Attention Steering (STAR) architecture provided in the embodiments of this application. The text generation method provided in the embodiments of this application is applied under the STAR architecture, such as... Figure 3 As shown, the STAR architecture adds a structure analysis and identification module and an attention steering module to the existing LLM.

[0077] In addition to being processed by the LLM, the input instructions to the LLM also undergo structural analysis by the structural analysis and identification module. This analysis yields the structural type of the input instruction and the multiple sub-instructions it includes. These sub-instructions are denoted as T = {T1, T2, ..., T...} m The structure type of the input instruction and the multiple sub-instructions included in the input instruction are sent to the attention guidance module.

[0078] The attention guidance module is used to obtain the attention matrix A of the h-th attention head in the l-th layer of the large language model during the processing of input instructions. (l,h)Based on the attention matrix, a key sub-instruction is determined from the multiple sub-instructions included in the input instruction. This key sub-instruction is the one that contributes the most to the prediction of the next token. The attention scores of irrelevant sub-instructions in the attention matrix are masked to obtain the guided attention matrix. The guided attention matrix is ​​then sent to the LLM. This guided attention matrix can also be called the corrected attention matrix.

[0079] The attention guidance module sends the guided attention matrix to the h-th attention head of the l-th layer of the large language model. The h-th attention head calculates the attention weights based on the guided attention matrix and performs subsequent processing.

[0080] In this embodiment, when generating each token, the LLM uses the above-described method to guide the attention weights. By dynamically guiding the attention weights, attention to sub-instructions that are not related to the key sub-instructions (or active sub-instructions) is shielded, enabling the model to focus on the key sub-instructions, avoiding interference from irrelevant information, improving the quality of the LLM's output, and making the model's output more in line with expectations.

[0081] Example 1

[0082] Figure 4 This document provides a flowchart of a text generation method for Embodiment 1 of this application. The method in this embodiment can be executed by a terminal device, which may be a smartphone, tablet, laptop, desktop computer, smart TV, gaming device, smart home device, in-vehicle terminal, aircraft, etc. Figure 4 As shown, the method provided in this embodiment includes the following steps.

[0083] S101. Obtain the attention matrix of the h-th attention head of the l-th layer of the LLM, wherein the attention matrix is ​​obtained by the LLM processing the input instructions. The LLM includes L layers of processing units, each of which includes a multi-head attention module, and the multi-head attention module includes H attention heads.

[0084] Text generation software runs on the terminal device, capable of performing text generation tasks. This text generation software is implemented using an LLM (Limited Language Management System). The text generation software can be an application running on the terminal device, or it can be implemented as a small program or application module within a host application.

[0085] The user interface (UI) of text generation software includes an information input box. Users input LLM input instructions through this information input box. The input instructions can be complex instructions or other input instructions containing multiple sub-instructions.

[0086] The LLM comprises L layers of processing units, where L is a positive integer. Each layer includes a multi-head attention module. This processing unit may also include other modules such as a feedforward network. The specific structure of the processing units is not limited in this embodiment. The multi-head attention module includes H attention heads, where H is an integer greater than or equal to 2. The structure of the LLM is as described above. Figure 2 As shown, it will not be elaborated further here.

[0087] The LLM processes the input instructions to obtain the attention matrix of the h-th attention head in the l-th layer. The attention guidance module can obtain the attention matrix of the h-th attention head in the l-th layer from the LLM, where l takes at least one value from 1 to L, and h takes at least one value from 1 to H.

[0088] The LLM processes the input commands according to the normal processing flow. When it passes through the multi-head attention module of the l-th layer, the multi-head attention module of this layer sends the attention matrix of the h-th attention head to the attention guidance module. The attention matrix of the h-th attention head of the l-th layer can be represented as A. (l,h) LLM can send the attention matrix of the h-th attention head in layer l to the attention guidance module through an intermediate feature sharing mechanism.

[0089] S102. Perform structural analysis on the input instruction to obtain the multiple sub-instructions included in the input instruction.

[0090] Reference Figure 3 As shown, the input instruction has two processing paths: one is processed by the LLM, and the other is processed by the structural analysis and recognition module. These two processing paths can be processed simultaneously in parallel.

[0091] The structural analysis and identification module performs structural analysis on the input instruction, obtaining multiple sub-instructions included in the input instruction. Some instructions are mutually exclusive. Mutually exclusive sub-instructions are those that cannot be executed simultaneously logically or whose execution processes are mutually exclusive. In other words, mutually exclusive sub-instructions have no dependency relationship during execution. That is, when executing a sub-instruction, if the sub-instruction does not depend on another sub-instruction, then the sub-instruction and the other sub-instruction are mutually exclusive. It should be noted that whether or not it depends on other sub-instructions here refers to depending on the other sub-instructions themselves, not on the generated results of other sub-instructions.

[0092] Optionally, the structure analysis and identification module also outputs the structure type of the input instruction. The structure type of the input instruction includes, but is not limited to, chain structure, branch structure, or parallel structure. The descriptions of various structure types are as described in the aforementioned content and will not be repeated here.

[0093] The structure analysis and recognition module can first identify the structure type of the input instruction, and then break it down into multiple sub-instructions based on the structure type. Taking a chain structure as an example, the input instruction is broken down into multiple sub-instructions that are executed sequentially based on the keywords in the input instruction. These sequentially executed sub-instructions are mutually exclusive in pairs. For example, if the input instruction in this chain structure includes three sequentially executed sub-instructions: A, B, and C, then A and B are mutually exclusive sub-instructions, B and C are mutually exclusive sub-instructions, and A and C are mutually exclusive sub-instructions.

[0094] For branching structures, multiple branches are determined based on keywords in the input instruction, and each branch is broken down into sub-instructions. These sub-instructions are mutually exclusive. Taking a parallel structure as an example, the input instruction is split into multiple sub-instructions based on keywords, and these sub-instructions are mutually exclusive.

[0095] Optionally, the structure analysis and recognition module can use traditional analysis methods, traditional machine learning methods, or large language models to determine the structure type and sub-instructions of the input instruction. When the structure analysis and recognition module uses traditional machine learning methods or large language models, the input instruction is input into the structure analysis and recognition model, which outputs the structure type of the input instruction and the multiple mutually exclusive sub-instructions included in the input instruction.

[0096] The structure type and sub-instructions of the input instructions identified by the structure analysis and recognition module serve as both guidance and constraints for the subsequent attention guidance mechanism. In this embodiment, attention guidance is limited to the scope of multiple sub-instructions to maintain the integrity of global information.

[0097] In this embodiment, l takes at least one value from 1 to L, and h takes at least one value from 1 to H. That is, attention guidance can be performed by selecting some attention heads in the model or by selecting all attention heads in the model.

[0098] When l takes values ​​from 1 to L (i.e., all values ​​from 1 to L) and h takes values ​​from 1 to H (i.e., all values ​​from 1 to H), attention head guidance is required for all attention heads in the model. When l takes values ​​from 1 to L, and / or h takes values ​​from 1 to H, attention head guidance is required for some attention heads in the model.

[0099] When attention guidance is required for some attention heads in the model, it can be pre-configured which attention heads to guide. When the token generation stage reaches that attention head, that attention head sends the attention matrix to the attention guidance module.

[0100] For example, attention guidance can be performed by selecting the multi-head attention module of the layer with the highest activation response value, or by selecting a shallow, intermediate, or deep multi-head attention module. After selecting a layer, attention guidance can be performed on all attention heads of the selected layer, or attention guidance can be further selected to perform attention guidance on only a subset of the attention heads of the selected layer, for example, by selecting a subset of the numbered attention heads. Optionally, attention guidance can also be performed on a subset of attention heads of all layers.

[0101] In this embodiment, attention heads that need attention guidance can be flexibly selected according to the needs of the usage scenario, and precise attention guidance can be given to the selected attention heads to improve the model's understanding ability.

[0102] S103. Based on the attention matrix, determine a key sub-instruction from the multiple sub-instructions included in the input instruction. The key sub-instruction is the one that contributes the most to predicting the next label.

[0103] During each token generation phase, only one sub-instruction is active and dominates the output. This sub-instruction is called the key sub-instruction or active sub-instruction. Among these multiple sub-instructions, only one is the key sub-instruction, which is the sub-instruction that contributes the most to the prediction of the next token. Let T denote the key sub-instruction. key Then T key Belonging to {T1,T2,…,T m}

[0104] In this embodiment, during the generation stage of each token, the attention guidance module dynamically identifies key sub-instructions in that generation stage. Key sub-instructions refer to the sub-instructions that contribute the most to predicting the next token. Here, the next token can be understood as the token currently being predicted or the next token to be predicted. In an autoregressive language model, the next token needs to be predicted based on all the tokens that have already been predicted.

[0105] Each sub-instruction includes multiple tokens, and the tokens included in a sub-instruction are also called the token set of the sub-instruction. Since each token in the multiple sub-instructions has a different position and relationship relative to the next token to be predicted, the contribution of each token in the multiple sub-instructions to predicting the next token is also different, resulting in different contributions from each sub-instruction to the prediction of the next token. Some sub-instructions contribute significantly to the prediction of the next token, while others contribute less. In this embodiment, the sub-instruction with the largest contribution to the prediction of the next token can be selected as the key sub-instruction.

[0106] S104. Mask the attention scores of irrelevant sub-instructions of key sub-instructions in the attention matrix to obtain the guided attention matrix.

[0107] To eliminate interference from irrelevant sub-instructions and ensure the model focuses primarily on key sub-instructions, the attention scores of irrelevant sub-instructions to the key sub-instructions can be masked from the attention matrix, resulting in a guided attention matrix. Irrelevant sub-instructions to the key sub-instructions refer to sub-instructions unrelated to the key sub-instructions. Before masking the attention scores of irrelevant sub-instructions to the key sub-instructions in the attention matrix, these irrelevant sub-instructions must first be identified.

[0108] Optionally, irrelevant sub-instructions of key sub-instructions can be determined based on the contribution value of each sub-instruction to the prediction of the next label. This contribution value can be an attention score calculated based on the attention matrix.

[0109] In one implementation, a subinstruction whose contribution value is less than that of the critical subinstruction is identified as an irrelevant subinstruction of the critical subinstruction. In other words, as long as the contribution value of a subinstruction is less than that of the critical subinstruction, the subinstruction is identified as an irrelevant subinstruction.

[0110] In another implementation, sub-instructions whose contribution value is less than a preset contribution value threshold are identified as irrelevant sub-instructions to the key sub-instruction. This contribution value threshold is less than the contribution value corresponding to the key sub-instruction. For example, when the contribution value corresponding to the key sub-instruction is 0.8, the contribution value threshold can be 0.2. Sub-instructions with contribution values ​​less than 0.2 are identified as irrelevant sub-instructions, while sub-instructions with contribution values ​​between 0.2 and 0.8 can be considered as sub-instructions related to the key sub-instruction.

[0111] Optionally, a subinstruction mutually exclusive with the critical subinstruction can be identified from multiple subinstructions, and the subinstruction mutually exclusive with the critical subinstruction can be identified as an irrelevant subinstruction of the critical subinstruction.

[0112] In one implementation, after determining the irrelevant sub-instructions of the key sub-instruction, a second bias matrix is ​​determined based on the irrelevant sub-instructions of the key sub-instruction and the attention matrix. The second bias matrix has the same size as the attention matrix. The attention score corresponding to the token of the irrelevant sub-instruction of the key sub-instruction in the second bias matrix is ​​a preset negative mask factor. The remaining attention scores in the second bias matrix are 0, and the mask factor is greater than 0. The second bias matrix and the attention matrix are added together to obtain the guided attention matrix.

[0113] For example, the second bias matrix can be determined by the following formula (1):

[0114]

[0115] Among them, T key T represents the set of tokens for key sub-instructions. j T represents the set of tokens for irrelevant sub-instructions of the critical sub-instruction. j ⊥T key T represents j With T key Unrelated; α is the masking factor, which controls the degree of masking. The value of α is greater than 0. This can be understood as being caused by T j The bias matrix of the token set index, obtained through formula (1), when sub-instruction T j With key sub-instruction T key When unrelated, the sub-instruction T in the second bias matrix will be... j The attention score corresponding to the token set is set to -α, and in other cases, the remaining attention scores in the second bias matrix are set to 0.

[0116] T key It can also represent the index of the token set of key sub-instructions, T j It can also represent the index of the set of tokens of irrelevant sub-instructions of the key sub-instruction. It can be understood that each element in the attention matrix is ​​the attention score corresponding to a token, and the attention scores in the attention matrix are identified and referenced by the index of the corresponding token.

[0117] After determining the second bias matrix, the second bias matrix is ​​added to the attention matrix to obtain the guided attention matrix, which can be expressed by the following formula (2):

[0118]

[0119] Among them, A (l,h)' A represents the attention matrix after guidance from the h-th attention head in layer l.(l,h) This represents the attention matrix of the h-th attention head in the l-th layer. A represents (l,h) The second transpose matrix.

[0120] Since the attention score corresponding to the token set of irrelevant sub-instructions of the key sub-instruction in the second bias matrix is ​​set to -α, and the remaining attention scores are 0, after adding the attention matrix to the second bias matrix, the attention score corresponding to the token set of irrelevant sub-instructions of the key sub-instruction in the attention matrix changes due to the addition of -α, while the remaining attention scores remain unchanged due to the addition of 0. Therefore, in the guided attention matrix obtained by addition, the attention score corresponding to the token set of irrelevant sub-instructions of the key sub-instruction changes, while the remaining attention scores remain unchanged.

[0121] In this embodiment, the attention scores of irrelevant sub-instructions of key sub-instructions in the attention matrix are selectively masked through the attention guidance mechanism. By adjusting the distribution of attention scores, the model can pay more attention to key sub-instructions during the generation process and avoid interference from other irrelevant sub-instructions, thereby making the generated results more relevant and consistent with the input instructions.

[0122] S105. Based on the guided attention matrix, the LLM generates the output result corresponding to the input instruction.

[0123] Specifically, the h-th attention head of the l-th layer of the LLM generates the output vector of the h-th attention head of the l-th layer based on the guided attention matrix. Subsequent processing continues based on this output vector until the output result corresponding to the input instruction is generated.

[0124] When the attention guidance module determines the guided attention matrix, it sends the guided attention matrix to the LLM. The h-th attention head of the l-th layer of the LLM generates the output vector of the h-th attention head of the l-th layer based on the guided attention matrix. For example, the h-th attention head of the l-th layer generates the output vector H of the h-th attention head of the l-th layer using the following formula (3). (l,h) :

[0125]

[0126] Where V represents the value vector or value matrix of the h-th attention head in the l-th layer, and the softmax() function represents the normalization exponential function. After applying the Softmax() function, the attention scores of the token set of irrelevant sub-instructions are calculated according to e. αScaling reduces the attention scores of irrelevant sub-instructions, allowing the model to focus more on key sub-instructions and improve its understanding of the input. α controls the degree of masking; a value greater than 0 indicates a stronger masking effect, resulting in more significant suppression or masking of the attention scores of the token set of irrelevant sub-instructions. For example, α is set to log0.01.

[0127] The multi-head attention module in layer l concatenates the output vectors of H attention heads and uses them as input to the next module in layer l, for example, as input to the feedforward network in layer l. The output vector of the feedforward network can be used as input to the multi-head attention module in layer l+1. Following this process, the module passes through the processing units of each layer in sequence to obtain the output result of LLM.

[0128] In this embodiment, the attention score of irrelevant sub-instructions in the attention matrix is ​​selectively masked by the attention guidance mechanism. By adjusting the distribution of attention scores, the model can pay more attention to key sub-instructions during the generation process and avoid interference from other irrelevant sub-instructions, thereby making the generated result more relevant and consistent with the input instruction. Figure 5 The diagram illustrates the existing output of the large language model and the guided output obtained using the method described in this application. Figure 5 The input command is as follows:

[0129] Please introduce the following painting:

[0130] -if the work contains any animal, the description should be inEnglish;

[0131] -otherwise, the description should be in Chinese;

[0132] Painting: "Mona Lisa"

[0133] The input instruction has a branching structure, with the branch selection condition being whether the drawing contains an animal. This input instruction contains two sub-instructions: "the description should be in English" and "the description should be in Chinese." In existing technologies, attention scores are distributed across the entire input, and the two sub-instructions interfere with each other, leading to chaotic model output (e.g., the output includes Chinese characters). The output is disrupted by the instruction from the other branch, failing to meet the expected output of the input instruction. However, by using the method of this application embodiment to guide attention, "the description should be in English" is identified as the key sub-instruction, thereby masking the attention score of the other sub-instruction. (Refer to...) Figure 5 As shown, after guidance, the attention score for the sub-instruction "the description should be in Chinese" becomes 0. The model only focuses on the sub-instruction "the description should be in English", thus ensuring that the output after guidance only focuses on the sub-instruction corresponding to the selected branch and does not focus on the sub-instruction of the other branch. The output result matches the expected result of the input instruction.

[0134] In this embodiment, the attention guidance mechanism operates within a single forward pass without requiring parameter modification. Attention modifications incur a slight runtime overhead, but this overhead is negligible. Attention modifications are limited to the span of the mutex sub-instruction, ensuring the integrity of the surrounding context and preventing the loss of global information.

[0135] Optionally, in one implementation, after determining the bias matrix of the attention matrix, the attention guidance module does not calculate the guided attention matrix, but instead sends the bias matrix of the attention matrix to the LLM. After receiving the bias matrix, the h-th attention head of the l-th layer calculates the output vector of the h-th attention head of the l-th layer using formula (3).

[0136] In this embodiment, by performing structural analysis on the input instructions of the LLM, multiple sub-instructions are obtained. During the generation of each token, the attention matrix of the h-th attention head of the l-th layer of the LLM is obtained. Based on the attention matrix, the key sub-instructions in the current generation stage are dynamically identified from the multiple sub-instructions. The key sub-instructions are those that contribute the most to predicting the next token. The attention scores of irrelevant sub-instructions of the key sub-instructions in the attention matrix are masked to obtain the guided attention matrix, which is then sent to the LLM for subsequent processing. Because the irrelevant sub-instructions of the key sub-instructions are masked in the guided attention matrix, the LLM can focus on the key sub-instructions in the current generation stage when generating the next token, avoiding interference from irrelevant information, and generating more coherent, accurate, and input instruction-compliant outputs, thus improving the quality of the LLM's output results.

[0137] The method described in this application can exist as a plug-in, and is compatible with various LLMs in a plug-in manner. Figure 3 Taking the architecture shown as an example, the structure analysis and recognition module and the attention guidance module can be integrated with LLM as a whole or as a plug-in. This method does not require modification of the structure and parameters of LLM, nor does it require intervention in the training of the model, thus lowering the application threshold and improving the ability of LLM to follow complex instructions with an efficient and flexible technology.

[0138] Example 2

[0139] Based on Embodiment 1, Embodiment 2 of this application provides a method for determining key sub-instructions, which is used to illustrate step S104 in Embodiment 1. In one optional implementation, according to the attention matrix, the attention score of each sub-instruction is calculated to contribute to the prediction of the next token. The attention score is the contribution value of the sub-instruction to the prediction of the next token. Based on the contribution values ​​of multiple sub-instructions to the prediction of the next token, the key sub-instruction is determined from multiple sub-instructions.

[0140] In this embodiment, the attention score contributed by each sub-instruction to the prediction of the next token is used as the contribution value to measure the contribution of each sub-instruction to the prediction of the next token. The attention score contributed by a sub-instruction to the prediction of the next token can be either the total attention score of the sub-instruction's token set to the prediction of the next token, or the average attention score of the sub-instruction's token set to the prediction of the next token.

[0141] Suppose that the token set of the sub-instruction includes N sub-instructions. Each token in the token set of the sub-instruction has a different position relative to the next token to be predicted, and a different relationship with the next token to be predicted. Therefore, each token in the token set of the sub-instruction contributes differently to the prediction of the next token to be predicted.

[0142] Optionally, the attention guidance module can select all or part of the tokens in the token set of the sub-instruction to calculate the contribution value of the sub-instruction.

[0143] Figure 6 This is a flowchart of one implementation of step S104 in Embodiment 1, as follows: Figure 6 As shown, the method provided in this embodiment includes the following steps.

[0144] S1041. Based on the structure type of the input instruction and the multiple sub-instructions included in the input instruction, determine the mutually exclusive sub-instruction pairs in the input instruction. Each mutually exclusive sub-instruction pair consists of two mutually exclusive sub-instructions.

[0145] Input instructions include multiple sub-instructions. When two sub-instructions are mutually exclusive, their information exchange can introduce interference. For example, in a chained input instruction structure, multiple sub-instructions must be executed sequentially; a sub-instruction executed earlier should not consider subsequent sub-instructions. In a branched input instruction structure, only one sub-instruction corresponding to a different branch will be executed; mutual attention between sub-instructions in different branches can lead to ambiguous results. In a parallel input instruction structure, multiple sub-instructions are executed independently; any sub-instruction paying attention to other sub-instructions may cause confusion regarding its own meaning.

[0146] Therefore, in this embodiment, attention between the token sets corresponding to the mutex sub-instruction pairs is first masked to reduce potential mutual interference. Before masking, the mutex sub-instruction pairs in the input instruction are determined based on the structure type of the input instruction and the multiple sub-instructions included in the input instruction. The structure type of the input instruction can be identified by the structure analysis and recognition module.

[0147] S1042. By masking the attention scores corresponding to the interference markers between mutually exclusive sub-instruction pairs in the attention matrix, the corrected attention matrix is ​​obtained.

[0148] A mutex subinstruction pair consists of two mutually exclusive subinstructions. Each subinstruction's token set includes multiple tokens. Some tokens in the token sets of the two mutually exclusive subinstructions will interact and interfere with each other. These tokens are called interference tokens. The attention scores corresponding to interference tokens between the mutex subinstruction pairs need to be masked to reduce the potential mutual interference between interference tokens.

[0149] Figure 7 This is a schematic diagram illustrating the changes in the attention matrix during the guidance process, such as... Figure 7 As shown in Figure (a), it is a schematic diagram of the attention matrix of the h-th attention head in the l-th layer. Each box in the figure represents the attention score corresponding to a token. Taking the input instruction as an example that includes three sub-instructions, the attention scores shown in the gray-filled boxes are the attention scores corresponding to the token set of sub-instruction T1, the attention scores shown in the horizontal-filled boxes are the attention scores corresponding to the token set of sub-instruction T2, the attention scores shown in the diagonal-filled boxes are the attention scores corresponding to the token set of sub-instruction T3, and the attention scores shown in the white boxes are the attention scores corresponding to some uplink and downlink information.

[0150] Reference Figure 7 In (a), the boxes in the first dashed area represent the attention scores corresponding to the interference tokens of instruction T1 and sub-instruction T2. ​​The boxes in the second virtual area represent the attention scores corresponding to the interference tokens of instruction T1 and sub-instruction T3. The boxes in the third virtual area represent the attention scores corresponding to the interference tokens of instruction T2 and sub-instruction T3. The attention scores corresponding to these interference tokens in the attention matrix need to be masked. (b) is the attention matrix after the interference is masked, that is, the corrected attention matrix.

[0151] In one implementation, a first bias matrix is ​​determined based on the mutex sub-instruction pairs and the attention matrix. The first bias matrix has the same size as the attention matrix. The attention score between the mutex sub-instruction pairs in the first bias matrix is ​​a preset negative masking factor. The remaining attention score in the first bias matrix is ​​0, and the masking factor is greater than 0. The first bias matrix and the attention matrix are added together to obtain the corrected attention matrix.

[0152] For example, the first bias matrix can be determined by the following formula (4):

[0153]

[0154] Among them, T i T represents the set of tokens for the i-th sub-instruction. j T represents the set of tokens for the j-th sub-instruction. j ⊥T key T represents j With T j Mutually exclusive, α is the masking factor, α is used to control the degree of masking, and the value of α is greater than 0. It should be noted that the value of the masking factor in formula (4) can be the same as or different from the masking factor in formula (1). This application embodiment does not limit this.

[0155] A represents the attention matrix A of the h-th attention head in the l-th layer. (l,h) The first bias matrix can be understood as being composed of T i The token set and T j The bias matrix of the token set index, obtained through formula (4), when sub-instruction T i AND sub-instruction T j When mutually exclusive, the sub-instruction T in the first bias matrix is... i Sub-instruction T j The attention score corresponding to the interfering token is set to -α; otherwise, the attention score in the first bias matrix is ​​set to 0.

[0156] T i It can also represent the index of the token set of the i-th sub-instruction, T. j It can also represent the index of the token set of the j-th sub-instruction. It can be understood that each element in the attention matrix is ​​the attention score corresponding to a token, and the attention scores in the attention matrix are identified and referenced by the index of the corresponding token.

[0157] After determining the first bias matrix, the first bias matrix is ​​added to the attention matrix to obtain the corrected attention matrix. The corrected attention matrix is ​​the attention matrix after filtering out interference. The corrected attention matrix can be expressed by the following formula (5):

[0158]

[0159] Among them, A (l,h)” Let A represent the corrected attention matrix for the h-th attention head in the l-th layer. (l,h) This represents the attention matrix of the h-th attention head in the l-th layer. A represents (l,h) The first transpose of the matrix.

[0160] Since the attention score corresponding to the interference token between the mutual exclusion sub-instruction pairs in the first bias matrix is ​​set to -α, and the remaining attention scores are set to 0, after the attention matrix is ​​added to the first bias matrix, the attention score corresponding to the interference token between the mutual exclusion sub-instruction pairs in the attention matrix changes due to the addition of -α, while the remaining attention scores do not change due to the addition of 0. Therefore, in the corrected attention matrix, the attention score corresponding to the interference token between the mutual exclusion sub-instruction pairs changes, while the remaining attention scores remain unchanged.

[0161] In this embodiment, by masking the attention scores corresponding to interfering tokens between mutually exclusive sub-instruction pairs in the attention matrix, unexpected interactions between mutually exclusive sub-instructions are avoided, thereby making the contribution values ​​of each sub-instruction calculated based on the corrected attention matrix to predicting the next token more accurate.

[0162] S1043. Based on the corrected attention matrix, calculate the attention score of each sub-instruction's contribution to predicting the next label. This attention score is the contribution value of the sub-instruction to predicting the next label.

[0163] Optionally, the contribution of a sub-instruction to predicting the next token is the average attention score of the contribution of the sub-instruction's set of tokens to predicting the next token.

[0164] In one implementation, the average attention score of the token set of a sub-instruction contributing to the prediction of the next token can be calculated as follows: For each sub-instruction, the average attention score of all tokens between the k-th token and the next token in the modified attention matrix is ​​calculated to obtain the average attention score of the k-th token contributing to the prediction of the next token, where k takes the value 1...N, and N is the number of tokens included in the token set of the sub-instruction; the average of the average attention scores of the N tokens included in the token set of the sub-instruction contributing to the prediction of the next token is calculated to obtain the average attention score of the sub-instruction contributing to the prediction of the next token.

[0165] This implementation can be represented by the following formula (6):

[0166]

[0167] Among them, score(T) i ,t) represents the i-th subinstruction T i The average attention score for predicting the contribution of the next token, |T i | indicates the i-th subinstruction T i The number of tokens in |T i The value of | is equal to N, t represents the index of the next token to be predicted, and k represents T. i The index of the k-th token, tk represents T. i The number of attention scores between the k-th token and the next token t, A (l,h) (q,k) represents T i The attention scores corresponding to the k-th to q-th tokens.

[0168] In this embodiment, the attention guidance module selects all tokens in the token set of the sub-instruction to calculate the contribution value of the sub-instruction. Optionally, in other embodiments of this application, the attention guidance module may also select a portion of the tokens in the token set of the sub-instruction to calculate the contribution value of the sub-instruction.

[0169] In this embodiment, the attention guidance module uses the average attention score of the sub-instruction's token set contributing to the prediction of the next token as the contribution value of the sub-instruction. Optionally, in other embodiments of this application, the total attention score of the sub-instruction's token set contributing to the prediction of the next token can also be used as the contribution value of the sub-instruction.

[0170] S1044. Based on the contribution values ​​of multiple sub-instructions to predicting the next flag, determine the key sub-instruction from the multiple sub-instructions.

[0171] In one implementation, based on the contribution values ​​of multiple sub-instructions to predicting the next token, the sub-instruction with the largest contribution value is selected as the key sub-instruction.

[0172] In another implementation, the information entropy of each sub-instruction is calculated based on its contribution to the prediction of the next token. Then, a key sub-instruction is determined from the multiple sub-instructions based on their contribution and information entropy.

[0173] This implementation method borrows the entropy formula from information theory. The information entropy of each sub-instruction is calculated. Information entropy is used to measure the uncertainty of the distribution, thereby improving the robustness of identifying key sub-instructions.

[0174] Optionally, based on the contribution value and information entropy of multiple sub-instructions to the prediction of the next token, a key sub-instruction is determined from the multiple sub-instructions, including: when the information entropy of each sub-instruction is less than the confidence threshold, the sub-instruction with the largest contribution value to the prediction of the next token is determined as the key sub-instruction; when the information entropy of at least one sub-instruction is not less than the confidence threshold, the sub-instructions with information entropy not less than the confidence threshold are filtered out, and the sub-instruction with the largest contribution value to the prediction of the next token is determined as the key sub-instruction from the remaining sub-instructions.

[0175] This determination method can also be expressed by the following formula (7):

[0176] T key =argmax Ti (score(T i ,t)),stH(score(T,t))<ε (7)

[0177] Among them, T key This indicates a key sub-instruction, score(T) i ,t) represents the i-th subinstruction T i The average attention score for predicting the contribution of the next token, i.e., the i-th sub-instruction T i The contribution value, H(score(T,t)), represents the set of information entropy of multiple sub-instructions. ε represents the confidence threshold, which can be a preset fixed value, or it can be related to the number of sub-instructions included in the input instruction; the more sub-instructions included in the input instruction, the smaller the confidence threshold. When the probability of the distribution is close to 0 or 1, the entropy reaches its minimum, indicating that the uncertainty of the distribution of multiple sub-instructions is low. Using H(score(T,t))<ε as a constraint to measure the confidence of key sub-instructions can ensure the reliability of key sub-instructions.

[0178] Reference Figure 7 As shown in Figure (c), the contributions of three sub-instructions T1, T2, and T3 to the prediction of the next token are illustrated. Among them, T2 has the largest contribution, therefore, T2 is selected as the key sub-instruction. T1 and T3 are both irrelevant sub-instructions to the key sub-instruction. Therefore, the attention scores corresponding to the token sets of T1 and T3 that are irrelevant to T2 in the attention matrix are masked to obtain the guided attention matrix, which is shown in Figure (d).

[0179] In this embodiment, based on the structure type of the input instruction and the multiple sub-instructions included in the input instruction, mutually exclusive sub-instruction pairs are determined. Attention scores corresponding to interfering tokens between mutually exclusive sub-instruction pairs in the attention matrix are masked to obtain a corrected attention matrix. Based on the corrected attention matrix, the attention score contributing to the prediction of the next token for each sub-instruction is calculated; the attention score is the contribution value of the sub-instruction to the prediction of the next token. Based on the contribution values ​​of multiple sub-instructions to the prediction of the next token, a key sub-instruction is determined from the multiple sub-instructions. This method ensures the accuracy of the determined key sub-instruction by calculating the attention score contributing to the prediction of the next token for each sub-instruction and using this attention score as the contribution value of the sub-instruction to the prediction of the next token, thus determining a key sub-instruction based on the contribution values ​​of multiple sub-instructions to the prediction of the next token. Furthermore, by masking the attention scores corresponding to interfering tokens between mutually exclusive sub-instruction pairs in the attention matrix, this method avoids unexpected interactions between mutually exclusive sub-instructions, making the subsequent contribution values ​​of each sub-instruction to the prediction of the next token calculated based on the corrected attention matrix more accurate, thereby improving the accuracy of the determined key sub-instruction.

[0180] In Example 2, the attention scores corresponding to interfering tokens between mutually exclusive sub-instruction pairs in the attention matrix are first masked to obtain a corrected attention matrix. Based on the corrected attention matrix, the attention score contributed by each sub-instruction to predicting the next token is calculated. Optionally, in other embodiments of this application, the attention score contributed by each sub-instruction to predicting the next token can also be calculated directly based on the attention matrix. Specifically, for each sub-instruction, the average attention score of all tokens between the k-th token and the next token in the attention matrix is ​​calculated to obtain the average attention score contributed by the k-th token to predicting the next token, where k takes the value 1...N, and N is the number of tokens included in the token set of the sub-instruction; the average of the average attention scores contributed by the N tokens included in the token set of the sub-instruction to predicting the next token is calculated to obtain the average attention score contributed by the sub-instruction to predicting the next token.

[0181] Example 3

[0182] To facilitate better implementation of the text generation method of this application embodiment, this application embodiment also provides a text generation apparatus. Figure 8 This is a schematic diagram of the structure of the text generation device provided in Embodiment 3 of this application, as shown below. Figure 8 As shown, the text generation device 100 may include: an acquisition module 11, a large language model 12, a structure analysis and recognition module 13, and an attention guidance module 14.

[0183] The acquisition module 11 is used to acquire the attention matrix of the h-th attention head of the l-th layer of the large language model 12. The attention matrix is ​​obtained by the large language model processing the input instructions. The large language model includes L layers of processing units. Each layer of processing units includes a multi-head attention module. The multi-head attention module includes H attention heads. L is a positive integer, H is an integer greater than or equal to 2, l takes at least one value from 1 to L, and h takes at least one value from 1 to H.

[0184] The structural analysis and identification module 13 is used to perform structural analysis on the input instruction to obtain multiple sub-instructions included in the input instruction;

[0185] The attention guidance module 14 is used to determine a key sub-instruction from multiple sub-instructions included in the input instruction according to the attention matrix, wherein the key sub-instruction is the sub-instruction that contributes the most to predicting the next label; and to mask the attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix to obtain the guided attention matrix.

[0186] The large language model 12 is used to generate the output result corresponding to the input instruction based on the guided attention matrix.

[0187] In some exemplary implementations, the attention guidance module 14 is specifically configured to: calculate an attention score for each sub-instruction contributing to the prediction of the next label based on the attention matrix, wherein the attention score is the contribution value of the sub-instruction to the prediction of the next label; and determine the key sub-instruction from the plurality of sub-instructions based on the contribution values ​​of the plurality of sub-instructions to the prediction of the next label.

[0188] In some exemplary implementations, calculating the attention score for each sub-instruction pair's contribution to predicting the next label based on the attention matrix includes: determining mutually exclusive sub-instruction pairs in the input instruction based on the structure type of the input instruction and the multiple sub-instructions included in the input instruction, each mutually exclusive sub-instruction pair consisting of two mutually exclusive sub-instructions; masking the attention scores corresponding to interfering labels between the mutually exclusive sub-instruction pairs in the attention matrix to obtain a modified attention matrix; and calculating the attention score for each sub-instruction pair's contribution to predicting the next label based on the modified attention matrix.

[0189] In some exemplary implementations, the contribution of the sub-instruction to the prediction of the next label is the average attention score of the contribution of the label set of the sub-instruction to the prediction of the next label; the step of calculating the attention score of each sub-instruction to the prediction of the next label according to the modified attention matrix includes: for each sub-instruction, calculating the average of the attention scores of all labels between the k-th label of the sub-instruction and the next label in the modified attention matrix to obtain the average attention score of the contribution of the k-th label to the prediction of the next label, where k takes the value of 1...N, and N is the number of labels included in the label set of the sub-instruction; calculating the average of the average attention scores of the N labels included in the label set of the sub-instruction to the prediction of the next label to obtain the average attention score of the contribution of the sub-instruction to the prediction of the next label.

[0190] In some exemplary implementations, the contribution of the sub-instruction to predicting the next label is the average attention score of the contribution of the label set of the sub-instruction to predicting the next label; the step of calculating the attention score of each sub-instruction to predicting the next label according to the attention matrix includes: for each sub-instruction, calculating the average of the attention scores of all labels between the k-th label of the sub-instruction and the next label in the attention matrix to obtain the average attention score of the contribution of the k-th label to predicting the next label, where k takes the value of 1...N, and N is the number of labels included in the label set of the sub-instruction; calculating the average of the average attention scores of the N labels included in the label set of the sub-instruction to predicting the next label to obtain the average attention score of the contribution of the sub-instruction to predicting the next label.

[0191] In some exemplary implementations, determining the key sub-instruction from the plurality of sub-instructions based on their contribution values ​​includes: calculating the information entropy of each sub-instruction based on its contribution value to predicting the next marker; and determining the key sub-instruction from the plurality of sub-instructions based on their contribution value and information entropy.

[0192] In some exemplary implementations, determining the key sub-instruction from the plurality of sub-instructions based on their contribution values ​​and information entropy to predicting the next marker includes: when the information entropy of each sub-instruction is less than a confidence threshold, determining the sub-instruction with the largest contribution value to predicting the next marker as the key sub-instruction; when the information entropy of at least one sub-instruction is not less than the confidence threshold, filtering out sub-instructions with information entropy not less than the confidence threshold, and determining the sub-instruction with the largest contribution value to predicting the next marker as the key sub-instruction from the remaining sub-instructions.

[0193] In some exemplary implementations, the confidence threshold is related to the number of sub-instructions included in the input instruction; the more sub-instructions included in the input instruction, the smaller the confidence threshold.

[0194] In some exemplary implementations, the step of masking the attention scores between the mutually exclusive sub-instruction pairs in the attention matrix to obtain a modified attention matrix includes: determining a first bias matrix based on the mutually exclusive sub-instruction pairs and the attention matrix, wherein the first bias matrix has the same size as the attention matrix, the attention scores between the mutually exclusive sub-instruction pairs in the first bias matrix are preset negative masking factors, the remaining attention scores in the first bias matrix are 0, and the masking factor is greater than 0; and adding the first bias matrix and the attention matrix to obtain the modified attention matrix.

[0195] In some exemplary implementations, the attention guidance module 14 is specifically used to: determine the irrelevant sub-instructions of the key sub-instruction based on the contribution value of each sub-instruction to the prediction of the next label; determine a second bias matrix based on the irrelevant sub-instructions of the key sub-instruction and the attention matrix, wherein the second bias matrix has the same size as the attention matrix, the attention score corresponding to the label of the irrelevant sub-instruction of the key sub-instruction in the second bias matrix is ​​a preset negative masking factor, the remaining attention score in the second bias matrix is ​​0, and the masking factor is greater than 0; and add the second bias matrix and the attention matrix to obtain the guided attention matrix.

[0196] In some exemplary implementations, the structure analysis and recognition module 13 is a structure analysis and recognition model, which is specifically used to: input the input instruction into the structure analysis and recognition model, and output the structure type of the input instruction and the multiple sub-instructions included in the input instruction.

[0197] In some exemplary embodiments, the attention guidance module 14 and the structure analysis and recognition module 13 interact with the large language model 12 as plug-ins.

[0198] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here.

[0199] The apparatus 100 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly manifested as execution by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0200] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0201] This application also provides an electronic device. Figure 9 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application, as shown below. Figure 9 As shown, the electronic device 200 may include:

[0202] The system includes a memory 202 for storing computer programs and a processor 201 for transferring program code to the processor 201. In other words, the processor 201 can retrieve and run the computer program from the memory 202 to implement the method described below:

[0203] Obtain the attention matrix of the h-th attention head in the l-th layer of the large language model, where the attention matrix is ​​obtained by the large language model processing the input instruction. The large language model includes L layers of processing units, each of which includes a multi-head attention module. The multi-head attention module includes H attention heads, where L is a positive integer, H is an integer greater than or equal to 2, l takes at least one value from 1 to L, and h takes at least one value from 1 to H. Perform structural analysis on the input instruction to obtain multiple sub-instructions included in the input instruction. Based on the attention matrix, determine a key sub-instruction from the multiple sub-instructions included in the input instruction. The key sub-instruction is the sub-instruction that contributes the most to predicting the next label. Mask the attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix to obtain a guided attention matrix. Generate the output result corresponding to the input instruction through the large language model based on the guided attention matrix.

[0204] The specific implementation of the method is described in the foregoing method embodiments and will not be repeated here.

[0205] In some embodiments of this application, the processor 201 may include, but is not limited to: a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0206] In some embodiments of this application, the memory 202 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0207] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 202 and executed by the processor 201 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device 200.

[0208] Optional, such as Figure 9 As shown, the electronic device 200 also includes: a display screen 203, a radio frequency circuit 204, an audio circuit 205, an input unit 206, and a power supply 207. The processor 201 is electrically connected to the display screen 203, the radio frequency circuit 204, the audio circuit 205, the input unit 206, and the power supply 207. Those skilled in the art will understand that... Figure 9 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0209] The display screen 203 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The display screen 203 can be a touch screen, which may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the computer device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 201. It can also receive and execute commands from the processor 201. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 201 to determine the type of touch event. Subsequently, the processor 201 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into a touch display screen to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen can be implemented as two independent components to achieve input and output functions. That is, the touch display screen can also be used as part of the input unit 206 to achieve input functions.

[0210] The radio frequency circuit 204 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other computer devices, and to transmit and receive signals with network devices or other computer devices.

[0211] Audio circuitry 205 can be used to provide an audio interface between a user and a computer device via a speaker and a microphone. Audio circuitry 205 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 205, converted back into audio data, and then processed by processor 201 before being transmitted via radio frequency circuitry 204 to, for example, another computer device, or output to memory 302 for further processing. Audio circuitry 205 may also include an earphone jack to facilitate communication between peripheral headphones and the computer device.

[0212] The input unit 206 can be used to receive input numbers, character information or object feature information (such as fingerprints, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0213] Power supply 207 is used to supply power to various components of electronic device 200. Optionally, power supply 207 can be logically connected to processor 201 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 207 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0214] although Figure 9 As not shown in the diagram, the electronic device 200 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0215] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0216] This application also provides a computer program product comprising a computer program stored in a computer-readable storage medium. The processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the corresponding processes in the above method embodiments; for brevity, these will not be elaborated further here.

[0217] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0218] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0219] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A text generation method, characterized in that, The method includes: Obtain the attention matrix of the h-th attention head in the l-th layer of the large language model, wherein the attention matrix is ​​obtained by the large language model processing the input instruction. The large language model includes L layers of processing units, each of which includes a multi-head attention module. The multi-head attention module includes H attention heads, where L is a positive integer, H is an integer greater than or equal to 2, l takes at least one value from 1 to L, and h takes at least one value from 1 to H. Structural analysis is performed on the input instruction to obtain multiple sub-instructions included in the input instruction; Based on the attention matrix, a key sub-instruction is determined from the multiple sub-instructions included in the input instruction. The key sub-instruction is the sub-instruction that contributes the most to predicting the next label. The attention scores of irrelevant sub-instructions of the key sub-instructions in the attention matrix are masked to obtain the guided attention matrix; The large language model generates the output corresponding to the input instruction based on the guided attention matrix.

2. The method according to claim 1, characterized in that, The step of determining a key sub-instruction from multiple sub-instructions included in the input instruction based on the attention matrix includes: Based on the attention matrix, calculate the attention score for each sub-instruction's contribution to predicting the next label, where the attention score is the contribution value of the sub-instruction to predicting the next label; The key sub-instruction is determined from the plurality of sub-instructions based on their contribution to the prediction of the next tag.

3. The method according to claim 2, characterized in that, The step of calculating the attention score for each sub-instruction's contribution to predicting the next label based on the attention matrix includes: Based on the structure type of the input instruction and the multiple sub-instructions included in the input instruction, determine the mutually exclusive sub-instruction pairs in the input instruction, each of which consists of two mutually exclusive sub-instructions; By masking the attention scores corresponding to the interference markers between the mutually exclusive sub-instruction pairs in the attention matrix, a corrected attention matrix is ​​obtained. Based on the modified attention matrix, calculate the attention score for each sub-instruction's contribution to predicting the next label.

4. The method according to claim 3, characterized in that, The contribution of the sub-instruction to the prediction of the next label is the average attention score of the contribution of the label set of the sub-instruction to the prediction of the next label; The step of calculating the attention score for each sub-instruction's contribution to predicting the next label based on the corrected attention matrix includes: For each sub-instruction, the average attention score of all the labels between the k-th label and the next label in the modified attention matrix is ​​calculated to obtain the average attention score of the contribution of the k-th label to the prediction of the next label, where k takes the value of 1, 2, ..., N, and N is the number of labels included in the label set of the sub-instruction; The average attention score of the sub-instruction's contribution to predicting the next label is calculated by averaging the N labels included in the label set of the sub-instruction.

5. The method according to claim 2, characterized in that, The contribution of the sub-instruction to the prediction of the next label is the average attention score of the contribution of the label set of the sub-instruction to the prediction of the next label; The step of calculating the attention score for each sub-instruction's contribution to predicting the next label based on the attention matrix includes: For each sub-instruction, the average attention score of all the tags between the k-th tag of the sub-instruction and the next tag in the attention matrix is ​​calculated to obtain the average attention score of the contribution of the k-th tag to the prediction of the next tag, where k takes the value 1, 2, ..., N, and N is the number of tags included in the tag set of the sub-instruction; The average attention score of the sub-instruction's contribution to predicting the next label is calculated by averaging the N labels included in the label set of the sub-instruction.

6. The method according to any one of claims 2-5, characterized in that, The step of determining the key sub-instruction from the plurality of sub-instructions based on their contribution values ​​includes: Based on the contribution values ​​of the multiple sub-instructions to the prediction of the next tag, calculate the information entropy of each sub-instruction; The key sub-instruction is determined from the plurality of sub-instructions based on their contribution to the prediction of the next tag and the information entropy.

7. The method according to claim 6, characterized in that, The step of determining the key sub-instruction from the plurality of sub-instructions based on their contribution to the prediction of the next marker and the information entropy includes: When the information entropy of each sub-instruction is less than the confidence threshold, the sub-instruction with the largest contribution to predicting the next label is determined from the plurality of sub-instructions as the key sub-instruction; When at least one sub-instruction has an information entropy not less than the confidence threshold, the sub-instructions with information entropy not less than the confidence threshold are filtered out, and the sub-instruction with the largest contribution to predicting the next label is determined from the remaining sub-instructions as the key sub-instruction.

8. The method according to claim 7, characterized in that, The confidence threshold is related to the number of sub-instructions included in the input instruction; the more sub-instructions included in the input instruction, the smaller the confidence threshold.

9. The method according to claim 3 or 4, characterized in that, The process of masking the attention scores between the mutually exclusive sub-instruction pairs in the attention matrix to obtain a modified attention matrix includes: Based on the mutual exclusion sub-instruction pairs and the attention matrix, a first bias matrix is ​​determined, wherein the first bias matrix has the same size as the attention matrix, the attention score between the mutual exclusion sub-instruction pairs in the first bias matrix is ​​a preset negative masking factor, the remaining attention score in the first bias matrix is ​​0, and the masking factor is greater than 0. The first bias matrix and the attention matrix are added together to obtain the corrected attention matrix.

10. The method according to any one of claims 2-5, characterized in that, The process of masking the attention scores of irrelevant sub-instructions of the key sub-instructions in the attention matrix to obtain the guided attention matrix includes: Based on the contribution value of each sub-instruction to the prediction of the next marker, determine the irrelevant sub-instructions of the key sub-instruction; Based on the irrelevant sub-instructions of the key sub-instruction and the attention matrix, a second bias matrix is ​​determined, wherein the second bias matrix has the same size as the attention matrix, the attention score corresponding to the tag of the irrelevant sub-instruction of the key sub-instruction in the second bias matrix is ​​a preset negative masking factor, the remaining attention score in the second bias matrix is ​​0, and the masking factor is greater than 0. The second bias matrix and the attention matrix are added together to obtain the guided attention matrix.

11. The method according to any one of claims 1-5, characterized in that, The structural analysis of the input instruction yields multiple sub-instructions, including: The input instruction is input into the structure analysis and recognition model, and the structure analysis and recognition model outputs the structure type of the input instruction and the multiple sub-instructions included in the input instruction.

12. The method according to any one of claims 1-5, characterized in that, The following steps are performed by the structural analysis and identification module: Structural analysis is performed on the input instruction to obtain multiple sub-instructions included in the input instruction; The following steps are performed by the attention guidance module: Obtain the attention matrix of the h-th attention head in the l-th layer of the large language model; Based on the attention matrix, a key sub-instruction is determined from the multiple sub-instructions included in the input instruction; The attention scores of irrelevant sub-instructions of the key sub-instructions in the attention matrix are masked to obtain the guided attention matrix, and the guided attention matrix is ​​sent to the large language model; The attention guidance module and the structure analysis and recognition module interact with the large language model as plug-ins.

13. A text generation device, characterized in that, The device includes: The acquisition module is used to acquire the attention matrix of the h-th attention head of the l-th layer of the large language model. The attention matrix is ​​obtained by the large language model processing the input instructions. The large language model includes L layers of processing units. Each layer of processing units includes a multi-head attention module. The multi-head attention module includes H attention heads. L is a positive integer, H is an integer greater than or equal to 2, l takes at least one value from 1 to L, and h takes at least one value from 1 to H. The structural analysis and identification module is used to perform structural analysis on the input instruction to obtain multiple sub-instructions included in the input instruction; An attention guidance module is used to determine a key sub-instruction from multiple sub-instructions included in the input instruction based on the attention matrix. The key sub-instruction is the sub-instruction that contributes the most to predicting the next label. The attention scores of irrelevant sub-instructions of the key sub-instruction in the attention matrix are masked to obtain a guided attention matrix. The large language model is used to generate the output result corresponding to the input instruction based on the guided attention matrix.

14. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to invoke and run the computer program stored in the memory to perform the method of any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 12.

Images