Toxic thinking chain analysis and optimization method and device for large language model

Abstract

The invention provides a toxic thinking chain analysis and optimization method and device for a large language model. The method comprises the steps that a question text is obtained and input into the large language model for text question answering; reasoning drift information loss existing when the toxic thinking chain is generated is analyzed through attribution tracking, or answer offset information loss existing when an answer text is generated according to the toxic thinking chain is analyzed through causal tracking; performing residual decoding on the toxic thinking chain to obtain an optimized thinking chain when reasoning drift information loss is analyzed, and performing text sequence position exchange on the question text and the toxic thinking chain in a thinking chain text fragment to obtain an optimized text sequence when answer drift information loss is analyzed; and generating an answer text by the large language model based on the optimized thinking chain or the optimized text sequence. Through the method and the device, the defect of low reasoning performance caused by the fact that an optimization method of a large language model toxic thinking chain cannot be universally adapted in a question and answer scene is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and device for analyzing and optimizing toxic thought chains for large language models. Background Art

[0002] As their size increases, large language models (LLMs) perform well in different reasoning tasks, among which commonsense reasoning has attracted much attention due to its importance to general intelligence. A series of enhancement methods such as Chain of Thoughts (CoT) are proposed in reasoning tasks to stimulate the potential capabilities of large language models. Through the Chain of Thoughts technique, LLMs can generate reasonable inferences and improve their reasoning performance.

[0003] However, the large language model based on thought chain has an overlooked problem, which is called the toxicity of CoT, that is, toxic thought chain. Specifically, if thought chain is not used, LLMs can directly provide the correct answer to the question, but after applying methods such as thought chain, it brings additional reasoning paths to LLMs, causing LLMs to produce wrong reasoning answers. Toxic thought chain will lead to two main types of errors - reasoning drift and answer drift. Reasoning drift is that the semantics of the answer result deviates from the semantics of the question asked, causing LLMs to produce wrong reasoning results. Answer drift means that LLMs can give the correct reasoning result based on the question asked, and can also give the correct reasoning result after generating the correct thought chain, but if the wrong toxic thought chain is generated, the answer will produce an incorrect reasoning result.

[0004] In the prior art, for the problem of LLMs generating toxic thought chains, the prompt words are extracted, intervened or rewritten and the answer results are compared to analyze the reasons why LLMs generate incorrect answers due to toxic thought chains, or additional supervision signals and training processes are designed for LLMs, or external knowledge base resources are referenced to improve the performance of LLMs. However, these methods lack research on the internal mechanisms of LLMs, resulting in the analysis and optimization methods of toxic thought chains not being universally applicable, making the reasoning performance of LLMs still low. Summary of the invention

[0005] The present invention provides a toxic thought chain analysis and optimization method and device for a large language model, so as to overcome the defect that the optimization method of the toxic thought chain of a large language model in the prior art cannot be universally adapted in question-answering scenarios, resulting in low reasoning performance.

[0006] The present invention provides a toxic thought chain analysis and optimization method for a large language model, the method comprising the following steps: Obtaining a question text, and inputting the question text into a large language model for text question answering; When conducting text question answering, the reasoning drift information loss that exists when the large language model generates a toxic thought chain based on the question text is analyzed by attribution tracking, or the answer offset information loss that exists when the large language model generates an answer text based on the toxic thought chain is analyzed by causal tracking; When it is analyzed that there is reasoning drift information loss in the text question and answer, residual decoding processing is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model generates an answer text for the question text based on the optimized thought chain; When it is analyzed that there is answer drift information loss in the text question and answer, the question text and the toxic thought chain are exchanged in the text sequence position in the thought chain text segment to obtain an optimized text sequence, so that the large language model generates an answer text for the question text based on the optimized text sequence.

[0007] In some embodiments, the analyzing, by attribution tracking, the loss of reasoning drift information when the large language model generates a toxic thought chain according to the question text includes: In the process of generating the toxic thought chain according to the question text, determining the attribution score of each attention layer in the large language model, and summing the attribution scores of each attention layer to obtain a final attribution score; In each step of generating a toxic thinking chain, determining a step information flow between the question text and the toxic thinking chain according to the final attribution score, and taking an average of the step information flow as a first average information flow for generating the toxic thinking chain; Based on the question text, generate a correct thought chain through a large language model, and determine a second average information flow in the process of generating the correct thought chain; When the first average information flow is less than the second average information flow, it is determined that there is reasoning drift information loss when the large language model generates the toxic thought chain.

[0008] In some embodiments, the analysis of the answer deviation information loss when the large language model generates the answer text according to the toxic thought chain by causal tracing includes: In a forward propagation process before the big language model generates an answer text according to the toxic thought chain, the first latent vector of each attention layer of the question text in the Transformer module of the big language model is determined, and the normal prediction probability when generating the answer text is determined according to the first latent vector; Preset a fixed Gaussian noise vector, sum the first latent vector and the Gaussian noise vector to obtain a second latent vector, and determine the interference prediction probability when generating the answer text based on the second latent vector; Determining a direct effect value of the Transformer module for the answer text according to the normal prediction probability and the interference prediction probability, and determining a first average direct effect value of the Transformer module based on the direct effect value; When the large language model generates a response text according to the correct thought chain, determining a second average direct effect value of the Transformer module; When the first average direct effect value is less than the second average direct effect value, it is determined that there is answer offset information loss when the large language model generates answer text according to the toxic thought chain.

[0009] In some embodiments, the residual decoding process is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, including: For each original thought chain word in the generated toxic thought chain, determine the model prediction score of the original thought chain word generated by the large language model, and select multiple target thought chain words with the highest model prediction scores; For each target thought chain word, determine the attention score of the context of the question text and the original thought chain word, and weight the attention score by a preset attention weight to obtain a reward score for the target thought chain word; The model prediction score of the target thought chain word is summed with the reward score to obtain a final prediction score of the target thought chain word; By finally predicting the target thought chain word with the highest score, the original thought chain words in the toxic thought chain are updated to obtain an optimized thought chain.

[0010] In some embodiments, in the thought chain text segment, the question text and the toxic thought chain are exchanged in text sequence position to obtain an optimized text sequence, including: When the large language model generates the answer text according to the toxic thought chain, the position of the question text sequence in the thought chain text segment is swapped with the position of the toxic thought chain sequence to obtain an optimized text sequence; Among them, the thought chain text fragments are spliced ​​in sequence according to the question text sequence, the toxic thought chain sequence, the last word sequence and the answer text sequence, and the optimized text sequence is spliced ​​in sequence according to the toxic thought chain sequence, the question text sequence, the last word sequence and the answer text sequence.

[0011] In some embodiments, the method further comprises: When it is analyzed that there is reasoning drift information loss or answer drift information loss in the text question and answer, the toxicity rate of the toxic thinking chain is determined, and the process of determining the toxicity rate includes: Acquire multiple candidate question texts, call the large language model to generate a test thought chain, generate a first correct answer text or a first wrong answer text based on the test thought chain, and select a first target question text corresponding to the first correct answer text from the candidate question texts; Optimizing the test thought chain, calling the large language model to generate a second correct answer text or a second wrong answer text based on the optimized thought chain, and selecting a second target question text corresponding to the second wrong answer text from the candidate question texts, wherein the optimization process includes residual decoding and text sequence position exchange; Determine the intersection of the first target question text and the second target question text, and use the ratio of the intersection to the second target question text as the toxicity rate of the test thinking chain.

[0012] The present invention also provides a toxic thought chain analysis and optimization device for a large language model, the device comprising the following modules: A question-answering module is used to obtain question text and input the question text into a large language model for text question answering; A tracking module is used to analyze the reasoning drift information loss when the large language model generates a toxic thought chain according to the question text by attribution tracking, or to analyze the answer offset information loss when the large language model generates an answer text according to the toxic thought chain by causal tracking when performing text question answering; An optimization module, for performing residual decoding processing on the toxic thought chain generated by the question text to obtain an optimized thought chain when it is analyzed that there is reasoning drift information loss in the text question answering, so that the large language model generates an answer text for the question text based on the optimized thought chain; The optimization module is also used to exchange the text sequence position of the question text and the toxic thought chain when it is analyzed that there is answer drift information loss in the text question and answer, so as to obtain an optimized text sequence, so that the large language model can generate an answer text for the question text based on the optimized text sequence.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the toxic thought chain analysis and optimization method for a large language model as described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for analyzing and optimizing toxic thought chains for large language models.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned toxic thought chain analysis and optimization methods for a large language model.

[0016] The present invention provides a toxic thought chain analysis and optimization method and device for a large language model. When the large language model performs text question and answer, the method uses attribution tracking and causal tracking methods to analyze the reasoning drift information loss and answer offset information loss of the large language model that may be caused by toxic thought chains, and optimizes the two losses in a targeted manner through residual decoding and text sequence position exchange, thereby eliminating the impact of toxic thought chains on the reasoning performance of the large language model, thereby overcoming the defect that the prior art method for optimizing the toxic thought chains of large language models in question and answer scenarios cannot be universally adapted, resulting in the still low reasoning performance of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the toxic thought chain analysis and optimization method for a large language model provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the change of the average information flow of the toxic thinking chain and the normal thinking chain provided by the present invention at the attention layer.

[0020] Figure 3 It is a visualization diagram of the attention score differences of the attention heads provided by the present invention.

[0021] Figure 4 It is a schematic diagram comparing the average direct effect value of the attention layer and the MLP layer provided by the present invention.

[0022] Figure 5 It is a schematic diagram of the thought chain text fragments provided by the present invention executing text sequence position exchange.

[0023] Figure 6 It is a structural schematic diagram of a toxic thought chain analysis and optimization device for a large language model provided by the present invention.

[0024] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Combine the following Figures 1 to 6 The present invention describes the toxic thought chain analysis and optimization method and device for large language models. Figure 1 is a flow chart of the toxic thought chain analysis and optimization method for a large language model provided by the present invention, such as Figure 1 As shown, the method includes the following steps 101 to 104.

[0027] Step 101: Obtain question text and input the question text into a large language model for text question answering.

[0028] In the question-answering scenario, the corresponding question text is input to the large language model as a question, and the large language model answers the question according to the question and generates the answer text. The question text can be a question-answering question or a multiple-choice question. The question text of the question-answering question is generally a question sentence, without options to choose from, and the large language model needs to answer according to the question sentence, such as a conversation chat. For multiple-choice questions, multiple candidate answers are provided based on the question text (question sentence). The large language model selects the correct answer from multiple candidate answers based on the question sentence and outputs it as the answer text.

[0029] In the embodiment of the present application, the question text is obtained and input into the large language model for text answering. The question text can be a question-and-answer question or a multiple-choice question. If the thought chain method is adopted, the large language model will generate a corresponding thought chain based on the question text. The thought chain is essentially a piece of text, which is generated by the large language model based on the contextual semantic features of the question text to reflect the large language model's understanding and reasoning process of the semantics of the question text. The answer text generated by the large language model is also the answer obtained by reasoning based on this thought chain text.

[0030] Step 102: When conducting text question and answer, analyze the reasoning drift information loss that exists when the large language model generates toxic thought chains based on the question text through attribution tracking, or analyze the answer offset information loss that exists when the large language model generates answer text based on the toxic thought chain through causal tracking.

[0031] Following the above step 101, when the large language model performs text question and answer, if a toxic thought chain is generated, it means that the large language model may have reasoning drift and answer drift when answering the text, resulting in the generation of incorrect answer text. Reasoning drift occurs in the reasoning stage when the large language model generates the toxic thought chain, and answer drift occurs in the answer generation stage when the large language model outputs the answer text based on the toxic thought chain.

[0032] According to the above description, if the large language model has reasoning drift information loss when conducting text question answering, it means that the thought chain generated in the reasoning stage is a toxic thought chain. In order to optimize the toxic thought chain, it is necessary to analyze whether the large language model generates toxic thought chains in the reasoning stage. Therefore, when conducting text question answering, the embodiment of the present invention analyzes the reasoning drift information loss when the large language model generates toxic thought chains based on the question text through attribution tracking, which is explained in detail below.

[0033] In the reasoning stage of the large language model, it is a semantic understanding process for the question text. The large language model is generally composed of a transformer module, in which the transformer module contains important multi-head attention and consists of attention layers. Here, in the process of generating toxic thought chains based on the question text, the attribution score of each attention layer in the large language model is determined, and the attribution scores of each attention layer are summed to obtain the final attribution score.

[0034] In the process of generating toxic thought chains based on the question text, it is necessary to calculate the attribution score of each attention layer in the large language model. When calculating, for each neuron in the attention layer ( Indicates The attribution score Attr is approximately calculated by the Riemann method, which is expressed as the following formula (1): (1)

[0035] In the above formula (1), Represents neurons in the attention layer , F() represents the output of the attention layer, m represents the number of approximate integration steps, , k are parameters, Represents a logical operator.

[0036] Next, the attribution scores Attr of each attention layer are summed to obtain the final attribution score.

[0037] In each step of generating a toxic thinking chain, the step information flow between the question text and the toxic thinking chain is determined according to the final attribution score, and the average of the step information flow is taken as the first average information flow for generating a toxic thinking chain.

[0038] Here, we consider that when generating each word in the toxic thought chain, the attention operation of the attention layer involves the interaction of multiple words, and the attribution score of each attention layer can characterize the information flow intensity of the generated toxic thought chain words. Therefore, in each step of generating the toxic thought chain, the step information flow between the question text and the toxic thought chain is calculated according to the final attribution score, that is, the information flow between the question context q of the question text and the toxic thought chain c, which is recorded as , expressed as the following formula (2) and formula (3): (2)

[0039] (3)

[0040] In the above formula (2), Indicated in The information flow between the question context q of the question text and the toxic thought chain c in the attention layer. N represents the number of steps to generate the toxic thought chain. It means that from the i-th word in the problem context q to the j-th word in the toxic thinking chain, The information flow strength of the attention layer (i.e., the final attribution score).

[0041] In the above formula (3), represents the information reasoning association relationship between the problem context q and the toxic thinking chain c, represents the starting word of the question context q, i represents the i-th word of the question context q, Indicates the end word of the question context q. represents the starting word of the generated toxic thought chain, j represents the jth word in the generated toxic thought chain, Indicates the ending word in a generated chain of toxic thoughts.

[0042] In this way, each attention layer can calculate the corresponding step information flow, and then use the average value of the step information flow as the first average information flow to generate the toxic thinking chain, that is, the average of the step information flow of each attention layer is taken to obtain the first average information flow.

[0043] In order to conduct analysis and verification, the embodiment of the present application also generates a correct chain of thoughts based on the question text through a large language model, and determines the second average information flow in the process of generating the correct chain of thoughts. Here, based on the same question text, the large language model is called to generate a correct chain of thoughts and a correct answer text, but in the process of generating the correct chain of thoughts, the corresponding second average information flow is also calculated according to the above formulas (1), (2) and (3) for reference.

[0044] See also Figure 2 ,exist Figure 2 The figure shows how the average information flow changes with the attention layer under the toxic thought chain (Drift) and the normal thought chain (Correct). By comparing the first average information flow and the second average information flow, when the first average information flow is smaller than the second average information flow, it is determined that there is reasoning drift information loss when the large language model generates the toxic thought chain.

[0045] Here, if Figure 2 As shown in the figure, when the first average information flow of the toxic thought chain (Drift) is less than the second average information flow of the normal thought chain (Correct), it means that when the large language model generates a thought chain based on the question context of the question text, there is information loss in the attention layer, and at the 15th attention layer, the difference in the average information flow between the two is the largest, indicating that the information loss is the most serious. This difference in information flow causes the thought chain to receive less information from the question context, which means that the thought chain generated by the large language model in the reasoning stage is an incorrect toxic thought chain, and there is information loss in reasoning drift.

[0046] In some embodiments, for the question context q of the same question text, a toxic thought chain can also be generated. For each word of , the attention score of the attention head in the attention layer is calculated, and then the attention score of each word when the normal thinking chain c is generated is calculated. Finally, the difference between the attention scores of the two is compared and recorded as . By this attention score difference , it can be determined whether there is any information loss in the reasoning drift when the attention head in the large language model is performing reasoning. The calculation formula for the attention score difference is as follows (4): (4)

[0047] In the above formula (4), Represents the large language model The attention weight calculated by the ℎth attention head in the attention layer. Represents the information reasoning relationship between the problem context q and the normal thinking chain c, Representing Problem Context q and Toxic Thinking Chain The information reasoning association relationship between them, i represents the i-th word in the problem context q, and j represents the j-th word in the generated toxic thinking chain.

[0048] See also Figure 3 , Figure 3 A visualization showing the differences in attention scores of the attention heads in the attention layer, given by Figure 3It can be seen that there are differences in average information flow in multiple attention heads of each attention layer, and the information flow difference is the largest at the 15th attention layer, which is consistent with the above Figure 2 The same conclusions were drawn in .

[0049] In other embodiments, if the large language model suffers from answer drift information loss when conducting text question and answer, it means that an incorrect answer text was generated according to the toxic thinking chain during the answer generation stage. Therefore, in order to determine whether the large language model has generated an incorrect answer text according to the toxic thinking chain, it is necessary to determine whether the large language model has answer drift information loss. When conducting text question and answer, the embodiment of the present invention analyzes the answer drift information loss when the large language model generates the answer text according to the toxic thinking chain by causal tracing, which is specifically explained below in conjunction with the case where the question text is a multiple-choice question.

[0050] In the answer generation stage of the large language model, the answer text is generated according to the thought chain generated in the reasoning stage. Among them, the generation of the answer text requires the large language model to predict each word in the thought chain, determine whether the thought chain word can be used as the answer text through the model prediction score, and then select the corresponding answer from the given options. The model prediction process is also a process of activating the semantic features of the word in real time and mapping the prediction probability. The semantic features are propagated in the attention layer and MLP (multi-layer perceptron) layer of the Transformer module of the large language model in the form of latent vectors. The propagation process can be expressed as the following formulas (5) to (7): (5)

[0051] (6)

[0052] (7)

[0053] In the above formulas (5), (6), and (7), represents the hidden vector of the 𝑖th word in the lth attention layer of the Transformer module, represents the attention activation value of the hidden vector of the i-th word in the l-th attention layer of the Transformer module, represents the MLP activation value of the hidden vector of the i-th word in the l-th attention layer of the Transformer module, represents the attention activation function of the lth attention layer, represents the MLP activation function of the lth attention layer.

[0054] In order to determine whether there is answer drift information loss when the large language model predicts and generates answer text, it is necessary to pay attention to the forward propagation process of the large language model before predicting and generating answer text, and add Gaussian noise to the latent vector during this forward propagation process to interfere and verify the impact of noise on the prediction results.

[0055] Specifically, in the forward propagation process before the big language model generates the answer text according to the toxic thought chain, the first latent vector of the question text in each attention layer in the Transformer module of the big language model is determined, and the normal prediction probability when generating the answer text is determined based on the first latent vector. Here, the first latent vector of the thought chain word in each attention layer can be recorded as z. Among them, , Represents the first latent vector of the i-th word in the thought chain at the l-th attention layer. Through normal forward propagation of these latent vectors z in the Transformer module, we can Determine the normal prediction probability when generating the answer text, that is, the final prediction probability for a given option, recorded as .

[0056] Next, we add noise interference to the latent vector. We first preset a fixed Gaussian noise vector, denoted as , which is essentially a noise vector, and then the first latent vector With Gaussian noise vector Sum and get the second latent vector (i.e. ), and finally, the interference prediction probability when generating the answer text is determined based on the second latent vector, recorded as .

[0057] To ultimately verify the impact of noise on the prediction results, an effect index needs to be set to quantify the contribution of noise in the Transformer module to the final prediction, so as to track the information flow in the answer generation stage. This efficiency index is the direct effect value of the Transformer module on the answer options, denoted as DE. and interference prediction probability , determine the direct effect value DE of the Transformer module for the answer text, and the calculation formula is expressed as follows (8): (8)

[0058] In the above formula (8), z represents the first latent vector of the thought chain word in each attention layer of the Transformer module. It represents the direct effect value of the Transformer module on the answer text.

[0059] After calculating the direct effect value DE of the Transformer module for the answer text, the first average direct effect value ADE of the Transformer module is determined next, that is, the average of the direct effect values ​​DE of the Transformer module is calculated as the first average direct effect value.

[0060] Similar to attribution tracking, here in the process of causal tracking, when the large language model generates the answer text according to the correct thinking chain, the second average direct effect value of the Transformer module is determined as a reference. Then the first average direct effect value under the toxic thinking chain (Drifting Case) is compared with the second average direct effect value under the normal thinking chain (Correct Case). However, when the large language model answers the question text of the multiple-choice question, the thinking chain text fragment generated in the reasoning stage includes the question context context, the question option option, the thinking chain CoT, and the last word last before the label prediction. Therefore, the first average direct effect value and the second average direct effect value when the attention layer Attn is activated can be compared in these four parts. The visual comparison results can be seen in Figure 4 (a) and (c) in Figure 2. At the same time, we also need to compare the first average direct effect value and the second average direct effect value when the MLP layer is activated. The visual comparison results can be found in Figure 4 (b) and (d) in the figure.

[0061] By comparison, when the first average direct effect value is less than the second average direct effect value, it is determined that there is answer offset information loss when the large language model generates answer text according to the toxic thought chain. Figure 4 It can be seen that in the problem context of the attention module, the first average direct effect value of the toxic thinking chain is smaller than the second average direct effect value. Figure 4 As shown in (a) of , in the answer generation stage, when the normal thinking chain (Correct Case) generates the correct answer text, the average direct effect value on the question context and question options is very high, while Figure 4 In (c), when the toxic thinking chain (Drifting Case) generates an incorrect answer text, the average direct effect values ​​on the question context and the question options are significantly reduced. This shows that the toxic thinking chain (Drifting Case) loses information from the question text, causing the answer to drift information loss, thus generating an incorrect answer text. Figure 4As shown in (b) and (d), in the MLP module, for the four parts of question context, question options, thinking chain CoT, and the last word last before label prediction, there is a small difference between the first average direct effect value of the toxic thinking chain (Drifting Case) and the second average direct effect value of the normal thinking chain (Correct Case).

[0062] The embodiment of the present invention uses the attribution tracking and causal tracking methods and the white box tracking method to analyze the reasoning drift information loss and answer offset information loss caused by toxic thinking chains in the large language model in the black box, thereby accurately locating the reasons why the large language model generates toxic thinking chains and causes reasoning offset and answer offset, laying the foundation for subsequent optimization of toxic thinking chains.

[0063] Step 103: When it is analyzed that there is reasoning drift information loss in the text question and answer, residual decoding processing is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model can generate an answer text for the question text based on the optimized thought chain.

[0064] Here, through the analysis process for reasoning offset in step 102, when it is analyzed that there is reasoning drift information loss in the text question and answer, it means that the large language model has generated an erroneous toxic thought chain during the reasoning stage. Therefore, the embodiment of the present invention performs residual decoding processing on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model can generate the correct answer text corresponding to the question text based on the optimized thought chain.

[0065] The residual decoding process is mainly aimed at generating words of toxic thought chains. The problem context q is connected with the thought chain word tokens through a similar residual method, and the attention score Attn_score of certain thought chain words and the problem context is strengthened, so that the thought chain generated according to the problem context of the question text can retain more information without causing information loss, and eliminate the information flow difference between the question text and the thought chain.

[0066] Therefore, in the residual decoding process, firstly, for each original thought chain word token in the generated toxic thought chain, the model prediction score scores of the original thought chain words generated by the large language model are determined, and multiple target thought chain words with the highest model prediction scores (top) are selected. Therefore, the higher the model prediction score, the smaller the information difference between this word and the question context, and the higher the information utilization rate. Here, the number of target thought chain words is recorded as n, which is expressed as the following formula (9): (9)

[0067] In the above formula (9), x represents the question text input into the large language model M, M() represents the processing function of the large language model, and n represents the number of original thought chain words in the generated toxic thought chain.

[0068] Next, for each target thought chain word, determine the attention score Attn_score of the question context q and the original thought chain word token, and weight the attention score by the preset attention weight w to obtain the reward score of the target thought chain word. Finally, sum the model prediction score of the target thought chain word with the reward score to obtain the final prediction score Score of the target thought chain word, which is expressed as the following formula (10) and formula (11): (10)

[0069] (11)

[0070] In the above formula (10), token represents the original thought chain word, j represents the jth word among the n target thought chain words, q represents the context of the question text x, represents the attention score of the original thought chain word token for the question context q, It represents the attention score of the original thought chain word token for the question text x. Represents the attention score of the context q of the question and the original thought chain word token, and w represents the preset attention weight w. represents the model prediction score of the jth word among the n target thought chain words, Represents the final prediction score of the jth word among the n target thought chain words.

[0071] Finally, the target thought chain word with the highest final prediction score is used to update the original thought chain words in the toxic thought chain to obtain an optimized thought chain. After determining the target thought chain word with the highest final prediction score, the target thought chain word is used to update the original thought chain words in the toxic thought chain. Thus, when generating each original thought chain word of the toxic thought chain, the above process is repeated, and the target thought chain word with the highest final prediction score is determined by formulas (9), (10), and (11) to replace the words of the generated original thought chain. In this way, the optimized thought chain is finally updated, so that the large language model can generate the answer text of the question text based on the optimized thought chain.

[0072] In the embodiment of the present invention, when generating the original thought chain words, the target thought chain words with the highest model prediction score are selected, and then the attention scores of all the original thought chain words are used as the reward scores of the target thought chain in a similar residual manner, and added to the model prediction scores of the target thought chain words, so that when the large language model generates thought chain words, it focuses on generating the thought chain words with the highest model prediction scores. In this way, the information flow difference between the question text and the thought chain can be eliminated, so that the information difference between the generated thought chain and the question context is as small as possible, avoiding the generation of toxic thought chains, resulting in information loss and thus causing reasoning offset information loss.

[0073] Step 104: When it is analyzed that there is answer drift information loss in the text question and answer, in the thought chain text fragment, the question text and the toxic thought chain are exchanged in the text sequence position to obtain an optimized text sequence, so that the large language model can generate an answer text for the question text based on the optimized text sequence.

[0074] In other embodiments, through the analysis process for answer offset in step 102, when it is analyzed that there is answer drift information loss in the text question and answer, it means that the large language model generated an incorrect answer text according to the toxic thought chain during the answer generation stage. Therefore, in the embodiment of the present invention, in the thought chain text fragment, the question text and the toxic thought chain are exchanged in the text sequence position to obtain an optimized text sequence, so that the large language model generates an answer text corresponding to the question text based on the optimized text sequence.

[0075] The text sequence position exchange is mainly for thought chain text fragments. The thought chain text fragments are spliced ​​in sequence according to the question text sequence, toxic thought chain sequence, last word sequence and answer text sequence. This splicing order is fixed in the large language model. According to the research on the sequence position effect in the prior art, the large language model tends to make better use of the information at the beginning and end of the input. In an embodiment of the present invention, the beginning of the input is the prompt word, and the question text and the generated toxic thought chain are both at the end. Therefore, when the question text sequence is closer to the last word sequence, its key information is more easily used in the final prediction of the answer text. Figure 5 As shown, the lengths of the question text sequence and the toxic thought chain sequence are recorded as and , and assumes that the key information is located in the position of the question text sequence Toxic Thought Chain Sequence (Similar to the center of mass in physics). When the large language model generates the answer text according to the toxic thought chain, the position of the question text sequence in the thought chain text fragment is swapped with the position of the toxic thought chain sequence to obtain the optimized text sequence, which is spliced ​​in sequence according to the toxic thought chain sequence, the question text sequence, the last word sequence and the answer text sequence.

[0076] After the above swapping operation, the sequence distance from the question text sequence to the final word sequence is reduced. In addition, if the total distance from the question text and the toxic thought chain to the final word sequence is considered, the following formulas (12) to (14) can be calculated: (12)

[0077] (13)

[0078] (14)

[0079] In the above formulas (12) to (14), The total distance between the question text sequence and the thought chain sequence to the final word sequence before position exchange is performed for the thought chain text fragment, After performing position exchange for the thought chain text fragment to obtain the optimized text sequence, the total distance between the problem text sequence and the toxic thought chain sequence to the final word sequence, is the distance difference value. In most cases, the length of the question text sequence is smaller than the length of the toxic thought chain sequence, i.e. ,therefore This means that after performing the above-mentioned position swap, that is, replacing the order of "question text sequence + toxic thinking chain sequence" with the order of "toxic thinking chain sequence + question text sequence", the information flow intensity from the question text to the final prediction can be increased, thereby reducing the information loss of the answer offset.

[0080] In some embodiments, when it is analyzed that there is reasoning drift information loss or answer drift information loss in the text question and answer, the toxicity rate of the toxic thought chain is determined, recorded as TR. This toxicity rate is used to describe the severity of the toxic thought chain. Therefore, when the large language model generates the answer text, it is necessary not only to output the answer accuracy (ACC) as an evaluation indicator of the large language model, but also to reflect the severity of the toxic thought chain through the toxicity rate. The following describes the process of determining the toxicity rate.

[0081] First, multiple candidate question texts are obtained (which can be either essay questions or multiple-choice questions), and the large language model is called to generate a test thinking chain. Based on the test thinking chain, the first correct answer text or the first wrong answer text is generated, and the first target question text corresponding to the first correct answer text is selected from the candidate question texts, which is recorded as .

[0082] Here, in a question-answering scenario, under the condition of using the thought chain technology, the large language model generates a test thought chain, and then directly generates an answer to the candidate question text based on the test thought chain. Of course, there are right and wrong answers, so the first correct answer text or the first wrong answer text is generated. Then, the first target question text corresponding to the first correct answer text is selected from the candidate question texts. , that is, determine the first target question text with the correct answer.

[0083] Next, the test thought chain is optimized, and the large language model is called to generate a second correct answer text or a second wrong answer text based on the optimized thought chain, and the second target question text of the second wrong answer text is selected from the candidate question texts, which is recorded as .

[0084] Here, under the condition of using the thought chain technology in the same question-answering scenario, the test thought chain is optimized to obtain the optimized thought chain f. The optimization process here is the residual decoding process and the text sequence position exchange to eliminate the possible reasoning drift information loss or answer drift information loss. Then the large language model is called to generate the second correct answer text or the second wrong answer text based on the optimized thought chain f. Because the toxicity of the optimized thought chain f cannot be determined, the generated answers are both right and wrong, that is, the second correct answer text or the second wrong answer text is generated. Then the second target question text corresponding to the second wrong answer text is screened out from the candidate question texts. , that is, determine the second target question text with an incorrect answer.

[0085] Finally, the intersection of the first target question text and the second target question text is determined, and the ratio of the intersection to the second target question text is used as the toxicity rate TR(f) of the test thinking chain f. The calculation formula can be expressed as the following formula (15): (15)

[0086] It can be seen from the results calculated by the above formula that the lower the toxicity rate TR is, the fewer toxicity problems are introduced by the thinking chain after optimization, which also indirectly shows that the optimization effect of the toxic thinking chain in the embodiment of the present application is better.

[0087] The embodiment of the present invention provides a toxic thought chain analysis and optimization method for a large language model. When the large language model performs text question and answer, the attribution tracking and causal tracking methods are used to analyze the reasoning drift information loss and answer offset information loss of the large language model that may be caused by the toxic thought chain, and the two losses are optimized through residual decoding and text sequence position exchange, thereby eliminating the impact of the toxic thought chain on the reasoning performance of the large language model. The process of optimizing the toxic thought chain is a white box operation and can be adapted to any large language model. This overcomes the defect that the prior art method of optimizing the toxic thought chain of a large language model in the question and answer scenario cannot be universally adapted, resulting in the still low reasoning performance of the large language model.

[0088] The following is a description of the toxic thought chain analysis and optimization device for a large language model provided by the present invention. The toxic thought chain analysis and optimization device for a large language model described below and the toxic thought chain analysis and optimization method for a large language model described above can be referred to each other. Figure 6 As shown, the toxic thought chain analysis and optimization device for a large language model provided by the present invention includes: a question and answer module 601, a tracking module 602, and an optimization module 603. Among them, the question-answering module 601 is used to obtain a question text and input the question text into a large language model for text question-answering; the tracking module 602 is used to analyze, when performing text question-answering, the reasoning drift information loss that exists when the large language model generates a toxic thought chain based on the question text by attribution tracking, or to analyze, when performing text question-answering, the answer offset information loss that exists when the large language model generates an answer text based on the toxic thought chain by causal tracking; the optimization module 603 is used to, when analyzing that there is a reasoning drift information loss in the text question-answering, perform residual decoding processing on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model generates an answer text for the question text based on the optimized thought chain; the optimization module 603 is also used to, when analyzing that there is an answer drift information loss in the text question-answering, exchange the text sequence position of the question text with the toxic thought chain to obtain an optimized text sequence, so that the large language model generates an answer text for the question text based on the optimized text sequence.

[0089] It should be noted that the beneficial effects of the toxic thought chain analysis and optimization device for large language models here and the toxic thought chain analysis and optimization method for large language models mentioned above can correspond to each other, so the beneficial effects of the toxic thought chain analysis and optimization device for large language models will not be repeated here.

[0090] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor (processor) 710 , a communication interface (Communications Interface) 720 , a memory (memory) 730 and a communication bus 740 , wherein the processor 710 , the communication interface 720 , and the memory 730 communicate with each other through the communication bus 740 . The processor 710 can call the logic instructions in the memory 730 to execute a toxic thought chain analysis and optimization method for a large language model, the method comprising: obtaining a question text, inputting the question text into a large language model for text question and answering; when performing text question and answering, analyzing the reasoning drift information loss when the large language model generates a toxic thought chain based on the question text by attribution tracking, or analyzing the answer offset information loss when the large language model generates an answer text based on the toxic thought chain by causal tracking; when it is analyzed that there is reasoning drift information loss in the text question and answer, residual decoding processing is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model generates an answer text for the question text based on the optimized thought chain; when it is analyzed that there is answer drift information loss in the text question and answer, in the thought chain text fragment, the text sequence position of the question text and the toxic thought chain is exchanged to obtain an optimized text sequence, so that the large language model generates an answer text for the question text based on the optimized text sequence.

[0091] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0092] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the toxic thought chain analysis and optimization method for a large language model provided by the above-mentioned methods, the method including: obtaining a question text, and inputting the question text into a large language model for text question and answer; when performing text question and answer, analyzing the reasoning drift information loss existing when the large language model generates a toxic thought chain according to the question text by attribution tracking, or analyzing the answer offset information loss existing when the large language model generates an answer text according to the toxic thought chain by causal tracking; when it is analyzed that there is reasoning drift information loss in the text question and answer, residual decoding processing is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model generates an answer text for the question text based on the optimized thought chain; when it is analyzed that there is answer drift information loss in the text question and answer, in the thought chain text fragment, the question text and the toxic thought chain are exchanged for text sequence position to obtain an optimized text sequence, so that the large language model generates an answer text for the question text based on the optimized text sequence.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the toxic thought chain analysis and optimization method for a large language model provided by the above-mentioned methods, the method comprising: obtaining a question text, and inputting the question text into a large language model for text question and answer; when performing text question and answer, analyzing the reasoning drift information loss existing when the large language model generates a toxic thought chain according to the question text by attribution tracking, or analyzing the answer offset information loss existing when the large language model generates an answer text according to the toxic thought chain by causal tracking; when it is analyzed that there is reasoning drift information loss in the text question and answer, residual decoding processing is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model generates an answer text for the question text based on the optimized thought chain; when it is analyzed that there is answer drift information loss in the text question and answer, in the thought chain text fragment, the text sequence position of the question text and the toxic thought chain is exchanged to obtain an optimized text sequence, so that the large language model generates an answer text for the question text based on the optimized text sequence.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A toxic thought chain analysis and optimization method for a large language model, characterized in that: The method comprises: Obtaining a question text, and inputting the question text into a large language model for text question answering; When conducting text question answering, the reasoning drift information loss that exists when the large language model generates a toxic thought chain based on the question text is analyzed by attribution tracking, or the answer offset information loss that exists when the large language model generates an answer text based on the toxic thought chain is analyzed by causal tracking; When it is analyzed that there is reasoning drift information loss in the text question and answer, residual decoding processing is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, so that the large language model generates an answer text for the question text based on the optimized thought chain; When it is analyzed that there is answer drift information loss in the text question and answer, the question text and the toxic thought chain are exchanged in the text sequence position in the thought chain text segment to obtain an optimized text sequence, so that the large language model generates an answer text for the question text based on the optimized text sequence.

2. The toxic thought chain analysis and optimization method for a large language model according to claim 1, characterized in that: The attribution tracking method is used to analyze the reasoning drift information loss when the large language model generates a toxic thought chain according to the question text, including: In the process of generating the toxic thought chain according to the question text, determining the attribution score of each attention layer in the large language model, and summing the attribution scores of each attention layer to obtain a final attribution score; In each step of generating a toxic thinking chain, determining a step information flow between the question text and the toxic thinking chain according to the final attribution score, and taking an average of the step information flow as a first average information flow for generating the toxic thinking chain; Based on the question text, generate a correct thought chain through a large language model, and determine a second average information flow in the process of generating the correct thought chain; When the first average information flow is less than the second average information flow, it is determined that there is reasoning drift information loss when the large language model generates the toxic thought chain.

3. The toxic thought chain analysis and optimization method for a large language model according to claim 1, characterized in that: The causal tracing method is used to analyze the answer deviation information loss when the large language model generates the answer text according to the toxic thought chain, including: In a forward propagation process before the big language model generates an answer text according to the toxic thought chain, the first latent vector of each attention layer of the question text in the Transformer module of the big language model is determined, and the normal prediction probability when generating the answer text is determined according to the first latent vector; Preset a fixed Gaussian noise vector, sum the first latent vector and the Gaussian noise vector to obtain a second latent vector, and determine the interference prediction probability when generating the answer text based on the second latent vector; Determining a direct effect value of the Transformer module for the answer text according to the normal prediction probability and the interference prediction probability, and determining a first average direct effect value of the Transformer module based on the direct effect value; When the large language model generates a response text according to the correct thought chain, determining a second average direct effect value of the Transformer module; When the first average direct effect value is less than the second average direct effect value, it is determined that there is answer offset information loss when the large language model generates answer text according to the toxic thought chain.

4. The toxic thought chain analysis and optimization method for a large language model according to claim 1, characterized in that: The residual decoding process is performed on the toxic thought chain generated by the question text to obtain an optimized thought chain, including: For each original thought chain word in the generated toxic thought chain, determine the model prediction score of the original thought chain word generated by the large language model, and select multiple target thought chain words with the highest model prediction scores; For each target thought chain word, determine the attention score of the context of the question text and the original thought chain word, and weight the attention score by a preset attention weight to obtain a reward score for the target thought chain word; The model prediction score of the target thought chain word is summed with the reward score to obtain a final prediction score of the target thought chain word; By finally predicting the target thought chain word with the highest score, the original thought chain words in the toxic thought chain are updated to obtain an optimized thought chain.

5. The toxic thought chain analysis and optimization method for a large language model according to claim 1, characterized in that: In the thought chain text segment, the question text and the toxic thought chain are exchanged in text sequence position to obtain an optimized text sequence, including: When the large language model generates the answer text according to the toxic thought chain, the position of the question text sequence in the thought chain text segment is swapped with the position of the toxic thought chain sequence to obtain an optimized text sequence; Among them, the thought chain text fragments are spliced ​​in sequence according to the question text sequence, the toxic thought chain sequence, the last word sequence and the answer text sequence, and the optimized text sequence is spliced ​​in sequence according to the toxic thought chain sequence, the question text sequence, the last word sequence and the answer text sequence.

6. The toxic thought chain analysis and optimization method for a large language model according to claim 1, characterized in that: The method further comprises: When it is analyzed that there is reasoning drift information loss or answer drift information loss in the text question and answer, the toxicity rate of the toxic thinking chain is determined, and the process of determining the toxicity rate includes: Acquire multiple candidate question texts, call the large language model to generate a test thought chain, generate a first correct answer text or a first wrong answer text based on the test thought chain, and select a first target question text corresponding to the first correct answer text from the candidate question texts; Optimizing the test thought chain, calling the large language model to generate a second correct answer text or a second wrong answer text based on the optimized thought chain, and selecting a second target question text corresponding to the second wrong answer text from the candidate question texts, wherein the optimization process includes residual decoding and text sequence position exchange; Determine the intersection of the first target question text and the second target question text, and use the ratio of the intersection to the second target question text as the toxicity rate of the test thinking chain.

7. A toxic thought chain analysis and optimization device for a large language model, characterized in that: The device comprises: A question-answering module is used to obtain question text and input the question text into a large language model for text question answering; A tracking module is used to analyze the reasoning drift information loss when the large language model generates a toxic thought chain according to the question text by attribution tracking, or to analyze the answer offset information loss when the large language model generates an answer text according to the toxic thought chain by causal tracking when performing text question answering; An optimization module, for performing residual decoding processing on the toxic thought chain generated by the question text to obtain an optimized thought chain when it is analyzed that there is reasoning drift information loss in the text question answering, so that the large language model generates an answer text for the question text based on the optimized thought chain; The optimization module is also used to exchange the text sequence position of the question text and the toxic thought chain when it is analyzed that there is answer drift information loss in the text question and answer, so as to obtain an optimized text sequence, so that the large language model can generate an answer text for the question text based on the optimized text sequence.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the toxic thought chain analysis and optimization method for a large language model is implemented as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the toxic thought chain analysis and optimization method for a large language model is implemented as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the toxic thought chain analysis and optimization method for a large language model is implemented as described in any one of claims 1 to 6.

Images