Content generation method and device, computer equipment and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to a content generation method and device, computer equipment and a storage medium, and the method comprises the steps: employing a content generation model to process a target problem containing context information, and calculating an attribution score of each neuron in the model when the context information is processed; dividing each neuron into a contextual neuron and a non-contextual neuron according to the attribution score; increasing the weight of the context neuron according to a preset enhancement strength value, and carrying out feature extraction on the target problem by adopting the weight-enhanced context neuron and the non-context neuron to obtain a feature vector; an attention head and a decoder in the model are adopted, reply content for the target question is generated according to the feature vector, and the attention head comprises a to-be-intervened attention head subjected to fact correlation evaluation in advance. The problem that the content generation accuracy is low when internal and external knowledge conflicts can be effectively solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a content generation method, apparatus, computer equipment and storage medium. Background Art

[0002] In intelligent customer service and question-answering systems, it is crucial to ensure the accuracy and real-time nature of answers in a field such as financial services, which is rapidly changing and has diverse customer needs. Currently, retrieval-augmented language models (RALMs), as core components of intelligent question-answering systems, generate answers by integrating information from external knowledge bases. However, such models have significant defects in dealing with internal and external knowledge conflicts: they tend to rely on inherent knowledge encoded internally and ignore the latest information input from external knowledge bases. This bias may not only lead to content generation errors, but may even cause "hallucination" phenomena (i.e., the content generated by the model does not match the actual situation), resulting in reduced accuracy of the generated content. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a content generation method, apparatus, computer device and storage medium, which can effectively solve the problem of low accuracy of content generation when internal and external knowledge conflicts.

[0004] In order to solve the above technical problems, the present application provides a content generation method, which adopts the following technical solution:

[0005] Get the target question containing context information;

[0006] Inputting the target question into a content generation model, and calculating an attribution score of each neuron included in the content generation model when processing the context information;

[0007] According to the attribution score, each of the neurons is divided into a context neuron and a non-context neuron;

[0008] Increasing the weight of the context neuron according to a preset enhancement strength value to obtain a weight-enhanced context neuron;

[0009] Using the weight-enhanced context neurons and the non-context neurons to extract features of the target problem to obtain a feature vector;

[0010] The attention heads and decoders in the content generation model are used to generate a response content for the target question according to the feature vector, wherein the attention heads include attention heads to be intervened that have been pre-evaluated for factual relevance.

[0011] In order to solve the above technical problems, the embodiment of the present application further provides a content generation device, which adopts the following technical solution:

[0012] An acquisition module is used to obtain the target question containing context information;

[0013] A calculation module, used for inputting the target question into a content generation model, and calculating an attribution score of each neuron included in the content generation model when processing the context information;

[0014] A division module, used for dividing each neuron into a context neuron and a non-context neuron according to the attribution score;

[0015] A weighting module, used for increasing the weight of the context neuron according to a preset enhancement strength value to obtain a weight-enhanced context neuron;

[0016] An extraction module, used for extracting features of the target problem using the weight-enhanced context neurons and the non-context neurons to obtain a feature vector;

[0017] A generation module is used to use the attention heads and decoders in the content generation model to generate a response content for the target question according to the feature vector, wherein the attention heads include attention heads to be intervened that have been pre-evaluated for factual relevance.

[0018] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, which adopts the technical solution described below: the computer device includes a memory and a processor, the memory stores computer-readable instructions, and the processor implements the steps of the content generation method described in any one of the above items when executing the computer-readable instructions.

[0019] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which adopts the following technical solution: the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the content generation method described in any one of the above items are implemented.

[0020] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0021] The embodiment of the present application processes the target problem containing context information by adopting a content generation model, calculates the attribution score of each neuron in the model when processing the context information, and can accurately identify neurons that are more sensitive to context information, namely, context neurons, thereby ensuring that the model can focus on key context information when processing problems; by increasing the weight of the context neurons according to a preset enhancement strength value, the model can more keenly capture and utilize context information when processing problems, which helps the model to give priority to the latest external knowledge when dealing with internal and external knowledge conflicts and effectively resolve knowledge conflicts; by adopting weight-enhanced context neurons and non-context neurons to jointly process the target problem, the logic and coherence of the generated content are ensured; by pre-evaluating the fact relevance of the attention heads in the model to determine the attention heads to be intervened, all attention heads and decoders including the attention heads to be intervened are used, and the response content for the target question is generated according to the feature vector extracted by the neurons, so that the generated response content is more in line with the actual situation, effectively reducing the occurrence of the "hallucination" phenomenon and improving the accuracy of the generated content. The embodiment of the present application is based on contextual neuron enhancement and factual reasoning intervention mechanism, which ensures that the model can give priority to the latest external knowledge when facing knowledge conflicts, reduces the occurrence of "hallucination" phenomenon, and avoids generating wrong answers, especially when processing complex knowledge, thereby improving the credibility of generated content. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0024] Figure 2 It is a flowchart of an embodiment of the content generation method of the present application;

[0025] Figure 3 is a structural diagram of an embodiment of a content generation device of the present application;

[0026] Figure 4 It is a structural diagram of an embodiment of a computer device of the present application. DETAILED DESCRIPTION

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0028] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0030] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0031] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0032] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0033] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0034] It should be noted that the content generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the content generation device is generally arranged in the server / terminal device.

[0035] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0036] Continue to refer Figure 2 , shows a flow chart of an embodiment of a question-answering method according to the present application. The content generation method comprises the following steps:

[0037] Step S201, obtaining a target question including context information;

[0038] In this embodiment, the electronic device (eg, Figure 1 The server / terminal device shown in the figure can obtain the target question containing context information through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 2G / 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0039] Specifically, the target question containing context information can be obtained by obtaining the target question through a user interface and extracting relevant context information using technical means such as a database or knowledge graph. The target question refers to a specific question or statement raised by a user for which the system needs to generate an answer; the context information refers to background information related to the target question, which may include historical questions asked by users, historical answers from the system, and specific domain knowledge.

[0040] Step S202, inputting the target question into a content generation model, and calculating an attribution score of each neuron included in the content generation model when processing the context information;

[0041] In this embodiment, a content generation model is pre-constructed based on deep learning (such as Transformer, BERT, etc.), and the content generation model is used to generate corresponding reply content according to the input target question.

[0042] Specifically, the acquired target question and its context information are used as the input of the content generation model, and the context information of the target question is processed by each neuron contained in the content generation model, and the attribution score of each neuron when processing the context information is recorded. The attribution score is a score that quantifies the importance of each input unit (neuron) to the output in the neural network model. More specifically, the activation value generated by each neuron when processing the context information is calculated through the forward propagation of the content generation model, and then the attribution score is calculated based on the activation value using attribution analysis technology (such as Layer-wise Relevance Propagation, LRP).

[0043] In some optional implementations, the above-mentioned calculation of the attribution score of each neuron included in the content generation model when processing the context information may include the following steps:

[0044] Using each neuron to process the context information, and recording a neuron activation value of each neuron when processing the context information;

[0045] Specifically, during the forward propagation of the model, a recording mechanism is pre-set for each neuron, which is used to capture and store the activation value of the neuron when processing context information. After the acquired target question and its context information are input into the content generation model, during the forward propagation of the model, each neuron in the model is used to process the context information, such as performing nonlinear transformation and feature extraction on the context information, and recording the neuron activation value of each neuron when processing the context information. The neuron activation value is used to reflect the response strength of the neuron to the input context information.

[0046] The contribution of each neuron to the context is calculated according to the neuron activation value to obtain an attribution score.

[0047] Specifically, a predefined attribution method for calculating the contribution of neurons is obtained, which may include gradient-based attribution (such as Gradient×Input, Integrated Gradients), perturbation-based attribution (such as Occlusion / Perturbation Sensitivity), decomposition-based attribution (such as Layer-wise RelevancePropagation, LRP), etc. Using the predefined attribution method, the contribution of each neuron to the context is calculated according to the recorded neuron activation value to obtain the attribution score.

[0048] The embodiment of the present application can identify the neurons that play a key role in processing contextual information by calculating the attribution score of each neuron in the content generation model when processing contextual information, which helps to enhance the understanding of the internal working mechanism of the model. Among them, the attribution score can be used as a basis for adjusting the weight of the neuron, and the performance of the model can be improved by enhancing or weakening the contribution of specific neurons.

[0049] Step S203, dividing each neuron into a context neuron and a non-context neuron according to the attribution score;

[0050] In this embodiment, a score threshold is pre-set, and the score threshold is used to divide the context neurons and non-context neurons in the model. Among them, context neurons refer to neurons that pay more attention to information related to the context of the target problem when processing the target problem and its context information. That is, context neurons can play an important role in the processing and transmission of context information. Non-context neurons refer to neurons that pay more attention to other information that is not directly related to the context of the target problem when processing the target problem and its context information. That is, non-context neurons play a smaller role in the processing and transmission of context information.

[0051] Specifically, the attribution score calculated by traversing each neuron is matched with a preset score threshold. When the attribution score meets the score threshold, the neuron corresponding to the attribution score is classified as a context neuron; when the attribution score does not meet the score threshold, the neuron corresponding to the attribution score is classified as a non-context neuron.

[0052] In some optional implementations, the dividing each neuron into a context neuron and a non-context neuron according to the attribution score may include the following steps:

[0053] Obtaining a preset score threshold, and comparing the attribution score with the score threshold;

[0054] If the attribution score is greater than or equal to the score threshold, determining the neuron corresponding to the attribution score as a context neuron;

[0055] If the attribution score is less than the score threshold, the neuron corresponding to the attribution score is determined as a non-context neuron.

[0056] Specifically, a preset score threshold is obtained, and the score threshold is used to divide the context neurons and non-context neurons in the model. The attribution score calculated by each neuron is traversed, and the attribution score is compared with the preset score threshold. When the attribution score is greater than or equal to the score threshold, it indicates that the neuron corresponding to the attribution score has a higher contribution in processing context information, and the neuron corresponding to the attribution score is determined as a context neuron. When the attribution score is less than the score threshold, it indicates that the neuron corresponding to the attribution score contributes less to the processing and transmission of context information, and the neuron corresponding to the attribution score is determined as a non-context neuron.

[0057] Step S204, increasing the weight of the context neuron according to a preset enhancement strength value to obtain a weight-enhanced context neuron;

[0058] Specifically, an enhancement strength value is preset, and the enhancement strength value is used to control the degree of enhancement of the context neuron weight. The weight of the context neuron is increased according to the preset enhancement strength value, for example, the weight of the context neuron is superimposed or multiplied with the enhancement strength value to obtain a weight-enhanced context neuron, so as to increase the importance of the context neuron in subsequent calculations.

[0059] In some optional implementations, the step of increasing the weight of the context neuron according to the preset enhancement strength value to obtain a weight-enhanced context neuron may include the following steps:

[0060] Obtaining a preset enhancement strength value and an original weight of the context neuron;

[0061] The enhancement strength value is multiplied by the original weight to increase the weight of the context neuron to obtain a weight-enhanced context neuron.

[0062] Specifically, a preset enhancement strength value is obtained, and the enhancement strength value is used to control the degree of context neuron weight enhancement. The original weight of each context neuron is obtained. The preset enhancement strength value is multiplied by the original weight of the context neuron to increase the weight of the context neuron, thereby obtaining a weight-enhanced context neuron.

[0063] Exemplarily, a weight enhancement formula is used to adjust the context neuron weight, wherein the weight enhancement formula is as follows:

[0064] w enh ance =α·w context_neuron_i ;

[0065] where w context_neuron_i represents the original weight of the i-th context neuron, α represents the enhancement value, and w enh ance Represents the enhanced weight of the i-th context neuron.

[0066] Step S205, using the weight-enhanced context neurons and the non-context neurons to extract features of the target question to obtain a feature vector;

[0067] Specifically, after weight enhancement of context neurons, all neurons in the content generation model (including weight enhanced context neurons and non-context neurons) are used to extract features of the input target question through forward propagation to obtain a feature vector.

[0068] In some optional implementations, the above-mentioned use of the weight-enhanced context neurons and the non-context neurons to extract features of the target problem to obtain a feature vector may include the following steps:

[0069] Combining the weight-enhanced context neurons and the non-context neurons to obtain a target neuron set;

[0070] Convert the target problem into a vector representation, and use the target neuron set to perform weighted summation on the vector representation to obtain a weighted summation result;

[0071] A preset nonlinear activation function is used to perform nonlinear transformation on the weighted summation result to obtain a feature vector of the target problem.

[0072] Specifically, after replacing the original weights of the context neurons with the weight-enhanced values, the context neurons and non-context neurons with various weight enhancements in the model are combined to obtain a target neuron set. The target problem is mapped into a high-dimensional vector space, converted into a vector representation, and the target neuron set is used to perform weighted summation on the vector representation, that is, the weight of each neuron in the target neuron set (including the weight of the context neurons after weight enhancement and the original weight of the non-context neurons) is used to perform weighted summation on the vector representation to obtain a weighted summation result. A preset nonlinear activation function is obtained, which may include a ReLU function, a sigmoid function, a tanh function, etc. The weighted summation result is nonlinearly transformed using a preset nonlinear activation function to obtain a feature vector of the target problem, which contains feature information of the target problem after vector conversion, weighted summation, and nonlinear transformation.

[0073] The embodiment of the present application can more accurately capture context-related information by enhancing the weight of context neurons in the model, thereby highlighting these key information in the feature extraction process. By combining context neurons and non-context neurons to perform weighted summation on the vector representation, it is ensured that the contribution of each feature in the final feature vector is proportional to its importance, so that the model not only pays attention to context information, but also considers broader or global features, achieves a balance between global and local features, and improves the comprehensiveness and accuracy of feature extraction. The nonlinear expression ability of the model is increased by the application of nonlinear activation functions, so that the model can capture more complex feature relationships, further improving the performance of the model.

[0074] Step S206, using the attention heads and decoders in the content generation model to generate a response content for the target question according to the feature vector, wherein the attention heads include attention heads to be intervened that have been pre-evaluated for factual relevance.

[0075] In this embodiment, the content generation model is also provided with several attention heads, which are components in the model for focusing on different parts of the input sequence when generating a response. The decoder is a component in the model for converting a feature vector into a natural language response.

[0076] In this embodiment, during the model training or tuning stage, the attention heads that pay more attention to the actual facts are determined in advance by performing factual evaluation on each attention head in the model. The attention heads that pay more attention to the actual facts are used as the attention heads to be intervened, and in the content generation stage for the target question, the attention heads to be intervened are intervened (for example, the weight is increased), which can make the response content generated by the model closer to the actual facts and avoid the occurrence of the "hallucination" phenomenon (that is, the content generated by the model does not match the actual situation).

[0077] Specifically, the feature vectors extracted by all neurons in the content generation model are input into the attention heads (including the attention heads to be intervened that have been pre-evaluated for factual relevance and other attention heads in the model) through forward propagation. The attention mechanism is used to focus on the key information in the input feature vectors, and the focused key information is passed to the decoder, which is used to convert the focused key information into response content for the target question.

[0078] In some optional implementations, before using the attention head and the decoder in the content generation model to generate the answer content for the target question according to the feature vector, the following steps may also be included:

[0079] Obtain a preset training data set, wherein the training data set includes a plurality of training questions and annotated answers corresponding to the training questions;

[0080] Specifically, before using the content generation model to answer the input target question, each attention head in the content generation model is evaluated for factuality to determine the attention head that pays more attention to the actual facts. To achieve this goal, a pre-collected training data set is obtained, which includes a number of training questions and the labeled answers corresponding to the training questions. The labeled answers can be true answers and wrong answers.

[0081] Inputting the training problem into the content generation model for processing, and recording the attention activation value of each attention head included in the content generation model when processing the training problem;

[0082] Specifically, the training problem contained in the training data set is input into the content generation model for processing, and the attention activation value of each attention head contained in the content generation model when processing the training problem is recorded through a preset recording mechanism. The attention activation value is used to reflect the attention of the attention head to different parts of the training problem.

[0083] Obtain a pre-trained classifier, input the attention activation value and the labeled answer into the classifier, and calculate the fact correlation value between each attention head and the labeled answer according to the attention activation value and the labeled answer;

[0084] Specifically, a pre-trained classifier is obtained, which is used to calculate and output the fact correlation value between the attention activation value and the labeled answer. The recorded attention activation value and the labeled answer in the training data set are input into the classifier, and the fact correlation value between each attention head and the labeled answer is calculated by the classifier. The fact correlation value is used to reflect the correlation between the information captured by the attention head and the labeled answer.

[0085] Each of the attention heads is sorted according to the fact relevance value, and a preset number of attention heads with the highest sorting order are selected as the attention heads to be intervened after fact relevance evaluation.

[0086] Specifically, each attention head is sorted according to the calculated fact relevance value, wherein the attention head with a higher fact relevance value indicates that the information it captures is more consistent with the labeled answer, that is, the attention head pays more attention to the actual facts. From the sorted attention heads, a preset number (i.e., N) of the top-ranked attention heads are selected as the attention heads to be intervened after the fact relevance evaluation.

[0087] The embodiment of the present application records the activation values ​​of the attention heads and uses a classifier to calculate the factual correlation value between the activation values ​​of the attention heads and the annotated answers, so as to identify which attention heads are more capable of capturing information related to the authenticity of the answers when processing training questions. Selecting the top-ranked attention heads as the attention heads to be intervened helps to focus on relevant information more accurately when generating the reply content later.

[0088] In some optional implementations, the above-mentioned using the attention head and the decoder in the content generation model to generate the answer content for the target question according to the feature vector may include the following steps:

[0089] Obtaining a preset intervention strength value, intervening the attention head to be intervened according to the intervention strength value, and obtaining an attention head after intervention;

[0090] Specifically, a preset intervention strength value is obtained, which reflects the degree of intervention on the attention head to be intervened. According to the preset intervention strength value, the degree of attention of the attention head to be intervened to its attention direction is adjusted to achieve intervention on the attention head to be intervened, and obtain the attention head after intervention.

[0091] Exemplarily, an intervention formula is used to adjust the attention degree of the attention head to be intervened on its attention direction, so as to implement intervention on the attention head to be intervened, wherein the intervention formula is as follows:

[0092]

[0093] Among them, x l represents the attention activation value of the attention head to be intervened in the lth layer of the model; x l′ represents the activation value of the attention head after intervention in the lth layer of the model; β represents the intervention intensity value; Indicates the focus direction of the attention head to be intervened, which is used to guide the attention head to be intervened to pay more attention to factual knowledge.

[0094] Using the intervention attention head and other attention heads in the content generation model, weighting the feature vector to obtain a weighted feature vector;

[0095] Specifically, the intervention attention head is combined with other attention heads in the content generation model to form an attention head set, and each attention head in the attention set is used to weight the feature vector, that is, the attention activation value of each attention head is used to weight the feature vector to obtain a weighted feature vector.

[0096] The weighted feature vector is passed to a decoder of the content generation model, and the decoder is used to generate a reply content for the target question according to the weighted feature vector.

[0097] Specifically, the weighted feature vector is passed to the decoder of the content generation model, and the decoder converts the received weighted feature vector into a text sequence, adjusts the generated text to conform to grammatical rules, and generates a response to the target question.

[0098] The embodiment of the present application sets the intervention intensity value and performs precise intervention on the attention head to be intervened, so as to guide the model to pay more attention to key information related to the facts and generate response content that is more in line with the actual situation, thereby effectively reducing the occurrence of the "hallucination" phenomenon and improving the accuracy of the generated content.

[0099] The embodiment of the present application adopts the above scheme, specifically by using a content generation model to process the target problem containing context information, and calculating the attribution score of each neuron in the model when processing the context information, it can accurately identify neurons that are more sensitive to context information, namely context neurons, and ensure that the model can focus on key context information when processing problems; by increasing the weight of context neurons according to a preset enhancement strength value, the model can more keenly capture and utilize context information when processing problems, which helps the model to give priority to the latest external knowledge when dealing with internal and external knowledge conflicts and effectively resolve knowledge conflicts; by using weight-enhanced context neurons and non-context neurons to jointly process the target problem, the logic and coherence of the generated content are ensured; by pre-evaluating the fact relevance of the attention heads in the model to determine the attention heads to be intervened, all attention heads and decoders including the attention heads to be intervened are used, and the response content for the target question is generated according to the feature vector extracted by the neurons, so that the generated response content is more in line with the actual situation, effectively reducing the occurrence of the "hallucination" phenomenon and improving the accuracy of the generated content. The embodiment of the present application is based on contextual neuron enhancement and factual reasoning intervention mechanism, which ensures that the model can give priority to the latest external knowledge when facing knowledge conflicts, reduces the occurrence of "hallucination" phenomenon, and avoids generating wrong answers, especially when processing complex knowledge, thereby improving the credibility of generated content.

[0100] In addition, compared with the traditional dual decoding method (such as CAD and CD2), the embodiment of the present application avoids repeated decoding processes and significantly reduces the consumption of computing resources by combining contextual neuron enhancement and factual reasoning intervention mechanism. This enables the model to improve the quality of generated content without adding additional computing burden, especially in real-time tasks and large-scale deployment. At the same time, the model generates reply content by combining contextual neuron enhancement and factual reasoning intervention mechanism, which can improve the utilization efficiency of contextual information, thereby improving the overall efficiency of the system without adding additional computing burden. Especially in application scenarios that require real-time feedback and high concurrent response, the model can quickly respond to user query needs while ensuring high-quality output.

[0101] It should be emphasized that in order to further ensure the privacy and security of the target questions, attribution scores, reply content and other information of the above-mentioned context information and other relevant data, the target questions, attribution scores, reply content and other information of the above-mentioned context information and other relevant data can also be stored in a blockchain node.

[0102] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0103] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0104] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0105] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a content generation device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0106] like Figure 3 As shown, the content generation device 400 described in this embodiment includes: an acquisition module 401, a calculation module 402, a division module 403, a weighting module 404, an extraction module 405 and a generation module 406. Among them:

[0107] An acquisition module 401 is used to acquire a target question including context information;

[0108] A calculation module 402 is used to input the target question into a content generation model, and calculate an attribution score of each neuron included in the content generation model when processing the context information;

[0109] A division module 403, configured to divide each neuron into a context neuron and a non-context neuron according to the attribution score;

[0110] A weighting module 404, configured to increase the weight of the context neuron according to a preset enhancement strength value to obtain a weight-enhanced context neuron;

[0111] An extraction module 405 is used to extract features of the target problem using the weight-enhanced context neurons and the non-context neurons to obtain a feature vector;

[0112] The generation module 406 is used to use the attention heads and decoders in the content generation model to generate the response content for the target question according to the feature vector, wherein the attention heads include the attention heads to be intervened which have been pre-evaluated for factual relevance.

[0113] In this embodiment, the acquisition module 401 is used to acquire the target question containing context information by acquiring the target question through the user interface and extracting relevant context information by using technical means such as database or knowledge graph. The target question refers to a specific question or statement raised by the user and requires the system to generate an answer; the context information refers to background information related to the target question, which may include historical questions asked by the user, historical answers from the system, and specific domain knowledge.

[0114] Specifically, a content generation model is pre-built based on deep learning (such as Transformer, BERT, etc.), and the content generation model is used to generate corresponding reply content according to the input target question.

[0115] Specifically, the calculation module 402 is used to use the acquired target question and its context information as the input of the content generation model, process the context information of the target question through each neuron contained in the content generation model, and record the attribution score of each neuron when processing the context information. The attribution score is a score that quantifies the importance of each input unit (neuron) to the output in the neural network model. More specifically, the activation value generated by each neuron when processing the context information is calculated through the forward propagation of the content generation model, and then the attribution score is calculated based on the activation value using attribution analysis technology (such as Layer-wise Relevance Propagation, LRP).

[0116] Specifically, a score threshold is pre-set, and the score threshold is used to divide the context neurons and non-context neurons in the model. Among them, context neurons refer to neurons that pay more attention to information related to the context of the target problem when processing the target problem and its context information. That is, context neurons can play an important role in the processing and transmission of context information. Non-context neurons refer to neurons that pay more attention to other information that is not directly related to the context of the target problem when processing the target problem and its context information. That is, non-context neurons play a smaller role in the processing and transmission of context information.

[0117] Specifically, the division module 403 is used to traverse the attribution score calculated by each neuron, match the attribution score with a preset score threshold, and when the attribution score meets the score threshold, the neuron corresponding to the attribution score is divided into a context neuron; when the attribution score does not meet the score threshold, the neuron corresponding to the attribution score is divided into a non-context neuron.

[0118] Specifically, an enhancement strength value is preset, and the enhancement strength value is used to control the degree of enhancement of the context neuron weight. The weighting module 404 is used to increase the weight of the context neuron according to the preset enhancement strength value, for example, the weight of the context neuron is superimposed or multiplied with the enhancement strength value to obtain a weight-enhanced context neuron, so as to increase the importance of the context neuron in subsequent calculations.

[0119] Specifically, the extraction module 405 is used to extract features of the input target question by forward propagation using all neurons in the content generation model (including weight-enhanced context neurons and non-context neurons) after weight-enhancing the context neurons to obtain a feature vector.

[0120] Specifically, the content generation model is also provided with several attention heads, which are components in the model used to focus on different parts of the input sequence when generating a response. The decoder is a component in the model used to convert the feature vector into a natural language response.

[0121] Specifically, during the model training or tuning phase, the attention heads that pay more attention to the actual facts are determined by performing factual evaluation on each attention head in the model in advance. The attention heads that pay more attention to the actual facts are used as the attention heads to be intervened, and in the content generation phase for the target question, the attention heads to be intervened are intervened (for example, the weight is increased), which can make the response content generated by the model closer to the actual facts and avoid the occurrence of "hallucination" phenomenon (that is, the content generated by the model does not match the actual situation).

[0122] Specifically, generation module 406 is used to input the feature vectors extracted by all neurons in the content generation model into the attention heads (including the attention heads to be intervened that have been pre-evaluated for factual relevance and other attention heads in the model) through forward propagation, use the attention mechanism to focus on the key information in the input feature vectors, and pass the focused key information to the decoder, which converts the focused key information into the response content for the target question.

[0123] The content generation device 400 of the embodiment of the present application processes the target problem containing context information by adopting a content generation model, calculates the attribution score of each neuron in the model when processing the context information, and can accurately identify neurons that are more sensitive to context information, namely, context neurons, thereby ensuring that the model can focus on key context information when processing problems; by increasing the weight of context neurons according to a preset enhancement strength value, the model can more keenly capture and utilize context information when processing problems, which helps the model to give priority to the latest external knowledge when dealing with internal and external knowledge conflicts and effectively resolve knowledge conflicts; by adopting weight-enhanced context neurons and non-context neurons to jointly process the target problem, the logic and coherence of the generated content are ensured; by pre-evaluating the fact relevance of the attention heads in the model to determine the attention heads to be intervened, all attention heads and decoders including the attention heads to be intervened are used, and the response content for the target question is generated according to the feature vector extracted by the neurons, so that the generated response content is more in line with the actual situation, effectively reducing the occurrence of the "hallucination" phenomenon and improving the accuracy of the generated content. The embodiment of the present application is based on contextual neuron enhancement and factual reasoning intervention mechanism, which ensures that the model can give priority to the latest external knowledge when facing knowledge conflicts, reduces the occurrence of "hallucination" phenomenon, and avoids generating wrong answers, especially when processing complex knowledge, thereby improving the credibility of generated content.

[0124] In some optional implementations of this embodiment, the calculation module 402 may include a first recording submodule and a score calculation submodule.

[0125] A first recording submodule, configured to process the context information using each neuron and record a neuron activation value of each neuron when processing the context information;

[0126] The score calculation submodule is used to calculate the contribution of each neuron to the context according to the neuron activation value to obtain an attribution score.

[0127] Specifically, the first recording submodule is used to set a recording mechanism for each neuron in advance during the forward propagation process of the model, and the recording mechanism is used to capture and store the activation value of the neuron when processing context information. After the acquired target question and its context information are input into the content generation model, during the forward propagation process of the model, each neuron in the model is used to process the context information, such as performing nonlinear transformation and feature extraction on the context information, and recording the neuron activation value of each neuron when processing the context information. The neuron activation value is used to reflect the response strength of the neuron to the input context information.

[0128] Specifically, the score calculation submodule is used to obtain a predefined attribution method for calculating the contribution of neurons, which may include gradient-based attribution (such as Gradient×Input, Integrated Gradients), perturbation-based attribution (such as Occlusion / Perturbation Sensitivity), decomposition-based attribution (such as Layer-wise Relevance Propagation, LRP), etc. Using the predefined attribution method, the contribution of each neuron to the context is calculated according to the recorded neuron activation value to obtain the attribution score.

[0129] The calculation module 402 of the embodiment of the present application can identify the neurons that play a key role in processing context information by calculating the attribution score of each neuron in the content generation model when processing context information, which helps to enhance the understanding of the internal working mechanism of the model. Among them, the attribution score can be used as a basis for adjusting the weight of the neuron, and the performance of the model can be improved by enhancing or weakening the contribution of specific neurons.

[0130] In some optional implementations of this embodiment, the division module 403 may include a score comparison submodule, a first determination submodule, and a second determination submodule.

[0131] A score comparison submodule, used to obtain a preset score threshold and compare the attribution score with the score threshold;

[0132] A first determination submodule, configured to determine the neuron corresponding to the attribution score as a context neuron if the attribution score is greater than or equal to the score threshold;

[0133] The second determination submodule is configured to determine the neuron corresponding to the attribution score as a non-context neuron if the attribution score is less than the score threshold.

[0134] Specifically, the score comparison submodule is used to obtain a preset score threshold, which is used to divide the context neurons and non-context neurons in the model. The attribution score calculated by each neuron is traversed and compared with the preset score threshold. The first determination submodule is used to determine the neuron corresponding to the attribution score as a context neuron when the attribution score is greater than or equal to the score threshold, indicating that the neuron corresponding to the attribution score has a higher contribution in processing context information. The second determination submodule is used to determine the neuron corresponding to the attribution score as a non-context neuron when the attribution score is less than the score threshold, indicating that the neuron corresponding to the attribution score contributes less to the processing and transmission of context information.

[0135] In some optional implementations of this embodiment, the weighting module 404 may include an information acquisition submodule and a weight enhancement submodule.

[0136] An information acquisition submodule, used to acquire a preset enhancement strength value and an original weight of the context neuron;

[0137] The weight enhancement submodule is used to multiply the enhancement strength value by the original weight to increase the weight of the context neuron to obtain a weight-enhanced context neuron.

[0138] Specifically, the information acquisition submodule is used to obtain a preset enhancement strength value, which is used to control the degree of context neuron weight enhancement. The original weight of each context neuron is obtained. The weight enhancement submodule is used to multiply the preset enhancement strength value with the original weight of the context neuron to increase the weight of the context neuron to obtain a weight-enhanced context neuron.

[0139] Exemplarily, a weight enhancement formula is used to adjust the context neuron weight, wherein the weight enhancement formula is as follows:

[0140] w enh ance =α·w context_neuron_i ;

[0141] where w context_neuron_i represents the original weight of the i-th context neuron, α represents the enhancement value, and w enh ance Represents the enhanced weight of the i-th context neuron.

[0142] In some optional implementations of this embodiment, the extraction module 405 may include a neuron combination submodule, a weighted summation submodule, and a linear conversion submodule, wherein:

[0143] A neuron combination submodule, used for combining the weight-enhanced context neurons and the non-context neurons to obtain a target neuron set;

[0144] A weighted summation submodule, used to convert the target problem into a vector representation, and perform weighted summation on the vector representation using the target neuron set to obtain a weighted summation result;

[0145] The linear conversion submodule is used to perform nonlinear conversion on the weighted summation result by using a preset nonlinear activation function to obtain a feature vector of the target problem.

[0146] Specifically, the neuron combination submodule is used to replace the original weights of the context neurons with the weight-enhanced values, and then combine the context neurons and non-context neurons in the model to obtain the target neuron set. The weighted summation submodule is used to map the target problem into a high-dimensional vector space, convert it into a vector representation, and use the target neuron set to perform weighted summation on the vector representation, that is, use the weight of each neuron in the target neuron set (including the weight of the context neurons after weight enhancement and the original weight of the non-context neurons) to perform weighted summation on the vector representation to obtain the weighted summation result. The linear conversion submodule is used to obtain a preset nonlinear activation function, which may include a ReLU function, a sigmoid function, a tanh function, etc. The weighted summation result is nonlinearly converted using a preset nonlinear activation function to obtain a feature vector of the target problem, which contains feature information of the target problem after vector conversion, weighted summation, and nonlinear conversion.

[0147] The extraction module 405 of the embodiment of the present application can more accurately capture context-related information by enhancing the weights of context neurons in the model, thereby highlighting these key information in the feature extraction process. By combining context neurons and non-context neurons to perform weighted summation on the vector representation, it is ensured that the contribution of each feature in the final feature vector is proportional to its importance, so that the model not only pays attention to context information, but also considers broader or global features, achieves a balance between global and local features, and improves the comprehensiveness and accuracy of feature extraction. The nonlinear expression ability of the model is increased by the application of nonlinear activation functions, so that the model can capture more complex feature relationships, further improving the performance of the model.

[0148] In some optional implementations of this embodiment, the content generation device 400 may further include an evaluation module.

[0149] An evaluation module is used to obtain a preset training data set, where the training data set includes a number of training questions and labeled answers corresponding to the training questions; input the training questions into a content generation model for processing, and record the attention activation value of each attention head included in the content generation model when processing the training questions; obtain a pre-trained classifier, input the attention activation value and the labeled answer into the classifier, and calculate the fact correlation value between each attention head and the labeled answer based on the attention activation value and the labeled answer; sort each attention head according to the fact correlation value, and select a preset number of attention heads with the highest sorting as the attention heads to be intervened after the fact correlation evaluation.

[0150] Specifically, before using the content generation model to answer the input target question, each attention head in the content generation model is evaluated for factuality to determine the attention head that pays more attention to the actual facts. To achieve this goal, a pre-collected training data set is obtained, which includes a number of training questions and the labeled answers corresponding to the training questions. The labeled answers can be true answers and wrong answers.

[0151] Specifically, the training problem contained in the training data set is input into the content generation model for processing, and the attention activation value of each attention head contained in the content generation model when processing the training problem is recorded through a preset recording mechanism. The attention activation value is used to reflect the attention of the attention head to different parts of the training problem.

[0152] Specifically, a pre-trained classifier is obtained, which is used to calculate and output the fact correlation value between the attention activation value and the labeled answer. The recorded attention activation value and the labeled answer in the training data set are input into the classifier, and the fact correlation value between each attention head and the labeled answer is calculated by the classifier. The fact correlation value is used to reflect the correlation between the information captured by the attention head and the labeled answer.

[0153] Specifically, each attention head is sorted according to the calculated fact relevance value, wherein the attention head with a higher fact relevance value indicates that the information it captures is more consistent with the labeled answer, that is, the attention head pays more attention to the actual facts. From the sorted attention heads, a preset number (i.e., N) of the top-ranked attention heads are selected as the attention heads to be intervened after the fact relevance evaluation.

[0154] The content generation device 400 of the embodiment of the present application can identify which attention heads are more capable of capturing information related to the authenticity of the answer when processing training questions by recording the activation values ​​of the attention heads and using a classifier to calculate the factual correlation value of the attention head activation values ​​and the annotated answers. Selecting the top-ranked attention heads as the attention heads to be intervened helps to focus on relevant information more accurately when generating the reply content later.

[0155] In some optional implementations of this embodiment, the generation module 406 may include an attention intervention submodule, a weighted processing submodule, and a vector decoding submodule.

[0156] An attention intervention submodule, used to obtain a preset intervention strength value, and intervene in the attention head to be intervened according to the intervention strength value to obtain an attention head after intervention;

[0157] A weighted processing submodule, configured to perform weighted processing on the feature vector using the intervention attention head and other attention heads in the content generation model to obtain a weighted feature vector;

[0158] The vector decoding submodule is used to transfer the weighted feature vector to the decoder of the content generation model, and use the decoder to generate a response content for the target question according to the weighted feature vector.

[0159] Specifically, the attention intervention submodule is used to obtain a preset intervention strength value, which reflects the degree of intervention on the attention head to be intervened. According to the preset intervention strength value, the degree of attention of the attention head to be intervened to its attention direction is adjusted to achieve intervention on the attention head to be intervened, and obtain the attention head after intervention.

[0160] Exemplarily, an intervention formula is used to adjust the attention degree of the attention head to be intervened on its attention direction, so as to implement intervention on the attention head to be intervened, wherein the intervention formula is as follows:

[0161]

[0162] Among them, x l represents the attention activation value of the attention head to be intervened in the lth layer of the model; x l′ represents the activation value of the attention head after intervention in the lth layer of the model; β represents the intervention intensity value; Indicates the focus direction of the attention head to be intervened, which is used to guide the attention head to be intervened to pay more attention to factual knowledge.

[0163] Specifically, the weighted processing submodule is used to combine the intervention attention head with other attention heads in the content generation model to form an attention head set, and use each attention head in the attention set to weight the feature vector, that is, use the attention activation value of each attention head to weight the feature vector to obtain a weighted feature vector.

[0164] Specifically, the vector decoding submodule is used to pass the weighted feature vector to the decoder of the content generation model, and the decoder converts the received weighted feature vector into a text sequence, adjusts the generated text to comply with grammatical rules, and generates a response content for the target question.

[0165] The generation module 406 of the embodiment of the present application can guide the model to pay more attention to key information related to the facts and generate response content that is more in line with the actual situation by setting the intervention intensity value and accurately intervening the attention head to be intervened, thereby effectively reducing the occurrence of the "hallucination" phenomenon and improving the accuracy of the generated content.

[0166] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0167] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 6 with a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0168] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0169] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of the content generation method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0170] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the computer-readable instructions stored in the memory 61 or process data, such as computer-readable instructions for running the content generation method.

[0171] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0172] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the content generation method as described above.

[0173] The computer device, computer-readable storage medium and computer-readable instructions thereof provided in the embodiments of the present application process a target problem containing context information by executing a content generation model through a processor, calculate the attribution score of each neuron in the model when processing context information, and accurately identify neurons that are more sensitive to context information, namely, context neurons, thereby ensuring that the model can focus on key context information when processing problems; by increasing the weight of context neurons according to a preset enhancement strength value, the model can more keenly capture and utilize context information when processing problems, which helps the model to give priority to the latest external knowledge when processing internal and external knowledge conflicts and effectively resolve knowledge conflicts; by using weight-enhanced context neurons and non-context neurons to jointly process the target problem, the logic and coherence of the generated content are ensured; by pre-evaluating the fact relevance of the attention heads in the model to determine the attention heads to be intervened, all attention heads and decoders including the attention heads to be intervened are used to generate the reply content for the target question according to the feature vector extracted by the neurons, so that the generated reply content is more in line with the actual situation, effectively reducing the occurrence of the "hallucination" phenomenon and improving the accuracy of the generated content. The embodiment of the present application is based on contextual neuron enhancement and factual reasoning intervention mechanism, which ensures that the model can give priority to the latest external knowledge when facing knowledge conflicts, reduces the occurrence of "hallucination" phenomenon, and avoids generating wrong answers, especially when processing complex knowledge, thereby improving the credibility of generated content.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0175] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0176] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A content generation method, characterized in that: The steps include: Get the target question containing context information; Inputting the target question into a content generation model, and calculating an attribution score of each neuron included in the content generation model when processing the context information; According to the attribution score, each of the neurons is divided into a context neuron and a non-context neuron; Increasing the weight of the context neuron according to a preset enhancement strength value to obtain a weight-enhanced context neuron; Using the weight-enhanced context neurons and the non-context neurons to extract features of the target problem to obtain a feature vector; The attention heads and decoders in the content generation model are used to generate a response content for the target question according to the feature vector, wherein the attention heads include attention heads to be intervened that have been pre-evaluated for factual relevance.

2. The content generation method according to claim 1, characterized in that: The step of calculating the attribution score of each neuron included in the content generation model when processing the context information includes: Using each neuron to process the context information, and recording a neuron activation value of each neuron when processing the context information; The contribution of each neuron to the context is calculated according to the neuron activation value to obtain an attribution score.

3. The content generation method according to claim 1, characterized in that: The step of dividing each neuron into a context neuron and a non-context neuron according to the attribution score comprises: Obtaining a preset score threshold, and comparing the attribution score with the score threshold; If the attribution score is greater than or equal to the score threshold, determining the neuron corresponding to the attribution score as a context neuron; If the attribution score is less than the score threshold, the neuron corresponding to the attribution score is determined as a non-context neuron.

4. The content generation method according to claim 1, characterized in that: The step of increasing the weight of the context neuron according to the preset enhancement strength value to obtain a weight-enhanced context neuron includes: Obtaining a preset enhancement strength value and an original weight of the context neuron; The enhancement strength value is multiplied by the original weight to increase the weight of the context neuron to obtain a weight-enhanced context neuron.

5. The content generation method according to claim 1, characterized in that: The step of extracting features of the target problem using the weight-enhanced context neurons and the non-context neurons to obtain a feature vector comprises: Combining the weight-enhanced context neurons and the non-context neurons to obtain a target neuron set; Convert the target problem into a vector representation, and use the target neuron set to perform weighted summation on the vector representation to obtain a weighted summation result; A preset nonlinear activation function is used to perform nonlinear transformation on the weighted summation result to obtain a feature vector of the target problem.

6. The content generation method according to claim 1, characterized in that: Before the step of using the attention head and the decoder in the content generation model to generate the answer content for the target question according to the feature vector, the step further includes: Obtain a preset training data set, wherein the training data set includes a plurality of training questions and annotated answers corresponding to the training questions; Inputting the training problem into the content generation model for processing, and recording the attention activation value of each attention head included in the content generation model when processing the training problem; Obtain a pre-trained classifier, input the attention activation value and the labeled answer into the classifier, and calculate the fact correlation value between each attention head and the labeled answer according to the attention activation value and the labeled answer; Each of the attention heads is sorted according to the fact relevance value, and a preset number of attention heads with the highest sorting order are selected as the attention heads to be intervened after fact relevance evaluation.

7. The content generation method according to claim 1, characterized in that: The step of using the attention head and the decoder in the content generation model to generate the answer content for the target question according to the feature vector comprises: Obtaining a preset intervention strength value, intervening the attention head to be intervened according to the intervention strength value, and obtaining an attention head after intervention; Using the intervention attention head and other attention heads in the content generation model, weighting the feature vector to obtain a weighted feature vector; The weighted feature vector is passed to a decoder of the content generation model, and the decoder is used to generate a response content for the target question according to the weighted feature vector.

8. A content generating device, characterized in that: The device comprises: An acquisition module is used to obtain the target question containing context information; A calculation module, used for inputting the target question into a content generation model, and calculating an attribution score of each neuron included in the content generation model when processing the context information; A division module, used for dividing each neuron into a context neuron and a non-context neuron according to the attribution score; A weighting module, used for increasing the weight of the context neuron according to a preset enhancement strength value to obtain a weight-enhanced context neuron; An extraction module, used for extracting features of the target problem using the weight-enhanced context neurons and the non-context neurons to obtain a feature vector; A generation module is used to use the attention heads and decoders in the content generation model to generate a response content for the target question according to the feature vector, wherein the attention heads include attention heads to be intervened that have been pre-evaluated for factual relevance.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the content generation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the content generation method according to any one of claims 1 to 7 are implemented.

Images