Architecture method of user mental perception question and answer model based on knowledge reasoning technology

Abstract

The invention discloses an architecture method of a user mental perception question and answer model based on a knowledge reasoning technology, and relates to the technical field of multi-modal intelligent question and answer. The method specifically comprises the following steps: S1, constructing an information fusion module: fusing multi-modal information of a scene graph and a problem text, generating a fusion embedded representation, and obtaining multi-modal information fusion; s2, constructing a strategy network based on reinforcement learning: constructing the strategy network by adopting a reinforcement learning normal form, and modeling a process of searching a reasoning path in the scene graph; and S3, constructing a user feedback module based on question similarity: judging question semantic repetition based on word vector cosine similarity, converting the question semantic repetition into a user feedback signal, and returning the user feedback signal to the strategy network to guide path reasoning. According to the method, the interpretability of the question and answer model can be improved by utilizing the knowledge reasoning technology under the reinforcement learning normal form, and the feedback of the user is considered, so that the perception of the mind of the user is realized, and the personalized service is provided for the user.

Description

[0001] This application is a divisional application. The application number of the original application is 202311071584.7, the application date is August 24, 2023, and the name of the invention is “A method for constructing a multimodal user mind-perception question-and-answer model”. Technical Field

[0002] The present invention relates to the technical field of multimodal intelligent question answering, and in particular to a method for structuring a user mind perception question answering model based on knowledge reasoning technology. Background Art

[0003] In the era of big data, data presents the characteristics of massive multi-source, diverse modalities, and obvious fragmentation. In the field of fiscal and tax data, traditional fiscal and tax services have accumulated huge data assets that have not been fully utilized. The services provided are limited to simple queries in form and fixed databases in scope. The overall functions are single and the flexibility is not high. There are often individual differences between users, and the service requirements of the system are different. The traditional service system provides relatively unified objective results for users of different groups. It is impossible to perceive the subjective mental differences of users, and it is difficult to detect the real needs of users. Therefore, it is very necessary to build an intelligent user mental perception model to support the future smart fiscal and tax service system.

[0004] The concept of mental model comes from the field of interactive design. The interaction between mental model and the real world is completed through feedback. The feedback implies the characteristics of the user, which can be understood as the user's view of the current system. Therefore, the mental model should change with the change of the user. The purpose of introducing user feedback to build a mental model is to enable the model to dynamically adjust the answer strategy and give answers that are more in line with the user's personal mind.

[0005] The current financial and taxation platforms mainly provide users with Q&A queries. This is because with the development of natural language processing technology, intelligent Q&A, especially conversational Q&A, has become a popular form of human-computer interaction. Conversational Q&A is also called multi-round Q&A, which requires the machine to have a continuous conversation with the user around a certain topic based on the context. The current development status of Q&A systems can be summarized from two perspectives.

[0006] From the perspective of model input, this intelligent question-answering system can be divided into a unimodal information input system and a multimodal information input system. The unimodal information input question-answering system mainly uses text input, and the machine only understands the user's intention through the input text content. The multimodal information input question-answering system can receive information in multiple modes such as images, voice, text, knowledge graphs, and tables as input. The existing research hotspots of multimodal question-answering systems are concentrated on the system category with "image + text" as input. The system needs to extract features from data of two modes. Early multimodal tasks took splicing operations on these features from different modal data and then passed them into the classifier for training. The multimodal model constructed in this way heavily relies on multiple operations on features, the process is cumbersome and complicated, multimodal features are difficult to align, and the key information of local features is easily ignored, resulting in poor overall effect. In recent years, Transformer has made great achievements in the field of natural language processing. More and more studies have tried to apply Transformer structure to visual tasks, which has also opened a path for Transformer in the field of multimodal tasks. Since the Transformer's self-attention structure can adapt to different types of data, data of various modalities can perform better in mode alignment, fundamentally reducing the heterogeneity between modalities while also preserving the semantic integrity of the modality itself as much as possible. Therefore, applying the Transformer architecture to a multimodal question-answering system is an effective way to integrate multimodal data features, and its attention mechanism can also help the model capture more important semantic features in the input information.

[0007] From the perspective of model knowledge source, intelligent question-answering systems can be divided into machine reading comprehension-based and knowledge-based question-answering systems. Machine reading comprehension-based question-answering systems use unstructured knowledge sources. This type of question-answering system generally answers questions based on the information given by the user. The difficulty lies in the machine reading comprehension task. The existing methods mainly train recurrent neural networks and convolutional networks through supervised learning, or give machines the ability to understand and answer based on large-scale pre-trained language models. Knowledge-based question-answering systems use structured knowledge sources, such as knowledge graphs, which are essentially a kind of structured representation of knowledge. This type of question-answering system extracts important entities and relationships from the information given by the user, and uses the structural relationship of the knowledge graph to understand and reason about the problem. The rapid development of the information industry has made massive data a valuable asset that needs to be mined and utilized. These fragmented massive information often contains rich associations, and knowledge reasoning technology can use explicit knowledge to mine implicit relationships and fill in knowledge gaps. Therefore, it has gradually been applied to question-answering systems. Knowledge reasoning algorithms are mainly divided into algorithms based on association rules, path sorting algorithms based on graph structures, algorithms based on distributed representation learning, algorithms based on neural networks, and hybrid reasoning methods. Algorithms based on association rules mine and reason by defining or learning rules in knowledge. Path sorting algorithms based on graph structures predict possible relationships between entities by discovering link paths between them. Methods based on distributed representation learning map triple representations to vector space through mapping functions so that their new representations satisfy certain relationships as much as possible. Reasoning algorithms based on neural networks are represented by relational graph convolutional networks. The more popular reasoning algorithm at present is the hybrid reasoning method, which is mainly based on the idea of ​​reinforcement learning, treating fact judgment as a path-finding problem between head and tail entities, and using policy-based reinforcement learning methods to solve sequential decision problems.

[0008] Therefore, it is an urgent problem for technical personnel in this field to propose an architectural method for a user mind-aware question-answering model based on knowledge reasoning technology to solve the existing problems of how to use multimodal information to build an intelligent multimodal question-answering system, how to use knowledge reasoning technology under the paradigm of reinforcement learning to improve the interpretability of the question-answering model, and take into account user feedback to achieve perception of the user's mind, thereby providing personalized services to users. Summary of the invention

[0009] In view of this, the present invention provides an architecture method for a user mind-aware question-answering model based on knowledge reasoning technology, so that the reasoning of the model can be carried out in a direction closer to the user's mind, providing users with a better human-computer service experience.

[0010] In order to achieve the above object, the present invention adopts the following technical solution:

[0011] A method for constructing a user mind-aware question-answering model based on knowledge reasoning technology includes the following steps:

[0012] S1. Construct information fusion module: fuse the multimodal information of scene graph and question text, generate fused embedding representation, and obtain multimodal information fusion;

[0013] S2. Construct a policy network based on reinforcement learning: Use the reinforcement learning paradigm to construct a policy network to model the process of finding reasoning paths in the scene graph;

[0014] S3. Build a user feedback module based on question similarity: judge the semantic repetition of questions based on the cosine similarity of word vectors, convert it into user feedback signals and return them to the policy network to guide path reasoning.

[0015] Optionally, the specific steps of constructing the information fusion module in S1 are:

[0016] S11. GAT-based scene graph encoding: The scene graph data is encoded through the GAT graph attention network to obtain node embedding representation and relationship embedding representation;

[0017] S12. Performing contextual historical information fusion based on the attention mechanism to obtain a contextual historical information fusion representation;

[0018] S13. Multimodal information fusion based on Transformer decoder architecture: The Transformer decoder architecture is used to fuse the contextual historical information fusion representation, scene graph node embedding representation and relationship embedding representation to obtain a multimodal fusion output.

[0019] Optionally, the specific content of the GAT-based scene graph encoding in S11 is:

[0020] Get the scene graph node representation;

[0021] Calculate the correlation coefficient between the node and all neighboring nodes;

[0022] Normalize the correlation coefficient to get the attention coefficient;

[0023] The information of neighbor nodes is weighted and summed according to the attention coefficient to obtain a new representation of the node.

[0024] Optionally, the specific content of the context history information fusion based on the attention mechanism in S12 is:

[0025] Use the Glove pre-trained model to obtain text word vector representation;

[0026] The multi-head self-attention mechanism is used to calculate the attention coefficient of the current question and the context history information to obtain the output of the multi-head self-attention;

[0027] Perform the first residual connection and normalization on the output of multi-head self-attention;

[0028] After the feed-forward fully connected layer, a second residual connection and normalization are performed to obtain a fusion representation of contextual historical information.

[0029] Optionally, the specific content of the multimodal information fusion based on the Transformer decoder architecture in S13 is:

[0030] Input the output of the context fusion module into the first multi-head self-attention layer to calculate the Q, K, and V matrices to obtain the output representation of the first multi-head self-attention layer;

[0031] Input the output of the scene graph encoder and the output of the previous multi-head self-attention layer into the second multi-head self-attention layer to calculate the Q, K, V matrices;

[0032] The output of the second multi-head self-attention is residually connected and normalized, and then passed through a feed-forward fully connected layer to obtain the multimodal fusion output;

[0033] Among them, Q is the output from the previous multi-head self-attention, and K and V are the outputs from the scene graph encoder.

[0034] Optionally, the specific steps for constructing a policy network based on reinforcement learning in S2 are:

[0035] S21. Based on the Markov decision process modeling path finding process, the reasoning process of the model on the scene graph is modeled as the process of the wandering agent completing multi-hop reasoning;

[0036] S22. Build a strategy network based on the feedforward connection network structure and output the multi-hop action probability distribution of the intelligent agent;

[0037] S23. Optimize network parameters based on the policy gradient algorithm to maximize the reward of the agent's action trajectory.

[0038] Optionally, in S22, a policy network is built based on a feedforward connection network structure, and the specific content of the multi-hop action probability distribution of the output agent is:

[0039] Concatenate the output of the information fusion module and the embedded representation of the agent’s historical decisions and input them into the policy network;

[0040] Construct a two-layer feedforward network activated by the nonlinear ReLU function;

[0041] The probability distribution of an action space is obtained through the softmax operation;

[0042] The agent samples the action trajectory according to the probability distribution until it reaches the end of the path in the scene graph.

[0043] Optionally, the specific steps for constructing a user feedback module based on question similarity in S3 are:

[0044] S31. Encode the current question and the previous question into word vector representations using a fine-tuned BERT pre-trained model;

[0045] S32. Calculate the cosine similarity between two word vector representations:

[0046]

[0047] Among them, q t-1 is the word vector of the previous question, q t is the word vector of the current question, i is the i-th dimension of the vector;

[0048] S33. Determine the semantic similarity between the previous and next questions by using cosine similarity;

[0049] S34. judging the user's feedback on the answer to the previous question based on the relationship between the semantic similarity and the set threshold, and judging that the user has asked a repeated question and is dissatisfied with the answer to the previous question when the semantic similarity between the previous and subsequent questions is too high;

[0050] S35. Return corresponding rewards to the mind-aware question-answering model based on user feedback.

[0051] Optionally, it also includes a feasibility evaluation of the mind-aware question answering model. The specific content of the feasibility evaluation is to evaluate the performance of the mind-aware question answering model on the scene graph question answering dataset through three link prediction indicators.

[0052] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides an architectural method for a user mind-aware question-answering model based on knowledge reasoning technology, which has the following beneficial effects: 1) Introducing visual information in the form of scene graphs into the model can effectively integrate multi-round dialogue history information and image scene graph information, so that the model can obtain information at the image semantic level, and the structured data form of the scene graph also enhances the interpretability of the model reasoning; 2) Proposing a user feedback acquisition method based on question similarity introduces user feedback signals into the model, which can more intelligently infer user feedback on answers than traditional question-answering models. The feedback can be used to guide the subsequent policy network training of the model, so that the model's reasoning is closer to the user's mind, providing users with a better human-computer service experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0054] Figure 1 A schematic diagram of the architecture of a multimodal user mind perception question-answering model provided by the present invention;

[0055] Figure 2 A schematic diagram of a context information fusion module provided by the present invention;

[0056] Figure 3 A schematic diagram of the information fusion module provided by the present invention;

[0057] Figure 4 A schematic diagram of a reinforcement learning strategy network provided by the present invention;

[0058] Figure 5 A flow chart of a method for constructing a multimodal user mind perception question and answer model provided by the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] A multimodal user mind-aware question-answering model model architecture Figure 1 As shown in the figure, the entire mind-aware question-answering model consists of three parts: an information fusion module, a reinforcement learning reasoning module based on a policy network, and a user feedback module based on question similarity.

[0061] The information fusion module consists of three parts: scene graph encoder, context fusion module and multimodal information fusion module. It mainly completes the fusion of context information and multimodal information based on the Transformer decoder architecture. The input of the module consists of two parts: scene graph and question and answer text, and the output of the module is a fusion representation of the two modal information.

[0062] The reinforcement learning reasoning module based on the policy network uses the fusion representation of multimodal information to perform policy-based reasoning on the scene graph, models the dialogue task as a path-finding problem on the scene graph, and uses the final node of the path as the model reasoning answer.

[0063] The user feedback module based on question similarity combines the pre-trained language model to judge the similarity of context questions, returns the similarity as a feedback signal to the policy network, and adjusts the policy to give reasoning results that are more in line with the user's psychological expectations.

[0064] See also Figure 5 As shown, the present invention discloses a method for constructing a multimodal user mind perception question-answering model, comprising the following steps:

[0065] S1. Construct information fusion module: fuse the multimodal information of scene graph and question text, generate fused embedding representation, and obtain multimodal information fusion;

[0066] S2. Construct a policy network based on reinforcement learning: Use the reinforcement learning paradigm to construct a policy network to model the process of finding reasoning paths in the scene graph;

[0067] S3. Build a user feedback module based on question similarity: judge the semantic repetition of questions based on the cosine similarity of word vectors, convert it into user feedback signals and return them to the policy network to guide path reasoning.

[0068] Furthermore, the specific steps of constructing the information fusion module in S1 are:

[0069] S11. GAT-based scene graph encoding: The scene graph data is encoded through the GAT graph attention network to obtain node embedding representation and relationship embedding representation;

[0070] S12. Performing contextual historical information fusion based on the attention mechanism to obtain a contextual historical information fusion representation;

[0071] S13. Multimodal information fusion based on Transformer decoder architecture: The Transformer decoder architecture is used to fuse the contextual historical information fusion representation, scene graph node embedding representation and relationship embedding representation to obtain a multimodal fusion output.

[0072] Furthermore, the specific content of the GAT-based scene graph encoding in S11 is:

[0073] Get the scene graph node representation;

[0074] Calculate the correlation coefficient between the node and all neighboring nodes;

[0075] Normalize the correlation coefficient to get the attention coefficient;

[0076] The information of neighbor nodes is weighted and summed according to the attention coefficient to obtain a new representation of the node.

[0077] Furthermore, the specific content of contextual historical information fusion based on the attention mechanism in S12 is:

[0078] Use the Glove pre-trained model to obtain text word vector representation;

[0079] The multi-head self-attention mechanism is used to calculate the attention coefficient of the current question and the context history information to obtain the output of the multi-head self-attention;

[0080] Perform the first residual connection and normalization on the output of multi-head self-attention;

[0081] After the feed-forward fully connected layer, a second residual connection and normalization are performed to obtain a fusion representation of contextual historical information.

[0082] Furthermore, the specific content of multimodal information fusion based on the Transformer decoder architecture in S13 is:

[0083] Input the output of the context fusion module into the first multi-head self-attention layer to calculate the Q, K, and V matrices to obtain the output representation of the first multi-head self-attention layer;

[0084] Input the output of the scene graph encoder and the output of the previous multi-head self-attention layer into the second multi-head self-attention layer to calculate the Q, K, V matrices;

[0085] The output of the second multi-head self-attention is residually connected and normalized, and then passed through a feed-forward fully connected layer to obtain the multimodal fusion output;

[0086] Among them, Q is the output from the previous multi-head self-attention, and K and V are the outputs from the scene graph encoder.

[0087] Specifically, scene graph encoding: the scene graph data is encoded through the GAT (Graph Attention Network) graph attention network to generate node embedding representation and relationship embedding representation.

[0088] The scene graph is a directed graph, and each node corresponds to an entity or an attribute of an object in a real image. The entity itself also has category attributes. The edges between the nodes in the scene graph describe the association relationship between the entities corresponding to the nodes, such as spatial position relationship, category affiliation, etc. In the scene graph, the important features of any node come from both the structural relationship with the neighboring nodes and the features or attributes of the node itself, so it is necessary to use a graph convolutional neural network to encode it. In addition, the scene graph is a directed graph, and the relationship between entities is directional. It is necessary to use a graph attention network with an attention mechanism to aggregate the features of the node's neighbor nodes so that the correlation between the node features can be better integrated into the new embedding representation.

[0089] Suppose there are several nodes in the scene graph, and a node i has N i neighbor nodes, the feature of node i is represented by h i ,The steps to generate scene graph node embedding representation and relationship embedding representation using graph attention network are as follows:

[0090] Calculate the attention coefficient: First calculate the correlation coefficient between nodes, and then normalize the correlation coefficient to obtain the attention coefficient.

[0091] Calculate the correlation coefficient: For node i, calculate its neighbor nodes j∈N one by one i The correlation coefficient between it and itself is:

[0092] e ij =a([Wh i ][Wh j ])

[0093] First, we use a linear mapping W to increase the dimension of the features of nodes i and j, and then concatenate the increased dimensional features to obtain higher dimensional features. We use a single-layer feedforward neural network a(·) to map this higher dimensional feature to a real number and obtain the correlation coefficient e between nodes i and j. ij .

[0094] Normalization: Use the softmax operation to normalize the correlation coefficient to obtain the attention coefficient α ij :

[0095]

[0096] Among them, e ik is the correlation coefficient between node i and its kth neighbor node.

[0097] Weighted summation: According to the attention coefficient calculated above, the features are weighted summed and a new feature h' of node i is obtained through a layer of activation function. i :

[0098]

[0099] Among them, σ is the activation function, h j is the feature before node i.

[0100] Contextual historical information fusion: Generate word vector representation and use multi-head attention mechanism to calculate attention coefficient. The architecture is as follows Figure 2 shown.

[0101] Generate word vector representation: First, use the Glove (GlobalVector) pre-trained model to initialize the text information to generate the word vector representation of the current question q tWord vector representation of historical dialogue Among them, Glov completes word representation based on global word frequency statistics. The pre-trained Glov model can directly convert words into word vectors. These vector representations already contain the semantic characteristics of the words. By calculating the distance or cosine similarity between word vectors, the semantic similarity of the words can be obtained.

[0102] Use the multi-head self-attention mechanism to calculate the attention coefficient: send the word vector representation of the current question and historical information to N multi-head attention modules to calculate the attention coefficient. The multi-head self-attention module is mainly implemented by scaling the dot product attention. The implementation process of the first head's self-attention is used as an example:

[0103] Calculate Query, Key, Value matrices: Use linear transformation matrices Q t , H t Transform into a new matrix representation:

[0104]

[0105] Compute the self-attention output of the first head:

[0106]

[0107] Among them, Att1 is the self-attention output of the first self-attention head, Q is the input vector q t The generated Query matrix, K is the input vector H t The generated Key matrix, d K is the dimension of the Value matrix.

[0108] Similarly, calculate the self-attention outputs of other heads and concatenate them together, and get the output of multi-head self-attention after a linear layer:

[0109]

[0110] Among them, Linear(·) is a linear mapping function, Att1...Att n is the self-attention output of the 1st to nth self-attention heads, For splicing operation.

[0111] The first residual connection and normalization: The residual connection refers to connecting the input and output of the multi-head self-attention. This connection method allows the network to focus on the current difference part. Normalization uses the layer normalization method to standardize the input of each layer of neurons in the network:

[0112] Z' t =LayerNorm(Zt +q t )

[0113] Among them, LayerNorm(·) is the normalization operation function, Z t is the output of multi-head self-attention.

[0114] Feedforward fully connected layer: It consists of two fully connected layers, and the parameters are and Indicates that the first fully connected layer is activated by the ReLU function:

[0115]

[0116] Second residual connection and normalization:

[0117] X' t =LayerNorm(X t +Z' t )

[0118] The final contextual historical information fusion is expressed as:

[0119]

[0120] After integrating the contextual historical information, the current question q t We get a new representation f t , due to the use of the self-attention mechanism, the new representation of the problem f t Be context-aware.

[0121] Multimodal information fusion: Transformer decoder architecture is used to represent historical information perception t and scene graph node embedding representation fusion. Figure 3 As shown in Figure 1, this architecture includes two multi-head self-attention layers. Compared with the structure of the context information fusion module, the multimodal information fusion module has one more multi-head self-attention layer, and the subsequent operations are similar to those described above. Among them, the K and V matrices of the second multi-head self-attention layer are calculated using the output of the scene graph encoder, and the Q matrix is ​​calculated using the output of the context history information fusion module. This layer allows the information of the scene graph modality to complement the information of the text modality, and the two can be effectively fused.

[0122] The first multi-head self-attention layer: The Q, K, and V matrices of the first layer all use the output f of the context fusion module t calculate:

[0123] Q=f t W q , K = f t W k , V = f tW v

[0124] Calculate the QK of each head T After that, it is bitwise multiplied with the masking matrix to obtain the masked QK T , and then perform subsequent softmax calculations, so the self-attention output is expressed as:

[0125]

[0126] Among them, Att masked is the masked self-attention output and Masked(.) is the masking operation.

[0127] The masked self-attention outputs of multiple heads are concatenated to get the output of the masked multi-head self-attention layer.

[0128] The second multi-head self-attention layer: calculates the Q, K, V matrices. The object of the linear transformation here is the output e of the scene graph encoder t And the output f' of the previous multi-head self-attention layer t :

[0129]

[0130] The following steps are similar to the contextual history information fusion module. After using Q and K to calculate the attention coefficient, it is residually connected and normalized, and then passed through a layer of feedforward fully connected layer to obtain the final multimodal fusion output.

[0131] Furthermore, the specific steps of constructing a policy network based on reinforcement learning in S2 are:

[0132] S21. Based on the Markov decision process modeling path finding process, the reasoning process of the model on the scene graph is modeled as the process of the wandering agent completing multi-hop reasoning;

[0133] S22. Build a strategy network based on the feedforward connection network structure and output the multi-hop action probability distribution of the intelligent agent;

[0134] S23. Optimize network parameters based on the policy gradient algorithm to maximize the reward of the agent's action trajectory.

[0135] Furthermore, in S22, a strategy network is built based on the feedforward connection network structure, and the specific content of the multi-hop action probability distribution of the output agent is:

[0136] Concatenate the output of the information fusion module and the embedded representation of the agent’s historical decisions and input them into the policy network;

[0137] Construct a two-layer feedforward network activated by the nonlinear ReLU function;

[0138] The probability distribution of an action space is obtained through the softmax operation;

[0139] The agent samples the action trajectory according to the probability distribution until it reaches the end of the path in the scene graph.

[0140] Specifically, Figure 4 As shown in the figure, we construct a policy network, model the path-finding process, and use the reinforcement learning paradigm to complete the model's reasoning on the scene graph. The specific steps are:

[0141] Modeling the path-finding process: The reasoning process of the model on the scene graph is modeled as a process of a wandering agent completing multi-hop reasoning. The purpose of multi-hop reasoning is to find a reliable prediction path between entities and relationships. Generally, multiple reasonings are modeled as a Markov sequence decision process MDP (Markov Decision Process). The Markov sequence decision process is defined by a quaternion (S, A, P, R), which represents the state space, action space, state transition probability, and reward function, respectively. For a scene graph Define E to represent the set of all entities, and R to represent the set of all relationships, then the scene graph It can be represented as a set of triples (s, p, o), representing subject, predicate, and object respectively.

[0142] The core idea of ​​Markov decision process is the state S at the next moment t+1 Only with the current state S t and the action to be taken in the current state A t Therefore, to obtain the state at the next moment, you only need to go back to the previous step.

[0143] In the process of reinforcement learning, the agent needs to interact with the environment. Specifically, the agent obtains the state from the environment, and uses this state to select an action output. The process of outputting an action is a decision-making process. After the decision is returned to the environment, the environment will give the next state and the reward that the current decision should get. The state space, action space and reward function are explained below.

[0144] State space: At time step t, the state S = (e t ,h t ,q), where e t is the entity node currently reached by the agent, h t is the historical decision of the agent at time step t, that is, the decision at time step t-1. When t=1, h t is an empty set, and q represents the current given problem. For the agent, q is a global information, but for e tThe choice depends on h t of.

[0145] Action space: At time step t, when the agent is in state S, the next action A it will take t From an action set A(S t Specifically, an action of the agent includes the edge selected in the next step and the node selected in the next step, so A(S t ) should be all the nodes related to the current node e t The set of associated tuples (r,e),

[0146] Reward function: According to the previous description of the reinforcement learning process, each decision of the agent should be rewarded by environmental feedback, and the ultimate goal of the agent is to obtain as many rewards as possible. However, in our question-answering reasoning task scenario, only when the agent reaches the final correct node will its series of decisions be given positive feedback. Correspondingly, the reward function value is 1 in this case, otherwise it is 0.

[0147] Constructing a policy network: The input of the policy network is composed of two parts. The first part is the multimodal information fusion representation M output by the previous information fusion module. t , the second part is the embedding representation H of the agent’s historical decisions t The policy network is a two-layer feedforward network structure, which is activated by a nonlinear ReLu function and finally obtains a probability distribution P(A t ). The parameters of the policy network mainly include two weight matrices W1 and W2:

[0148] P(A t )=softmax(A t (W2ReLU(W1[H t ,M t ])))

[0149] in, represents all actions at time t, H t It is the historical decision representation processed by LSTM:

[0150] H t =LSTM(a t-1 )

[0151] Probability distribution P(A t) reflects the probability of the agent selecting each action in the next time step. As the policy network is updated, the probability of selecting actions with high feedback value will be increased. Conversely, the probability of selecting actions with low feedback value will be reduced. The agent will sample the next action according to this probability distribution until it reaches the end of the path:

[0152]

[0153] Training policy network: Use the policy gradient algorithm REINFORCE to train the policy network constructed above. The training goal is to learn a policy network parameter π that can maximize the reward. θ .

[0154] Define the maximization objective function:

[0155]

[0156] In the actual reasoning process, it is difficult for the intelligent agent to enumerate all complete reasoning paths, so it is necessary to sample as many action trajectories τ as possible and obtain the expectation to replace the original summation operation.

[0157] Among them, P(τ; θ) represents the product of the state transition profile and the action selection probability:

[0158]

[0159] Policy network parameter update rule: In order to maximize the feedback, the policy gradient is updated in the upward direction:

[0160]

[0161] Furthermore, the specific steps for constructing a user feedback module based on question similarity in S3 are:

[0162] S31. Encode the current question and the previous question into word vector representations using a fine-tuned BERT pre-trained model;

[0163] S32. Calculate the cosine similarity between two word vector representations:

[0164]

[0165] Among them, q t-1 is the word vector of the previous question, q t is the word vector of the current question, d is the dimension of the word vector, and i is the i-th dimension of the vector;

[0166] S33. Determine the semantic similarity between the previous and next questions by using cosine similarity;

[0167] S34. judging the user's feedback on the answer to the previous question based on the relationship between the semantic similarity and the set threshold, and judging that the user has asked a repeated question and is dissatisfied with the answer to the previous question when the semantic similarity between the previous and subsequent questions is too high;

[0168] S35. Return corresponding rewards to the mind-aware question-answering model based on user feedback.

[0169] Specifically, in a real human-computer dialogue, when the machine gives an incorrect answer or an answer that does not meet human expectations, the human response is usually to rephrase the question in a different form and input it back to the machine. Based on this common fact, the present invention constructs a user feedback module based on question similarity judgment to simulate the human-computer interaction process in real scenarios and realize the model's perception of the user's mind.

[0170] Question similarity calculation: Use the fine-tuned BERT model to obtain the word vector representation of the two question texts, and calculate the cosine similarity of the two word vectors:

[0171]

[0172] Among them, q t-1 is the word vector of the previous question, q t is the word vector of the current question, and d is the dimension of the word vector.

[0173] Feedback function design: Set a judgment threshold δ. When the similarity between the previous and next questions is higher than the threshold, it is considered that the user has asked a semantically repeated question, and the model reasoning result does not meet the user's expectations. The feedback given is 0. On the contrary, when the similarity between the previous and next questions is higher than the threshold, it is considered that the user has entered a new dialogue process, and the model reasoning result should be given positive feedback 1:

[0174]

[0175] cos_sim is the cosine similarity, and threshold is the set threshold.

[0176] Furthermore, it also includes a feasibility evaluation of the mind-aware question answering model. The specific content of the feasibility evaluation is to evaluate the performance of the mind-aware question answering model on the scene graph question answering dataset through three link prediction indicators.

[0177] In a specific embodiment, the specific contents of the feasibility verification of the scheme are:

[0178] The scene graph question answering dataset GQA was selected to experimentally verify the above design scheme: the GQA dataset contains 11,000 pictures, each of which corresponds to a well-labeled scene graph structure data. The question and answer content is mainly designed around the object relationship in the picture and the object's own attributes, and is semantically rich. The entire vocabulary contains 3,097 words, including 1,702 object classes, 310 relationship classes, and 610 object attributes. There are 3 to 20 question and answer pairs around each picture.

[0179] Filter data: Filter out all images containing 10 question-answer pairs, and arrange their related conversation contents into multiple rounds of dialogue. Finally, 2,982 images are screened.

[0180] Verify the feasibility of the user feedback module based on question similarity judgment: Use ChatGPT to rewrite some questions without changing the semantics of the sentences. In the specific model training process, when the model's reasoning result for the input question does not match the standard answer, the rewritten question will be input as supplementary corpus to the model to continue learning based on the corresponding feedback.

[0181] Model reasoning is essentially a triple link prediction task. For a missing entity o in a triple (s, p, o), the model needs to use all entities in the scene graph as candidates for calculation and ranking. Therefore, the link prediction index HITS@n is used as the evaluation index of model performance. It indicates how many correct triplets are finally ranked in the top n. Commonly used indicators are HITS@1, HITS@3, and HITS@10. The larger the index value, the better the inference prediction effect of the model:

[0182]

[0183] Among them, II is the conditional truth function, if the condition is true, the function value is 1, otherwise it is 0, |S| is the number of triples, rank i is the ranking of the correct missing entity of the i-th triple in the model prediction results.

[0184] As shown in Table 1, the model performs well in testing binary questions, where the answer forms are only "yes" and "no". The model without the contextual historical information fusion module and the user feedback module based on question similarity judgment is used as a common multimodal question-answering model to compare with the user mind-aware question-answering model. It can be seen that the user mind-aware question-answering model performs better than the common multimodal question-answering model in all three indicators, which shows the necessity of contextual historical information fusion and user feedback.

[0185] Table 1 Test results of binary questions

[0186] Model HITS@1 HITS@3 HITS@10 Multimodal user mind-aware question-answering model 0.7782 0.9097 0.9222 General Multimodal Question Answering Model 0.7653 0.9021 0.9214

[0187] As shown in Table 2, open-ended questions are tested. The answers to open-ended questions are not limited to "yes" and "no". The answers are more diversified and the reasoning difficulty for the model is higher than that of binary questions. Therefore, the overall performance of the model is lower than the results in Table 1. However, compared with the ordinary multimodal question-answering model, the user mind perception question-answering model still performs better.

[0188] Table 2 Test results of open questions

[0189] Model HITS@1 HITS@3 HITS@10 Multimodal user mind-aware question-answering model 0.6898 0.8734 0.9018 General Multimodal Question Answering Model 0.6721 0.8684 0.8980

[0190] After testing different types of questions, the multimodal user mind-aware question-answering model can achieve better performance than the ordinary multimodal question-answering model, verifying the feasibility of the above design scheme.

[0191] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a user mind perception question-answering model based on knowledge reasoning technology, characterized in that: The following steps are involved: S1. Construct information fusion module: fuse the multimodal information of scene graph and question text, generate fused embedding representation, and obtain multimodal information fusion; S2. Construct a policy network based on reinforcement learning: Use the reinforcement learning paradigm to construct a policy network to model the process of finding reasoning paths in the scene graph; S3. Build a user feedback module based on question similarity: judge the semantic repetition of questions based on the cosine similarity of word vectors, convert it into user feedback signals and return them to the policy network to guide path reasoning; The specific steps of constructing the information fusion module in S1 are: S11. GAT-based scene graph encoding: The scene graph data is encoded through the GAT graph attention network to obtain node embedding representation and relationship embedding representation; S12. Performing contextual historical information fusion based on the attention mechanism to obtain a contextual historical information fusion representation; S13. Multimodal information fusion based on Transformer decoder architecture: The Transformer decoder architecture is used to fuse the context history information fusion representation, scene graph node embedding representation and relationship embedding representation to obtain a multimodal fusion output; The specific content of contextual historical information fusion based on the attention mechanism in S12 is: Generate text word vector representation: Use the Glove pre-trained model to initialize the text information to generate the word vector representation of the current question q t Word vector representation of historical dialogue Use the multi-head self-attention mechanism to calculate the attention coefficient: send the word vector representation of the current question and historical information to N multi-head attention modules respectively to calculate the attention coefficient; the specific implementation process of self-attention is: Calculate Query, Key, Value matrices: Use linear transformation matrices Q t , H t Transform into a new matrix representation: Compute the self-attention output of the first head: Among them, Att1 is the self-attention output of the first self-attention head, and Q is the input vector q t The generated Query matrix, K is the input vector H t The generated Key matrix, d K is the dimension of the Value matrix; Calculate the self-attention outputs of other self-attention heads and concatenate them together. After a linear layer, we get the output of multi-head self-attention: Among them, Linear(·) is a linear mapping function, Att1...Att n is the self-attention output of the 1st to nth self-attention heads, For splicing operation; First residual connection and normalization: The first residual connection and normalization are performed on the output of the multi-head self-attention. The residual connection is to connect the input and output of the multi-head self-attention. The normalization adopts the layer normalization method to standardize the input of each layer of neurons in the network: Z' t =LayerNorm(Z t +q t ) Among them, LayerNorm(·) is the normalization operation function, Z t is the output of multi-head self-attention; After the feedforward fully connected layer, a second residual connection and normalization are performed to obtain the contextual historical information fusion representation; the feedforward fully connected layer consists of two fully connected layers, and the parameters are respectively and Indicates that the first fully connected layer is activated by the ReLU function: Second residual connection and normalization: X' t =LayerNorm(X t +Z' t ) The final contextual historical information fusion is expressed as: After integrating the contextual historical information, the current question q t We obtain a new context-aware representation f t .

2. According to the method for constructing a user mind perception question-answering model based on knowledge reasoning technology in claim 1, it is characterized in that: The specific contents of the GAT-based scene graph encoding in S11 are: Get the scene graph node representation: Suppose there are several nodes in the scene graph, and a node i has N i neighbor nodes, the feature of node i is represented by h i ; Calculate the correlation coefficient between a node and all neighboring nodes: For node i, calculate its neighboring nodes j∈N one by one i The correlation coefficient between it and itself is: e ij =a([Wh i ][Wh j ]) First, we use a linear mapping W to increase the dimension of the features of nodes i and j, and then concatenate the increased dimensional features to obtain higher dimensional features. We use a single-layer feedforward neural network a(·) to map this higher dimensional feature to a real number, and obtain the correlation coefficient e between nodes i and j. ij ; Use the softmax operation to normalize the correlation coefficient to obtain the attention coefficient α ij : Among them, e ik is the correlation coefficient between node i and its kth neighbor node; According to the attention coefficient, the features are weighted and summed up, and a new feature h' of node i is obtained through a layer of activation function. i : Among them, σ is the activation function, h j is the feature before node i.

3. According to the method for constructing a user mind perception question-answering model based on knowledge reasoning technology in claim 1, it is characterized in that: The specific contents of multimodal information fusion based on Transformer decoder architecture in S13 are: The output of the context fusion module f t Input into the first multi-head self-attention layer to calculate the Q, K, V matrices: Q=f t W q ,K=f t W k ,V=f t W v Calculate the QK of each head T After that, it is bitwise multiplied with the masking matrix to obtain the masked QK T , and then perform the subsequent softmax calculation, the self-attention output is expressed as: Among them, Att masked is the masked self-attention output, Masked(.) is the masking operation; Concatenate the masked self-attention outputs of multiple heads to get the output of the masked multi-head self-attention layer; The output of the scene graph encoder is t And the output f' of the previous multi-head self-attention t Input into the second multi-head self-attention layer to calculate the Q, K, V matrices: The output of the second multi-head self-attention is residually connected and normalized, and then passed through a feed-forward fully connected layer to obtain the multimodal fusion output; Among them, Q is the output from the previous multi-head self-attention, and K and V are the outputs from the scene graph encoder.

4. According to the method for structuring a user mind perception question-answering model based on knowledge reasoning technology in claim 1, it is characterized in that: The specific steps for constructing a policy network based on reinforcement learning in S2 are: S21. Based on the Markov decision process modeling path finding process, the reasoning process of the model on the scene graph is modeled as the process of the wandering agent completing multi-hop reasoning; S22. Build a strategy network based on the feedforward connection network structure and output the multi-hop action probability distribution of the intelligent agent; S23. Optimize network parameters based on the policy gradient algorithm to maximize the reward of the agent's action trajectory.

5. The method for constructing a user mind perception question-answering model based on knowledge reasoning technology according to claim 4 is characterized in that: In S21, the Markov decision process is defined by a quaternion (S, A, P, R), which represents the state space, action space, state transition probability, and reward function respectively; For a scene graph Define E to represent the set of all entities and R to represent the set of all relationships, then the scene graph It can be represented as a set of triples (s, p, o), representing subject, predicate, and object respectively; State space: At time step t, the state S = (e t ,h t ,q), where e t is the entity node currently reached by the agent, h t is the historical decision of the agent at time step t, that is, the decision at time step t-1. When t=1, h t is an empty set, q represents the current given problem; for the agent, q is a global information, but for e t The choice depends on h t of; Action space: At time step t, when the agent is in state S, the next action A it will take t From an action set A(S t ); Specifically, an action of the agent includes the edge selected in the next step and the node selected in the next step, so A(S t ) are all nodes related to the current node e t The set of associated tuples (r,e), Reward function: Only when the agent reaches the final correct node will its series of decisions be given positive feedback. Correspondingly, the reward function value is 1 in this case, otherwise it is 0.

6. The method for constructing a user mind perception question-answering model based on knowledge reasoning technology according to claim 4 is characterized in that: In S22, the input of the policy network is represented by the multimodal information fusion output by the information fusion module M t and the embedding representation H of the agent’s historical decisions t Spliced ​​together; The policy network is a two-layer feedforward network structure, which is activated by a nonlinear ReLU function and finally obtains a probability distribution P(A) of the action space through a softmax operation. t ); The parameters of the policy network mainly include two weight matrices W1 and W2: P(A t )=softmax(A t (W2ReLU(W1[H t ,M t ]))) in, represents all actions at time t, H t It is the historical decision representation processed by LSTM: H t =LSTM(a t-1 ) Probability distribution P(A t ) reflects the probability of the agent selecting each action in the next time step. As the policy network is updated, the probability of actions with high feedback value being selected will be increased. Conversely, the probability of actions with low feedback value being selected will be reduced. The agent samples the next action according to this probability distribution until it reaches the end of the path in the scene graph:

7. The method for constructing a user mind perception question-answering model based on knowledge reasoning technology according to claim 4 is characterized in that: In S23, the network parameters are optimized based on the policy gradient algorithm optimization strategy, specifically: The policy gradient algorithm REINFORCE is used to train the policy network. The training goal is to learn a policy network parameter π that can maximize the reward. θ ; Define the maximization objective function: Sample the action trajectory τ and obtain the expectation to replace the original summation operation; Among them, P(τ; θ) represents the product of the state transition profile and the action selection probability: In order to maximize the feedback, the policy gradient is updated in the upward direction:

8. The method for constructing a user mind perception question-answering model based on knowledge reasoning technology according to claim 1 is characterized in that: The specific steps for building a user feedback module based on question similarity in S3 are: S31. Encode the current question and the previous question into word vector representations using a fine-tuned BERT pre-trained model; S32. Calculate the cosine similarity between two word vector representations: Among them, q t-1 is the word vector of the previous question, q t is the word vector of the current question, i is the i-th dimension of the vector, and d is the dimension of the word vector; S33. Determine the semantic similarity between the previous and next questions by using cosine similarity; S34. Set a judgment threshold δ. When the similarity between the previous and next questions is higher than the threshold, it is considered that the user has asked a semantically repeated question, and the model reasoning result does not meet the user's expectations. The feedback given is 0. On the contrary, when the similarity between the previous and next questions is lower than the threshold, it is considered that the user has entered a new dialogue process, and the model reasoning result should be given positive feedback 1: Among them, cos_sim is the cosine similarity, and threshold is the set threshold; S35. Return corresponding rewards to the mind-aware question-answering model based on user feedback.

9. The method for constructing a user mind perception question-answering model based on knowledge reasoning technology according to claim 1 is characterized in that: It also includes a feasibility evaluation of the mind-aware question answering model. The specific content of the feasibility evaluation is to evaluate the performance of the mind-aware question answering model on the scene graph question answering dataset through three link prediction indicators.

Images