An architecture method of a user mental perception question and answer model based on knowledge reasoning technology

Abstract

The application discloses a kind of based on knowledge reasoning technology's user mental perception question and answer model's architecture method, it is related to multimodal intelligent question and answer technical field.The specific content is: S1.build information fusion module: the multimodal information of fusion scene graph, question text, generate fusion embedding representation, obtain multimodal information fusion;S2.build strategy network based on reinforcement learning: strategy network is constructed using reinforcement learning paradigm, modeling in scene graph finds the process of reasoning path;S3.build user feedback module based on question similarity: based on word vector cosine similarity judges question semantic repeatability, converts into user feedback signal and returns to strategy network to guide path reasoning.The application can use knowledge reasoning technology to improve the explainability of question and answer model under the paradigm of reinforcement learning, and consider the feedback of user, realize the perception to user's mind, to provide personalized service for user.

Description

[0001] This application is a divisional application, the original case's application number is 202311071584.7, the application date is 2023-08-24, and the invention name is "a construction method of multi-modal user mental perception question and answer model". TECHNICAL FIELD

[0002] The present application relates to the technical field of multi-modal intelligent question and answer, in particular to a construction method of user mental perception question and answer model based on knowledge reasoning technology. BACKGROUND

[0003] In the big data era, data presents the characteristics of massive multi-source, modal diversity and obvious fragmentation. In the field of financial and tax data, the traditional financial and tax services have accumulated a large amount of data assets that have not been fully applied. The service provided is limited to simple query in form and fixed database in scope, with single overall function and low flexibility. There are individual differences among users, and the traditional service system provides relatively unified objective results for different groups of users, which cannot perceive the subjective mental differences of users and is difficult to explore the real needs of users. Therefore, it is necessary to construct an intelligent user mental perception model to support the future intelligent financial and tax service system.

[0004] The concept of mental model comes from the field of interactive design. The interaction between mental model and the real world is completed through feedback, and the user's characteristics are implied in the feedback, which can be understood as the user's view of the current system, so the mental model should change with the user. The purpose of introducing user feedback to construct the mental model is to enable the model to dynamically adjust the answering strategy and give answers that are more in line with the user's personal mind.

[0005] The current financial and tax platform provides interaction for users mainly in the form of question and answer query, which is also because with the development of natural language processing technology, intelligent question and answer, especially conversational question and answer, has become a popular form of human-computer interaction. Conversational question and answer, also known as multi-round question and answer, requires the machine to have continuous dialogue with the user around a certain topic based on the context content. The current development and research status of question and answer system can be summarized from two angles.

[0006] From the perspective of model input, this kind of intelligent question and answer system can be divided into single modal information input system and multi-modal information input system. The single modal information input question and answer system is mainly based on text input, and the machine only understands the user's intention through the input text content. The multi-modal information input question and answer system can receive image, voice, text, knowledge graph, table and other multi-modal information as input. The existing research hotspots of multi-modal question and answer system are concentrated on the system category with "image + text" as input. The system needs to extract features from two modal data. Early multi-modal tasks take splicing operation on these features from different modal data, and then transmit them into the classifier for training. This way of constructing multi-modal model seriously depends on multiple operations on features, and the process is complicated. The multi-modal features are difficult to align, and the key information of local features is easy to be ignored, so the overall effect is not good. In recent years, Transformer has made great achievements in natural language processing. More and more research attempts to apply Transformer structure to visual tasks, which also opens up a path for Transformer in the field of multi-modal tasks. Because the self-attention structure of Transformer can adapt to different types of data, the performance of various modal data in modal alignment is better, which fundamentally reduces the heterogeneity difference between modalities, and also tries to preserve the semantic integrity of the modal itself. Therefore, applying Transformer architecture to multi-modal question and answer system is an effective method to fuse multi-modal data features, and its attention mechanism can also help the model capture more important semantic features in the input information.

[0007] From the perspective of model knowledge source, intelligent question answering system can be divided into machine reading comprehension based question answering system and knowledge based question answering system. The question answering system based on machine reading comprehension utilizes unstructured knowledge source. This kind of question answering system generally answers the question according to the information given by the user, and the difficulty lies in the machine reading comprehension task. The existing method mainly trains recurrent neural network and convolutional network through supervised learning, or gives the machine the ability of understanding and answering based on large-scale pre-training language model. The knowledge based question answering system utilizes structured knowledge source, such as knowledge graph. The essence is a kind of structured knowledge. This kind of question answering system extracts important entities and relations from the information given by the user, and understands and reasons the question by using the structural relationship of the knowledge graph. The rapid development of information industry makes massive data become valuable assets that need to be mined and utilized. These fragmented massive information often contains rich correlations, and the knowledge reasoning technology can utilize explicit knowledge to mine implicit relationships and complete knowledge gaps, so it is gradually applied to the question answering system. The knowledge reasoning algorithm mainly includes association rule based algorithm, graph structure based path ranking algorithm, distributed representation learning based algorithm, neural network based algorithm and hybrid reasoning method. The association rule based algorithm mines and reasons by defining or learning the rules existing in the knowledge. The graph structure based path ranking algorithm predicts the possible relationship between entities by discovering the link path between them. The distributed representation learning based method maps the triplets to vector space by mapping function so that their new representations can meet certain relationship as much as possible. The neural network based reasoning algorithm is represented by relation graph convolution network. The current popular reasoning algorithm is hybrid reasoning method, which mainly uses the idea of reinforcement learning to regard the fact judgment as the path finding problem between the head and tail entities, and uses the strategy based reinforcement learning method to solve the sequence decision problem.

[0008] Therefore, an architecture method of a user mind perception question answering model based on knowledge reasoning technology is proposed to solve the problems of how to use multi-modal information to construct an intelligent multi-modal question answering system, how to improve the explainability of the question answering model by using knowledge reasoning technology under the paradigm of reinforcement learning, and how to consider the user feedback to realize the perception of the user's mind and provide personalized services for the user, which are problems that need to be solved by those skilled in the art. SUMMARY

[0009] Therefore, the architecture method of the user mind perception question answering model based on knowledge reasoning technology is provided, which makes the reasoning of the model closer to the user's mind, and provides better human-computer service experience for the user.

[0010] In order to achieve the above purpose, the technical scheme is adopted as follows:

[0011] A user mental perception question and answer model architecture method based on knowledge reasoning technology, comprising the following steps:

[0012] S1. Constructing an information fusion module: fusing the multi-modal information of the scene graph and the question text, generating a fusion embedding representation, and obtaining multi-modal information fusion;

[0013] S2. Constructing a strategy network based on reinforcement learning: constructing a strategy network using the reinforcement learning paradigm to model the process of finding a reasoning path in the scene graph;

[0014] S3. Constructing a user feedback module based on question similarity: determining the semantic redundancy of the question based on the cosine similarity of word vectors and converting it into a user feedback signal to return to the strategy network to guide path reasoning.

[0015] Optionally, the specific steps for constructing the information fusion module in S1 are:

[0016] S11. Scene graph encoding based on GAT: encoding the scene graph data through the GAT graph attention network to obtain node embedding representation and relationship embedding representation;

[0017] S12. Context and historical information fusion based on attention mechanism to obtain context and historical information fusion representation;

[0018] S13. Multi-modal information fusion based on Transformer decoder architecture: using the decoder architecture of Transformer to fuse the context and historical information fusion representation, scene graph node embedding representation and relationship embedding representation to obtain multi-modal fusion output.

[0019] Optionally, the specific content of the scene graph encoding based on GAT in S11 is:

[0020] Obtain the scene graph node representation;

[0021] Calculate the correlation coefficient between the node and all neighbor nodes;

[0022] Normalize the correlation coefficient to obtain the attention coefficient;

[0023] According to the attention coefficient, the information of the neighbor nodes is weighted and summed to obtain the new representation of the node.

[0024] Optionally, the specific content of the context and historical information fusion based on the attention mechanism in S12 is:

[0025] Obtain the text word vector representation using the Glove pre-training model;

[0026] Calculate the current question and context and historical information attention coefficient using the multi-head self-attention mechanism to obtain the output of the multi-head self-attention;

[0027] The first residual connection and normalization are performed on the output of the multi-head self-attention;

[0028] The second residual connection and normalization are performed after the feedforward fully connected layer, and the context history information fusion representation is obtained.

[0029] Optionally, the specific content of the multi-modal information fusion based on the Transformer decoder architecture in S13 is:

[0030] The output of the context fusion module is input into the first multi-head self-attention layer to calculate the Q, K, and V matrices, and the output of the first multi-head self-attention is obtained.

[0031] The output of the scene graph encoder and the output of the previous multi-head self-attention are input into the second multi-head self-attention layer to calculate the Q, K, and V matrices.

[0032] The output of the second multi-head self-attention is connected in residual and normalized, and then a multi-modal fusion output is obtained through a feedforward fully connected layer.

[0033] Wherein, Q is the output from the previous multi-head self-attention, K and V are the output from the scene graph encoder.

[0034] Optionally, the specific steps of constructing the policy network based on reinforcement learning in S2 are:

[0035] S21. Model the pathfinding process based on Markov decision process, and model the reasoning process of the model on the scene graph as the process of a walking agent completing multi-hop reasoning.

[0036] S22. Build a policy network based on a feedforward connection network structure, and output the multi-hop action probability distribution of the agent.

[0037] S23. Optimize the policy optimization network parameters based on the policy gradient algorithm to maximize the reward of the agent's action trajectory.

[0038] Optionally, the specific content of building a policy network based on a feedforward connection network structure in S22 to output the multi-hop action probability distribution of the agent is:

[0039] The output of the information fusion module and the embedded representation of the agent's historical decision are spliced and input into the policy network.

[0040] A two-layer feedforward network is constructed, and a nonlinear ReLU function is activated.

[0041] A probability distribution of the action space is obtained through a softmax operation.

[0042] The agent samples the action trajectory according to the probability distribution until it reaches the path endpoint of the scene graph.

[0043] Optionally, the specific steps of constructing the user feedback module based on the similarity of the question in S3 are as follows:

[0044] S31. Encode the current question and the previous question into word vector representations through the fine-tuned BERT pre-training model;

[0045] S32. Calculate the cosine similarity between the two word vector representations:

[0046]

[0047] wherein q t-1 is the word vector of the previous question, q t is the word vector of the current question, and i is the i-th dimension of the vector.

[0048] S33. Determine the semantic similarity between the previous question and the current question through the cosine similarity;

[0049] S34. Determine the user feedback on the answer to the previous question according to the relationship between the semantic similarity and the set threshold value. When the semantic similarity between the previous question and the current question is too high, it is determined that the user has asked a repeated question and is not satisfied with the answer to the previous question.

[0050] S35. Return a corresponding reward to the mental perception question and answer model according to the user feedback.

[0051] Optionally, the mental perception question and answer model is also subjected to feasibility evaluation. The specific content of the feasibility evaluation is to evaluate the performance of the mental perception question and answer model on the scene graph question and answer dataset through three kinds of link prediction indicators.

[0052] According to the technical solution described above, compared with the prior art, the application provides an architecture method of a user mental perception question and answer model based on knowledge reasoning technology, which has the following beneficial effects: 1) visual information in the form of a scene graph is introduced into the model, which can effectively fuse multi-round dialogue historical information and image scene graph information, so that the model can obtain image semantic level information, and the structured data form of the scene graph also enhances the explainability of the model reasoning; 2) the user feedback acquisition method based on the similarity of the question is proposed, which introduces the user feedback signal into the model, which can more intelligently infer the user feedback on the answer compared with the traditional question and answer model, and the feedback can guide the subsequent strategy network training of the model, so that the reasoning of the model is closer to the user's mind, and better human-computer service experience is provided for the user. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative effort based on the provided drawings also belong to the protection scope of the present application.

[0054] Figure 1 A multi-modal user mental perception question and answer model architecture schematic diagram provided by the present application;

[0055] Figure 2 A context information fusion module schematic diagram provided by the present application;

[0056] Figure 3 An information fusion module schematic diagram provided by the present application;

[0057] Figure 4 A reinforcement learning strategy network schematic diagram provided by the present application;

[0058] Figure 5 A multi-modal user mental perception question and answer model construction method flowchart provided by the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described clearly and completely below with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments only represent some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0060] A multi-modal user mental perception question and answer model architecture is shown in Figure 1 The whole mental perception question and answer model is composed of three parts, namely, an information fusion module, a reinforcement learning reasoning module based on a strategy network, and a user feedback module based on question similarity.

[0061] The information fusion module includes a scene graph encoder, a context fusion module, and a multi-modal information fusion module. The fusion of context information and multi-modal information is mainly completed based on a decoder architecture of the Transformer. The input of the module is composed of a scene graph and question and answer text, and the output of the module is the fusion representation of the two modalities.

[0062] The reinforcement learning reasoning module based on the strategy network uses the fusion representation of multi-modal information to perform strategy-based reasoning on the scene graph. The dialogue task is modeled as a path finding problem on the scene graph, and the final node of the path is used as the model reasoning answer.

[0063] The user feedback module based on question similarity judges the similarity of the context question in combination with the pre-trained language model, returns the similarity as a feedback signal to the strategy network, and adjusts the strategy to give a reasoning result that is more in line with the user's psychological expectation.

[0064] Referring to Figure 5 As shown in the figure, the application discloses a construction method of a multi-modal user mind perception question and answer model, comprising the following steps:

[0065] S1. Constructing an information fusion module: fuse the multi-modal information of the scene graph and the question text, generate a fusion embedding representation, and obtain multi-modal information fusion;

[0066] S2. Constructing a strategy network based on reinforcement learning: a strategy network is constructed using the reinforcement learning paradigm to model the process of finding a reasoning path in the scene graph;

[0067] S3. Constructing a user feedback module based on question similarity: determine the semantic repetition of the question based on the cosine similarity of the word vector, and convert it into a user feedback signal to return to the strategy network to guide path reasoning.

[0068] Further, the specific steps for constructing the information fusion module in S1 are:

[0069] S11. Scene graph encoding based on GAT: encode the scene graph data through the GAT graph attention network to obtain node embedding representation and relationship embedding representation;

[0070] S12. Context history information fusion based on attention mechanism, to obtain context history information fusion representation;

[0071] S13. Multi-modal information fusion based on the decoder architecture of the Transformer: the context history information fusion representation, the scene graph node embedding representation and the relationship embedding representation are fused using the decoder architecture of the Transformer to obtain multi-modal fusion output.

[0072] Further, the specific content of the scene graph encoding based on GAT in S11 is:

[0073] Obtain the scene graph node representation;

[0074] Calculate the correlation coefficient between the node and all neighbor nodes;

[0075] Normalize the correlation coefficient to obtain the attention coefficient;

[0076] According to the attention coefficient, the information of the neighbor nodes is weighted and summed to obtain the new representation of the node.

[0077] Further, the specific content of S12 based on the attention mechanism for context history information fusion is:

[0078] The Glove pre-training model is used to obtain text word vector representation.

[0079] The current question and context history information attention coefficients are calculated by using the multi-head self-attention mechanism, and the multi-head self-attention output is obtained.

[0080] The first residual connection and normalization are performed on the multi-head self-attention output.

[0081] After passing through the feedforward fully connected layer, the second residual connection and normalization are performed, and the context history information fusion representation is obtained.

[0082] Further, the specific content of S13 based on the Transformer decoder architecture for multi-modal information fusion is:

[0083] The output of the context fusion module is input into the first multi-head self-attention layer to calculate the Q, K, and V matrices, and the first multi-head self-attention output representation is obtained.

[0084] The output of the scene graph encoder and the previous multi-head self-attention output are input into the second multi-head self-attention layer to calculate the Q, K, and V matrices.

[0085] The second multi-head self-attention output is connected in residual and normalized, and then a multi-modal fusion output is obtained through a layer of feedforward fully connected layer.

[0086] Wherein, Q is the output from the previous multi-head self-attention, K and V are the output from the scene graph encoder.

[0087] Specifically, scene graph encoding: the scene graph data is encoded by GAT (Graph Attention Network) graph attention network to generate node embedding representation and relationship embedding representation.

[0088] Wherein, the scene graph is a directed graph, each node corresponds to an entity or an attribute of an object in a real image, and the entity itself also has a category attribute. The edges between the nodes in the scene graph describe the association relationship between the entities corresponding to the nodes, such as spatial position relationship, category attribution relationship, etc. In the scene graph, the important features of any node come from both the structural relationship with the neighbor nodes and the features or attributes of the node itself, so it is necessary to use a graph convolutional neural network to encode it. Moreover, the scene graph is a directed graph, and the relationship between entities has directionality, so it is necessary to use a graph attention network with attention mechanism to aggregate the neighbor node features of the node, so that the correlation between node features is better integrated into the new embedding representation.

[0089] Suppose there are several nodes in the scene graph, a node i has N i neighbors, and the feature of node i is represented as h i . The steps of generating scene graph node embedding representation and relationship embedding representation using a graph attention network are as follows:

[0090] Calculate the attention coefficient: first calculate the correlation coefficient between nodes, and then normalize the correlation coefficient to obtain the attention coefficient.

[0091] Calculate the correlation coefficient: for node i, calculate the correlation coefficient between its neighbor node j∈N i and itself one by one:

[0092] e ij =a([Wh i ][Wh j ])

[0093] First, use a linear mapping of W to increase the dimension of the features of nodes i and j, then concatenate the increased features to obtain higher-dimensional features, and use a single-layer feedforward neural network a(·) to map the higher-dimensional features to a real number to obtain the correlation coefficient e ij between nodes i and j.

[0094] Normalization: normalize the correlation coefficient using the softmax operation to obtain the attention coefficient α ij :

[0095]

[0096] where e ik is the correlation coefficient between node i and its kth neighbor node.

[0097] Weighted summation: weight the features according to the attention coefficients calculated above and pass them through an activation function to obtain the new features h' i of node i:

[0098]

[0099] where σ is the activation function and h j is the previous feature of node i.

[0100] Context history information fusion: generate word vector representation, use multi-head attention mechanism to calculate attention coefficient, and the architecture is shown in Figure 2 .

[0101] Generating word vector representation: first, use the Glove (Global Vector) pre-training model to initialize the text information to generate the word vector representation q tWord vector representation of historical dialogue Wherein, Glove completes word representation based on global word frequency statistics, and the pre-trained Glove model can directly convert words into word vectors. These vector representations already contain the semantic characteristics of words. By calculating the distance or cosine similarity between word vectors, the semantic similarity of words can be obtained.

[0102] Calculate attention coefficients using multi-head self-attention mechanism: send the word vector representation of the current question and historical information into N multi-head attention modules to calculate attention coefficients. Multi-head self-attention module mainly realizes through scaled dot-product attention. Take the implementation process of the first head self-attention as an example for illustration:

[0103] Calculate Query, Key, Value matrix: use linear transformation matrix Transform q t , H t into new matrix representation:

[0104]

[0105] Calculate the self-attention output of the first head:

[0106]

[0107] Wherein, Att1 is the self-attention output of the first self-attention head, Q is the input vector q t produced Query matrix, K is the input vector H t produced Key matrix, d K is the dimension of the Value matrix.

[0108] Similarly, calculate the self-attention output of other heads and concatenate them together. After a linear layer, the output of multi-head self-attention is obtained:

[0109]

[0110] Wherein, Linear(·) is a linear mapping function, Att1... Att n is the self-attention output of the first to nth self-attention head, is the concatenation operation.

[0111] First residual connection and normalization: residual connection refers to connecting the input and output of multi-head self-attention. This connection method can make the network pay attention to the difference between the current and the output. Normalization uses layer normalization method to standardize the input of each neuron in the network:

[0112] Z' t = LayerNorm(Zt +q t )

[0113] where LayerNorm(·) is a normalization operation function, Z t is the output of multi-head self-attention.

[0114] Feed-forward full connection layer: composed of two full connection layers, parameters are represented by and respectively, the first full connection layer is activated by ReLU function:

[0115]

[0116] Second residual connection and normalization:

[0117] X' t =LayerNorm(X t +Z' t )

[0118] The final context history information fusion representation is:

[0119]

[0120] After the fusion of context history information, the current question q t gets a new representation f t . Due to the use of self-attention mechanism, the new representation f t of the question has the perception of context.

[0121] Multi-modal information fusion: the decoder architecture of Transformer is used to fuse the history information perception representation f t and the scene graph node embedding representation. As shown in Figure 3 , this architecture contains two multi-head self-attention layers. Compared with the structure of the context information fusion module, the multi-modal information fusion module has an additional multi-head self-attention layer, and the subsequent operations are similar to the above. Among them, the K and V matrices of the second multi-head self-attention layer are calculated using the output of the scene graph encoder, and the Q matrix is calculated using the output of the context history information fusion module. This layer makes the information of the scene graph modality supplement the information of the text modality, and the two can be effectively fused.

[0122] First multi-head self-attention layer: the Q, K, and V matrices of the first layer are all calculated using the output f t of the context fusion module:

[0123] Q=f t W q , K=f t W k , V=f tW v

[0124] QK of each head is calculated T Then, it is multiplied by the mask matrix bit by bit to obtain the masked QK T And then the subsequent softmax calculation is performed, so the representation of the self-attention output is:

[0125]

[0126] Where, Att masked is the masked self-attention output, and Masked(.) is the masking operation.

[0127] The masked self-attention outputs of multiple heads are spliced to obtain the output of the masked multi-head self-attention layer.

[0128] The second multi-head self-attention layer: calculate the Q, K and V matrices, and the object of linear transformation here is the output e of the scene graph encoder t and the output f' of the previous multi-head self-attention layer t :

[0129]

[0130] The following steps are similar to the context history information fusion module. After the attention coefficients are calculated using Q and K, the residual connection and normalization are performed, and then the final multi-modal fusion output is obtained through a layer of feedforward fully connected layer.

[0131] Further, the specific steps of constructing the policy network based on reinforcement learning in S2 are:

[0132] S21. Model the pathfinding process based on Markov decision process, and model the reasoning process of the model on the scene graph as the process of a walking agent completing multi-hop reasoning;

[0133] S22. Build a policy network based on a feedforward connection network structure, and output the multi-hop action probability distribution of the agent;

[0134] S23. Optimize the network parameters of the policy optimization network based on the policy gradient algorithm to maximize the reward of the action trajectory of the agent.

[0135] Further, the specific content of building a policy network based on a feedforward connection network structure in S22 to output the multi-hop action probability distribution of the agent is:

[0136] The output of the information fusion module and the embedding representation of the historical decision of the agent are spliced and input to the policy network;

[0137] A two-layer feedforward network is constructed and activated by a nonlinear ReLU function;

[0138] get a probability distribution of action space by softmax operation;

[0139] The agent samples an action trajectory according to the probability distribution until reaching the path endpoint of the scene graph.

[0140] Specifically, as shown in Figure 4 , a strategy network is constructed to model the pathfinding process, and the model completes reasoning on the scene graph using the reinforcement learning paradigm. The specific steps are as follows:

[0141] Modeling the pathfinding process: the reasoning process of the model on the scene graph is modeled as the process of a walking agent completing multi-hop reasoning. The purpose of multi-hop reasoning is to find a reliable prediction path between entities and relationships. Generally, multiple reasoning is modeled as a Markov Decision Process (MDP). The Markov Decision Process is defined by a four-tuple (S, A, P, R), representing the state space, the action space, the state transition probability, and the reward function, respectively. For a scene graph Define E to represent the set of all entities, and R to represent the set of all relationships. Then the scene graph can be represented as a set of triples (s, p, o), representing subject, predicate, and object, respectively.

[0142] The core idea of the Markov Decision Process is that the next state S t+1 is only related to the current state S t and the action A t to be taken under the current state, so that the next state can be obtained by backtracking one step.

[0143] In the reinforcement learning process, the agent needs to interact with the environment. Specifically, the agent obtains the state from the environment, and using this state, the agent can select an action output. The output action process is a decision-making process. After the decision is returned to the environment, the environment will give the next state and the reward that the current decision should get. The state space, action space, and reward function are described below.

[0144] State space: at time step t, the state S = (e t , h t , q), where e t is the entity node currently reached by the agent, h t is the history decision of the agent at time step t, that is, the decision at time step t-1. When t = 1, h t is an empty set, and q represents the given question. For the agent, q is global information, but for e tThe selection of h t .

[0145] Action space: At time step t, when the agent is in state S, the next action A t to be taken by the agent t is from a set of actions A(S t ). Specifically, an action of the agent includes the next selected edge and the next selected node, so A(S t ) should be a set of all binary tuples (r, e) associated with the current node e

[0146] Reward function: According to the description of the reinforcement learning process above, each decision of the agent should be rewarded by the environment feedback, and the ultimate goal of the agent is to obtain as much reward as possible. But in the scene of our question-answering reasoning task, the agent's series of decisions will only be given positive feedback when it reaches the final correct node, and accordingly the reward function is 1 in this case, otherwise 0.

[0147] Building the policy network: The input of the policy network is spliced from two parts, the first part is the multi-modal information fusion representation M t output by the information fusion module above, and the second part is the embedding representation H t of the agent's historical decisions. The policy network is a two-layer feedforward network structure, activated by a nonlinear ReLu function between them, and finally obtains a probability distribution P(A t ) through the softmax operation. The parameters of the policy network mainly include two weight matrices W1, W2:

[0148] P(A t )=softmax(A t (W2ReLU(W1[H t ,M t ])))

[0149] where, represents all action representations at time t, and H t is the historical decision representation processed by LSTM:

[0150] H t =LSTM(a t-1 )

[0151] Probability distribution P(A t) reflects the probability of the agent selecting each action at the next time step, and as the policy network is updated, actions with high feedback values are increased in probability of selection, and conversely, actions with low feedback values are decreased in probability of selection. The agent samples from this probability distribution to obtain the next action until the end of the path is reached:

[0152]

[0153] Training the policy network: the policy gradient algorithm REINFORCE is used to train the above constructed policy network, and the training goal is to learn a policy network parameter π that maximizes the reward θ .

[0154] Define the maximization objective function:

[0155]

[0156] In actual reasoning process, it is difficult for the agent to exhaust all complete reasoning paths, so as much as possible, the action trajectory τ is sampled, and the expectation is calculated instead of the original summation operation.

[0157] Wherein, P(τ; θ) represents the product of state transition probability and action selection probability:

[0158]

[0159] Policy network parameter update rule: in order to maximize the feedback, the policy gradient is updated in the upward direction:

[0160]

[0161] Further, the specific steps of constructing the user feedback module based on problem similarity in S3 are:

[0162] S31. The current question and the last question are encoded into word vector representation by fine-tuning the BERT pre-training model;

[0163] S32. Calculate the cosine similarity between the two word vector representations:

[0164]

[0165] Wherein, q t-1 is the word vector of the last question, q t is the word vector of the current question, d is the dimension of the word vector, and i is the i-th dimension of the vector;

[0166] S33. Determine the semantic similarity between the two questions by cosine similarity;

[0167] S34. Determine the user's feedback on the answer to the previous question according to the relationship between semantic similarity and the set threshold. When the semantic similarity between the current and previous questions is too high, it is determined that the user has asked a repeated question and is not satisfied with the answer to the previous question;

[0168] S35. Return the corresponding reward to the mental perception question and answer model according to the user's feedback.

[0169] Specifically, in the real human-computer dialogue process, when the machine gives a wrong answer or an answer that does not meet the human expectation, the human reaction is usually to restate the question in a different form and input it to the machine again. Based on this common fact, the present application constructs a user feedback module based on question similarity judgment to simulate the human-computer interaction process in the real scene and realize the perception of the user's mind by the model.

[0170] Question similarity calculation: the fine-tuned BERT model is used to obtain the word vector representation of the two question texts, and the cosine similarity of the two word vectors is calculated:

[0171]

[0172] where q t-1 is the word vector of the previous question, q t is the word vector of the current question, and d is the dimension of the word vector.

[0173] Feedback function design: set the judgment threshold δ. When the similarity between the current and previous questions is higher than the threshold, it is considered that the user has asked a semantically repeated question, and the model reasoning result does not meet the user's expectation, and the feedback is 0. Otherwise, when the similarity between the current and previous questions is higher than the threshold, it is considered that the user has entered a new dialogue process, and the model reasoning result should be given a positive feedback of 1:

[0174]

[0175] cos_sim is the cosine similarity, and threshold is the set threshold.

[0176] Further, it also includes feasibility evaluation of the mental perception question and answer model. The specific content of the feasibility evaluation is to evaluate the performance of the mental perception question and answer model on the scene graph question and answer dataset through three link prediction indicators.

[0177] In a specific embodiment, the specific content of the scheme feasibility verification is:

[0178] The GQA dataset is selected to verify the above design scheme: the GQA dataset contains 11000 pictures, each picture has complete scene graph structure data, and the question and answer content is mainly designed around the object relationship and object attribute in the picture. The whole vocabulary contains 3097 words, including 1702 object classes, 310 relationship classes and 610 object attributes. There are 3-20 question and answer pairs around each picture.

[0179] Screening data: all images containing 10 question and answer pairs are screened out, and the related dialogue content is arranged in the form of multi-round dialogue. Finally, 2982 pictures are selected.

[0180] Verify the feasibility of the user feedback module based on question similarity judgment: use ChatGPT to rewrite part of the questions without changing the semantics of the sentences. In the specific model training process, when the model's reasoning result for the input question does not match the standard answer, the rewritten question of this question will be input to the model as supplementary corpus for the model to continue learning according to the corresponding feedback.

[0181] The essence of model reasoning is a triple link prediction task. For a missing entity o in a triple (s, p, o), the model needs to use all entities in the scene graph as candidates for calculation and ranking. Therefore, the link prediction indicator HITS@n is used as the evaluation indicator of the model performance, which represents how many correct triples are finally ranked within the top n. Common indicators are HITS@1, HITS@3 and HITS@10. The larger the indicator value, the better the reasoning and prediction effect of the model:

[0182]

[0183] Where, II is the conditional true value function, 1 if the condition is true, otherwise 0, |S| is the number of triples, rank i is the ranking of the correct missing entity of the ith triple in the model prediction result.

[0184] As shown in Table 1, the binary type question is tested, and the answer form of the binary type question is only two, "yes" and "no". The model performs well. The model without the context history information fusion module in the information fusion module and the user feedback module based on question similarity judgment is used as a common multi-modal question and answer model to compare with the user's mental perception question and answer model. It can be seen that the effect of the user's mental perception question and answer model is better than that of the common multi-modal question and answer model in three indicators, which shows the necessity of introducing context history information fusion and user feedback.

[0185] Table 1 Test results of binary type questions

[0186] Model HITS@1 HITS@3 HITS@10 Multimodal user mind perception question answering model 0.7782 0.9097 0.9222 Ordinary multimodal question answering model 0.7653 0.9021 0.9214

[0187] As shown in Table 2, open questions are tested, the answer form of open questions is not limited to two forms of "yes" and "no", the answer is more diversified, the inference difficulty for the model is higher than that of binary questions, and therefore the overall performance of the model is lower than the result in Table 1. However, in comparison with the ordinary multi-modal question and answer model, the effect of the user mental perception question and answer model is still better.

[0188] Table 2 Test results of open questions

[0189] Model HITS@1 HITS@3 HITS@10 Multimodal user mind perception question answering model 0.6898 0.8734 0.9018 Ordinary multimodal question answering model 0.6721 0.8684 0.8980

[0190] After testing different types of questions, the multi-modal user mental perception question and answer model can achieve better performance than the ordinary multi-modal question and answer model, verifying the feasibility of the above design scheme.

[0191] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An architecture method of a user mental perception question and answer model based on knowledge reasoning technology, characterized in that, The method comprises the following steps: S1. Constructing an information fusion module: fusing the multi-modal information of the scene graph and the question text, generating a fusion embedding representation, and obtaining multi-modal information fusion; S2. Constructing a strategy network based on reinforcement learning: constructing a strategy network using the reinforcement learning paradigm to model the process of finding a reasoning path in the scene graph; S3. Constructing a user feedback module based on question similarity: determining the semantic repetition of the question based on the cosine similarity of word vectors and converting it into a user feedback signal to return to the strategy network to guide path reasoning; The specific steps for constructing the information fusion module in S1 are as follows: S11. Scene graph encoding based on GAT: encoding the scene graph data through the GAT graph attention network to obtain node embedding representation and relationship embedding representation; S12. Contextual historical information fusion based on attention mechanism to obtain contextual historical information fusion representation; S13. Multi-modal information fusion based on Transformer decoder architecture: using the decoder architecture of Transformer to fuse the contextual historical information fusion representation, scene graph node embedding representation and relationship embedding representation to obtain multi-modal fusion output; The specific content of contextual historical information fusion based on attention mechanism in S12 is as follows: Generating textual word vector representations: using a Glove pre-trained model to initialize the textual information to generate word vector representations for the current question and word vector representations of historical dialogues ; Using multi-head self-attention mechanism to calculate attention coefficients: input the word vector representation of the current question and historical information into N multi-head attention modules to calculate attention coefficients; wherein the specific implementation process of self-attention is as follows: Compute Query, Key, Value matrices: Use linear transformation matrices , Convert , to new matrix representation: ， Calculate the self-attention output of the first head: wherein, is the self-attention output for the first self-attention head, is the input vector is the Query matrix generated, K is the input vector is the Key matrix generated, is the dimension of the Value matrix; Calculate the self-attention output of other self-attention heads and concatenate them together, and then pass through a linear layer to obtain the output of multi-head self-attention: wherein, is a linear mapping function, is a self-attention output of the 1st to nth self-attention head, is a concatenation operation; First residual connection and normalization: the output of multi-head self-attention is connected through the first residual connection and normalized; wherein the residual connection is to connect the input and output of multi-head self-attention, and the normalization adopts the layer normalization method to standardize the input of each layer of neurons in the network: wherein, is a normalization operation function, is the output of multi-head self-attention; After the feedforward fully connected layer, the second residual connection and normalization are performed to obtain the context history information fusion representation; wherein the feedforward fully connected layer is composed of two fully connected layers, and the parameters are represented by and respectively, and the first layer of the fully connected layer is activated by a ReLU function: Second residual connection and normalization: The final context history information fusion is represented as: ; By fusing the context history information, the current question a new representation with perception of context is obtained ; The specific steps for constructing the strategy network based on reinforcement learning in S2 are as follows: S21. Modeling the path finding process based on Markov decision process, modeling the reasoning process of the model on the scene graph as the process of a walking agent completing multi-hop reasoning; S22. Building a strategy network based on a feedforward connection network structure, outputting the multi-hop action probability distribution of the agent; S23. Optimizing network parameters based on the policy gradient algorithm to maximize the reward of the agent's action trajectory.

2. The method of claim 1, wherein the method is characterized by: The specific content of scene graph encoding based on GAT in S11 is as follows: Obtaining the node representation of the scene graph: Suppose there are several nodes in the scene graph, and a certain node... have There are n neighboring nodes, and the feature of node i is represented as: ; Correlation coefficient between the computing node and all the neighbor nodes: for the node , compute the correlation coefficient between each of its neighbor nodes and itself: First use one Linear mappings to nodes The features are augmented, and then the augmented features are concatenated to obtain higher-dimensional features using a single-layer feedforward neural network. Mapping this higher-dimensional feature to a real number yields the node. Correlation coefficient between ; The correlation coefficient is normalized by using a softmax operation to obtain an attention coefficient : wherein, is the correlation coefficient for node i and its kth neighbor node; The features are weighted and summed according to the attention coefficients and passed through an activation function to get the node new features : where, is an activation function, is a feature before node i.

3. The method of claim 1, wherein the method further comprises: determining a user's mental state based on the user's input; and providing a response to the user's input based on the user's mental state. The specific content of multi-modal information fusion based on Transformer decoder architecture in S13 is as follows: The output of the contextual fusion module Compute Q, K, V matrices input to the first multi-head self-attention layer: ， ，V= The calculation of each head is After that, it is multiplied by the masking matrix bit by bit to obtain the masked The representation of the self-attention output is: wherein, is a masked self-attention output, is a masking operation; Concatenate the masked self-attention outputs of multiple heads to obtain the output of the masked multi-head self-attention layer; The output of the scene graph encoder is encoded and the output of the previous multi-headed self-attention Input to the second multi-headed self-attention layer to compute Q, K, V matrices: ， ，V= Residual connection and normalization are performed on the output of the second multi-head self-attention, and then a multi-modal fusion output is obtained through a layer of feedforward fully connected layer; Wherein, Q is the output from the previous multi-head self-attention, K and V are the outputs from the scene graph encoder.

4. The method of claim 1, wherein the method further comprises: In S21, the Markov decision process is defined by a four-tuple , representing the state space, the action space, the state transition probabilities, and the reward function, respectively. For a scene graph , define E represents the set of all entities, R represents the set of all relationships, then the scene graph can be represented as a set of triples , respectively representing subject, predicate, and object. State space: At time step t, the state ,in It is the entity node that the agent has currently reached. It refers to the agent's historical decisions at time step t, that is, the decisions at time step t-1. When t=1, Let q be an empty set, representing the current given problem; for the agent, q is global information, but for the specific problem... The choice depends on of; Action space: the next action to be taken by the agent when in state S at time step t from a set of actions ; in particular, an action of the agent comprises a next selected edge and a next selected node, so that is the set of all pairs (r, e) associated with the current node , ; Reward function: the agent's series of decisions will be given positive feedback only if it reaches the final correct node, and the reward function is 1 in this case, otherwise 0.

5. The method of claim 1, wherein the method further comprises: In S22, the input of the policy network is represented by the multi-modal information fusion output by the information fusion module and the embedded representation of the historical decisions of the agent are spliced together; The policy network is a two-layer feedforward network structure, which is activated by a nonlinear ReLU function between the layers, and finally obtains a probability distribution of the action space through a softmax operation ; The parameters of the policy network mainly include two weight matrices , : where, denotes all actions at time t, is the history of decisions processed by LSTM: Probability distribution The probability distribution reflects the probability of the agent selecting each action at the next time step. As the policy network is updated, actions with high feedback values are increased in the probability of being selected, and conversely, actions with low feedback values are decreased in the probability of being selected. The agent samples according to this probability distribution to obtain the next action until it reaches the path endpoint of the scene graph: 。 6. The method of claim 1, wherein the method further comprises: In S23, the policy optimization network parameters are optimized based on the policy gradient algorithm, specifically: The policy network is trained by using a strategy gradient algorithm REINFORCE, and a training target is to learn a policy network parameter capable of maximizing a reward ; Define the maximization objective function: on the motion trajectory sampling is performed and expectation is taken instead of the original summation operation; wherein, denotes the product of the state transition probabilities and the action selection probabilities: In order to maximize the feedback, update the policy gradient in the upward direction: 。 7. The method of claim 1, wherein the method further comprises: determining a user's mental state based on the user's input; and providing a response to the user's input based on the user's mental state. The specific steps of constructing the user feedback module based on the similarity of the problem in S3 are: S31. Encode the current question and the previous question into word vector representation through the fine-tuned BERT pre-training model; S32. Calculate the cosine similarity between the two word vector representations: wherein, is the word vector of the previous question, is the word vector of the current question, i is the i-th dimension of the vector, and d is the dimension of the word vector; S33. Determine the semantic similarity between the two questions through the cosine similarity; S34. Set a judgment threshold When the similarity of the current and previous questions is higher than the threshold, it is considered that the user raises a semantically repeated question, the model inference result does not meet the user's expectation, and the feedback given is 0. Otherwise, when the similarity of the current and previous questions is lower than the threshold, it is considered that the user enters a new dialogue process, and the model inference result should be given positive feedback 1. Where cos_sim is the cosine similarity, and threshold is the set threshold value; S35. Return the corresponding reward to the mental perception question and answer model according to the user feedback.

8. The architecture method of a user mental perception question and answer model based on knowledge reasoning technology according to claim 1, characterized in that, Further comprising a feasibility evaluation of the mental perception question and answer model, and the specific content of the feasibility evaluation is to evaluate the performance of the mental perception question and answer model on the scene graph question and answer dataset through three link prediction indicators.

Images