A decision optimization method fusing reinforced multi-modal learning and knowledge graph

Abstract

The application discloses a decision optimization method fusing reinforced multi-modal learning and a knowledge graph, and relates to the technical field of artificial intelligence and knowledge graphs. The decision optimization method fusing reinforced multi-modal learning and the knowledge graph realizes dynamic fusion of multi-modal features through a dynamic weight adjustment and semantic alignment constraint mode, improves semantic precision and interpretability, effectively solves the semantic deviation problem caused by traditional static feature fusion, and can perform interactive graph structure display by recording an intelligent reasoning path selected by a layered reinforcement learning agent, has the advantages of transparency, and further improves the interpretability of the path in cooperation with a multi-target reward function, intelligently updates the knowledge graph in cooperation with comprehensive confidence, realizes intelligent growth of the knowledge graph, generates a fine-grained and interpretable report through an adversarial training mechanism and counterfactual reasoning, further reduces the false positive rate of output results, and improves decision transparency.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and knowledge graph technology, specifically to a decision optimization method that integrates reinforcement multimodal learning and knowledge graphs. Background Technology

[0002] Knowledge graphs, also known as scientific knowledge graphs, are a concept in the field of library and information science. They are used to draw, analyze, and display the interrelationships between disciplines or academic research subjects. They are visualization tools that reveal the development process and structural relationships of scientific knowledge. In most cases, knowledge graphs are visualized using a graph structure, with nodes representing authors, academic institutions, scientific documents, or keywords, and lines representing the relationships between nodes.

[0003] Existing knowledge graphs do indeed suffer from poor interpretability in applications, especially when using complex models such as deep neural networks. The reasoning process is often treated as a black box, lacking transparency. Furthermore, multimodal features are often fused using static feature fusion, which can easily lead to semantic bias and lack of traceability. For example, the TransAE model uses weighted averaging or concatenation to fuse multimodal data. In addition, knowledge graphs are usually supplemented manually, which is labor-intensive and lacks intelligent growth potential.

[0004] In view of this, a decision optimization method that integrates reinforcement multimodal learning and knowledge graph is proposed. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a decision optimization method that integrates enhanced multimodal learning and knowledge graphs. This method solves the problems of poor interpretability, static feature fusion leading to semantic bias and lack of traceability, and lack of intelligent growth in existing knowledge graphs.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a decision optimization method integrating reinforcement multimodal learning and knowledge graphs, specifically comprising the following steps:

[0007] S1. Construct a multimodal embedding space containing text, images, and time-series data, and dynamically align the feature vectors of each modality through a meta-learning framework;

[0008] S2. Construct a hierarchical reinforcement learning agent whose state space includes the topological structure of entity relationships in the knowledge graph and the joint encoding vector of multimodal features;

[0009] S3. Define a dual action space: the main action space is used to control the selection of reasoning paths for knowledge graph relationships, and the auxiliary action space is used to adjust the weights of multimodal data fusion.

[0010] S4. Construct a multi-objective reward function to drive a hierarchical reinforcement learning agent to generate intelligent reasoning paths;

[0011] S5. Construct an adversarial training mechanism to verify the rationality of the intelligent reasoning path, generate a difference explanation report through counterfactual reasoning, and perform incremental knowledge graph updates.

[0012] The present invention is further configured such that: in step S1, a gated attention mechanism is used to construct a cross-modal feature interaction channel and calculate the semantic correlation degree of text-image features. The calculation formula for the gated attention mechanism is as follows:

[0013]

[0014] In the formula, Attention weights for the text-image modality. ,when At that time, the cross-modal attention mechanism is activated. For the sigmoid function, This is a trainable weight matrix for text, used for linear transformation of text features. For text feature vectors, This is a trainable weight matrix for image features, used for linear transformations. For image feature vectors, These are the bias term parameters.

[0015] The present invention is further configured such that: the auxiliary action space in S3 is used to adjust the multimodal data fusion weights in the following ways:

[0016]

[0017] In the formula, For the first The fusion weights of each modality It is an exponential function used for Softmax normalization. For the first An importance scoring function for each modal feature. Let this be the current state vector. For the total number of modes, For modal traversal index, , For the first An importance scoring function for each modality.

[0018] The present invention is further configured such that the calculation formula for the multi-objective reward function in S4 is:

[0019]

[0020] In the formula, For the total reward score, Scoring is applied to the path complexity penalty term. Rate the reward items based on user feedback. Scoring based on semantic consistency metrics;

[0021] The formula for calculating the path complexity penalty is:

[0022]

[0023] In the formula, The number of path nodes. This is the penalty weighting coefficient;

[0024] The formula for calculating user feedback rewards is:

[0025]

[0026] In the formula, For human scoring, , Provide weighting coefficients for user feedback;

[0027] The formula for calculating semantic consistency is:

[0028]

[0029] In the formula, Scoring for intelligent reasoning paths, This represents the semantic weight coefficient.

[0030] The present invention is further configured such that the adversarial training mechanism in S5 includes:

[0031] Construct a discriminator network to distinguish between intelligent inference paths and expert-annotated paths, and define an adversarial loss function:

[0032]

[0033] In the formula, To counteract the training loss value, For the discriminator network, For parameters, Mark the path for experts. This is the intelligent reasoning path.

[0034] The present invention is further configured such that: the method of generating the difference explanation report through counterfactual reasoning in S5 includes:

[0035] The contribution of each triple to the prediction is calculated using the SHAP value.

[0036] Remove the triplet with the highest contribution and observe the change in the predicted probability.

[0037] If the predicted probability change exceeds the threshold, the triple is marked as a decisive reasoning basis and a natural language explanation is generated.

[0038] The present invention is further configured such that the incremental knowledge graph update method in S5 includes:

[0039] When a new entity is detected in unstructured data, the semantic similarity between the new entity and existing entities is calculated, the three candidate related entities with the highest similarity are selected, the potential relationship is predicted using a graph convolutional network, multiple candidate insertion paths are generated based on a reinforcement learning agent, the path with the highest comprehensive confidence is selected, and knowledge graph expansion is performed when its comprehensive confidence exceeds a preset reference threshold.

[0040] The present invention is further configured such that the comprehensive confidence level is calculated as follows:

[0041]

[0042] In the formula, the graph structure score is the reasonableness score of predicting potential relationships through graph convolutional networks.

[0043] This invention provides a decision optimization method that integrates reinforcement multimodal learning and knowledge graphs. It has the following beneficial effects:

[0044] (1) This invention achieves dynamic fusion of multimodal features through dynamic weight adjustment and semantic alignment constraints, improves semantic accuracy and interpretability, effectively solves the semantic bias problem caused by traditional static feature fusion, and can interactively display the graph structure by recording the intelligent reasoning path selected by the hierarchical reinforcement learning agent. It has the advantage of transparency, and further improves the interpretability of the path by combining with the multi-objective reward function. It also achieves intelligent growth of the knowledge graph by combining with the comprehensive confidence.

[0045] (2) This invention generates fine-grained interpretable reports by combining adversarial training mechanism with counterfactual reasoning, thereby further reducing the false alarm rate of the output results and improving decision transparency. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0047] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0048] Please see Figure 1 The present invention provides the following technical solution: a decision optimization method integrating reinforcement multimodal learning and knowledge graph, specifically including the following steps:

[0049] S1. Construct a multimodal embedding space containing text, images, and time-series data. Dynamically align the feature vectors of each modality using a meta-learning framework. The processing methods include:

[0050] S11, Data Preprocessing

[0051] Text Modality: The semantic vectors of entity description text are extracted using the BERT model. Global semantics are captured through the hidden state marked by the last layer [CLS], generating a 768-dimensional sentence vector that preserves semantic information.

[0052] Image Modal: Visual features are extracted using the CLIP model, and a 512-dimensional image feature vector is output to capture visual details;

[0053] Time series data: The data is encoded into time series features using bidirectional LSTM, and a sliding window mechanism is used to extract time dependency patterns. The window step size is set to 10 seconds, and 128-dimensional time series features are output.

[0054] S12, Meta-learning framework dynamic alignment

[0055] Independent autoencoders are trained for text modality, image modality, and time-series data, respectively, and a reconstruction loss is pre-trained.

[0056] The initial parameters are learned through the MAML algorithm, enabling rapid adaptation to new tasks after a small number of gradient updates;

[0057] During inference, the meta-learner dynamically adjusts the encoder parameters based on the current input, enabling the feature vectors of each modality to be semantically aligned in the angle space through cosine similarity, thus ensuring cross-modal semantic consistency.

[0058] To further explain, S12 employs a gated attention mechanism for semantic alignment, constructs a cross-modal feature interaction channel, and calculates the semantic correlation between text and image features. The calculation formula for the gated attention mechanism is as follows:

[0059]

[0060] In the formula, Attention weights for the text-image modality. ,when At that time, the cross-modal attention mechanism is activated. For the sigmoid function, This is a trainable weight matrix for text, used for linear transformation of text features. For text feature vectors, This is a trainable weight matrix for image features, used for linear transformations. For image feature vectors, These are the bias term parameters.

[0061] S2. Construct a hierarchical reinforcement learning agent whose state space includes the topological structure of entity relationships in the knowledge graph and the joint encoding vector of multimodal features.

[0062] The methods for constructing the state space include:

[0063] The entity relations are encoded into an adjacency matrix. The vectors corresponding to the dynamically aligned text modality, image modality and time series data in S12 are concatenated and jointly represented.

[0064] To further explain, the hierarchical reinforcement learning agent consists of an upper-layer agent and a lower-layer agent. The upper-layer agent is used to receive the task objective and output a sequence of subtasks, while the lower-layer agent is used to execute specific actions for each subtask.

[0065] S3. To separate path selection from modality fusion weight adjustment and improve the flexibility and interpretability of decision-making, a dual action space is defined:

[0066] A. The active action space is used to control the selection of reasoning paths for knowledge graph relationships, including:

[0067] Action set: Contains all possible relationships in the knowledge graph;

[0068] Policy network: Using a DQN network, the input is a state vector, and the output is the probability of relation selection;

[0069] B. The auxiliary action space is used to adjust the weights for multimodal data fusion, including:

[0070] Action set: Defines the modal fusion weight vector;

[0071] Policy Network: Employs a Softmax output layer and encourages exploration of diversity through entropy regularization.

[0072] To further explain, the methods for adjusting the multimodal data fusion weights in the auxiliary action space include:

[0073]

[0074] In the formula, For the first The fusion weights of each modality It is an exponential function used for Softmax normalization. For the first An importance scoring function for each modal feature. Let this be the current state vector. For the total number of modes, For modal traversal index, , For the first An importance scoring function for each modality.

[0075] S4. Construct a multi-objective reward function to drive the hierarchical reinforcement learning agent to generate intelligent inference paths. The calculation formula for the multi-objective reward function is as follows:

[0076]

[0077] In the formula, For the total reward score, Scoring is applied to the path complexity penalty term. Rate the reward items based on user feedback. Scoring based on semantic consistency metrics;

[0078] The formula for calculating the path complexity penalty is:

[0079]

[0080] In the formula, The number of path nodes. The penalty weighting coefficient is preferably 0.3;

[0081] The formula for calculating user feedback rewards is:

[0082]

[0083] In the formula, For human scoring, , The preferred weighting factor for user feedback is 0.5.

[0084] The formula for calculating semantic consistency is:

[0085]

[0086] In the formula, Scoring for intelligent reasoning paths, This is the semantic weight coefficient, preferably 0.2.

[0087] S5. Construct an adversarial training mechanism to verify the rationality of the intelligent inference path. The adversarial training mechanism includes:

[0088] Construct a discriminator network to distinguish between intelligent inference paths and expert-annotated paths, and define an adversarial loss function:

[0089]

[0090] In the formula, To counteract the training loss value, For the discriminator network, For parameters, Mark the path for experts. For intelligent reasoning paths;

[0091] The method of generating discrepancy explanation reports through counterfactual reasoning includes:

[0092] The contribution of each triple to the prediction is calculated using the SHAP value.

[0093] Remove the triplet with the highest contribution and observe the change in the predicted probability.

[0094] If the predicted probability change exceeds the threshold, preferably 0.4, the triple is marked as a decisive reasoning basis and a natural language explanation is generated.

[0095] Incremental knowledge graph updates can be performed in the following ways:

[0096] When a new entity is detected in unstructured data, the semantic similarity between the new entity and existing entities is calculated. The three candidate related entities with the highest similarity are selected, and a graph convolutional network is used to predict potential relationships. Multiple candidate insertion paths are generated based on a reinforcement learning agent. The path with the highest overall confidence is selected. When its overall confidence exceeds a preset reference threshold, knowledge graph expansion is performed. The preferred reference threshold is 0.8. The overall confidence is calculated as follows:

[0097]

[0098] In the formula, the graph structure score is the reasonableness score of predicting potential relationships through graph convolutional networks.

[0099] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A decision optimization method integrating reinforcement multimodal learning and knowledge graphs, characterized in that, Specifically, the following steps are included: S1. Construct a multimodal embedding space containing text, images, and time-series data, and dynamically align the feature vectors of each modality through a meta-learning framework; S2. Construct a hierarchical reinforcement learning agent whose state space includes the topological structure of knowledge graph entity relationships and the joint encoding vector of multimodal features; S3. Define a dual action space: the main action space is used to control the selection of knowledge graph relationship reasoning paths, and the auxiliary action space is used to adjust the weights of multimodal data fusion. S4. Construct a multi-objective reward function to drive a hierarchical reinforcement learning agent to generate intelligent reasoning paths; S5. Construct an adversarial training mechanism to verify the rationality of the intelligent reasoning path, generate a difference explanation report through counterfactual reasoning, and perform incremental knowledge graph updates; in S1, a gated attention mechanism is used to construct a cross-modal feature interaction channel, calculate the semantic correlation degree of text-image features, and the calculation formula of the gated attention mechanism is: In the formula, Attention weights for the text-image modality. ,when At that time, the cross-modal attention mechanism is activated. For the sigmoid function, This is a trainable weight matrix for text, used for linear transformation of text features. For text feature vectors, This is a trainable weight matrix for image features, used for linear transformations. For image feature vectors, The bias term parameter; the auxiliary action space in S3 is used to adjust the multimodal data fusion weights in the following ways: In the formula, For the first The fusion weights of each modality It is an exponential function used for Softmax normalization. For the first An importance scoring function for each modal feature. Let this be the current state vector. For the total number of modes, For modal traversal index, , For the first The importance scoring function for each modality; the calculation formula for the multi-objective reward function in S4 is: In the formula, For the total reward score, Scoring is applied to the path complexity penalty term. Rate the reward items based on user feedback. The semantic consistency metric is scored; the formula for calculating the path complexity penalty is: In the formula, The number of path nodes. The penalty weighting coefficient is used; the formula for calculating user feedback reward items is: In the formula, For human scoring, , The user feedback weight coefficient is used; the formula for calculating the semantic consistency metric is: In the formula, Scoring for intelligent reasoning paths, These are semantic weight coefficients; the adversarial training mechanism in S5 includes: constructing a discriminator network to distinguish between intelligent reasoning paths and expert-annotated paths, and defining an adversarial loss function: In the formula, To counteract the training loss value, For the discriminator network, For parameters, Mark the path for experts. This is the intelligent reasoning path.

2. The decision optimization method integrating reinforcement multimodal learning and knowledge graph as described in claim 1, characterized in that, The method of generating a difference explanation report through counterfactual reasoning in S5 includes: The contribution of each triple to the prediction is calculated using the SHAP value; the triple with the highest contribution is deleted, and the change in the prediction probability is observed; if the change in the prediction probability exceeds a threshold, the triple is marked as a decisive reasoning basis, and a natural language explanation is generated.

3. The decision optimization method integrating reinforcement multimodal learning and knowledge graph as described in claim 1, characterized in that, The incremental knowledge graph update method in S5 includes: when a new entity is detected in unstructured data, calculating the semantic similarity between the new entity and the existing entity, filtering the three candidate related entities with the highest similarity, using a graph convolutional network to predict potential relationships, generating multiple candidate insertion paths based on a reinforcement learning agent, selecting the path with the highest comprehensive confidence, and expanding the knowledge graph when its comprehensive confidence exceeds a preset reference threshold.

4. The decision optimization method integrating reinforcement multimodal learning and knowledge graph as described in claim 3, characterized in that, The overall confidence level is calculated as follows: In the formula, the graph structure score is the reasonableness score of predicting potential relationships through graph convolutional networks.

Images