Method and System for Constructing a Data Knowledge Graph Driven by a Complex Ecological Intelligence Brain

Abstract

The present invention provides a method and system for constructing a data knowledge graph driven by a complex ecological intelligent brain, which relates to the field of complex ecological technologies. It includes collecting scenario data through a deep feature perception network to generate an initial graph structure; calculating entity similarity based on graph feature fingerprints, screening relevant entity pairs and performing multi-hop path reasoning to generate an entity association confidence index; generating multi-modal knowledge representation vectors based on multi-modal feature extraction and cross-modal attention mechanisms; identifying influencing features, calculating propagation probabilities in combination with a problem evolution graph and causal reasoning, and comparing and analyzing with graph feature fingerprints to generate an optimization target, using an Adam optimizer to generate an error correction strategy, and dynamically adjusting the reasoning depth through a double DQN network to form an optimization closed loop and construct a data knowledge graph. The present invention can effectively improve the accuracy and integrity of the knowledge graph, enhance its reasoning ability, and better adapt to knowledge discovery and application in complex ecological scenarios.

Description

Technical Field

[0001] The present invention relates to complex ecological technologies, and particularly to a method and system for constructing a data knowledge graph driven by a complex ecological intelligent brain. Background Art

[0002] As a structured knowledge representation form, knowledge graphs play an increasingly important role in various fields of artificial intelligence, such as natural language processing, recommendation systems, and knowledge reasoning. Traditional methods for constructing knowledge graphs usually rely on manual construction or rule-based automated methods, which suffer from problems such as low efficiency, poor scalability, and difficulty in adapting to complex and changing real-world data. In recent years, with the rise of deep learning and graph neural networks, researchers have begun to explore using these technologies to automatically construct and reason about knowledge graphs.

[0003] Existing methods usually have difficulty effectively processing multimodal data. Data in the real world often contains information in multiple modalities such as text, images, and audio, while most existing methods only focus on data in a single modality and are difficult to fully utilize the rich semantic information in multimodal data.

[0004] Existing methods still have limitations in knowledge reasoning. Knowledge reasoning is one of the important functions of knowledge graphs, but the reasoning ability of existing methods is often limited by shallow reasoning paths and it is difficult to perform complex multi-hop reasoning.

[0005] Existing methods lack effective optimization mechanisms. The construction of knowledge graphs is an iterative optimization process that requires continuous correction and improvement of the graph structure and knowledge, while existing methods often lack effective optimization mechanisms and it is difficult to ensure the quality and accuracy of the graphs. Summary of the Invention

[0006] Embodiments of the present invention provide a method and system for constructing a data knowledge graph driven by a complex ecological intelligent brain, which can solve the problems in the prior art.

[0007] In the first aspect of the embodiments of the present invention,

[0008] A method for constructing a data knowledge graph driven by a complex ecological intelligent brain is provided, including:

[0009] Collecting scenario data through a deep feature perception network to obtain an original multi-dimensional sequence including business processes, processing rules, and asset associations, semantically parsing the original multi-dimensional sequence to generate an initial knowledge vector, constructing an initial graph structure including entity nodes and relationship edges based on the initial knowledge vector, and encoding the initial graph structure using a graph neural network to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery;

[0010] Calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with a correlation higher than a preset similarity threshold from the similarity matrix. Perform multi-hop path reasoning on the screened entity pairs to generate an inference path set. Input the inference path set into a recurrent neural network to generate an entity association confidence index. Based on the entity association confidence index, perform multi-modal feature extraction on the data in the initial graph structure and the inference path set respectively. The multi-modal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multi-modal features to generate a unified multi-modal knowledge representation vector;

[0011] Input the multi-modal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multi-modal knowledge representation vector. The multi-modal knowledge representation vector includes an entity feature vector, a relationship feature vector, and a subgraph feature vector. Based on the multi-modal knowledge representation vector, identify influencing features through the SHAP method. Input the influencing features into a problem evolution graph to calculate the propagation probability through causal reasoning. Compare and analyze the propagation probability with the graph feature fingerprint to generate an optimization target. Extract gradient information in the feature space according to the optimization target and input it into an Adam optimizer to generate an error correction strategy. Generate adversarial samples based on the error correction strategy for verification, and dynamically adjust the inference depth through a double DQN network according to the verification results. Finally, compare and verify the optimization effect by comparing the optimized graph features with the graph feature fingerprint to form a complete optimization closed-loop.

[0012] Calculating the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint and using the feature set to screen entity pairs with a correlation higher than a preset similarity threshold from the similarity matrix includes:

[0013] The graph feature fingerprint is characterized as a high-dimensional feature vector, which contains graph structure features, semantic distribution features, and topological attribute features. The knowledge feature contains the attribute feature matrix of entity nodes;

[0014] Use a linear transformation unit to perform feature transformation on the attribute feature matrix to generate a query matrix, a key matrix, and a value matrix. The query matrix is obtained by multiplying the query weight matrix by the attribute feature matrix. The key matrix is obtained by multiplying the key weight matrix by the attribute feature matrix. The value matrix is obtained by multiplying the value weight matrix by the attribute feature matrix;

[0015] Divide the query matrix, the key matrix, and the value matrix into multiple attention heads. Each attention head independently calculates attention scores. For each attention head, obtain the attention weights by multiplying the query matrix by the transpose of the key matrix and performing dimensional normalization. Multiply the attention weights by the value matrix to obtain the head output, and concatenate the multiple head outputs to form the multi-head attention output;

[0016] Based on the multi-head attention output, calculate the similarity relationship between entity pairs through cosine similarity to generate a similarity matrix. Each element of the similarity matrix represents the similarity score of the corresponding entity pair;

[0017] Set a dynamic similarity threshold according to the distribution characteristics in the graph feature fingerprint, and apply the dynamic similarity threshold to the similarity matrix for two-way verification and screening. The two-way verification and screening require that both the forward similarity and the reverse similarity of the entity pair satisfy the dynamic similarity threshold;

[0018] Combine the semantic constraint conditions in the knowledge features to filter the entity pairs that pass the two-way verification and generate the final relevant entity pairs.

[0019] Based on the entity association confidence index, extract multi-modal features from the data in the initial graph structure and the inference path set through the BERT-Large model, the Vision Transformer model, and the Wav2Vec model. The multi-modal features include text features, image features, and audio features, and use the cross-modal attention mechanism to align and fuse the multi-modal features to generate a unified multi-modal knowledge representation vector, including:

[0020] Use the BERT-Large pre-trained model to encode the input text data, and extract the context semantic features of the text data through a multi-layer Transformer structure. The context semantic features are represented as a text feature matrix, and generate a text pre-trained feature vector based on the text feature matrix;

[0021] Use the Vision Transformer model to divide the input image data into a sequence of image patches with a size of 16 by 16 pixels. The sequence of image patches is input into the self-attention layer after position encoding to extract the global visual features of the sequence of image patches, and jointly encode the global visual features with the text pre-trained feature vector to obtain a text-image joint feature vector;

[0022] Construct an attention guidance matrix based on the combined text and image feature vector. The attention guidance matrix is used to process the input audio data. Input the audio data into the Wav2Vec model, extract audio features through a multi-layer convolutional network and a Transformer structure, align the audio features with the combined text and image feature vector, and generate multi-modal alignment features;

[0023] Calculate the inter-modal similarity matrix for the multi-modal alignment features, construct a cross-modal attention network based on the similarity matrix, and the cross-modal attention network outputs an attention weight distribution;

[0024] Input the attention weight distribution into a linear projection layer. The linear projection layer maps the features of different modalities to the unified semantic space to generate a unified multi-modal knowledge representation vector.

[0025] Identify influential features based on the multi-modal knowledge representation vector through the SHAP method. Input the influential features into the problem evolution graph and calculate the propagation probability through causal reasoning. Compare and analyze the propagation probability with the graph feature fingerprint to generate an optimization objective, including:

[0026] Input the multi-modal knowledge representation vector into an improved SHAP model. The improved SHAP model calculates the synergy effect between features based on a feature combination evaluation mechanism to obtain feature importance scores; screen the multi-modal knowledge representation vector according to the feature importance scores, and select features with importance scores exceeding a preset feature threshold as influential features;

[0027] Construct a problem evolution graph with the influential features as nodes. The node attributes of the problem evolution graph include feature values and importance scores, and construct directed edges based on the temporal relationship between features; input the problem evolution graph into a causal reasoning network. The causal reasoning network calculates the conditional probability between nodes through variational inference and uses Monte Carlo sampling to obtain the probability distribution of the propagation path;

[0028] Compare the probability distribution of the propagation path with the pre-stored graph feature fingerprint, and obtain the state difference distribution by calculating the KL divergence; construct an optimization objective function based on the state difference distribution. The optimization objective function includes a distribution difference term and an integrity constraint term, and obtain the optimization objective of the knowledge graph through weighted summation.

[0029] Input the multi-modal knowledge representation vector into an improved SHAP model. The improved SHAP model calculates the synergy effect between features based on a feature combination evaluation mechanism to obtain feature importance scores; screen the multi-modal knowledge representation vector according to the feature importance scores, and select features with importance scores exceeding a preset feature threshold as influential features, including:

[0030] Input the multi-modal knowledge representation vector into an improved SHAP model. The improved SHAP model uses a sliding window to generate a feature combination pool, and calculates feature combination weights based on the mutual information, correlation, and temporal correlation of the features in the feature combination pool. The improved SHAP model uses a non-linear mapping layer to map the feature combination weights, and calculates the synergy effect between features based on a feature combination evaluation mechanism.

[0031] Construct a feature contribution matrix based on the synergy effect. The matrix elements of the feature contribution matrix represent the contribution degree of features to the sample. Use a temporal decay factor to perform temporal weighting on the feature contribution matrix to obtain a temporal contribution degree, and calculate the initial SHAP value based on the temporal contribution degree. Calculate a modality adaptive weight according to the variance of different modality features, and use the modality adaptive weight to perform weighted fusion on the initial SHAP value to obtain a feature importance score.

[0032] Filter the multi-modal knowledge representation vector according to the feature importance score. Calculate a preset feature threshold based on the mean and standard deviation of the feature importance score, and dynamically adjust the preset feature threshold in combination with the change rate of the feature importance score. Construct a feature dependency graph to analyze the correlation relationship between features, and select features with importance scores exceeding the preset feature threshold as influencing features based on the feature group effect and stability constraint. The influencing features are used to guide subsequent feature optimization.

[0033] Extract gradient information in the feature space according to the optimization target and input it into an Adam optimizer to generate an error correction strategy. Generate adversarial samples based on the error correction strategy for verification, dynamically adjust the inference depth according to the verification result through a double DQN network, and finally compare the optimized graph features with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed-loop, including:

[0034] Extract gradient information in the feature space according to the optimization target, calculate local gradients through a multi-scale window, and use a gradient clipping mechanism and exponential moving average to obtain a smoothed gradient. Input the smoothed gradient into an Adam optimizer. The Adam optimizer adaptively adjusts the learning rate based on the gradient norm and generates an error correction strategy through second-order momentum correction.

[0035] Generate adversarial samples based on the error correction strategy. The adversarial samples ensure effectiveness through perturbation boundary constraints and prevent sample degradation through diversity constraints. Input the adversarial samples into a multi-layer verification network for quality assessment to obtain a verification result.

[0036] Input the verification result into the double DQN network. The double DQN network constructs a state space based on the current inference depth, the verification result, and resource occupancy, constructs an action space based on the depth adjustment step size and skip connection parameters, and dynamically adjusts the inference depth through the state space and the action space to obtain an adjusted inference depth;

[0037] Optimize the knowledge graph according to the adjusted inference depth to obtain optimized graph features; compare the optimized graph features with the graph feature fingerprint, calculate the KL divergence of the feature distribution and the topological structure similarity as evaluation metrics; feedback the evaluation metrics to the Adam optimizer for optimizing parameter updates to form a complete optimization loop.

[0038] Optimize the knowledge graph according to the adjusted inference depth to obtain optimized graph features; compare the optimized graph features with the graph feature fingerprint, calculate the KL divergence of the feature distribution and the topological structure similarity as evaluation metrics; feedback the evaluation metrics to the Adam optimizer for optimizing parameter updates. The complete optimization loop includes:

[0039] Construct a knowledge graph optimization model according to the adjusted inference depth. The knowledge graph optimization model calculates the node attention weight and the edge relationship attention weight through a hierarchical attention mechanism, and performs feature recombination on the knowledge graph based on the node attention weight and the edge relationship attention weight to obtain optimized graph features;

[0040] Compare the optimized graph features with a pre-constructed graph feature fingerprint, calculate the KL divergence of the feature distribution using a kernel function estimator, and the kernel function estimator introduces an adaptive smoothing factor to prevent zero-probability problems; construct a multi-scale structure descriptor to characterize the graph structure features, and calculate the topological structure similarity based on the multi-scale structure descriptor; perform weighted fusion on the KL divergence and the topological structure similarity to obtain an evaluation metric;

[0041] Feedback the evaluation metric to the Adam optimizer. The Adam optimizer updates the parameters using an adaptive learning rate, and reduces the parameter update fluctuation through gradient accumulation; construct a convergence criterion based on the change value of the evaluation metric and the parameter change value, and judge whether the optimization converges according to the convergence criterion; if not converged, feedback the updated parameters to the knowledge graph optimization model for the next round of optimization until the convergence completes the optimization loop.

[0042] In the second aspect of the embodiments of the present invention,

[0043] Provide a data knowledge graph construction system driven by a complex ecological intelligent brain, including:

[0044] The first unit is used to collect scenario data through a deep feature perception network, obtain an original multi-dimensional sequence containing business processes, processing rules, and asset associations, perform semantic parsing on the original multi-dimensional sequence to generate an initial knowledge vector, construct an initial graph structure containing entity nodes and relationship edges based on the initial knowledge vector, and encode the initial graph structure using a graph neural network to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery;

[0045] The second unit is used to calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to filter entity pairs with a correlation higher than a preset similarity threshold from the similarity matrix. Perform multi-hop path reasoning on the filtered entity pairs to generate an inference path set, input the inference path set into a recurrent neural network to generate an entity association confidence index, and perform multi-modal feature extraction on the data in the initial graph structure and the inference path set based on the entity association confidence index. The multi-modal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multi-modal features to generate a unified multi-modal knowledge representation vector;

[0046] The third unit is used to input the multi-modal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multi-modal knowledge representation vector. The multi-modal knowledge representation vector includes an entity feature vector, a relationship feature vector, and a sub-graph feature vector. Identify influencing features based on the multi-modal knowledge representation vector through the SHAP method, input the influencing features into a problem evolution graph to calculate the propagation probability through causal reasoning, compare and analyze the propagation probability with the graph feature fingerprint to generate an optimization target, extract gradient information in the feature space according to the optimization target and input it into an Adam optimizer to generate an error correction strategy, generate adversarial samples for verification based on the error correction strategy, dynamically adjust the inference depth according to the verification result through a double DQN network, and finally compare and verify the optimization effect by comparing the optimized graph features with the graph feature fingerprint to form a complete optimization closed-loop.

[0047] In the third aspect of the embodiments of the present invention,

[0048] Provide an electronic device, including:

[0049] A processor;

[0050] A memory for storing instructions executable by the processor;

[0051] Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0052] In the fourth aspect of the embodiments of the present invention,

[0053] Provided is a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the foregoing method is implemented.

[0054] The beneficial effects of this application are as follows:

[0055] 1. Improve the efficiency and accuracy of knowledge graph construction: Through the deep feature perception network for data collection and semantic parsing, combined with graph neural network encoding, the initial graph structure can be efficiently constructed and key features can be extracted. Using multi-hop path reasoning and recurrent neural networks, entity associations can be more accurately identified, and through multi-modal feature extraction and cross-modal attention mechanisms, a more comprehensive knowledge representation can be constructed.

[0056] 2. Enhance knowledge discovery and reasoning capabilities: Based on the graph feature fingerprint for similarity calculation and entity screening, combined with multi-hop path reasoning, potential entity associations and knowledge can be discovered. Using the SHAP method to identify influencing features and calculating the propagation probability through causal reasoning, the causal relationship between knowledge can be better understood and more accurate reasoning can be performed.

[0057] 3. Achieve the continuous optimization and adaptation of the knowledge graph: By comparing and analyzing the propagation probability and the graph feature fingerprint, an optimization target is generated, and the Adam optimizer is used to generate an error correction strategy. Through adversarial sample verification and the double DQN network to dynamically adjust the reasoning depth, a complete optimization closed-loop is formed, enabling the knowledge graph to be continuously optimized and adapted, and improving its robustness and generalization ability. Description of the Drawings

[0058] Figure 1 It is a schematic flowchart of the method for constructing a data knowledge graph driven by a complex ecological intelligent brain according to an embodiment of the present invention;

[0059] Figure 2 It is a schematic structural diagram of a system for constructing a data knowledge graph driven by a complex ecological intelligent brain according to an embodiment of the present invention. Detailed Embodiments

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0061] The technical solution of the present invention will be described in detail below with specific embodiments. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0062] Figure 1 It is a schematic flowchart of a method for constructing a data knowledge graph driven by a complex ecological intelligent brain according to an embodiment of the present invention. As Figure 1 shown, the method includes:

[0063] S11. Collect scene data through a deep feature perception network to obtain an original multi-dimensional sequence containing business processes, processing rules, and asset associations. Parse the semantics of the original multi-dimensional sequence to generate an initial knowledge vector. Based on the initial knowledge vector, construct an initial graph structure containing entity nodes and relationship edges. Use a graph neural network to encode the initial graph structure to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery;

[0064] S12. Calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with a correlation higher than a preset similarity threshold from the similarity matrix. Perform multi-hop path reasoning on the screened entity pairs to generate an inference path set. Input the inference path set into a recurrent neural network to generate an entity association confidence index. Based on the entity association confidence index, perform multi-modal feature extraction on the data in the initial graph structure and the inference path set respectively. The multi-modal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multi-modal features to generate a unified multi-modal knowledge representation vector;

[0065] S13. Input the multi-modal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multi-modal knowledge representation vector. The multi-modal knowledge representation vector includes entity feature vectors, relationship feature vectors, and sub-graph feature vectors. Identify influencing features based on the multi-modal knowledge representation vector through the SHAP method. Input the influencing features into a problem evolution graph to calculate the propagation probability through causal reasoning. Compare the propagation probability with the graph feature fingerprint to generate an optimization target. Extract gradient information in the feature space according to the optimization target and input it into an Adam optimizer to generate an error correction strategy. Generate adversarial samples based on the error correction strategy for verification. Dynamically adjust the inference depth according to the verification result through a double DQN network. Finally, compare the optimized graph features with the graph feature fingerprint to verify the optimization effect and form a complete optimization closed-loop.

[0066] In an alternative embodiment, calculating a similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and screening entity pairs with a correlation higher than a preset similarity threshold from the similarity matrix includes:

[0067] The graph feature fingerprint is characterized as a high-dimensional feature vector, the high-dimensional feature vector includes graph structure features, semantic distribution features, and topological attribute features, and the knowledge feature includes an attribute feature matrix of entity nodes;

[0068] Using a linear transformation unit to perform feature transformation on the attribute feature matrix to generate a query matrix, a key matrix, and a value matrix. The query matrix is obtained by multiplying a query weight matrix by the attribute feature matrix, the key matrix is obtained by multiplying a key weight matrix by the attribute feature matrix, and the value matrix is obtained by multiplying a value weight matrix by the attribute feature matrix;

[0069] Dividing the query matrix, the key matrix, and the value matrix into multiple attention heads, each attention head independently calculates an attention score. For each attention head, the attention weight is obtained by multiplying the query matrix by the transpose of the key matrix and performing dimensional normalization, and the attention weight is multiplied by the value matrix to obtain a head output. The outputs of multiple heads are concatenated to form a multi-head attention output;

[0070] Based on the multi-head attention output, calculating the similarity relationship between entity pairs through cosine similarity to generate a similarity matrix, and each element of the similarity matrix represents the similarity score of the corresponding entity pair;

[0071] Setting a dynamic similarity threshold according to the distribution feature in the graph feature fingerprint, and applying the dynamic similarity threshold to the similarity matrix for two-way verification screening. The two-way verification screening requires that both the forward similarity and the reverse similarity of the entity pair satisfy the dynamic similarity threshold;

[0072] Combining the semantic constraint conditions in the knowledge feature to filter the entity pairs passing the two-way verification to generate the final relevant entity pairs.

[0073] A method for entity correlation analysis based on graph feature fingerprint, which is used to calculate the similarity between entities in the initial graph structure and screen entity pairs with a correlation higher than a preset threshold.

[0074] First, construct a graph feature fingerprint. The graph feature fingerprint is characterized as a high-dimensional feature vector, including graph structure features, semantic distribution features, and topological attribute features. For example, graph structure features can be the degree, centrality, etc. of entity nodes; semantic distribution features can be the probability distribution of entity nodes under different topics; topological attribute features can be information such as the position of entity nodes in the graph and neighbor nodes. At the same time, extract knowledge features, including the attribute feature matrix of entity nodes. For example, the attributes of a movie entity can include director, actors, genre, release time, etc.

[0075] Next, use a linear transformation unit to perform feature transformation on the attribute feature matrix to generate a query matrix, a key matrix, and a value matrix. Specifically, multiply the attribute feature matrix by three different weight matrices respectively to obtain the query matrix, the key matrix, and the value matrix. For example, assume the query weight matrix is [[0.1, 0.2, 0.3, 0.4], [0.2, 0.3, 0.4,0.1]], and multiply it with the first row [Zhang San, Li Si, Plot, 2022] of the attribute feature matrix to obtain the first row of the query matrix.

[0076] Then, divide the query matrix, the key matrix, and the value matrix into multiple attention heads. Each attention head independently calculates attention scores. For each attention head, obtain the attention weights by multiplying the query matrix with the transpose of the key matrix and performing dimensional normalization. Multiply the attention weights with the value matrix to obtain the head output. Concatenate the outputs of multiple heads to form a multi-head attention output. For example, assume the query matrix, the key matrix, and the value matrix are all divided into two attention heads. Multiply the attention weights calculated by the first attention head with the value matrix to obtain the first head output. Multiply the attention weights calculated by the second attention head with the value matrix to obtain the second head output. Concatenate the two head outputs to form a multi-head attention output.

[0077] Based on the multi-head attention output, calculate the similarity relationship between entity pairs through cosine similarity to generate a similarity matrix. Each element of the similarity matrix represents the similarity score of the corresponding entity pair. For example, if the multi-head attention outputs of movie A and movie B are [0.1, 0.2, 0.3] and [0.2, 0.3, 0.4] respectively, then the cosine similarity between them is the dot product of these two vectors divided by the product of their norms.

[0078] Set a dynamic similarity threshold according to the distribution characteristics in the graph feature fingerprint. Apply the dynamic similarity threshold to the similarity matrix for two-way verification and screening. The two-way verification and screening require that both the forward similarity and the reverse similarity of the entity pair satisfy the dynamic similarity threshold. For example, assume that the similarity between movie A and movie B is 0.8, the similarity between movie B and movie A is 0.9, and the dynamic similarity threshold is 0.7. Then movie A and movie B satisfy the two-way verification.

[0079] Filter the entity pairs that pass the two-way verification by combining the semantic constraint conditions in the knowledge features to generate the final relevant entity pairs. For example, assume that the semantic constraint condition is that the movie types must be the same. Then movie A and movie C are relevant because they are both of the drama type. Although movie A and movie B pass the two-way verification, they are finally filtered out because their types are different.

[0080] The solution of this application can:

[0081] Improve the accuracy of entity relevance analysis: By constructing a graph feature fingerprint by combining graph structure features, semantic distribution features, and topological attribute features, the relevance between entities can be more comprehensively characterized, thereby improving the accuracy of entity relevance analysis. Enhance the robustness of entity relevance analysis: The dynamic similarity threshold can be adaptively adjusted according to the distribution characteristics of different entities, thereby enhancing the robustness of entity relevance analysis and avoiding the limitations brought by a fixed threshold. Improve the efficiency of entity relevance analysis: The multi-head attention mechanism can calculate the similarity between entities in parallel, thereby improving the efficiency of entity relevance analysis, especially being more prominent in large-scale graph data.

[0082] In an optional implementation manner, based on the entity association confidence index, extract multi-modal features from the data in the initial graph structure and the inference path set through the BERT-Large model, the Vision Transformer model, and the Wav2Vec model. The multi-modal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multi-modal features to generate a unified multi-modal knowledge representation vector, including:

[0083] Use the BERT-Large pre-trained model to encode the input text data, extract the context semantic features of the text data through a multi-layer Transformer structure. The context semantic features are represented as a text feature matrix, and generate a text pre-trained feature vector based on the text feature matrix;

[0084] The Vision Transformer model is used to divide the input image data into a sequence of image patches of size sixteen by sixteen pixels. After position encoding, the sequence of image patches is input into the self-attention layer to extract the global visual features of the sequence of image patches. The global visual features are jointly encoded with the text pre-training feature vector to obtain a text-image joint feature vector;

[0085] An attention guidance matrix is constructed based on the text-image joint feature vector. The attention guidance matrix is used to process the input audio data. The audio data is input into the Wav2Vec model, and audio features are extracted through a multi-layer convolutional network and a Transformer structure. The audio features are aligned with the text-image joint feature vector to generate multi-modal aligned features;

[0086] The inter-modal similarity matrix is calculated for the multi-modal aligned features, and a cross-modal attention network is constructed based on the similarity matrix. The cross-modal attention network outputs an attention weight distribution;

[0087] The attention weight distribution is input into a linear projection layer, and the linear projection layer maps the different modal features to the unified semantic space to generate a unified multi-modal knowledge representation vector.

[0088] A knowledge graph reasoning method based on multi-modal knowledge representation aims to improve the accuracy and efficiency of knowledge graph reasoning. The core of this method is to use the BERT-Large model, the Vision Transformer model, and the Wav2Vec model to extract text, image, and audio features respectively, and fuse these multi-modal features through a cross-modal attention mechanism to generate a unified multi-modal knowledge representation vector for subsequent knowledge reasoning.

[0089] First, the entities and relationships in the knowledge graph are initialized to construct an initial graph structure. For example, if the graph contains entities "cat" and "animal", and the relationship "belongs to", an initial triple (cat, belongs to, animal) can be constructed. At the same time, according to the requirements of the reasoning task, a set of reasoning paths is generated. For example, if we want to reason about "what is the sound of a cat", the reasoning path "cat -> belongs to -> animal -> makes sound -> meows" can be generated. The data in these initial graph structures and sets of reasoning paths is associated with a confidence metric, which represents the reliability of the data in the reasoning process. For example, according to the existing knowledge base, the confidence of "a cat belongs to an animal" may be high, while the confidence of "a cat makes a sound" may be relatively low.

[0090] Next, perform multi-modal feature extraction on the data in the initial graph structure and the set of inference paths. Taking the entity "cat" as an example, extract its text description "A small mammal that is usually kept as a pet by humans", and use the BERT-Large model to encode the text to obtain a text feature vector. At the same time, extract the image of the "cat", such as a picture of a cat, and use the Vision Transformer model to extract the image features. In addition, extract the audio of the "cat", such as an audio of a cat's meow, and use the Wav2Vec model to extract the audio features.

[0091] Then, perform multi-modal feature fusion. Jointly encode the text feature vector generated by the BERT-Large model and the image features extracted by the Vision Transformer model to obtain a text-image joint feature vector. Input the audio data into the Wav2Vec model to extract the audio features, and align the features with the text-image joint feature vector to generate multi-modal aligned features. Calculate the inter-modal similarity matrix of the multi-modal aligned features, and construct a cross-modal attention network based on the similarity matrix to obtain the attention weight distribution. Finally, input the attention weight distribution into the linear projection layer to map the features of different modalities to a unified semantic space and generate a unified multi-modal knowledge representation vector. For example, the final multi-modal knowledge representation vector of the "cat" integrates text, image, and audio information and can represent the concept of "cat" more comprehensively.

[0092] Finally, perform knowledge inference based on the generated unified multi-modal knowledge representation vector. For example, to infer "What is the meow of a cat?", the multi-modal knowledge representation vector of the "cat" can be compared and calculated with the multi-modal knowledge representation vectors of other related entities (such as "animal", "sound"), and finally it can be inferred that the meow of the "cat" is "meow".

[0093] The solution of this application can:

[0094] Enhancing the comprehensiveness of knowledge representation: By integrating multi-modal information such as text, images, and audio, knowledge can be represented more comprehensively, avoiding biases caused by single-modal information, thereby improving the accuracy and integrity of knowledge representation. For example, fusing a picture of a cat, a textual description, and its meowing sound can express the concept of "cat" more completely than using only a textual description. Enhancing the accuracy of knowledge reasoning: Multi-modal knowledge representation vectors contain richer semantic information, which can better capture the associations between entities and relationships, thus improving the accuracy of knowledge reasoning. For example, when reasoning about "what is the meowing sound of a cat", a multi-modal representation vector that incorporates the meowing audio of the cat can infer the answer more accurately. Expanding the application scope of the knowledge graph: Multi-modal knowledge representation can better handle multimedia data, thereby expanding the applications of the knowledge graph in fields such as image recognition and speech recognition. For example, this method can be used to construct a multi-modal knowledge graph that includes images, sounds, and text for applications such as intelligent question answering and information retrieval.

[0095] In an alternative embodiment, based on the multi-modal knowledge representation vector, the influencing features are identified by the SHAP method, the influencing features are input into the problem evolution graph, and the propagation probability is calculated through causal reasoning. The comparison and analysis of the propagation probability with the graph feature fingerprint generate an optimization objective, including:

[0096] Input the multi-modal knowledge representation vector into an improved SHAP model. The improved SHAP model calculates the synergy effect between features based on a feature combination evaluation mechanism to obtain feature importance scores; screen the multi-modal knowledge representation vector according to the feature importance scores, and select the features with importance scores exceeding a preset feature threshold as influencing features;

[0097] Construct a problem evolution graph with the influencing features as nodes. The node attributes of the problem evolution graph include feature values and importance scores, and a directed edge is constructed based on the temporal relationship between features; input the problem evolution graph into a causal reasoning network. The causal reasoning network calculates the conditional probability between nodes through variational inference and uses Monte Carlo sampling to obtain the probability distribution of the propagation path;

[0098] Compare the probability distribution of the propagation path with the pre-stored graph feature fingerprint, and obtain the state difference distribution by calculating the KL divergence; construct an optimization objective function based on the state difference distribution. The optimization objective function includes a distribution difference term and an integrity constraint term, and the optimization objective of the knowledge graph is obtained through weighted summation.

[0099] A multi-modal knowledge graph optimization method aims to improve the integrity and accuracy of the knowledge graph. The core idea of this method is to analyze the propagation probability of influencing features in the problem evolution graph and compare it with the existing graph feature fingerprint to identify the parts that need to be optimized.

[0100] First, perform feature selection on the multi-modal knowledge representation vector. Input the multi-modal knowledge representation vector into the improved SHAP model. This improved SHAP model is not the traditional SHAP, but an evaluation mechanism that considers feature combinations and can capture the synergistic effects between features, thereby more accurately evaluating the importance of each feature. For example, in an image, there are features of "cat" and "sofa", and the combination of these two features may imply a higher-level semantic information such as "the cat is on the sofa", and the improved SHAP model can capture this synergistic effect. Through this model, obtain the importance score of each feature. Set a preset feature threshold, such as 0.05, and filter out the features whose importance scores exceed this threshold as influencing features. Suppose the multi-modal vector contains four features: "cat", "sofa", "table", and "dog", and their importance scores are 0.1, 0.08, 0.02, and 0.01 respectively, then "cat" and "sofa" will be selected as influencing features.

[0101] Next, construct a problem evolution graph. Use the influencing features selected in the previous step as the nodes of the graph. Each node contains two attributes: feature value and importance score. The nodes are connected by directed edges, and the direction of the edges represents the temporal relationship between features. For example, the "cat" node points to the "sofa" node, indicating that the action of the "cat" precedes the state of "the cat is on the sofa". Suppose the importance score of "cat" is 0.1 and the importance score of "sofa" is 0.08, then the edge from the "cat" node to the "sofa" node can be represented as ("cat", 0.1) → ("sofa", 0.08).

[0102] Then, perform causal reasoning and propagation probability calculation. Input the constructed problem evolution graph into the causal reasoning network. This network uses variational inference method to calculate the conditional probability between nodes and uses Monte Carlo sampling method to simulate the propagation path of features in the graph to obtain the probability distribution of the propagation path. For example, simulate 1000 samplings, and the results of 800 samplings show that the "cat" will jump to the "sofa", then the propagation probability from "cat" to "sofa" is 0.8.

[0103] Compare and analyze the calculated probability distribution of the propagation path with the pre-stored graph feature fingerprint. The graph feature fingerprint is a pre-stored probability distribution of feature propagation representing common patterns in the knowledge graph. By calculating the KL divergence between the propagation probability distribution and the graph feature fingerprint, obtain the state difference distribution. The KL divergence is used to measure the degree of difference between two probability distributions.

[0104] Finally, an optimization objective for generating the knowledge graph is defined. An optimization objective function is constructed based on the state difference distribution. This function consists of two terms: the distribution difference term and the integrity constraint term. The distribution difference term is used to measure the gap between the current knowledge graph and the ideal state, and the integrity constraint term is used to ensure the integrity and consistency of the knowledge graph. By performing a weighted sum of these two terms, the final optimization objective is obtained.

[0105] The solution of this application can:

[0106] Improve the accuracy of the knowledge graph: By identifying and correcting the feature propagation paths that do not conform to the preset patterns, the error information in the knowledge graph can be effectively corrected, thereby improving its accuracy. Enhance the integrity of the knowledge graph: This method can identify the missing information in the knowledge graph and supplement it, thus enhancing the coverage and information volume of the knowledge graph. Improve the interpretability of the knowledge graph: By analyzing the causal relationships and propagation paths between features, the internal logic and evolution rules of the knowledge graph can be better understood, thereby improving its interpretability.

[0107] In an alternative embodiment, the multi-modal knowledge representation vector is input into an improved SHAP model. The improved SHAP model calculates the synergy between features based on a feature combination evaluation mechanism to obtain feature importance scores; the multi-modal knowledge representation vector is screened according to the feature importance scores, and the features with importance scores exceeding a preset feature threshold are selected as influencing features, including:

[0108] The multi-modal knowledge representation vector is input into an improved SHAP model. The improved SHAP model uses a sliding window to generate a feature combination pool, and calculates feature combination weights based on the mutual information, correlation, and temporal correlation of the features in the feature combination pool; the improved SHAP model uses a non-linear mapping layer to map the feature combination weights and calculates the synergy between features based on a feature combination evaluation mechanism;

[0109] A feature contribution matrix is constructed based on the synergy. The matrix elements of the feature contribution matrix represent the contribution degree of features to the samples; a temporal decay factor is used to perform temporal weighting on the feature contribution matrix to obtain a temporal contribution degree, and an initial SHAP value is calculated based on the temporal contribution degree; a modal adaptive weight is calculated according to the variances of different modal features, and the initial SHAP value is weighted and fused using the modal adaptive weight to obtain a feature importance score;

[0110] Filter the multi-modal knowledge representation vectors according to the feature importance scores, calculate a preset feature threshold based on the mean and standard deviation of the feature importance scores, and dynamically adjust the preset feature threshold in combination with the change rate of the feature importance scores; construct a feature dependency graph to analyze the correlation relationships between features, and select the features with importance scores exceeding the preset feature threshold as influencing features based on the feature group effect and stability constraints, where the influencing features are used to guide subsequent feature optimization.

[0111] First, preprocess the multi-modal knowledge representation vectors. Convert data of different modalities, such as images, texts, and audios, into a unified vector representation form. For example, image data can extract features through a convolutional neural network, text data can be converted into vectors through a word embedding model, and audio data can be converted into vectors through spectrogram feature extraction. These vectors are concatenated together to form the multi-modal knowledge representation vectors. Suppose a multi-modal sample contains three modalities: image, text, and audio, and the vector dimensions of each modality are 128, 64, and 32 respectively, then the dimension of the final multi-modal knowledge representation vector is 224.

[0112] Next, input the preprocessed multi-modal knowledge representation vectors into the improved SHAP model. This model first uses a sliding window mechanism to generate a feature combination pool. For example, suppose the dimension of the multi-modal knowledge representation vector is 224 and the sliding window size is set to 3, then 222 feature combinations will be generated, and each combination contains 3 consecutive features. Then, calculate the feature combination weights based on the mutual information, correlation, and temporal correlation of the features in the feature combination pool. For example, the higher the frequency of occurrence of two features in time, the higher their temporal correlation and the higher the corresponding feature combination weight. Suppose the mutual information of a certain feature combination is 0.8, the correlation is 0.9, and the temporal correlation is 0.7, then the weight of this feature combination can be calculated according to a preset weight calculation method, such as weighted average. Suppose the weights of mutual information, correlation, and temporal correlation are 0.3, 0.4, and 0.3 respectively, then the weight of this feature combination is 0.3 * 0.8 + 0.4 * 0.9 + 0.3 * 0.7 = 0.78. The improved SHAP model uses a non-linear mapping layer to map the feature combination weights, such as using the Sigmoid function to map the weights to between 0 and 1. Then, calculate the synergy effect between features based on the feature combination evaluation mechanism, that is, the impact of the joint action of multiple features on the result. For example, if the combination weight of image features and text features is high, it indicates that there is a strong synergy effect between these two features.

[0113] After that, a feature contribution matrix is constructed based on the synergy effect. Each element of this matrix represents the contribution degree of a feature to a sample. For example, if a certain image feature has a great influence on the classification result of a sample, the corresponding contribution value of this feature is relatively high. A time series decay factor is used to perform time series weighting on the feature contribution matrix to obtain the time series contribution. For example, for time series data, the more recent features have a greater impact on the current result, so their corresponding time series decay factors are larger. Suppose the contribution degree of a certain feature at time t is 0.8 and the time series decay factor is 0.9, then the time series contribution degree of this feature at time t - 1 is 0.8 * 0.9 = 0.72. The initial SHAP values are calculated based on the time series contribution. The modality adaptive weights are calculated according to the variances of different modality features. For example, if the variance of image features is relatively large, it indicates that the image features have a higher discrimination degree for samples, and their corresponding modality adaptive weights are also higher. The initial SHAP values are weighted and fused using the modality adaptive weights to obtain the feature importance scores.

[0114] Finally, the multi-modal knowledge representation vectors are screened according to the feature importance scores. The preset feature threshold is calculated based on the mean and standard deviation of the feature importance scores. For example, the mean plus the standard deviation is used as the threshold. The preset feature threshold is dynamically adjusted in combination with the change rate of the feature importance scores. For example, if the change rate of the feature importance scores is relatively large, it indicates that the importance of the features is constantly changing, and the threshold needs to be dynamically adjusted. A feature dependency graph is constructed to analyze the association relationships between features. For example, if two features often appear simultaneously, there is a strong association relationship between them. Based on the feature group effect and stability constraint, the features with importance scores exceeding the preset feature threshold are selected as the influencing features. For example, if a certain feature group has a great influence on the classification result of a sample, the features in this group are more likely to be selected. Suppose the preset feature threshold is 0.8 and the importance score of a certain feature is 0.9, then this feature will be selected as an influencing feature. These influencing features are used to guide subsequent feature optimization.

[0115] The solution of this application can:

[0116] Improve the accuracy of feature screening: By considering the synergy effect, time series information, and modality differences between features, the features that have important impacts on the results can be more accurately identified, avoiding missing key information, thereby improving the accuracy of feature screening. Enhance the interpretability of the model: By constructing a feature dependency graph, the association relationships between features can be analyzed, and the impact mechanism of features on the results can be better understood, thereby enhancing the interpretability of the model. Optimize the model performance: By screening out important influencing features, redundant information can be reduced, the computational complexity can be lowered, and the efficiency and performance of the model can be improved.

[0117] In an alternative embodiment, gradient information is extracted in the feature space according to the optimization objective and input into the Adam optimizer to generate an error correction strategy. An adversarial sample is generated based on the error correction strategy for verification. The inference depth is dynamically adjusted by the double DQN network according to the verification result. Finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization loop, including:

[0118] Gradient information is extracted in the feature space according to the optimization objective. The local gradient is calculated through a multi-scale window, and a smoothed gradient is obtained by using a gradient clipping mechanism and exponential moving average. The smoothed gradient is input into the Adam optimizer, which adaptively adjusts the learning rate based on the gradient norm and generates an error correction strategy through second-order momentum correction;

[0119] An adversarial sample is generated based on the error correction strategy. The adversarial sample ensures effectiveness through perturbation boundary constraints and prevents sample degradation through diversity constraints. The adversarial sample is input into a multi-layer verification network for quality assessment to obtain a verification result;

[0120] The verification result is input into the double DQN network. The double DQN network constructs a state space based on the current inference depth, the verification result, and resource occupancy, constructs an action space based on the depth adjustment step size and skip connection parameters, and dynamically adjusts the inference depth through the state space and the action space to obtain an adjusted inference depth;

[0121] The knowledge graph is optimized according to the adjusted inference depth to obtain optimized graph features. The optimized graph features are compared with the graph feature fingerprint, and the KL divergence of the feature distribution and the topological structure similarity are calculated as evaluation metrics. The evaluation metrics are fed back to the Adam optimizer for optimizing parameter updates, forming a complete optimization loop.

[0122] The goal is to optimize the knowledge graph features to make them closer to the preset graph feature fingerprint. This method adopts an adversarial training and depth dynamic adjustment strategy and continuously optimizes through a closed-loop feedback mechanism.

[0123] Feature Gradient Extraction and Smoothing. First, according to the optimization objective, such as improving the performance of the knowledge graph in downstream tasks, gradient information is extracted in the feature space. To obtain more comprehensive gradient information, multi-scale windows are used to calculate local gradients. Specifically, windows of different sizes are slid on the feature map, the gradients within each window are calculated separately, and then these gradients are aggregated. To avoid gradient explosion or vanishing, a gradient clipping mechanism is adopted to limit the gradient magnitude within a certain range. Finally, to reduce gradient fluctuations, an exponential moving average method is used to smooth the gradients, obtaining smoothed gradients. For example, three window sizes of 3x3, 5x5, and 7x7 are set, the local gradients are calculated separately and then weighted averaged with weights of 0.2, 0.3, and 0.5 respectively. Then the gradient magnitude is limited between [-1, 1]. Finally, an exponential moving average with a decay rate of 0.9 is adopted to obtain smoothed gradients.

[0124] Error Correction Strategy Generation. The smoothed gradients are input into the Adam optimizer. The Adam optimizer adaptively adjusts the learning rate according to the gradient norm and corrects the gradient direction using the second-order momentum to generate an error correction strategy. For example, the initial learning rate is set to 0.001, the decay rate of the first-order momentum is set to 0.9, and the decay rate of the second-order momentum is set to 0.999.

[0125] Adversarial Sample Generation and Verification. Adversarial samples are generated based on the error correction strategy. The generation process of adversarial samples needs to satisfy perturbation boundary constraints and diversity constraints. The perturbation boundary constraints limit the difference between the adversarial samples and the original samples to ensure the effectiveness of the adversarial samples. The diversity constraints encourage the generation of different adversarial samples to prevent sample degradation. For example, the perturbation boundary constraint is set to 0.1, and the diversity constraint is achieved by adding random noise in the gradient direction. The generated adversarial samples are input into a multi-layer verification network for quality assessment to obtain verification results, such as the change in loss value caused by the adversarial samples.

[0126] Inference Depth Dynamic Adjustment. The verification results are input into a double DQN network. The state space of the double DQN network consists of the current inference depth, verification results, and resource occupancy. The action space consists of the depth adjustment step size and skip connection parameters. The double DQN network selects the best action according to the current state to dynamically adjust the inference depth. For example, the state space includes the current inference depth (e.g., 5), verification results (e.g., the loss value increases by 0.05), and resource occupancy (e.g., memory usage). The action space includes the depth adjustment step size (e.g., increase by 1 layer or decrease by 1 layer) and skip connection parameters (e.g., whether to enable skip connections).

[0127] Optimization and verification of graph features. Optimize the knowledge graph according to the adjusted inference depth to obtain the optimized graph features. Compare the optimized graph features with the preset graph feature fingerprints, and calculate the KL divergence of the feature distribution and the topological structure similarity as evaluation metrics. For example, the KL divergence between the optimized graph features and the fingerprint features is calculated to be 0.02, and the topological structure similarity is 0.95.

[0128] Closed-loop feedback and optimization. Feed the evaluation metrics back to the Adam optimizer to adjust the parameters of the optimizer, such as the learning rate and momentum, to form a complete optimization closed-loop. For example, if the KL divergence is large, reduce the learning rate; if the topological structure similarity is low, adjust the gradient calculation method.

[0129] The solution of this application can:

[0130] Improve the quality of the knowledge graph: Through adversarial training and dynamic depth adjustment, the features of the knowledge graph are made closer to the preset fingerprints, thereby improving the quality of the knowledge graph and its performance in downstream tasks. Achieve adaptive resource allocation: The double DQN network dynamically adjusts the inference depth according to the verification results and resource occupancy, achieving adaptive resource allocation and avoiding resource waste. Construct a complete optimization closed-loop: By feeding the evaluation metrics back to the optimizer, a complete optimization closed-loop is formed, realizing continuous optimization and continuously improving the quality of the knowledge graph.

[0131] In an optional implementation manner, optimize the knowledge graph according to the adjusted inference depth to obtain the optimized graph features; compare the optimized graph features with the graph feature fingerprints, and calculate the KL divergence of the feature distribution and the topological structure similarity as evaluation metrics; feed the evaluation metrics back to the Adam optimizer for optimizing parameter updates, and the formation of a complete optimization closed-loop includes:

[0132] Construct a knowledge graph optimization model according to the adjusted inference depth. The knowledge graph optimization model calculates the node attention weight and the edge relationship attention weight through a hierarchical attention mechanism, and performs feature recombination on the knowledge graph based on the node attention weight and the edge relationship attention weight to obtain the optimized graph features;

[0133] Compare the optimized graph features with the pre-constructed graph feature fingerprints, calculate the KL divergence of the feature distribution using a kernel function estimator, and the kernel function estimator introduces an adaptive smoothing factor to prevent zero-probability problems; construct a multi-scale structure descriptor to characterize the graph structure features, and calculate the topological structure similarity based on the multi-scale structure descriptor; perform weighted fusion on the KL divergence and the topological structure similarity to obtain the evaluation metrics;

[0134] Feed the evaluation metrics back to the Adam optimizer. The Adam optimizer updates the parameters using an adaptive learning rate and reduces the parameter update fluctuations through gradient accumulation. Construct a convergence criterion based on the change values of the evaluation metrics and the parameter change values, and determine whether the optimization converges according to the convergence criterion. If it does not converge, feed the updated parameters back to the knowledge graph optimization model for the next round of optimization until the convergence completes the optimization loop.

[0135] First, in the preparation stage, it is necessary to construct a knowledge graph and its corresponding graph feature fingerprint. The graph feature fingerprint can be understood as a description of the ideal graph features. It can be predefined, extracted from high-quality knowledge graphs, or constructed by experts based on domain knowledge. For example, if the goal is to construct a knowledge graph about movies, the graph feature fingerprint can include information such as movie genres, actors, directors, release years, etc. and the relationships between them. At the same time, it is necessary to set an initial inference depth, for example, set it to 3, indicating considering the relationships within three hops between nodes in the graph.

[0136] Next, enter the iterative optimization stage. According to the current inference depth, construct a knowledge graph optimization model. This model uses a hierarchical attention mechanism. Specifically, the model first calculates the attention weights of each node itself. For example, according to indicators such as the degree and centrality of the node, different nodes are given different importance. Then, the model calculates the attention weights of the edge relationships between nodes. For example, according to indicators such as the type and strength of the edge, different edges are given different importance. Based on the calculated attention weights of nodes and edges, the features of the knowledge graph are reorganized to obtain optimized graph features. For example, if there is a "co-director" relationship between movie A and movie B and the attention weight of this relationship is high, then in the optimized graph features, the feature representations of movie A and movie B will be more similar.

[0137] Then, compare the optimized graph features with the pre-constructed graph feature fingerprint. To evaluate the difference in feature distributions, use a kernel density estimator to calculate the KL divergence of the feature distributions. The kernel density estimator introduces an adaptive smoothing factor to prevent the zero-probability problem, that is, to avoid calculation errors caused by some feature values not appearing in the samples. For example, if the fingerprint contains a rare movie genre and there is no movie of this genre in the current graph, the adaptive smoothing factor can prevent the KL divergence calculation from resulting in infinity. At the same time, to evaluate the similarity of the graph topological structures, construct multi-scale structure descriptors to characterize the graph structure features, such as node degree distribution, clustering coefficient, etc. Based on these descriptors, calculate the topological structure similarity between the optimized graph and the fingerprint graph. Finally, weight and fuse the KL divergence and the topological structure similarity to obtain a comprehensive evaluation metric. The weights can be adjusted according to actual needs.

[0138] Feed the evaluation metric back to the Adam optimizer. The Adam optimizer updates the model parameters using an adaptive learning rate and reduces the fluctuations in parameter updates through gradient accumulation to make the optimization process more stable. Construct a convergence criterion based on the change value of the evaluation metric and the change value of the parameters. For example, a threshold can be set, and when both the change value of the evaluation metric and the change value of the parameters are less than this threshold, it is considered that the optimization process converges. If it does not converge, feedback the updated parameters to the knowledge graph optimization model, adjust the inference depth, for example, increase the inference depth to 4, and repeat the above steps for the next round of optimization until the convergence condition is met.

[0139] The solution of this application can:

[0140] Improve the quality of the knowledge graph. By comparing and optimizing with the preset graph feature fingerprint, the integrity, accuracy, and consistency of the knowledge graph can be effectively improved, making it more in line with expectations and better supporting downstream applications. Enhance controllability. By adjusting the inference depth and introducing the attention mechanism, the optimization process of the knowledge graph can be controlled more precisely, making the optimization results more in line with the requirements of a specific domain. Achieve automated optimization. This method uses the Adam optimizer and an adaptive learning rate, which can automatically adjust parameters without manual intervention, improving the optimization efficiency and reducing the usage threshold.

[0141] Figure 2 It is a schematic structural diagram of a data knowledge graph construction system driven by a complex ecological intelligent brain according to an embodiment of the present invention, as Figure 2 shown, the system includes:

[0142] A first unit for collecting scenario data through a deep feature perception network, obtaining an original multi-dimensional sequence including business processes, processing rules, and asset associations, performing semantic parsing on the original multi-dimensional sequence to generate an initial knowledge vector, constructing an initial graph structure including entity nodes and relationship edges based on the initial knowledge vector, and encoding the initial graph structure using a graph neural network to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery;

[0143] A second unit, configured to calculate a similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with a correlation higher than a preset similarity threshold from the similarity matrix, perform multi-hop path reasoning on the screened entity pairs to generate a set of reasoning paths, input the set of reasoning paths into a recurrent neural network to generate an entity association confidence index, respectively perform multi-modal feature extraction on the data in the initial graph structure and the set of reasoning paths based on the entity association confidence index, where the multi-modal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multi-modal features to generate a unified multi-modal knowledge representation vector;

[0144] A third unit, configured to input the multi-modal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multi-modal knowledge representation vector, where the multi-modal knowledge representation vector includes an entity feature vector, a relationship feature vector, and a sub-graph feature vector, identify influencing features based on the multi-modal knowledge representation vector through the SHAP method, input the influencing features into a problem evolution graph to calculate a propagation probability through causal reasoning, compare and analyze the propagation probability with the graph feature fingerprint to generate an optimization target, extract gradient information in the feature space according to the optimization target and input it into an Adam optimizer to generate an error correction strategy, generate adversarial samples based on the error correction strategy for verification, dynamically adjust the reasoning depth through a double DQN network according to the verification result, and finally compare and verify the optimization effect by comparing the optimized graph features with the graph feature fingerprint to form a complete optimization closed-loop.

[0145] In a third aspect of the embodiments of the present invention,

[0146] There is provided an electronic device, including:

[0147] A processor;

[0148] A memory for storing instructions executable by the processor;

[0149] Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0150] In a fourth aspect of the embodiments of the present invention,

[0151] There is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0152] The present invention may be a method, an apparatus, a system, and / or a computer program product. The computer program product may include a computer-readable storage medium, on which computer-readable program instructions for executing various aspects of the present invention are uploaded.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data knowledge graph construction method driven by a complex ecological intelligent brain, characterized in that: include: The scene data is collected through a deep feature perception network to obtain the original multidimensional sequence containing business processes, processing rules and asset associations, the original multidimensional sequence is semantically parsed to generate an initial knowledge vector, an initial graph structure containing entity nodes and relationship edges is constructed based on the initial knowledge vector, and the initial graph structure is encoded using a graph neural network to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; Based on the graph feature fingerprint, a similarity matrix between entities in the initial graph structure is calculated, and entity pairs whose correlation is higher than a preset similarity threshold are screened from the similarity matrix using the feature set, multi-hop path reasoning is performed on the screened entity pairs to generate a reasoning path set, and the reasoning path set is input into a recursive neural network to generate an entity association confidence index, and multimodal feature extraction is performed on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and the multimodal features are aligned and fused using a cross-modal attention mechanism to generate a unified multimodal knowledge representation vector; The multimodal knowledge representation vector and the entity association confidence index are input into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector. The influencing features are identified through the SHAP method based on the multimodal knowledge representation vector. The influencing features are input into the problem evolution graph to calculate the propagation probability through causal reasoning. The propagation probability is compared and analyzed with the graph feature fingerprint to generate an optimization target. According to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy. Based on the error correction strategy, an adversarial sample is generated for verification. The reasoning depth is dynamically adjusted according to the verification result through the dual DQN network. Finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed loop.

2. The method according to claim 1, characterized in that Calculating the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and using the feature set to filter entity pairs with correlations higher than a preset similarity threshold from the similarity matrix includes: The graph feature fingerprint is represented as a high-dimensional feature vector, which includes graph structure features, semantic distribution features and topological attribute features, and the knowledge features include the attribute feature matrix of the entity node; Performing feature transformation on the attribute feature matrix using a linear transformation unit to generate a query matrix, a key matrix and a value matrix, wherein the query matrix is ​​obtained by multiplying a query weight matrix and the attribute feature matrix, the key matrix is ​​obtained by multiplying a key weight matrix and the attribute feature matrix, and the value matrix is ​​obtained by multiplying a value weight matrix and the attribute feature matrix; Divide the query matrix, the key matrix and the value matrix into multiple attention heads, each of the attention heads independently calculates an attention score, for each of the attention heads, obtains an attention weight by multiplying the query matrix by the transpose of the key matrix and normalizing the dimension, multiplies the attention weight by the value matrix to obtain a head output, and concatenates multiple head outputs to form a multi-head attention output; Based on the multi-head attention output, the similarity relationship between the entity pairs is calculated by cosine similarity to generate a similarity matrix, wherein each element of the similarity matrix represents a similarity score of the corresponding entity pair; A dynamic similarity threshold is set according to the distribution characteristics in the graph feature fingerprint, and the dynamic similarity threshold is applied to the similarity matrix for bidirectional verification screening, wherein the bidirectional verification screening requires that both the forward similarity and the reverse similarity of the entity pair satisfy the dynamic similarity threshold; The entity pairs that pass the bidirectional verification are filtered in combination with the semantic constraints in the knowledge features to generate final related entity pairs.

3. The method according to claim 1, characterized in that Based on the entity association confidence index, the data in the initial graph structure and the reasoning path set are respectively extracted multimodal features through the BERT-Large model, the Vision Transformer model and the Wav2Vec model. The multimodal features include text features, image features and audio features, and the multimodal features are aligned and fused using the cross-modal attention mechanism to generate a unified multimodal knowledge representation vector including: The input text data is encoded using the BERT-Large pre-trained model, and contextual semantic features of the text data are extracted through a multi-layer Transformer structure, wherein the contextual semantic features are represented as a text feature matrix, and a text pre-trained feature vector is generated based on the text feature matrix; The Vision Transformer model is used to divide the input image data into a sequence of image blocks of size 16 by 16 pixels. The image block sequence is position-encoded and then input into the self-attention layer to extract the global visual features of the image block sequence. The global visual features are jointly encoded with the text pre-trained feature vector to obtain a joint image-text feature vector. An attention guidance matrix is ​​constructed based on the joint feature vector of the image and text, and the attention guidance matrix is ​​used to process the input audio data. The audio data is input into a Wav2Vec model, and audio features are extracted through a multi-layer convolutional network and a Transformer structure. The audio features are aligned with the joint feature vector of the image and text to generate a multimodal alignment feature. Calculating the inter-modal similarity matrix for the multimodal alignment features, constructing a cross-modal attention network based on the similarity matrix, and outputting an attention weight distribution through the cross-modal attention network; The attention weight distribution is input into a linear projection layer, and the linear projection layer maps the multimodal features into a unified semantic space to generate a unified multimodal knowledge representation vector.

4. The method according to claim 1, characterized in that Based on the multimodal knowledge representation vector, the influencing features are identified by the SHAP method, the influencing features are input into the problem evolution graph, the propagation probability is calculated by causal reasoning, and the propagation probability is compared and analyzed with the graph feature fingerprint to generate the optimization target, including: The multimodal knowledge representation vector is input into an improved SHAP model, and the improved SHAP model calculates the synergistic effect between features based on a feature combination evaluation mechanism to obtain a feature importance score; the multimodal knowledge representation vector is screened according to the feature importance score, and features whose importance scores exceed a preset feature threshold are selected as influencing features; The influencing features are used as nodes to construct a problem evolution graph, the node attributes of the problem evolution graph include feature values ​​and importance scores, and directed edges are constructed based on the temporal relationship between features; the problem evolution graph is input into a causal reasoning network, the causal reasoning network calculates the conditional probabilities between nodes through a variational reasoning method, and uses Monte Carlo sampling to obtain the probability distribution of the propagation path; The probability distribution of the propagation path is compared with the pre-stored graph feature fingerprint, and the state difference distribution is obtained by calculating the KL divergence; an optimization objective function is constructed based on the state difference distribution, and the optimization objective function includes a distribution difference term and an integrity constraint term, and the optimization target of the knowledge graph is obtained by weighted summation.

5. The method according to claim 4, characterized in that Inputting the multimodal knowledge representation vector into an improved SHAP model, wherein the improved SHAP model calculates the synergistic effect between features based on a feature combination evaluation mechanism to obtain a feature importance score; Screening the multimodal knowledge representation vector according to the feature importance score, and selecting features whose importance scores exceed a preset feature threshold as influencing features includes: The multimodal knowledge representation vector is input into an improved SHAP model, wherein the improved SHAP model uses a sliding window to generate a feature combination pool, and calculates feature combination weights based on the mutual information, correlation, and temporal correlation of features in the feature combination pool; the improved SHAP model uses a nonlinear mapping layer to map the feature combination weights, and calculates the synergistic effect between features based on a feature combination evaluation mechanism; A feature contribution matrix is ​​constructed based on the synergistic effect, wherein the matrix elements of the feature contribution matrix represent the degree of contribution of the feature to the sample; a temporal contribution is obtained by performing temporal weighting on the feature contribution matrix using a temporal attenuation factor, and an initial SHAP value is calculated based on the temporal contribution; a modal adaptive weight is calculated according to the variance of different modal features, and the initial SHAP value is weightedly fused using the modal adaptive weight to obtain a feature importance score; The multimodal knowledge representation vector is screened according to the feature importance score, a preset feature threshold is calculated based on the mean and standard deviation of the feature importance score, and the preset feature threshold is dynamically adjusted in combination with the rate of change of the feature importance score; a feature dependency graph is constructed to analyze the correlation between features, and features whose importance scores exceed the preset feature threshold are selected as influencing features based on feature group effects and stability constraints, and the influencing features are used to guide subsequent feature optimization.

6. The method according to claim 1, characterized in that According to the optimization goal, the gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy. Based on the error correction strategy, adversarial samples are generated for verification. The reasoning depth is dynamically adjusted according to the verification results through the dual DQN network. Finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed loop including: Extracting gradient information in the feature space according to the optimization objective, calculating local gradients through multi-scale windows and obtaining smooth gradients by using a gradient clipping mechanism and an exponential sliding average; inputting the smooth gradients into an Adam optimizer, which adaptively adjusts the learning rate based on the gradient norm and generates an error correction strategy through a second-order momentum correction; Generate adversarial samples based on the error correction strategy, wherein the adversarial samples ensure validity by perturbation boundary constraints and prevent sample degradation by diversity constraints; input the adversarial samples into a multi-layer verification network for quality assessment to obtain verification results; The verification result is input into a dual DQN network, the dual DQN network constructs a state space based on the current reasoning depth, the verification result and resource occupancy, constructs an action space based on the depth adjustment step size and the jump connection parameter, and dynamically adjusts the reasoning depth through the state space and the action space to obtain an adjusted reasoning depth; The knowledge graph is optimized according to the adjusted inference depth to obtain optimized graph features; the optimized graph features are compared with the graph feature fingerprint, and the KL divergence and topological structure similarity of the feature distribution are calculated as evaluation indicators; the evaluation indicators are fed back to the Adam optimizer for optimization parameter update to form a complete optimization closed loop.

7. The method according to claim 6, characterized in that Optimizing the knowledge graph according to the adjusted inference depth to obtain optimized graph features; comparing the optimized graph features with the graph feature fingerprint, and calculating the KL divergence and topological structure similarity of feature distribution as evaluation indicators; Feeding the evaluation index back to the Adam optimizer for optimizing parameter updates, forming a complete optimization closed loop including: Constructing a knowledge graph optimization model according to the adjusted reasoning depth, wherein the knowledge graph optimization model calculates node attention weights and edge relationship attention weights through a hierarchical attention mechanism, and performs feature reorganization on the knowledge graph based on the node attention weights and the edge relationship attention weights to obtain optimized graph features; The optimized graph features are compared with the pre-constructed graph feature fingerprints, and the KL divergence of the feature distribution is calculated using a kernel function estimator, and the kernel function estimator introduces an adaptive smoothing factor to prevent the zero probability problem; a multi-scale structure descriptor is constructed to characterize the graph structure features, and the topological structure similarity is calculated based on the multi-scale structure descriptor; the KL divergence and the topological structure similarity are weightedly fused to obtain an evaluation index; The evaluation index is fed back to the Adam optimizer, and the Adam optimizer uses an adaptive learning rate to update the parameters, and reduces the parameter update fluctuations through gradient accumulation; a convergence criterion is constructed based on the change value of the evaluation index and the change value of the parameter, and whether the optimization has converged is determined according to the convergence criterion; if not, the updated parameters are fed back to the knowledge graph optimization model for the next round of optimization until convergence completes the optimization closed loop.

8. A data knowledge graph construction system driven by a complex ecological intelligent brain, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect scene data through a deep feature perception network, obtain an original multidimensional sequence containing business processes, processing rules and asset associations, perform semantic analysis on the original multidimensional sequence to generate an initial knowledge vector, construct an initial graph structure containing entity nodes and relationship edges based on the initial knowledge vector, encode the initial graph structure using a graph neural network, and obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; The second unit is used to calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with correlation higher than a preset similarity threshold from the similarity matrix, perform multi-hop path reasoning on the screened entity pairs to generate a reasoning path set, input the reasoning path set into a recursive neural network to generate an entity association confidence index, and perform multimodal feature extraction on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multimodal features to generate a unified multimodal knowledge representation vector; The third unit is used to input the multimodal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector; based on the multimodal knowledge representation vector, the influencing features are identified by the SHAP method; the influencing features are input into the problem evolution graph to calculate the propagation probability by causal reasoning; the propagation probability is compared and analyzed with the graph feature fingerprint to generate an optimization target; according to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy; based on the error correction strategy, adversarial samples are generated for verification; the reasoning depth is dynamically adjusted according to the verification result through the dual DQN network; finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, so as to form a complete optimization closed loop.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images