Complex ecological smart brain-driven data knowledge graph construction method and system

Abstract

The invention provides a data knowledge graph construction method and system driven by a complex ecological smart brain, and relates to the technical field of complex ecology, and the method comprises the steps: collecting scene data through a depth feature sensing network, and generating an initial graph structure; entity similarity is calculated based on the map feature fingerprints, related entity pairs are screened, multi-hop path reasoning is carried out, and entity association confidence indexes are generated; generating a multi-modal knowledge representation vector based on multi-modal feature extraction and a cross-modal attention mechanism; the method comprises the following steps: identifying influence characteristics, calculating a propagation probability in combination with a problem evolution diagram and causal reasoning, performing comparative analysis on the propagation probability and a map characteristic fingerprint to generate an optimization target, generating an error correction strategy by using an Adam optimizer, dynamically adjusting the reasoning depth through a double-DQN network, forming an optimization closed loop, and constructing a data knowledge map. According to the method, the accuracy and integrity of the knowledge graph can be effectively improved, the reasoning ability is enhanced, and the method can better adapt to knowledge discovery and application in a complex ecological scene.

Description

Technical Field

[0001] The present invention relates to complex ecological technology, and in particular to a data knowledge graph construction method and system driven by a complex ecological intelligent brain. Background Art

[0002] As a structured form of knowledge representation, knowledge graphs play an increasingly important role in various fields of artificial intelligence, such as natural language processing, recommendation systems, and knowledge reasoning. Traditional knowledge graph construction methods usually rely on manual construction or rule-based automation methods, which have problems such as low efficiency, poor scalability, and difficulty in adapting to complex and changing real-world data. In recent years, with the rise of deep learning and graph neural networks, researchers have begun to explore the use of these technologies to automatically construct and reason about knowledge graphs.

[0003] Existing methods usually have difficulty in effectively processing multimodal data. Real-world data often contains information in multiple modalities, such as text, images, and audio, while existing methods mostly focus on data in a single modality and have difficulty making full use of the rich semantic information in multimodal data.

[0004] Existing methods still have limitations in knowledge reasoning. Knowledge reasoning is one of the important functions of knowledge graphs, but the reasoning ability of existing methods is often limited to shallow reasoning paths, making it difficult to perform complex multi-hop reasoning.

[0005] Existing methods lack effective optimization mechanisms. The construction of knowledge graphs is an iterative optimization process that requires continuous correction and improvement of graph structures and knowledge. However, existing methods often lack effective optimization mechanisms and it is difficult to ensure the quality and accuracy of graphs. Summary of the invention

[0006] The embodiments of the present invention provide a method and system for constructing a data knowledge graph driven by a complex ecological intelligent brain, which can solve the problems in the prior art.

[0007] According to a first aspect of the embodiments of the present invention, Provides a data knowledge graph construction method driven by a complex ecological intelligence brain, including: The scene data is collected through a deep feature perception network to obtain the original multidimensional sequence containing business processes, processing rules and asset associations, the original multidimensional sequence is semantically parsed to generate an initial knowledge vector, an initial graph structure containing entity nodes and relationship edges is constructed based on the initial knowledge vector, and the initial graph structure is encoded using a graph neural network to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; Based on the graph feature fingerprint, a similarity matrix between entities in the initial graph structure is calculated, and entity pairs whose correlation is higher than a preset similarity threshold are screened from the similarity matrix using the feature set, multi-hop path reasoning is performed on the screened entity pairs to generate a reasoning path set, and the reasoning path set is input into a recursive neural network to generate an entity association confidence index, and multimodal feature extraction is performed on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and the multimodal features are aligned and fused using a cross-modal attention mechanism to generate a unified multimodal knowledge representation vector; The multimodal knowledge representation vector and the entity association confidence index are input into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector. The influencing features are identified through the SHAP method based on the multimodal knowledge representation vector. The influencing features are input into the problem evolution graph to calculate the propagation probability through causal reasoning. The propagation probability is compared and analyzed with the graph feature fingerprint to generate an optimization target. According to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy. Based on the error correction strategy, an adversarial sample is generated for verification. The reasoning depth is dynamically adjusted according to the verification result through the dual DQN network. Finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed loop.

[0008] Calculating the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and using the feature set to filter entity pairs with correlations higher than a preset similarity threshold from the similarity matrix includes: The graph feature fingerprint is represented as a high-dimensional feature vector, which includes graph structure features, semantic distribution features and topological attribute features, and the knowledge features include an attribute feature matrix of entity nodes; Performing feature transformation on the attribute feature matrix using a linear transformation unit to generate a query matrix, a key matrix and a value matrix, wherein the query matrix is ​​obtained by multiplying a query weight matrix and the attribute feature matrix, the key matrix is ​​obtained by multiplying a key weight matrix and the attribute feature matrix, and the value matrix is ​​obtained by multiplying a value weight matrix and the attribute feature matrix; Divide the query matrix, the key matrix and the value matrix into multiple attention heads, each of the attention heads independently calculates an attention score, for each of the attention heads, obtains an attention weight by multiplying the query matrix by the transpose of the key matrix and normalizing the dimension, multiplies the attention weight by the value matrix to obtain a head output, and concatenates multiple head outputs to form a multi-head attention output; Based on the multi-head attention output, the similarity relationship between the entity pairs is calculated by cosine similarity to generate a similarity matrix, wherein each element of the similarity matrix represents a similarity score of the corresponding entity pair; A dynamic similarity threshold is set according to the distribution characteristics in the graph feature fingerprint, and the dynamic similarity threshold is applied to the similarity matrix for bidirectional verification screening, wherein the bidirectional verification screening requires that both the forward similarity and the reverse similarity of the entity pair satisfy the dynamic similarity threshold; The entity pairs that pass the bidirectional verification are filtered in combination with the semantic constraints in the knowledge features to generate final related entity pairs.

[0009] Based on the entity association confidence index, the data in the initial graph structure and the reasoning path set are respectively extracted multimodal features through the BERT-Large model, the Vision Transformer model and the Wav2Vec model. The multimodal features include text features, image features and audio features, and the multimodal features are aligned and fused using the cross-modal attention mechanism to generate a unified multimodal knowledge representation vector including: The input text data is encoded using the BERT-Large pre-trained model, and contextual semantic features of the text data are extracted through a multi-layer Transformer structure, wherein the contextual semantic features are represented as a text feature matrix, and a text pre-trained feature vector is generated based on the text feature matrix; The Vision Transformer model is used to divide the input image data into a sequence of image blocks of size 16 by 16 pixels. The image block sequence is position-encoded and then input into the self-attention layer to extract the global visual features of the image block sequence. The global visual features are jointly encoded with the text pre-trained feature vector to obtain a joint image-text feature vector. An attention guidance matrix is ​​constructed based on the joint feature vector of the image and text, and the attention guidance matrix is ​​used to process the input audio data. The audio data is input into a Wav2Vec model, and audio features are extracted through a multi-layer convolutional network and a Transformer structure. The audio features are aligned with the joint feature vector of the image and text to generate a multimodal alignment feature. Calculating the inter-modal similarity matrix for the multimodal alignment features, constructing a cross-modal attention network based on the similarity matrix, and outputting an attention weight distribution through the cross-modal attention network; The attention weight distribution is input into a linear projection layer, and the linear projection layer maps the different modal features into the unified semantic space to generate a unified multimodal knowledge representation vector.

[0010] Based on the multimodal knowledge representation vector, the influencing features are identified by the SHAP method, the influencing features are input into the problem evolution graph, the propagation probability is calculated by causal reasoning, and the propagation probability is compared and analyzed with the graph feature fingerprint to generate the optimization target, including: The multimodal knowledge representation vector is input into an improved SHAP model, and the improved SHAP model calculates the synergistic effect between features based on a feature combination evaluation mechanism to obtain a feature importance score; the multimodal knowledge representation vector is screened according to the feature importance score, and features whose importance scores exceed a preset feature threshold are selected as influencing features; The influencing features are used as nodes to construct a problem evolution graph, the node attributes of the problem evolution graph include feature values ​​and importance scores, and directed edges are constructed based on the temporal relationship between features; the problem evolution graph is input into a causal reasoning network, the causal reasoning network calculates the conditional probabilities between nodes through a variational reasoning method, and uses Monte Carlo sampling to obtain the probability distribution of the propagation path; The probability distribution of the propagation path is compared with the pre-stored graph feature fingerprint, and the state difference distribution is obtained by calculating the KL divergence; an optimization objective function is constructed based on the state difference distribution, and the optimization objective function includes a distribution difference term and an integrity constraint term, and the optimization target of the knowledge graph is obtained by weighted summation.

[0011] Inputting the multimodal knowledge representation vector into the improved SHAP model, the improved SHAP model calculates the synergistic effect between features based on the feature combination evaluation mechanism to obtain a feature importance score; screening the multimodal knowledge representation vector according to the feature importance score, and selecting features whose importance scores exceed a preset feature threshold as influencing features includes: The multimodal knowledge representation vector is input into an improved SHAP model, wherein the improved SHAP model uses a sliding window to generate a feature combination pool, and calculates feature combination weights based on the mutual information, correlation, and temporal correlation of features in the feature combination pool; the improved SHAP model uses a nonlinear mapping layer to map the feature combination weights, and calculates the synergistic effect between features based on a feature combination evaluation mechanism; A feature contribution matrix is ​​constructed based on the synergistic effect, wherein the matrix elements of the feature contribution matrix represent the degree of contribution of the feature to the sample; a temporal contribution is obtained by performing temporal weighting on the feature contribution matrix using a temporal attenuation factor, and an initial SHAP value is calculated based on the temporal contribution; a modal adaptive weight is calculated according to the variance of different modal features, and the initial SHAP value is weightedly fused using the modal adaptive weight to obtain a feature importance score; The multimodal knowledge representation vector is screened according to the feature importance score, a preset feature threshold is calculated based on the mean and standard deviation of the feature importance score, and the preset feature threshold is dynamically adjusted in combination with the rate of change of the feature importance score; a feature dependency graph is constructed to analyze the correlation between features, and features whose importance scores exceed the preset feature threshold are selected as influencing features based on feature group effects and stability constraints, and the influencing features are used to guide subsequent feature optimization.

[0012] According to the optimization goal, the gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy. Based on the error correction strategy, adversarial samples are generated for verification. The reasoning depth is dynamically adjusted according to the verification results through the dual DQN network. Finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed loop including: Extracting gradient information in the feature space according to the optimization objective, calculating local gradients through multi-scale windows and obtaining smooth gradients by using a gradient clipping mechanism and an exponential sliding average; inputting the smooth gradients into an Adam optimizer, which adaptively adjusts the learning rate based on the gradient norm and generates an error correction strategy through a second-order momentum correction; Generate adversarial samples based on the error correction strategy, wherein the adversarial samples ensure validity by perturbation boundary constraints and prevent sample degradation by diversity constraints; input the adversarial samples into a multi-layer verification network for quality assessment to obtain verification results; The verification result is input into a dual DQN network, the dual DQN network constructs a state space based on the current reasoning depth, the verification result and resource occupancy, constructs an action space based on the depth adjustment step size and the jump connection parameter, and dynamically adjusts the reasoning depth through the state space and the action space to obtain an adjusted reasoning depth; The knowledge graph is optimized according to the adjusted inference depth to obtain optimized graph features; the optimized graph features are compared with the graph feature fingerprint, and the KL divergence and topological structure similarity of the feature distribution are calculated as evaluation indicators; the evaluation indicators are fed back to the Adam optimizer for optimization parameter update to form a complete optimization closed loop.

[0013] The knowledge graph is optimized according to the adjusted inference depth to obtain optimized graph features; the optimized graph features are compared with the graph feature fingerprint, and the KL divergence and topological structure similarity of feature distribution are calculated as evaluation indicators; the evaluation indicators are fed back to the Adam optimizer for optimization parameter update, forming a complete optimization closed loop including: Constructing a knowledge graph optimization model according to the adjusted reasoning depth, wherein the knowledge graph optimization model calculates node attention weights and edge relationship attention weights through a hierarchical attention mechanism, and performs feature reorganization on the knowledge graph based on the node attention weights and the edge relationship attention weights to obtain optimized graph features; The optimized graph features are compared with the pre-constructed graph feature fingerprints, and the KL divergence of the feature distribution is calculated using a kernel function estimator, and the kernel function estimator introduces an adaptive smoothing factor to prevent the zero probability problem; a multi-scale structure descriptor is constructed to characterize the graph structure features, and the topological structure similarity is calculated based on the multi-scale structure descriptor; the KL divergence and the topological structure similarity are weightedly fused to obtain an evaluation index; The evaluation index is fed back to the Adam optimizer, and the Adam optimizer uses an adaptive learning rate to update the parameters, and reduces the parameter update fluctuations through gradient accumulation; a convergence criterion is constructed based on the change value of the evaluation index and the change value of the parameter, and whether the optimization has converged is determined according to the convergence criterion; if not, the updated parameters are fed back to the knowledge graph optimization model for the next round of optimization until convergence completes the optimization closed loop.

[0014] According to a second aspect of the embodiments of the present invention, Provides a data knowledge graph construction system driven by a complex ecological intelligence brain, including: The first unit is used to collect scene data through a deep feature perception network, obtain an original multidimensional sequence containing business processes, processing rules and asset associations, perform semantic analysis on the original multidimensional sequence to generate an initial knowledge vector, construct an initial graph structure containing entity nodes and relationship edges based on the initial knowledge vector, encode the initial graph structure using a graph neural network, and obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; The second unit is used to calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with correlation higher than a preset similarity threshold from the similarity matrix, perform multi-hop path reasoning on the screened entity pairs to generate a reasoning path set, input the reasoning path set into a recursive neural network to generate an entity association confidence index, and perform multimodal feature extraction on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multimodal features to generate a unified multimodal knowledge representation vector; The third unit is used to input the multimodal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector; based on the multimodal knowledge representation vector, the influencing features are identified by the SHAP method; the influencing features are input into the problem evolution graph to calculate the propagation probability by causal reasoning; the propagation probability is compared and analyzed with the graph feature fingerprint to generate an optimization target; according to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy; based on the error correction strategy, adversarial samples are generated for verification; the reasoning depth is dynamically adjusted according to the verification result through the dual DQN network; finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, so as to form a complete optimization closed loop.

[0015] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0016] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0017] The beneficial effects of this application are as follows: 1. Improve the efficiency and accuracy of knowledge graph construction: Through data collection and semantic analysis through deep feature perception network, combined with graph neural network encoding, it is possible to efficiently build the initial graph structure and extract key features. Using multi-hop path reasoning and recursive neural networks, it is possible to more accurately identify entity associations, and build a more comprehensive knowledge representation through multimodal feature extraction and cross-modal attention mechanism.

[0018] 2. Enhanced knowledge discovery and reasoning capabilities: Based on graph feature fingerprints, similarity calculation and entity screening are performed, and combined with multi-hop path reasoning, potential entity associations and knowledge can be discovered. Using the SHAP method to identify influencing features and calculating propagation probabilities through causal reasoning, the causal relationship between knowledge can be better understood and more accurate reasoning can be performed.

[0019] 3. Realize continuous optimization and adaptation of knowledge graph: Generate optimization targets by comparing and analyzing propagation probability and graph feature fingerprints, and use Adam optimizer to generate error correction strategies. Through adversarial sample verification and dynamic adjustment of reasoning depth by dual DQN network, a complete optimization closed loop is formed, so that knowledge graph can be continuously optimized and adapted, improving its robustness and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the process of constructing a data knowledge graph driven by a complex ecological intelligent brain according to an embodiment of the present invention; Figure 2 This is a structural diagram of the data knowledge graph construction system driven by the complex ecological intelligent brain in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0023] Figure 1 Schematic diagram of the process of constructing a data knowledge graph driven by a complex ecological intelligent brain according to an embodiment of the present invention. Figure 1 As shown, the method includes: S11. The scene data is collected through a deep feature perception network to obtain an original multidimensional sequence containing business processes, processing rules and asset associations, the original multidimensional sequence is semantically parsed to generate an initial knowledge vector, an initial graph structure containing entity nodes and relationship edges is constructed based on the initial knowledge vector, and the initial graph structure is encoded using a graph neural network to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; S12. Calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with correlations higher than a preset similarity threshold from the similarity matrix, perform multi-hop path reasoning on the screened entity pairs to generate a reasoning path set, input the reasoning path set into a recursive neural network to generate an entity association confidence index, and perform multimodal feature extraction on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multimodal features to generate a unified multimodal knowledge representation vector; S13. Input the multimodal knowledge representation vector and entity association confidence index into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector; identify influencing features through the SHAP method based on the multimodal knowledge representation vector; input the influencing features into the problem evolution graph to calculate the propagation probability through causal reasoning; compare and analyze the propagation probability with the graph feature fingerprint to generate an optimization target; extract gradient information in the feature space according to the optimization target and input it into the Adam optimizer to generate an error correction strategy; generate adversarial samples based on the error correction strategy for verification; dynamically adjust the reasoning depth according to the verification result through the dual DQN network; finally, compare the optimized graph features with the graph feature fingerprint to verify the optimization effect, so as to form a complete optimization closed loop.

[0024] In an optional implementation, calculating the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and using the feature set to filter entity pairs with correlations higher than a preset similarity threshold from the similarity matrix includes: The graph feature fingerprint is represented as a high-dimensional feature vector, which includes graph structure features, semantic distribution features and topological attribute features, and the knowledge features include an attribute feature matrix of entity nodes; Performing feature transformation on the attribute feature matrix using a linear transformation unit to generate a query matrix, a key matrix and a value matrix, wherein the query matrix is ​​obtained by multiplying a query weight matrix and the attribute feature matrix, the key matrix is ​​obtained by multiplying a key weight matrix and the attribute feature matrix, and the value matrix is ​​obtained by multiplying a value weight matrix and the attribute feature matrix; Divide the query matrix, the key matrix and the value matrix into multiple attention heads, each of the attention heads independently calculates an attention score, for each of the attention heads, obtains an attention weight by multiplying the query matrix by the transpose of the key matrix and normalizing the dimension, multiplies the attention weight by the value matrix to obtain a head output, and concatenates multiple head outputs to form a multi-head attention output; Based on the multi-head attention output, the similarity relationship between the entity pairs is calculated by cosine similarity to generate a similarity matrix, wherein each element of the similarity matrix represents a similarity score of the corresponding entity pair; A dynamic similarity threshold is set according to the distribution characteristics in the graph feature fingerprint, and the dynamic similarity threshold is applied to the similarity matrix for bidirectional verification screening, wherein the bidirectional verification screening requires that both the forward similarity and the reverse similarity of the entity pair satisfy the dynamic similarity threshold; The entity pairs that pass the bidirectional verification are filtered in combination with the semantic constraints in the knowledge features to generate final related entity pairs.

[0025] A method for entity correlation analysis based on graph feature fingerprints is used to calculate the similarity between entities in the initial graph structure and screen entity pairs whose correlation is higher than a preset threshold.

[0026] First, construct a graph feature fingerprint. The graph feature fingerprint is represented as a high-dimensional feature vector, which includes graph structure features, semantic distribution features, and topological attribute features. For example, graph structure features can be the degree and centrality of entity nodes; semantic distribution features can be the probability distribution of entity nodes under different topics; topological attribute features can be the location of entity nodes in the graph, neighbor nodes, and other information. At the same time, extract knowledge features, including the attribute feature matrix of entity nodes. For example, the attributes of a movie entity can include director, actor, type, release time, etc.

[0027] Next, the attribute feature matrix is ​​transformed using a linear transformation unit to generate a query matrix, a key matrix, and a value matrix. Specifically, the query matrix, the key matrix, and the value matrix are obtained by multiplying the attribute feature matrix with three different weight matrices. For example, assuming that the query weight matrix is ​​[[0.1, 0.2, 0.3, 0.4], [0.2, 0.3, 0.4,0.1]], the first row of the attribute feature matrix [Zhang San, Li Si, Plot, 2022] is multiplied with it to obtain the first row of the query matrix.

[0028] Then, the query matrix, key matrix, and value matrix are divided into multiple attention heads. Each attention head calculates the attention score independently. For each attention head, the attention weight is obtained by multiplying the query matrix with the transpose of the key matrix and normalizing the dimension. The attention weight is multiplied by the value matrix to obtain the head output. Multiple head outputs are concatenated to form a multi-head attention output. For example, suppose the query matrix, key matrix, and value matrix are all divided into two attention heads. The attention weight calculated by the first attention head is multiplied by the value matrix to obtain the first head output. The attention weight calculated by the second attention head is multiplied by the value matrix to obtain the second head output. The two head outputs are concatenated to form a multi-head attention output.

[0029] Based on the multi-head attention output, the similarity relationship between entity pairs is calculated by cosine similarity to generate a similarity matrix. Each element of the similarity matrix represents the similarity score of the corresponding entity pair. For example, the multi-head attention outputs of movie A and movie B are [0.1, 0.2, 0.3] and [0.2, 0.3, 0.4] respectively, then the cosine similarity between them is the dot product of the two vectors divided by the product of their moduli.

[0030] Set the dynamic similarity threshold according to the distribution characteristics in the graph feature fingerprint. Apply the dynamic similarity threshold to the similarity matrix for two-way verification screening. Two-way verification screening requires that both the forward similarity and reverse similarity of the entity pair meet the dynamic similarity threshold. For example, assuming that the similarity between movie A and movie B is 0.8, the similarity between movie B and movie A is 0.9, and the dynamic similarity threshold is 0.7, then movie A and movie B meet the two-way verification.

[0031] Combined with the semantic constraints in the knowledge features, the entity pairs that pass the two-way verification are filtered to generate the final related entity pairs. For example, assuming that the semantic constraint is that the movie genre must be the same, then Movie A and Movie C are related because their genres are both plot. Although Movie A and Movie B pass the two-way verification, they are ultimately filtered out due to their different genres.

[0032] The solution of this application can: Improve the accuracy of entity correlation analysis: By combining graph structure features, semantic distribution features, and topological attribute features to construct graph feature fingerprints, the correlation between entities can be more comprehensively characterized, thereby improving the accuracy of entity correlation analysis. Enhance the robustness of entity correlation analysis: The dynamic similarity threshold can be adaptively adjusted according to the distribution characteristics of different entities, thereby enhancing the robustness of entity correlation analysis and avoiding the limitations brought by fixed thresholds. Improve the efficiency of entity correlation analysis: The multi-head attention mechanism can calculate the similarity between entities in parallel, thereby improving the efficiency of entity correlation analysis, especially on large-scale graph data.

[0033] In an optional implementation, based on the entity association confidence index, multimodal features are extracted from the data in the initial graph structure and the reasoning path set through the BERT-Large model, the Vision Transformer model and the Wav2Vec model, respectively, and the multimodal features include text features, image features and audio features, and the multimodal features are aligned and fused using a cross-modal attention mechanism to generate a unified multimodal knowledge representation vector, including: The input text data is encoded using the BERT-Large pre-trained model, and contextual semantic features of the text data are extracted through a multi-layer Transformer structure, wherein the contextual semantic features are represented as a text feature matrix, and a text pre-trained feature vector is generated based on the text feature matrix; The Vision Transformer model is used to divide the input image data into a sequence of image blocks of size 16 by 16 pixels. The image block sequence is position-encoded and then input into the self-attention layer to extract the global visual features of the image block sequence. The global visual features are jointly encoded with the text pre-trained feature vector to obtain a joint image-text feature vector. An attention guidance matrix is ​​constructed based on the joint feature vector of the image and text, and the attention guidance matrix is ​​used to process the input audio data. The audio data is input into a Wav2Vec model, and audio features are extracted through a multi-layer convolutional network and a Transformer structure. The audio features are aligned with the joint feature vector of the image and text to generate a multimodal alignment feature. Calculating the inter-modal similarity matrix for the multimodal alignment features, constructing a cross-modal attention network based on the similarity matrix, and outputting an attention weight distribution through the cross-modal attention network; The attention weight distribution is input into a linear projection layer, and the linear projection layer maps the different modal features into the unified semantic space to generate a unified multimodal knowledge representation vector.

[0034] A knowledge graph reasoning method based on multimodal knowledge representation aims to improve the accuracy and efficiency of knowledge graph reasoning. The core of this method is to use the BERT-Large model, Vision Transformer model and Wav2Vec model to extract text, image and audio features respectively, and fuse these multimodal features through a cross-modal attention mechanism to generate a unified multimodal knowledge representation vector for subsequent knowledge reasoning.

[0035] First, initialize the entities and relationships in the knowledge graph and construct the initial graph structure. For example, if the graph contains the entities "cat" and "animal" and the relationship "belongs to", an initial triple (cat, belongs to, animal) can be constructed. At the same time, according to the needs of the reasoning task, generate a set of reasoning paths. For example, if you want to infer "what is the sound of a cat", you can generate a reasoning path "cat->belongs to->animal->makes a sound->meow". The data in these initial graph structures and reasoning path sets will be associated with a confidence indicator, which indicates the reliability of the data in the reasoning process. For example, according to the existing knowledge base, the confidence of "cat belongs to animal" may be very high, while the confidence of "cat makes a sound" is relatively low.

[0036] Next, perform multimodal feature extraction on the data in the initial graph structure and reasoning path set. Taking the entity "cat" as an example, extract its text description "a small mammal that is usually kept as a pet by humans", and use the BERT-Large model to encode the text to obtain a text feature vector. At the same time, extract the image of "cat", such as a picture of a cat, and use the Vision Transformer model to extract image features. In addition, extract the audio of "cat", such as an audio of a cat's meow, and use the Wav2Vec model to extract audio features.

[0037] Then, multimodal feature fusion is performed. The text feature vector generated by the BERT-Large model and the image features extracted by the VisionTransformer model are jointly encoded to obtain a joint feature vector of text and images. The audio data is input into the Wav2Vec model to extract audio features, and the features are aligned with the joint feature vector of text and images to generate multimodal alignment features. The inter-modal similarity matrix of the multimodal alignment features is calculated, and a cross-modal attention network is constructed based on the similarity matrix to obtain the attention weight distribution. Finally, the attention weight distribution is input into the linear projection layer to map the features of different modalities into a unified semantic space to generate a unified multimodal knowledge representation vector. For example, the final multimodal knowledge representation vector of "cat" integrates text, image, and audio information, and can more comprehensively represent the concept of "cat".

[0038] Finally, knowledge reasoning is performed based on the generated unified multimodal knowledge representation vector. For example, to reason about "what is the sound of a cat's meow", the multimodal knowledge representation vector of "cat" can be compared and calculated with the multimodal knowledge representation vectors of other related entities (such as "animal" and "sound"), and finally it can be inferred that the sound of a "cat" is "meow".

[0039] The solution of this application can: Improve the comprehensiveness of knowledge representation: By integrating multimodal information such as text, images, and audio, knowledge can be represented more comprehensively, avoiding the bias caused by single modal information, thereby improving the accuracy and completeness of knowledge representation. For example, integrating a cat's picture, text description, and cry can more completely express the concept of "cat" than using text description alone. Enhance the accuracy of knowledge reasoning: Multimodal knowledge representation vectors contain richer semantic information and can better capture the association between entities and relationships, thereby improving the accuracy of knowledge reasoning. For example, when reasoning "what is the sound of a cat's cry", a multimodal representation vector that incorporates the audio of a cat's cry can more accurately infer the answer. Expand the scope of application of knowledge graphs: Multimodal knowledge representation can better process multimedia data, thereby expanding the application of knowledge graphs in areas such as image recognition and speech recognition. For example, this method can be used to build a multimodal knowledge graph containing images, sounds, and text for applications such as intelligent question answering and information retrieval.

[0040] In an optional implementation, the SHAP method is used to identify influencing features based on the multimodal knowledge representation vector, the influencing features are input into the problem evolution graph to calculate the propagation probability through causal reasoning, and the propagation probability is compared and analyzed with the graph feature fingerprint to generate the optimization target, including: The multimodal knowledge representation vector is input into an improved SHAP model, and the improved SHAP model calculates the synergistic effect between features based on a feature combination evaluation mechanism to obtain a feature importance score; the multimodal knowledge representation vector is screened according to the feature importance score, and features whose importance scores exceed a preset feature threshold are selected as influencing features; The influencing features are used as nodes to construct a problem evolution graph, the node attributes of the problem evolution graph include feature values ​​and importance scores, and directed edges are constructed based on the temporal relationship between features; the problem evolution graph is input into a causal reasoning network, the causal reasoning network calculates the conditional probabilities between nodes through a variational reasoning method, and uses Monte Carlo sampling to obtain the probability distribution of the propagation path; The probability distribution of the propagation path is compared with the pre-stored graph feature fingerprint, and the state difference distribution is obtained by calculating the KL divergence; an optimization objective function is constructed based on the state difference distribution, and the optimization objective function includes a distribution difference term and an integrity constraint term, and the optimization target of the knowledge graph is obtained by weighted summation.

[0041] The multimodal knowledge graph optimization method aims to improve the integrity and accuracy of the knowledge graph. The core idea of ​​this method is to analyze the propagation probability of the influencing features in the problem evolution graph and compare them with the existing graph feature fingerprints to identify the parts that need to be optimized.

[0042] First, feature selection is performed on the multimodal knowledge representation vector. The multimodal knowledge representation vector is input into the improved SHAP model. This improved SHAP model is not a traditional SHAP, but considers the evaluation mechanism of feature combination, which can capture the synergistic effect between features and more accurately evaluate the importance of each feature. For example, an image contains both the features of "cat" and "sofa". The combination of these two features may imply a higher-level semantic information such as "cat on the sofa". The improved SHAP model can capture this synergistic effect. Through this model, the importance score of each feature is obtained. A preset feature threshold is set, such as 0.05, and features with importance scores exceeding the threshold are selected as influencing features. Assuming that the multimodal vector contains four features "cat", "sofa", "table", and "dog", and their importance scores are 0.1, 0.08, 0.02, and 0.01 respectively, then "cat" and "sofa" will be selected as influencing features.

[0043] Next, construct a problem evolution graph. The influencing features filtered out in the previous step are used as nodes of the graph. Each node contains two attributes: feature value and importance score. Nodes are connected by directed edges, and the direction of the edge represents the temporal relationship between features. For example, the "cat" node points to the "sofa" node, indicating that the action of "cat" precedes the state of "cat on the sofa". Assuming that the importance score of "cat" is 0.1 and the importance score of "sofa" is 0.08, the edge from the "cat" node to the "sofa" node can be expressed as ("cat", 0.1) → ("sofa", 0.08).

[0044] Then, causal reasoning and propagation probability calculation are performed. The constructed problem evolution graph is input into the causal reasoning network. The network uses the variational inference method to calculate the conditional probability between nodes, and uses the Monte Carlo sampling method to simulate the propagation path of the feature in the graph to obtain the probability distribution of the propagation path. For example, if 1000 samplings are simulated, and 800 sampling results show that the "cat" will jump onto the "sofa", then the propagation probability of the "cat" to the "sofa" is 0.8.

[0045] Compare and analyze the calculated propagation path probability distribution with the pre-stored graph feature fingerprint. The graph feature fingerprint is a pre-stored feature propagation probability distribution that represents common patterns in the knowledge graph. By calculating the KL divergence between the propagation probability distribution and the graph feature fingerprint, the state difference distribution is obtained. The KL divergence is used to measure the degree of difference between two probability distributions.

[0046] Finally, the optimization target of the knowledge graph is generated. The optimization objective function is constructed based on the state difference distribution. This function contains two items: distribution difference term and integrity constraint term. The distribution difference term is used to measure the gap between the current knowledge graph and the ideal state, and the integrity constraint term is used to ensure the integrity and consistency of the knowledge graph. The final optimization target is obtained by weighted summing these two items.

[0047] The solution of this application can: Improve the accuracy of the knowledge graph: By identifying and correcting the feature propagation paths that do not conform to the preset pattern, the erroneous information in the knowledge graph can be effectively corrected to improve its accuracy. Enhance the integrity of the knowledge graph: This method can identify the missing information in the knowledge graph and complete it, thereby enhancing the coverage and information volume of the knowledge graph. Improve the interpretability of the knowledge graph: By analyzing the causal relationship and propagation path between features, the internal logic and evolution law of the knowledge graph can be better understood, and its interpretability can be improved.

[0048] In an optional implementation, the multimodal knowledge representation vector is input into an improved SHAP model, and the improved SHAP model calculates the synergistic effect between features based on a feature combination evaluation mechanism to obtain a feature importance score; the multimodal knowledge representation vector is screened according to the feature importance score, and features whose importance scores exceed a preset feature threshold are selected as influencing features, including: The multimodal knowledge representation vector is input into an improved SHAP model, wherein the improved SHAP model uses a sliding window to generate a feature combination pool, and calculates feature combination weights based on the mutual information, correlation, and temporal correlation of features in the feature combination pool; the improved SHAP model uses a nonlinear mapping layer to map the feature combination weights, and calculates the synergistic effect between features based on a feature combination evaluation mechanism; A feature contribution matrix is ​​constructed based on the synergistic effect, wherein the matrix elements of the feature contribution matrix represent the degree of contribution of the feature to the sample; a temporal contribution is obtained by performing temporal weighting on the feature contribution matrix using a temporal attenuation factor, and an initial SHAP value is calculated based on the temporal contribution; a modal adaptive weight is calculated according to the variance of different modal features, and the initial SHAP value is weightedly fused using the modal adaptive weight to obtain a feature importance score; The multimodal knowledge representation vector is screened according to the feature importance score, a preset feature threshold is calculated based on the mean and standard deviation of the feature importance score, and the preset feature threshold is dynamically adjusted in combination with the rate of change of the feature importance score; a feature dependency graph is constructed to analyze the correlation between features, and features whose importance scores exceed the preset feature threshold are selected as influencing features based on feature group effects and stability constraints, and the influencing features are used to guide subsequent feature optimization.

[0049] First, the multimodal knowledge representation vector is preprocessed. Data of different modalities, such as images, text, and audio, are converted into a unified vector representation. For example, image data can be feature extracted through a convolutional neural network, text data can be converted into vectors through a word embedding model, and audio data can be converted into vectors through spectrogram feature extraction. These vectors are spliced ​​together to form a multimodal knowledge representation vector. Assuming that a multimodal sample contains three modalities: image, text, and audio, and the vector dimensions of each modality are 128, 64, and 32 respectively, the final multimodal knowledge representation vector dimension is 224.

[0050] Next, the preprocessed multimodal knowledge representation vector is input into the improved SHAP model. The model first uses a sliding window mechanism to generate a feature combination pool. For example, assuming that the dimension of the multimodal knowledge representation vector is 224 and the sliding window size is set to 3, 222 feature combinations will be generated, each containing 3 continuous features. Then, the feature combination weight is calculated based on the mutual information, correlation, and temporal correlation of the features in the feature combination pool. For example, the higher the frequency of two features appearing in time, the higher the temporal correlation between them, and the higher the corresponding feature combination weight. Here, assuming that the mutual information of a feature combination is 0.8, the correlation is 0.9, and the temporal correlation is 0.7, the weight of the feature combination can be calculated according to a preset weight calculation method, such as weighted average. Assuming that the weights of mutual information, correlation, and temporal correlation are 0.3, 0.4, and 0.3, respectively, the weight of the feature combination is 0.3 * 0.8 + 0.4 * 0.9 + 0.3 * 0.7 = 0.78. The improved SHAP model uses a nonlinear mapping layer to map the feature combination weights, such as using the Sigmoid function to map the weights between 0 and 1. Then, the synergistic effect between features is calculated based on the feature combination evaluation mechanism, that is, the impact of multiple features on the results. For example, if the combined weight of image features and text features is high, it means that there is a strong synergistic effect between the two features.

[0051] After that, a feature contribution matrix is ​​constructed based on the synergistic effect. Each element of the matrix represents the contribution of the feature to the sample. For example, if an image feature has a great influence on the classification result of the sample, the corresponding contribution value of the feature is high. The feature contribution matrix is ​​time-weighted by the time-series decay factor to obtain the time-series contribution. For example, for time series data, the most recent feature has a greater impact on the current result, so its corresponding time-series decay factor is larger. Assuming that the contribution of a feature at time t is 0.8 and the time-series decay factor is 0.9, the time-series contribution of the feature at time t-1 is 0.8 * 0.9 = 0.72. The initial SHAP value is calculated based on the time-series contribution. The modality adaptive weight is calculated based on the variance of different modality features. For example, if the variance of the image feature is large, it means that the image feature has a higher degree of discrimination for the sample, and its corresponding modality adaptive weight is also higher. The initial SHAP value is weighted and fused using the modality adaptive weight to obtain the feature importance score.

[0052] Finally, the multimodal knowledge representation vector is screened according to the feature importance score. The preset feature threshold is calculated based on the mean and standard deviation of the feature importance score, for example, the mean plus the standard deviation is used as the threshold. The preset feature threshold is dynamically adjusted in combination with the rate of change of the feature importance score. For example, if the rate of change of the feature importance score is large, it means that the importance of the feature is constantly changing and the threshold needs to be adjusted dynamically. A feature dependency graph is constructed to analyze the correlation between features. For example, if two features often appear at the same time, there is a strong correlation between them. Based on the feature group effect and stability constraint, features with importance scores exceeding the preset feature threshold are selected as influencing features. For example, if a feature group has a great influence on the classification result of the sample, the features in the group are more likely to be selected. Assuming that the preset feature threshold is 0.8 and the importance score of a feature is 0.9, the feature will be selected as an influencing feature. These influencing features are used to guide subsequent feature optimization.

[0053] The solution of this application can: Improve the accuracy of feature screening: By considering the synergy between features, timing information, and modal differences, you can more accurately identify features that have a significant impact on the results and avoid missing key information, thereby improving the accuracy of feature screening. Enhance the interpretability of the model: By constructing a feature dependency graph, you can analyze the correlation between features and better understand the mechanism of how features affect the results, thereby enhancing the interpretability of the model. Optimize model performance: By screening out important influencing features, you can reduce redundant information, reduce computational complexity, and improve the efficiency and performance of the model.

[0054] In an optional implementation, according to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy, adversarial samples are generated based on the error correction strategy for verification, and the reasoning depth is dynamically adjusted according to the verification result through the dual DQN network. Finally, the optimized graph feature is compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed loop including: Extracting gradient information in the feature space according to the optimization objective, calculating local gradients through multi-scale windows and obtaining smooth gradients by using a gradient clipping mechanism and an exponential sliding average; inputting the smooth gradients into an Adam optimizer, which adaptively adjusts the learning rate based on the gradient norm and generates an error correction strategy through a second-order momentum correction; Generate adversarial samples based on the error correction strategy, wherein the adversarial samples ensure validity by perturbation boundary constraints and prevent sample degradation by diversity constraints; input the adversarial samples into a multi-layer verification network for quality assessment to obtain verification results; The verification result is input into a dual DQN network, the dual DQN network constructs a state space based on the current reasoning depth, the verification result and resource occupancy, constructs an action space based on the depth adjustment step size and the jump connection parameter, and dynamically adjusts the reasoning depth through the state space and the action space to obtain an adjusted reasoning depth; The knowledge graph is optimized according to the adjusted inference depth to obtain optimized graph features; the optimized graph features are compared with the graph feature fingerprint, and the KL divergence and topological structure similarity of the feature distribution are calculated as evaluation indicators; the evaluation indicators are fed back to the Adam optimizer for optimization parameter update to form a complete optimization closed loop.

[0055] The goal is to optimize the knowledge graph features to make them closer to the preset graph feature fingerprint. This method uses adversarial training and deep dynamic adjustment strategies, and continuously optimizes through a closed-loop feedback mechanism.

[0056] Feature gradient extraction and smoothing. First, according to the optimization goal, such as improving the performance of the knowledge graph in downstream tasks, the gradient information is extracted in the feature space. In order to obtain more comprehensive gradient information, a multi-scale window is used to calculate the local gradient. Specifically, windows of different sizes are used to slide on the feature map, the gradients in each window are calculated separately, and then these gradients are aggregated. In order to avoid gradient explosion or disappearance, a gradient clipping mechanism is used to limit the gradient amplitude to a certain range. Finally, in order to reduce gradient fluctuations, the exponential sliding average method is used to smooth the gradient to obtain a smooth gradient. For example, the three window sizes are set to 3x3, 5x5 and 7x7, and the local gradients are calculated and weighted averaged respectively, with weights of 0.2, 0.3 and 0.5 respectively. The gradient amplitude is then limited to between [-1, 1]. Finally, an exponential sliding average is used with a decay rate set to 0.9 to obtain a smooth gradient.

[0057] Error correction strategy generation. The smoothed gradient is input into the Adam optimizer. The Adam optimizer adaptively adjusts the learning rate according to the gradient norm and uses the second-order momentum to correct the gradient direction to generate an error correction strategy. For example, the initial learning rate is set to 0.001, the first-order momentum decay rate is set to 0.9, and the second-order momentum decay rate is set to 0.999.

[0058] Adversarial sample generation and verification. Generate adversarial samples based on the error correction strategy. The generation process of adversarial samples needs to meet the perturbation boundary constraint and diversity constraint. The perturbation boundary constraint limits the difference between the adversarial sample and the original sample to ensure the effectiveness of the adversarial sample. The diversity constraint encourages the generation of different adversarial samples to prevent sample degradation. For example, the perturbation boundary constraint is set to 0.1, and the diversity constraint is implemented by adding random noise in the gradient direction. The generated adversarial samples are input into the multi-layer verification network for quality assessment to obtain verification results, such as the change in loss value caused by the adversarial sample.

[0059] The inference depth is adjusted dynamically. The verification results are input into the dual DQN network. The state space of the dual DQN network consists of the current inference depth, verification results, and resource usage. The action space consists of the depth adjustment step size and the skip connection parameters. The dual DQN network selects the best action based on the current state and dynamically adjusts the inference depth. For example, the state space includes the current inference depth (e.g., 5), verification results (e.g., the loss value increases by 0.05), and resource usage (e.g., memory usage). The action space includes the depth adjustment step size (e.g., increase 1 layer or decrease 1 layer) and the skip connection parameters (e.g., whether to enable the skip connection).

[0060] Graph feature optimization and verification. Optimize the knowledge graph according to the adjusted inference depth to obtain the optimized graph features. Compare the optimized graph features with the preset graph feature fingerprint, and calculate the KL divergence and topological structure similarity of the feature distribution as evaluation indicators. For example, the KL divergence between the optimized graph features and the fingerprint features is calculated to be 0.02, and the topological structure similarity is 0.95.

[0061] Closed-loop feedback and optimization. Feedback the evaluation indicators to the Adam optimizer to adjust the optimizer parameters, such as learning rate and momentum, to form a complete closed-loop optimization. For example, if the KL divergence is large, reduce the learning rate; if the topological structure similarity is low, adjust the gradient calculation method.

[0062] The solution of this application can: Improved the quality of the knowledge graph: Through adversarial training and dynamic depth adjustment, the features of the knowledge graph are closer to the preset fingerprint, thereby improving the quality of the knowledge graph and its performance in downstream tasks. Achieved adaptive allocation of resources: The dual DQN network dynamically adjusts the reasoning depth according to the verification results and resource occupancy, achieving adaptive allocation of resources and avoiding resource waste. Constructed a complete optimization closed loop: By feeding the evaluation indicators back to the optimizer, a complete optimization closed loop is formed, achieving continuous optimization and continuously improving the quality of the knowledge graph.

[0063] In an optional implementation, the knowledge graph is optimized according to the adjusted inference depth to obtain optimized graph features; the optimized graph features are compared with the graph feature fingerprint, and the KL divergence and topological structure similarity of the feature distribution are calculated as evaluation indicators; the evaluation indicators are fed back to the Adam optimizer for optimization parameter update, forming a complete optimization closed loop including: Constructing a knowledge graph optimization model according to the adjusted reasoning depth, wherein the knowledge graph optimization model calculates node attention weights and edge relationship attention weights through a hierarchical attention mechanism, and performs feature reorganization on the knowledge graph based on the node attention weights and the edge relationship attention weights to obtain optimized graph features; The optimized graph features are compared with the pre-constructed graph feature fingerprints, and the KL divergence of the feature distribution is calculated using a kernel function estimator, and the kernel function estimator introduces an adaptive smoothing factor to prevent the zero probability problem; a multi-scale structure descriptor is constructed to characterize the graph structure features, and the topological structure similarity is calculated based on the multi-scale structure descriptor; the KL divergence and the topological structure similarity are weightedly fused to obtain an evaluation index; The evaluation index is fed back to the Adam optimizer, and the Adam optimizer uses an adaptive learning rate to update the parameters, and reduces the parameter update fluctuations through gradient accumulation; a convergence criterion is constructed based on the change value of the evaluation index and the change value of the parameter, and whether the optimization has converged is determined according to the convergence criterion; if not, the updated parameters are fed back to the knowledge graph optimization model for the next round of optimization until convergence completes the optimization closed loop.

[0064] First, in the preparation phase, a knowledge graph and the corresponding graph feature fingerprint need to be constructed. The graph feature fingerprint can be understood as a description of the ideal graph feature. It can be pre-defined, extracted from a high-quality knowledge graph, or constructed by experts based on domain knowledge. For example, if the goal is to build a knowledge graph about movies, the graph feature fingerprint can contain information such as movie genre, actors, directors, release year, and the relationship between them. At the same time, the initial reasoning depth needs to be set, for example, set to 3, indicating that the relationship within three hops between nodes in the graph is considered.

[0065] Next, we enter the iterative optimization phase. Based on the current reasoning depth, we build a knowledge graph optimization model. This model uses a hierarchical attention mechanism. Specifically, the model first calculates the attention weight of each node itself, for example, assigning different importance to different nodes based on indicators such as the degree and centrality of the node. Then, the model calculates the attention weight of the edge relationship between nodes, for example, assigning different importance to different edges based on indicators such as the type and strength of the edge. Based on the calculated attention weights of the nodes and edges, the knowledge graph is reorganized to obtain optimized graph features. For example, if there is a "co-director" relationship between movie A and movie B, and the attention weight of this relationship is high, then in the optimized graph features, the feature representations of movie A and movie B will be more similar.

[0066] Then, the optimized graph features are compared with the pre-built graph feature fingerprint. In order to evaluate the difference in feature distribution, the kernel function estimator is used to calculate the KL divergence of feature distribution. The kernel function estimator introduces an adaptive smoothing factor to prevent the zero probability problem, that is, to avoid calculation errors caused by the absence of certain feature values ​​in the sample. For example, if the fingerprint contains a rare movie type, and there is no movie of this type in the current graph, the adaptive smoothing factor can avoid the infinite KL divergence calculation. At the same time, in order to evaluate the similarity of the graph topology, multi-scale structural descriptors are constructed to characterize the graph structure features, such as node degree distribution, clustering coefficient, etc. Based on these descriptors, the topological structure similarity between the optimized graph and the fingerprint graph is calculated. Finally, the KL divergence and topological structure similarity are weighted and fused to obtain a comprehensive evaluation index. The weight can be adjusted according to actual needs.

[0067] The evaluation index is fed back to the Adam optimizer. The Adam optimizer uses an adaptive learning rate to update the model parameters and reduces the fluctuation of parameter updates through gradient accumulation, making the optimization process more stable. Convergence criteria are constructed based on the change value of the evaluation index and the change value of the parameter. For example, a threshold can be set. When the change value of the evaluation index and the change value of the parameter are both less than the threshold, the optimization process is considered to have converged. If it has not converged, the updated parameters are fed back to the knowledge graph optimization model, the reasoning depth is adjusted, for example, the reasoning depth is increased to 4, and the above steps are repeated for the next round of optimization until the convergence conditions are met.

[0068] The solution of this application can: Improve the quality of the knowledge graph. By comparing and optimizing with the preset graph feature fingerprint, the integrity, accuracy and consistency of the knowledge graph can be effectively improved, making it more in line with expectations and better supporting downstream applications. Enhance controllability. By adjusting the reasoning depth and introducing the attention mechanism, the optimization process of the knowledge graph can be more finely controlled, making the optimization results more in line with the needs of specific fields. Achieve automated optimization. This method uses the Adam optimizer and adaptive learning rate to automatically adjust parameters without manual intervention, which improves optimization efficiency and lowers the threshold for use.

[0069] Figure 2 This is a structural diagram of a data knowledge graph construction system driven by a complex ecological intelligent brain according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to collect scene data through a deep feature perception network, obtain an original multidimensional sequence containing business processes, processing rules and asset associations, perform semantic analysis on the original multidimensional sequence to generate an initial knowledge vector, construct an initial graph structure containing entity nodes and relationship edges based on the initial knowledge vector, encode the initial graph structure using a graph neural network, and obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; The second unit is used to calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with correlation higher than a preset similarity threshold from the similarity matrix, perform multi-hop path reasoning on the screened entity pairs to generate a reasoning path set, input the reasoning path set into a recursive neural network to generate an entity association confidence index, and perform multimodal feature extraction on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multimodal features to generate a unified multimodal knowledge representation vector; The third unit is used to input the multimodal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector; based on the multimodal knowledge representation vector, the influencing features are identified by the SHAP method; the influencing features are input into the problem evolution graph to calculate the propagation probability by causal reasoning; the propagation probability is compared and analyzed with the graph feature fingerprint to generate an optimization target; according to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy; based on the error correction strategy, adversarial samples are generated for verification; the reasoning depth is dynamically adjusted according to the verification result through the dual DQN network; finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, so as to form a complete optimization closed loop.

[0070] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0071] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0072] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data knowledge graph construction method driven by a complex ecological intelligent brain, characterized in that: include: The scene data is collected through a deep feature perception network to obtain the original multidimensional sequence containing business processes, processing rules and asset associations, the original multidimensional sequence is semantically parsed to generate an initial knowledge vector, an initial graph structure containing entity nodes and relationship edges is constructed based on the initial knowledge vector, and the initial graph structure is encoded using a graph neural network to obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; Based on the graph feature fingerprint, a similarity matrix between entities in the initial graph structure is calculated, and entity pairs whose correlation is higher than a preset similarity threshold are screened from the similarity matrix using the feature set, multi-hop path reasoning is performed on the screened entity pairs to generate a reasoning path set, and the reasoning path set is input into a recursive neural network to generate an entity association confidence index, and multimodal feature extraction is performed on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and the multimodal features are aligned and fused using a cross-modal attention mechanism to generate a unified multimodal knowledge representation vector; The multimodal knowledge representation vector and the entity association confidence index are input into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector. The influencing features are identified through the SHAP method based on the multimodal knowledge representation vector. The influencing features are input into the problem evolution graph to calculate the propagation probability through causal reasoning. The propagation probability is compared and analyzed with the graph feature fingerprint to generate an optimization target. According to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy. Based on the error correction strategy, an adversarial sample is generated for verification. The reasoning depth is dynamically adjusted according to the verification result through the dual DQN network. Finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed loop.

2. The method according to claim 1, characterized in that Calculating the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and using the feature set to filter entity pairs with correlations higher than a preset similarity threshold from the similarity matrix includes: The graph feature fingerprint is represented as a high-dimensional feature vector, which includes graph structure features, semantic distribution features and topological attribute features, and the knowledge features include an attribute feature matrix of entity nodes; Performing feature transformation on the attribute feature matrix using a linear transformation unit to generate a query matrix, a key matrix and a value matrix, wherein the query matrix is ​​obtained by multiplying a query weight matrix and the attribute feature matrix, the key matrix is ​​obtained by multiplying a key weight matrix and the attribute feature matrix, and the value matrix is ​​obtained by multiplying a value weight matrix and the attribute feature matrix; Divide the query matrix, the key matrix and the value matrix into multiple attention heads, each of the attention heads independently calculates an attention score, for each of the attention heads, obtains an attention weight by multiplying the query matrix by the transpose of the key matrix and normalizing the dimension, multiplies the attention weight by the value matrix to obtain a head output, and concatenates multiple head outputs to form a multi-head attention output; Based on the multi-head attention output, the similarity relationship between the entity pairs is calculated by cosine similarity to generate a similarity matrix, wherein each element of the similarity matrix represents a similarity score of the corresponding entity pair; A dynamic similarity threshold is set according to the distribution characteristics in the graph feature fingerprint, and the dynamic similarity threshold is applied to the similarity matrix for bidirectional verification screening, wherein the bidirectional verification screening requires that both the forward similarity and the reverse similarity of the entity pair satisfy the dynamic similarity threshold; The entity pairs that pass the bidirectional verification are filtered in combination with the semantic constraints in the knowledge features to generate final related entity pairs.

3. The method according to claim 1, characterized in that Based on the entity association confidence index, the data in the initial graph structure and the reasoning path set are respectively extracted multimodal features through the BERT-Large model, the Vision Transformer model and the Wav2Vec model. The multimodal features include text features, image features and audio features, and the multimodal features are aligned and fused using the cross-modal attention mechanism to generate a unified multimodal knowledge representation vector including: The input text data is encoded using the BERT-Large pre-trained model, and contextual semantic features of the text data are extracted through a multi-layer Transformer structure, wherein the contextual semantic features are represented as a text feature matrix, and a text pre-trained feature vector is generated based on the text feature matrix; The Vision Transformer model is used to divide the input image data into a sequence of image blocks of size 16 by 16 pixels. The image block sequence is position-encoded and then input into the self-attention layer to extract the global visual features of the image block sequence. The global visual features are jointly encoded with the text pre-trained feature vector to obtain a joint image-text feature vector. An attention guidance matrix is ​​constructed based on the joint feature vector of the image and text, and the attention guidance matrix is ​​used to process the input audio data. The audio data is input into a Wav2Vec model, and audio features are extracted through a multi-layer convolutional network and a Transformer structure. The audio features are aligned with the joint feature vector of the image and text to generate a multimodal alignment feature. Calculating the inter-modal similarity matrix for the multimodal alignment features, constructing a cross-modal attention network based on the similarity matrix, and outputting an attention weight distribution through the cross-modal attention network; The attention weight distribution is input into a linear projection layer, and the linear projection layer maps the multimodal features into a unified semantic space to generate a unified multimodal knowledge representation vector.

4. The method according to claim 1, characterized in that: Based on the multimodal knowledge representation vector, the influencing features are identified by the SHAP method, the influencing features are input into the problem evolution graph, the propagation probability is calculated by causal reasoning, and the propagation probability is compared and analyzed with the graph feature fingerprint to generate the optimization target, including: The multimodal knowledge representation vector is input into an improved SHAP model, and the improved SHAP model calculates the synergistic effect between features based on a feature combination evaluation mechanism to obtain a feature importance score; the multimodal knowledge representation vector is screened according to the feature importance score, and features whose importance scores exceed a preset feature threshold are selected as influencing features; The influencing features are used as nodes to construct a problem evolution graph, the node attributes of the problem evolution graph include feature values ​​and importance scores, and directed edges are constructed based on the temporal relationship between features; the problem evolution graph is input into a causal reasoning network, the causal reasoning network calculates the conditional probabilities between nodes through a variational reasoning method, and uses Monte Carlo sampling to obtain the probability distribution of the propagation path; The probability distribution of the propagation path is compared with the pre-stored graph feature fingerprint, and the state difference distribution is obtained by calculating the KL divergence; an optimization objective function is constructed based on the state difference distribution, and the optimization objective function includes a distribution difference term and an integrity constraint term, and the optimization target of the knowledge graph is obtained by weighted summation.

5. The method according to claim 4, characterized in that Inputting the multimodal knowledge representation vector into an improved SHAP model, wherein the improved SHAP model calculates the synergistic effect between features based on a feature combination evaluation mechanism to obtain a feature importance score; Screening the multimodal knowledge representation vector according to the feature importance score, and selecting features whose importance scores exceed a preset feature threshold as influencing features includes: The multimodal knowledge representation vector is input into an improved SHAP model, wherein the improved SHAP model uses a sliding window to generate a feature combination pool, and calculates feature combination weights based on the mutual information, correlation, and temporal correlation of features in the feature combination pool; the improved SHAP model uses a nonlinear mapping layer to map the feature combination weights, and calculates the synergistic effect between features based on a feature combination evaluation mechanism; A feature contribution matrix is ​​constructed based on the synergistic effect, wherein the matrix elements of the feature contribution matrix represent the degree of contribution of the feature to the sample; a temporal contribution is obtained by performing temporal weighting on the feature contribution matrix using a temporal attenuation factor, and an initial SHAP value is calculated based on the temporal contribution; a modal adaptive weight is calculated according to the variance of different modal features, and the initial SHAP value is weightedly fused using the modal adaptive weight to obtain a feature importance score; The multimodal knowledge representation vector is screened according to the feature importance score, a preset feature threshold is calculated based on the mean and standard deviation of the feature importance score, and the preset feature threshold is dynamically adjusted in combination with the rate of change of the feature importance score; a feature dependency graph is constructed to analyze the correlation between features, and features whose importance scores exceed the preset feature threshold are selected as influencing features based on feature group effects and stability constraints, and the influencing features are used to guide subsequent feature optimization.

6. The method according to claim 1, characterized in that According to the optimization goal, the gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy. Based on the error correction strategy, adversarial samples are generated for verification. The reasoning depth is dynamically adjusted according to the verification results through the dual DQN network. Finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, forming a complete optimization closed loop including: Extracting gradient information in the feature space according to the optimization objective, calculating local gradients through multi-scale windows and obtaining smooth gradients by using a gradient clipping mechanism and an exponential sliding average; inputting the smooth gradients into an Adam optimizer, which adaptively adjusts the learning rate based on the gradient norm and generates an error correction strategy through a second-order momentum correction; Generate adversarial samples based on the error correction strategy, wherein the adversarial samples ensure validity by perturbation boundary constraints and prevent sample degradation by diversity constraints; input the adversarial samples into a multi-layer verification network for quality assessment to obtain verification results; The verification result is input into a dual DQN network, the dual DQN network constructs a state space based on the current reasoning depth, the verification result and resource occupancy, constructs an action space based on the depth adjustment step size and the jump connection parameter, and dynamically adjusts the reasoning depth through the state space and the action space to obtain an adjusted reasoning depth; The knowledge graph is optimized according to the adjusted inference depth to obtain optimized graph features; the optimized graph features are compared with the graph feature fingerprint, and the KL divergence and topological structure similarity of the feature distribution are calculated as evaluation indicators; the evaluation indicators are fed back to the Adam optimizer for optimization parameter update to form a complete optimization closed loop.

7. The method according to claim 6, characterized in that Optimizing the knowledge graph according to the adjusted inference depth to obtain optimized graph features; comparing the optimized graph features with the graph feature fingerprint, and calculating the KL divergence and topological structure similarity of feature distribution as evaluation indicators; Feeding the evaluation index back to the Adam optimizer for optimizing parameter updates, forming a complete optimization closed loop including: Constructing a knowledge graph optimization model according to the adjusted reasoning depth, wherein the knowledge graph optimization model calculates node attention weights and edge relationship attention weights through a hierarchical attention mechanism, and performs feature reorganization on the knowledge graph based on the node attention weights and the edge relationship attention weights to obtain optimized graph features; The optimized graph features are compared with the pre-constructed graph feature fingerprints, and the KL divergence of the feature distribution is calculated using a kernel function estimator, and the kernel function estimator introduces an adaptive smoothing factor to prevent the zero probability problem; a multi-scale structure descriptor is constructed to characterize the graph structure features, and the topological structure similarity is calculated based on the multi-scale structure descriptor; the KL divergence and the topological structure similarity are weightedly fused to obtain an evaluation index; The evaluation index is fed back to the Adam optimizer, and the Adam optimizer uses an adaptive learning rate to update the parameters, and reduces the parameter update fluctuations through gradient accumulation; a convergence criterion is constructed based on the change value of the evaluation index and the change value of the parameter, and whether the optimization has converged is determined according to the convergence criterion; if not, the updated parameters are fed back to the knowledge graph optimization model for the next round of optimization until convergence completes the optimization closed loop.

8. A data knowledge graph construction system driven by a complex ecological intelligent brain, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect scene data through a deep feature perception network, obtain an original multidimensional sequence containing business processes, processing rules and asset associations, perform semantic analysis on the original multidimensional sequence to generate an initial knowledge vector, construct an initial graph structure containing entity nodes and relationship edges based on the initial knowledge vector, encode the initial graph structure using a graph neural network, and obtain a graph feature fingerprint for feature distribution representation and a feature set for knowledge discovery; The second unit is used to calculate the similarity matrix between entities in the initial graph structure based on the graph feature fingerprint, and use the feature set to screen entity pairs with correlation higher than a preset similarity threshold from the similarity matrix, perform multi-hop path reasoning on the screened entity pairs to generate a reasoning path set, input the reasoning path set into a recursive neural network to generate an entity association confidence index, and perform multimodal feature extraction on the data in the initial graph structure and the reasoning path set based on the entity association confidence index, wherein the multimodal features include text features, image features, and audio features, and use a cross-modal attention mechanism to align and fuse the multimodal features to generate a unified multimodal knowledge representation vector; The third unit is used to input the multimodal knowledge representation vector and the entity association confidence index into a multi-layer feature network to generate a multimodal knowledge representation vector, wherein the multimodal knowledge representation vector includes an entity feature vector, a relationship feature vector and a subgraph feature vector; based on the multimodal knowledge representation vector, the influencing features are identified by the SHAP method; the influencing features are input into the problem evolution graph to calculate the propagation probability by causal reasoning; the propagation probability is compared and analyzed with the graph feature fingerprint to generate an optimization target; according to the optimization target, gradient information is extracted in the feature space and input into the Adam optimizer to generate an error correction strategy; based on the error correction strategy, adversarial samples are generated for verification; the reasoning depth is dynamically adjusted according to the verification result through the dual DQN network; finally, the optimized graph features are compared with the graph feature fingerprint to verify the optimization effect, so as to form a complete optimization closed loop.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images