Data intelligent association analysis and application method based on knowledge graph

Abstract

The invention discloses a data intelligent association analysis and application method based on a knowledge graph, and the method comprises the steps: S1, collecting and preprocessing multi-source heterogeneous data, and obtaining the preprocessed text data; s2, extracting entities and relationships from the preprocessed text data, and constructing a preliminary knowledge graph; s3, storing the knowledge graph by adopting a graph database; s4, expanding and supplementing the knowledge graph through an external data source; s5, performing data association analysis based on the knowledge graph, and identifying potential modes and association relationships in the data; s6, abnormal data in the knowledge graph are detected, and potential abnormal modes are found and marked; and S7, applying results of data association analysis and abnormal data detection to an actual scene. According to the method, an efficient and scientific optimization scheme can be provided in data intelligent correlation analysis and application design, and remarkable technical values and economic benefits are brought to practical application.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a data intelligent association analysis and application method based on a knowledge graph. Background Art

[0002] In the context of the rapid development of big data and artificial intelligence technology, knowledge graphs have become an important means of intelligent data analysis. By building a structured knowledge network, knowledge graphs can effectively represent entities and their relationships, thereby supporting tasks such as data integration, semantic reasoning, and intelligent decision-making. Existing knowledge graph technology has been widely used in the fields of medicine, finance, and recommendation systems.

[0003] However, there are still many technical bottlenecks in the existing knowledge graph construction and analysis methods. First, in data collection and preprocessing, traditional methods have difficulty processing multi-source heterogeneous data, and there are challenges in the fusion of data in different formats and structures. In particular, in the absence of unified standards, data consistency and integrity are difficult to ensure. Secondly, in the process of building a knowledge graph, the accuracy of entity recognition and relationship extraction is limited by data quality and the generalization ability of the training model. Traditional natural language processing methods have difficulty dealing with complex text structures, resulting in noise and information loss in the knowledge extraction process. In addition, knowledge graphs are usually stored in traditional graph databases, but the existing storage architecture has low retrieval efficiency when facing large-scale data, especially in real-time query and dynamic update scenarios. Query latency and storage optimization have become key issues restricting system performance.

[0004] In terms of knowledge expansion and knowledge fusion, existing methods mainly rely on external knowledge bases or rule-based knowledge alignment technology. However, artificially constructed knowledge alignment rules are difficult to adapt to the rapidly changing data environment, resulting in delayed knowledge updates and failure to meet the needs of highly dynamic scenarios. At the same time, in the process of data association analysis, traditional methods mostly use simple statistical associations or path search-based methods, lacking deep semantic reasoning capabilities, making it difficult to mine hidden knowledge relationships in complex scenarios, limiting the application effect of knowledge graphs in intelligent analysis tasks.

[0005] Anomaly detection and reasoning capabilities are also the shortcomings of existing knowledge graph analysis technologies. Traditional anomaly detection methods are mainly based on setting thresholds based on rules, or using traditional machine learning models for classification. These methods are difficult to adapt to abnormal patterns in large-scale complex knowledge graphs, especially in the presence of data noise and uncertainty, and the reliability of detection results is low. In addition, knowledge reasoning mostly relies on traditional logical reasoning or embedding-based graph neural network methods, but existing reasoning models have limited understanding of complex contexts and are difficult to support multi-hop reasoning and causal inference, resulting in insufficient reasoning accuracy.

[0006] Based on the defects of the above-mentioned prior art, the present invention proposes a data intelligent association analysis and application method based on knowledge graph, which improves the intelligence level of knowledge graph by optimizing key links such as data preprocessing, knowledge construction, storage management, association analysis, anomaly detection and reasoning optimization. This method can not only efficiently integrate multi-source heterogeneous data and improve the accuracy and automation of knowledge graph construction, but also optimize knowledge expansion, data association analysis and anomaly detection through graph neural network, causal reasoning and reinforcement learning, improve the intelligent analysis ability of the system, and make up for the shortcomings of prior art in dynamic knowledge update, anomaly detection accuracy and intelligent reasoning ability. Summary of the invention

[0007] One purpose of the present invention is to propose a data intelligent association analysis and application method based on knowledge graph. The present invention can provide an efficient and scientific optimization solution in data intelligent association analysis and application, bringing significant technical value and economic benefits to practical applications.

[0008] According to an embodiment of the present invention, a data intelligent association analysis and application method based on a knowledge graph includes the following steps:

[0009] S1, collect multi-source heterogeneous data and preprocess them to obtain preprocessed text data;

[0010] S2. Extract entities and relationships from the preprocessed text data, construct a preliminary knowledge graph, and optimize the constructed knowledge graph using causal reasoning methods and variational autoencoders;

[0011] S3, using graph database to store knowledge graph;

[0012] S4. Expand and supplement the knowledge graph through external data sources;

[0013] S5. Perform data association analysis based on knowledge graph to identify potential patterns and associations in the data;

[0014] S6. According to the results of data association analysis, a graph-based relationship scoring method is used to detect abnormal data in the knowledge graph, discover potential abnormal patterns and mark them.

[0015] Optionally, the S1 includes the following steps:

[0016] S11. Acquire multi-source heterogeneous data, including structured data, semi-structured data and unstructured data. Structured data includes tabular data in relational databases, semi-structured data includes data in JSON, XML and other formats, and unstructured data includes text, image, audio and video data.

[0017] S12. Perform format standardization processing on multi-source heterogeneous data:

[0018] For structured data, maintain field consistency;

[0019] For semi-structured data, perform data parsing and field mapping;

[0020] For unstructured data, use methods such as text vectorization, image feature extraction, and audio spectrum analysis for structured conversion;

[0021] S13. Perform data cleaning on the text data after format standardization to remove duplicate data and outliers;

[0022] S14. Perform preprocessing on the text data, including word segmentation, stop word removal, lemmatization, and entity recognition. Among them, the maximum matching algorithm is used for text word segmentation, and after word segmentation, a text sequence {w 1 , w 2 ,..., w n} is formed, where w i represents the i-th word segmentation unit;

[0023] S15. Store the preprocessed text data.

[0024] Optionally, S2 includes the following steps:

[0025] S21. Perform entity recognition on the preprocessed text data. Use a pre-trained language model with self-supervised learning combined with a named entity recognition method to extract an entity set E = {e 1 , e 2 ,..., e n} from the text sequence {w 1 , e 2 ,..., e m}, where e i represents the i-th recognized entity, and optimize the accuracy of entity recognition through a context relevance scoring mechanism;

[0026] S22. Perform entity disambiguation and normalization processing on the entity set E, and calculate entity similarity using an embedding matching method based on a graph neural network;

[0027] S23. Extract the relationships between entities from the preprocessed text data, and combine dependency syntax analysis, multi-hop relationship reasoning, and knowledge embedding methods to construct a relationship set R = {r 1 , r 2 ,..., r k}, where r k represents the k-th recognized relationship, and identify implicit relationships through cross-document reasoning;

[0028] S24. Generate a preliminary knowledge graph G = (V, E) based on the relational triple (e i , r k , e j ), where V = {e 1 , e 2 ,..., e m} represents the entity nodes in the knowledge graph, and E = {(e i , r k , e j ) | e i , e j ∈ V} represents the relational edges in the knowledge graph, and introduce a dynamic attention mechanism to optimize the correlation between entities and relations;

[0029] S25. For the preliminarily constructed knowledge graph, adopt a knowledge enhancement method based on causal reasoning for optimization, use a relational correction mechanism based on reinforcement learning to adjust low-confidence relational triples, and compress the knowledge representation through a variational autoencoder;

[0030] S26. Perform incremental update on the optimized knowledge graph, adopt a dynamic update strategy based on temporal knowledge embedding, automatically identify new entities and relations, and fuse new knowledge into the original graph through knowledge alignment technology.

[0031] Optionally, S25 includes the following steps:

[0032] S251. Conduct causal relationship modeling on the preliminarily constructed knowledge graph G = (V, E), and define a causal influence measurement function I(e i , r k , e j ):

[0033] I(e i , r k , e j ) = P(e j | e i , r k ) - P(e j | do(e i , r k ));

[0034] Among them, P(e j | e i , r k ) represents the probability that entity e i affects entity e k through relation r j under the condition of the existing knowledge graph, and P(e j | do(e i , r k)) represents that after applying external intervention, e j At e i and r k acting on the independent probability, the causal inference method is used to preliminarily screen the effectiveness of the knowledge graph relationships;

[0035] S252. Relationship correction mechanism based on reinforcement learning, define the reinforcement learning environment;

[0036] State space S: defined as the current knowledge graph structure G t =(V t , E t ), where V t is the current entity set of the knowledge graph, and E t is the current relationship set of the knowledge graph;

[0037] Action space A: includes relationship change operations A={A add , A del , A mod}, where A add is to add a new relationship, A del is to delete low-confidence relationships, and A mod is to modify the relationship category;

[0038] Reward function R: Calculate the consistency score F(G t+1 ) for the optimized knowledge graph G t+1 , and define the reward function:

[0039] R = F(G t+1 ) - F(G t );

[0040] Among them, F(G t ) is weighted and calculated by the relationship confidence, entity connectivity, and logical consistency metric functions. With the goal of maximizing R, optimize the knowledge graph relationships;

[0041] S253. Use variational autoencoders for knowledge graph embedding optimization. For each entity e i , perform low-dimensional embedding representation z i , and define the objective function of the variational autoencoder:

[0042] L VAE = E q(z|e) [logp(e|z)] - D KL (q(z|e)||p(z));

[0043] Among them, q(z|e) is the posterior distribution, p(z) is the prior distribution, and D KL is the KL divergence. By minimizing L VAETrain an autoencoder to improve the compactness and distinctiveness of entity embedding representations;

[0044] S254. Perform relation pruning and relation completion based on an entropy constraint optimization strategy, and calculate the confidence C(r k ) of each relation in the knowledge graph:

[0045] C(r k ) = -∑P(r k |e i , e j ) logP(r k |e i , e j );

[0046] Prune the relations with confidence lower than the set threshold to form a new knowledge graph structure G' = (V, E');

[0047] For the missing relations, use a relation prediction model R(e i , e j ) = f(e i , e j , θ) for completion, where f(e i , e j , θ) is the relation prediction function, θ is the model parameter, and use the gradient descent method to optimize θ to maximize the prediction probability of known relations;

[0048] S255. Perform consistency verification on the optimized knowledge graph, and perform constraint reasoning based on logical constraint rules to calculate the set E inconsistent of relations that violate the constraint rules. If , then perform the minimum modification operation:

[0049] ΔG = argmin ΔG ||G + ΔG||;

[0050] where ΔG is the minimum modification to the knowledge graph G, and replace or delete the least number of edges to correct inconsistent relations.

[0051] Optionally, the S4 includes the following steps:

[0052] S41. Expand the knowledge graph based on external data sources, screen the external data source set D = {D 1 , D 2 ,..., D n}, and extract the entity set V' = {e 1 ', e 2 ',..., em '} and the relationship set E' = {(e i ', r k ', e j ')}, where both V' and E' are from the external data source D;

[0053] S42. Align the extracted external entities V', and calculate the entity matching degree Sim(e i , e j ) using a matching method based on text similarity and entity embedding:

[0054] Sim(e i , e j ) = α · Sim text (e i , e j ) + β · Sim vec (e i , e j );

[0055] Among them, Sim text (e i , e j ) calculates the text similarity through the edit distance, and Sim vec (e i , e j ) calculates the similarity of entity embeddings using the vector cosine similarity. α and β are weight coefficients;

[0056] S43. Verify the extracted external relationship E', and calculate the matching degree Match(r k ', r k ) of the external relationship r k , r k ) using a relationship prediction model based on the graph attention network:

[0057] Match(r k , r k ) = softmax(W · [h(e i ) || h(e j )]);

[0058] Among them, h(e i ), h(e j ) are the graph neural network embeddings of entities e i , e j . W is a training parameter. If Match(r k , r k ) is greater than the set value λ, then determine that r k ' is a valid relationship and add it to the knowledge graph E;

[0059] S44. Merge external data using a confidence-based knowledge fusion strategy, and define the credibility score C(D i ) of the external data:

[0060]

[0061] where Trust(e j ) is weighted and calculated by source weight, data consistency, and entity matching degree. Filter the external data with a credibility lower than the threshold θ, and only retain high-confidence knowledge;

[0062] S45. Perform a consistency check on the extended knowledge graph G' = (V ∪ V', E ∪ E');

[0063] S46. Store the extended knowledge graph G' and update the index.

[0064] Optionally, the above S5 includes the following steps:

[0065] S51. Perform data association analysis based on the extended knowledge graph G', and define the entity association degree calculation function Rel(e i , e j ) as follows:

[0066]

[0067] where P(r k |e i , e j ) represents the probability that entities e i and e j are associated through relationship r k . Filter low-correlation entity pairs based on the association degree threshold τ r , and only retain the high-correlation entity set V r ;

[0068] S52. Perform clustering analysis on the entity set V r , and use the density-based DBSCAN clustering method;

[0069] Based on the clustering results, calculate the central node c k of each entity cluster as follows:

[0070]

[0071] where V k represents the k-th entity cluster, and c k is used as the representative entity of this cluster;

[0072] S53. Perform association rule mining, define the frequent relationship pattern FPM(R), and use the Apriori algorithm. Based on the relationship support degree Support(r k ) to calculate the frequent relationship set:

[0073]

[0074] If Support(r k ) > σ s , then determine that r k is a high-frequency relationship, add it to the frequent relationship set R f , and further generate the association rule R';

[0075] S54. Based on the association analysis result, construct the inference graph G inf = (V', E inf ), where E inf consists of high-confidence relationships. Define the inference credibility Conf(r k , e i , e j ) as:

[0076] Conf(r k , e i , e j ) = P(r k |e i , e j )·Trust(e i )·Trust(e j );

[0077] Among them, Trust(e i ) and Trust(e j ) are calculated from the knowledge source confidence. If Conf(r k , e i , e j ) > γ, then add (e i , r k , e j ) to the inference graph;

[0078] S55. Store the optimized knowledge association analysis result and provide the query interface Q(V', R f , G inf ).

[0079] Optionally, the S6 includes the following steps:

[0080] S61. Based on the knowledge association analysis result, for the knowledge graph G inf = (V', E inf) Perform anomaly detection and define the anomaly scoring function Anomaly(e i ):

[0081]

[0082] Where N(e i ) is the set of neighbor entities of entity e i , Sim(e i , e j ) is the entity similarity. If Anomaly(e i ) > τ a , then e i is classified into the abnormal entity set V a ;

[0083] S62. Detect anomalies in the relationship E inf in the knowledge graph. Adopt a graph structure-based relationship scoring method to calculate the relationship confidence Conf(r k , e i , e j ):

[0084] Conf(r k , e i , e j ) = P(r k |e i , e j )·Trust(e i )·Trust(e j );

[0085] Where P(r k |e i , e j ) is the knowledge graph inference probability, Trust(e i ), Trust(e j ) are entity credibility. If Conf(r k , e i , e j ) < θ c , then (e i , r k , e j ) is classified into the abnormal relationship set E a ;

[0086] S63. Based on time series analysis of abnormal change trends, construct the time-dependent relationship T(e i , t) = (e i , r k , e j , t), and calculate the time series anomaly score TS(e i , t):

[0087]

[0088] Among them, f t (e i ) represents the entity behavior feature at time t, μ(f(e i )) is the behavior mean of the entity, and σ(f(e i )) is the standard deviation. If TS(e i , t) > δ t , then it is determined that e i has an abnormal behavior at time t;

[0089] S64. Store the detected abnormal entity V a and abnormal relationship E a , and generate an abnormal knowledge graph G a = (V a , E a ), and provide an abnormal query interface Q(V a , E a , T).

[0090] The beneficial effects of the present invention are as follows:

[0091] (1) A data intelligent association analysis and application method based on a knowledge graph proposed by the present invention overcomes the limitations of the prior art in processing multi-source heterogeneous data, knowledge fusion, real-time update, anomaly detection, and inference optimization by optimizing key technical links such as data collection, knowledge construction, storage management, data analysis, and intelligent reasoning, and improves the application ability of the knowledge graph in a complex data environment. This method can establish a unified data representation framework between different data sources to ensure the standardization and consistency of data formats, thereby improving the accuracy of knowledge extraction and fusion and effectively solving the problem of knowledge fragmentation caused by data inconsistency or information loss in the process of constructing the existing knowledge graph.

[0092] (2) The present invention adopts an optimization method based on entity matching, relationship prediction, and knowledge completion to improve the integrity and coherence of knowledge representation. At the same time, by constructing an optimized graph database storage structure and introducing an efficient indexing mechanism, the access efficiency of large-scale knowledge graphs is improved, supporting real-time query and dynamic update of knowledge, so that the system can still maintain high response ability in the face of massive data. This storage optimization method not only solves the problem of high latency in retrieving traditional knowledge graphs but also ensures that complex query tasks can be quickly executed in a large-scale data environment.

[0093] (3) The present invention adopts an intelligent reasoning method based on multi-hop reasoning and causal analysis, enabling the system to not only mine the direct relationships between entities but also discover potential high-order associations through multi-step reasoning, enhancing the depth and breadth of knowledge reasoning. In addition, by optimizing the reasoning path selection through reinforcement learning, the credibility of the reasoning results is improved, enabling the knowledge graph to provide more accurate intelligent analysis capabilities in application scenarios such as decision support, intelligent recommendation, and risk warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0095] Figure 1 A method for data intelligent association analysis and application based on a knowledge graph proposed by the present invention;

[0096] Figure 2 A flowchart for optimizing the knowledge graph in a method for data intelligent association analysis and application based on a knowledge graph proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0097] The present invention will now be further described in detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, only showing the basic structure of the present invention in a schematic manner, and therefore only showing the components related to the present invention.

[0098] Referring to Figure 1-Figure 2 , a method for data intelligent association analysis and application based on a knowledge graph includes the following steps:

[0099] S1. Collect multi-source heterogeneous data and perform preprocessing to obtain preprocessed text data;

[0100] S2. Extract entities and relationships from the preprocessed text data, construct a preliminary knowledge graph, and optimize the constructed knowledge graph using a causal reasoning method and a variational autoencoder;

[0101] S3. Store the knowledge graph using a graph database;

[0102] S4. Expand and supplement the knowledge graph through external data sources;

[0103] S5. Perform data association analysis based on the knowledge graph to identify potential patterns and association relationships in the data;

[0104] S6. Detect abnormal data in the knowledge graph using a relationship scoring method based on a graph structure according to the data association analysis results, discover potential abnormal patterns and mark them.

[0105] In this embodiment, S1 includes the following steps:

[0106] S11. Obtain multi-source heterogeneous data, including structured data, semi-structured data, and unstructured data. The structured data includes tabular data in a relational database, the semi-structured data includes data in formats such as JSON and XML, and the unstructured data includes text, image, audio, and video data;

[0107] S12. Perform format standardization processing on the multi-source heterogeneous data:

[0108] For structured data, maintain field consistency;

[0109] For semi-structured data, perform data parsing and field mapping;

[0110] For unstructured data, use methods such as text vectorization, image feature extraction, and audio spectrum analysis for structured conversion;

[0111] S13. Clean the text data after format standardization to remove duplicate data and outliers;

[0112] S14. Preprocess the text data, including word segmentation, stop word removal, lemmatization, and entity recognition. Among them, text word segmentation uses the maximum matching algorithm, and after word segmentation, a text sequence {w 1 , w 2 ,..., w n} is formed, where w i represents the i-th word segmentation unit;

[0113] S15. Store the preprocessed text data.

[0114] In this embodiment, S2 includes the following steps:

[0115] S21. Perform entity recognition on the preprocessed text data. Use a pre-trained language model with self-supervised learning combined with a named entity recognition method to extract an entity set E = {e 1 , e 2 ,..., e n} from the text sequence {w 1 , w 2 ,..., w n}, where e 1 , e 2 ,..., e m represents the i-th recognized entity, and optimize the accuracy of entity recognition through a context relevance scoring mechanism; i

[0116] S22. Perform entity disambiguation and normalization processing on the entity set E, and calculate entity similarity using an embedding matching method based on a graph neural network;

[0117] S23. Extract the relationships between entities from the preprocessed text data, and construct a relationship set \(R = \{r 1 , r 2 , \cdots, r k \}\) by combining dependency syntactic analysis, multi-hop relationship reasoning, and knowledge embedding methods, where \(r k \) represents the \(k\)-th recognized relationship, and identify implicit relationships through cross-document reasoning;

[0118] S24. Generate a preliminary knowledge graph \(G=(V, E)\) based on the relational triple \((e i , r k , e j )\), where \(V = \{e 1 , e 2 , \cdots, e m \}\) represents the entity nodes in the knowledge graph, and \(E=\{(e i , r k , e j )|e i , e j \in V\}\) represents the relationship edges in the knowledge graph, and introduce a dynamic attention mechanism to optimize the correlation between entities and relationships;

[0119] S25. For the preliminarily constructed knowledge graph, adopt a knowledge enhancement method based on causal reasoning for optimization, use a relationship correction mechanism based on reinforcement learning to adjust low-confidence relational triples, and compress the knowledge representation through a variational autoencoder;

[0120] S26. Perform incremental update on the optimized knowledge graph, adopt a dynamic update strategy based on temporal knowledge embedding, automatically identify new entities and relationships, and fuse new knowledge into the original graph through knowledge alignment technology.

[0121] In this embodiment, S25 includes the following steps:

[0122] S251. Conduct causal relationship modeling on the preliminarily constructed knowledge graph \(G=(V, E)\), and define a causal influence measurement function \(I(e i , r k , e j )\):

[0123] I(e i , r k , e j ) = P(e j |e i , r k ) - P(e j |do(e i , r k ));

[0124] Among them, P(e j |e i ,r k ) represents the probability that entity e i influences entity e k through relationship r j under the condition of the existing knowledge graph. P(e j |do(e i ,r k )) represents the independent probability of e j under the action of e i and r k after applying external intervention. The causal inference method is used to preliminarily screen the effectiveness of the knowledge graph relationships;

[0125] S252. Relationship correction mechanism based on reinforcement learning, define the reinforcement learning environment;

[0126] State space S: It is defined as the current knowledge graph structure G t =(V t ,E t ), where V t is the current entity set of the knowledge graph, and E t is the current relationship set of the knowledge graph;

[0127] Action space A: It includes relationship change operations A={A add ,A del ,A mod}, where A add is to add a new relationship, A del is to delete low-confidence relationships, and A mod is to modify the relationship category;

[0128] Reward function R: Calculate the consistency score F(G t+1 ) for the optimized knowledge graph G t+1 and define the reward function:

[0129] R = F(G t+1 ) - F(G t );

[0130] Among them, F(G t ) is weighted and calculated by the relationship confidence, entity connectivity, and logical consistency metric functions. With the goal of maximizing R, optimize the knowledge graph relationships;

[0131] S253. Use variational autoencoders to optimize knowledge graph embedding, perform low-dimensional embedding representation z i for each entity e i , and define the objective function of the variational autoencoder:

[0132] LVAE = E q(z|e) [logp(e|z)] - D KL (q(z|e) || p(z));

[0133] where q(z|e) is the posterior distribution, p(z) is the prior distribution, and D KL is the KL divergence. By minimizing L VAE the autoencoder is trained to improve the compactness and distinguishability of the entity embedding representation;

[0134] S254. Based on the entropy constraint optimization strategy, perform relationship pruning and relationship completion, and calculate the confidence C(r k ):

[0135] C(r k ) = -∑P(r k |e i , e j ) logP(r k |e i , e j );

[0136] Prune the relationships with confidence lower than the set threshold to form a new knowledge graph structure G' = (V, E');

[0137] For the missing relationships, use the relationship prediction model R(e i , e j ) = f(e i , e j , θ) for completion, where f(e i , e j , θ) is the relationship prediction function, θ is the model parameter, and use the gradient descent method to optimize θ to maximize the prediction probability of the known relationships;

[0138] S255. Perform consistency verification on the optimized knowledge graph, and perform constraint reasoning based on the logical constraint rules to calculate the set of relationships E inconsistent that violate the constraint rules. If then perform the minimum modification operation:

[0139] ΔG = argmin ΔG ||G + ΔG||;

[0140] where ΔG is the minimum modification to the knowledge graph G, and the inconsistent relationships are corrected by replacing or deleting the fewest edges.

[0141] In this embodiment, S4 includes the following steps:

[0142] S41. Expand the knowledge graph based on external data sources, and filter the external data source set D = {D 1 , D 2 ,..., D n} to extract the entity set V' = {e 1 ', e 2 ',..., e m '} and the relationship set E' = {(e i ', r k ', e j ')} that are related to the existing knowledge graph G = (V, E), where both V' and E' are from the external data source D;

[0143] S42. Align the extracted external entities V', and calculate the entity matching degree Sim(e i , e j ') using a matching method based on text similarity and entity embedding:

[0144] Sim(e i , e j ') = α·Sim text (e i , e j ') + β·Sim vec (e i , e j ');

[0145] Among them, Sim text (e i , e j ') calculates the text similarity through the edit distance, and Sim vec (e i , e j ') calculates the similarity of entity embeddings using the vector cosine similarity. α and β are weight coefficients;

[0146] S43. Verify the extracted external relationships E', and calculate the matching degree Match(r k ', r k ) of the external relationship r k with the existing relationship r k ) using a relationship prediction model based on a graph attention network:

[0147] Match(r k , r k ) = softmax(W·[h(e i ) || h(e j )]);

[0148] Among them, h(e i ), h(ej ) is the graph neural network embedding of entity e i , e j . W is a training parameter. If Match(r k , r k ') is greater than the set value λ, then it is determined that r k ' is a valid relationship and is added to the knowledge graph E;

[0149] S44. Adopt a confidence-based knowledge fusion strategy to merge external data, and define the credibility score C(D i ) of external data:

[0150]

[0151] Among them, Trust(e j ) is weighted and calculated by source weight, data consistency, and entity matching degree. External data with a credibility lower than the threshold θ is filtered, and only high-confidence knowledge is retained;

[0152] S45. For the extended knowledge graph G'=(V∪V', E∪E'), perform a consistency check;

[0153] S46. Store the extended knowledge graph G' and update the index.

[0154] In this embodiment, S5 includes the following steps:

[0155] S51. Perform data association analysis based on the extended knowledge graph G', and define the entity association degree calculation function Rel(e i , e j ):

[0156]

[0157] Among them, P(r k |e i , e j ) represents the probability that entities e i and e j are associated through the relationship r k . Based on the association degree threshold τ r , filter low-correlation entity pairs, and only retain the high-correlation entity set V r ;

[0158] S52. Perform clustering analysis on the entity set V r , and adopt the density-based DBSCAN clustering method;

[0159] Based on the clustering result, calculate the central node c k of each entity cluster:

[0160]

[0161] Among them, V k represents the k-th entity cluster, and c k serves as the representative entity of this cluster;

[0162] S53. Perform association rule mining, define the frequent relationship pattern FPM(R), and use the Apriori algorithm to calculate the frequent relationship set based on the relationship support degree Support(r k ):

[0163]

[0164] If Support(r k ) > σ s , then determine that r k is a high-frequency relationship, add it to the frequent relationship set R f , and further generate the association rule R';

[0165] S54. Based on the association analysis results, construct the inference graph G inf =(V', E inf ), where E inf is composed of high-confidence relationships. Define the inference credibility Conf(r k , e i , e j ) as:

[0166] Conf(r k , e i , e j ) = P(r k |e i , e j )·Trust(e i ) · Trust(e j );

[0167] Among them, Trust(e i ) and Trust(e j ) are calculated from the confidence of the knowledge source. If Conf(r k , e i , e j ) > γ, then add (e i , r k , e j ) to the inference graph;

[0168] S55. Store the optimized knowledge association analysis results and provide the query interface Q(V', R f , G inf ).

[0169] In this embodiment, S6 includes the following steps:

[0170] S61. Based on the knowledge association analysis result, perform anomaly detection on the knowledge graph G inf =(V', E inf ) and define an anomaly scoring function Anomaly(e i ):

[0171]

[0172] where N(e i ) is the set of neighbor entities of entity e i , Sim(e i , e j ) is the entity similarity. If Anomaly(e i )>τ a , then classify e i into the abnormal entity set V a ;

[0173] S62. Perform anomaly detection on the relationships E inf in the knowledge graph. Adopt a graph-structure-based relationship scoring method to calculate the relationship confidence Conf(r k , e i , e j ):

[0174] Conf(r k , e i , e j ) = P(r k |e i , e j )·Trust(e i )·Trust(e j );

[0175] where P(r k |e i , e j ) is the knowledge graph inference probability, Trust(e i ), Trust(e j ) are entity credibility. If Conf(r k , e i , e j )<θ c , then classify (e i , r k , e j ) into the abnormal relationship set E a ;

[0176] S63. Based on the abnormal change trend analyzed by time series, construct a time-dependent relationship T(e i ,t) = (e i ,r k ,e j ,t), and calculate the time series anomaly score TS(e i ,t):

[0177]

[0178] Among them, f t (e i ) represents the entity behavior characteristics at time t, μ(f(e i )) is the behavior mean of the entity, σ(f(e i )) is the standard deviation. If TS(e i ,t) > δ t , then it is determined that e i has abnormal behavior at time t;

[0179] S64. Store the detected abnormal entities V a and abnormal relationships E a , and generate an abnormal knowledge graph G a = (V a ,E a ), and provide an abnormal query interface Q(V a ,E a ,T).

[0180] Example:

[0181] In the example, the loan application data of a large Internet financial platform is selected. Combining multi-source heterogeneous data such as the platform's historical transaction records, user social network, device information, and geographical location, a risk control system driven by a knowledge graph is constructed. This system can extract key entities (users, devices, bank cards, transaction records, etc.) and their associated relationships from different data sources, and establish a credit risk assessment system with a global perspective. By introducing intelligent reasoning and anomaly detection technologies, this system can accurately identify potential fraud gangs and effectively reduce credit losses.

[0182] In the actual application process, first, the system collects data such as the identity information, device information, transaction history, social relationships, and credit scores of loan applicants, and standardizes the data. Subsequently, natural language processing technology is used to identify key entities in the application materials, such as ID numbers, mobile phone numbers, bank card numbers, etc., and corresponding entity nodes are established in the knowledge graph. The system further extracts the association relationships between entities, including multiple loans applied for by the same device, multiple accounts bound to the same bank card, frequently interacting social relationships, etc., and integrates this data into the knowledge graph. Through graph neural network and causal inference technology, the system can calculate the similarity between each loan applicant and historical fraud cases, and analyze abnormal patterns, such as multiple loan applications by the same device within a short period of time, or multiple accounts trading using the same IP address, etc.

[0183] To verify the effectiveness of the system, the implementers compared the risk control performance in different time periods, and separately evaluated the performance of traditional methods (rule - based engines) and the method of the present invention (intelligent analysis based on knowledge graph) in terms of fraud recognition rate, false alarm rate, risk score accuracy, etc. Table 1 shows the experimental results based on real data:

[0184] Table 1 Performance of Different Risk Control Methods in Fraud Recognition

[0185] method Total dataset Identify fraudulent users False Positive Rate Processing time (seconds) Traditional rule engine 500,000 12,450 8.3％ 38.2 Machine Learning Models 500,000 14,700 6.1％ 45.6 Knowledge Graph + Intelligent Analysis 500,000 17,680 4.2％ 42.8

[0186] Throughout the embodiments, the intelligent risk control system based on the knowledge graph has demonstrated excellent effects in financial anti - fraud applications. Compared with traditional methods, it has a higher recognition rate, a lower false alarm rate, and stronger reasoning ability and dynamic adaptation ability.

[0187] The present invention constructs an intelligent risk control model that integrates multi - source heterogeneous data by introducing knowledge graph technology. Using entity recognition, relationship extraction, and causal inference methods, it globally models loan applications, transaction behaviors, and social networks, enabling the system to comprehensively understand the user's credit status and accurately identify potential fraud patterns. Compared with traditional rule - based engines and machine learning methods, through the association analysis and anomaly detection capabilities of the knowledge graph, the present invention can discover hidden fraud rings from different data dimensions, breaking through the limitations of existing methods that rely on static rules and limited feature sets.

[0188] In response to the actual needs of financial risk control, this invention combines knowledge fusion, graph neural networks, and reinforcement learning optimization strategies to establish a dynamic intelligent risk assessment system. Through entity matching, similarity calculation, and abnormal pattern recognition of data, high-precision fraud detection is achieved, and risk trends are predicted through time series analysis, achieving remarkable results in the interception and early warning of fraudulent transactions. The relationship analysis method based on intelligent reasoning proposed in this invention can automatically trace high-risk users and identify the organizational structure of fraud rings, ensuring that the risk control system has strong adaptability in dealing with new types of fraud behavior.

[0189] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered by the protection scope of the present invention.

Claims

1. A data intelligent association analysis and application method based on knowledge graph, characterized in that: The steps include: S1, collect multi-source heterogeneous data and preprocess them to obtain preprocessed text data; S2. Extract entities and relationships from the preprocessed text data, construct a preliminary knowledge graph, and optimize the constructed knowledge graph using causal reasoning methods and variational autoencoders; S3, using graph database to store knowledge graph; S4. Expand and supplement the knowledge graph through external data sources; S5. Perform data association analysis based on knowledge graph to identify potential patterns and associations in the data; S6. According to the results of data association analysis, a graph-based relationship scoring method is used to detect abnormal data in the knowledge graph, discover potential abnormal patterns and mark them.

2. According to claim 1, a method for intelligent data association analysis and application based on knowledge graph, characterized in that: The S1 comprises the following steps: S11. Acquire multi-source heterogeneous data, including structured data, semi-structured data, and unstructured data; S12. Standardize the format of multi-source heterogeneous data: For structured data, maintain field consistency; Perform data analysis and field mapping for semi-structured data; For unstructured data, text vectorization, image feature extraction, and audio spectrum analysis methods are used for structured conversion; S13, cleaning the text data after format standardization to remove duplicate data and outliers; S14, preprocess the text data and form a text sequence {w1,w2,...,w n }, where w i Represents the i-th word segmentation unit; S15: storing the preprocessed text data.

3. According to the method of intelligent data association analysis and application based on knowledge graph in claim 1, it is characterized in that: The S2 comprises the following steps: S21, perform entity recognition on the preprocessed text data, using a pre-trained language model for self-supervised learning combined with a named entity recognition method, from the text sequence {w1,w2,...,w n } extract the entity set E = {e1,e2,...,e m }, where e i Represents the i-th recognized entity and optimizes the accuracy of entity recognition through the context relevance scoring mechanism; S22, performing entity disambiguation and normalization processing on the entity set E, and calculating the entity similarity using an embedding matching method based on a graph neural network; S23, extract the relationship between entities from the preprocessed text data, combine dependency syntactic analysis, multi-hop relationship reasoning and knowledge embedding methods to construct a relationship set R = {r1, r2, ..., r k }, where r k Represents the kth identified relation and identifies implicit relations through cross-document reasoning; S24, based on the relationship triple (e i ,r k ,e j ) generates a preliminary knowledge graph G = (V, E), where V = {e1, e2, ..., e m } represents the entity node in the knowledge graph, E={(e i ,r k ,e j )|e i ,e j ∈V} represents the relationship edge in the knowledge graph, and a dynamic attention mechanism is introduced to optimize the correlation between entities and relationships; S25. For the initially constructed knowledge graph, a knowledge enhancement method based on causal reasoning is used for optimization, a relationship correction mechanism based on reinforcement learning is used to adjust the low-confidence relationship triples, and a variational autoencoder is used to compress the knowledge representation; S26. Incrementally update the optimized knowledge graph, adopt a dynamic update strategy based on temporal knowledge embedding, automatically identify new entities and relationships, and integrate new knowledge into the original graph through knowledge alignment technology.

4. According to claim 3, a method for intelligent data association analysis and application based on knowledge graph, characterized in that: The S25 comprises the following steps: S251. Model the causal relationship of the initially constructed knowledge graph G = (V, E) and define the causal influence measurement function I (e i ,r k ,e j ): I(e i ,r k ,e j )=P(e j |e i ,r k )-P(e j |do(e i ,r k )); Among them, P(e j |e i ,r k ) means that under the existing knowledge graph, entity e i Through the relationship k Affecting Entity j The probability, P(e j |do(e i ,r k )) indicates that when external intervention is applied, e j In e i 、r k The independent probability under the action of the causal inference method is used to preliminarily screen the validity of the knowledge graph relationship; S252, define the reinforcement learning environment based on the relationship correction mechanism of reinforcement learning; State space S: defined as the current knowledge graph structure G t =(V t ,E t ), where V t is the current entity set of the knowledge graph, E t is the current relation set of the knowledge graph; Action space A: includes relationship change operations A = {A add ,A del ,A mod }, where A add To add a new relationship, A del To remove low confidence relationships, A mod To modify the relationship category; Reward function R: for the optimized knowledge graph G t+1 Calculate the consistency score F(G t+1 ), define the reward function: R=F(G t+1 )-F(G t ); Among them, F(G t ) is calculated by weighted calculation of relationship confidence, entity connectivity and logical consistency measurement function, with the goal of maximizing R to optimize the knowledge graph relationship; S253, use variational autoencoder to optimize knowledge graph embedding, for each entity e i Perform low-dimensional embedding representation z i , define the objective function of the variational autoencoder: L VAE =E q(z|e) [logp(e|z)]-D KL (q(z|e)||p(z)); Among them, q(z|e) is the posterior distribution, p(z) is the prior distribution, and D KL is the KL divergence, by minimizing L VAE Train autoencoders to improve the compactness and discriminability of entity embedding representations; S254, perform relationship pruning and relationship completion based on entropy constraint optimization strategy, and calculate the confidence C (r k ): C(r k )=-∑P(r k |e i ,e j )logP(r k |e i ,e j ); Prune the relations whose confidence is lower than the set threshold to form a new knowledge graph structure G'=(V,E'); For missing relationships, a relationship prediction model R(e) based on graph neural network is used. i ,e j )=f(e i ,e j ,θ) to complete, where f(e i ,e j ,θ) is the relationship prediction function, θ is the model parameter, and the gradient descent method is used to optimize θ to maximize the prediction probability of the known relationship; S255: Check the consistency of the optimized knowledge graph based on logical constraint rules Perform constraint reasoning and calculate the set of relations E that violate the constraint rules inconsistent ,like Then perform minimal modification operations: ΔG=argmin ΔG ||G+ΔG||; Here, ΔG is the minimum modification made to the knowledge graph G, which corrects inconsistent relationships by replacing or deleting the least number of edges.

5. According to claim 1, a method for intelligent data association analysis and application based on knowledge graph, characterized in that: The S4 comprises the following steps: S41, based on the external data source to expand the knowledge graph, the external data source set D = {D1, D2, ..., D n } to filter and extract the entity set V'={e1',e2',...,e m '} and the relation set E'={(e i ',r k ',e j ')}, where V' and E' are both derived from an external data source D; S42, align the extracted external entity V', and calculate the entity matching degree Sim(e) using a matching method based on text similarity and entity embedding i ,e j '): Yes(and i ,and j ')=α·Yes text (and i ,and j ')+β·Yes vec (and i ,and j '); Among them, Sim text (e i ,e j ') Calculate text similarity by editing distance, Sim vec (e i ,e j ') The vector cosine similarity is used to calculate the similarity of entity embedding, and α and β are weight coefficients; S43, verify the extracted external relation E', and calculate the external relation r using the relation prediction model based on the graph attention network k 'With existing relationship k The matching degree Match(r k ,r k '): Match(r k ,r k ')=softmax(W·[h(e i )||h(e j )]); Among them, h(e i )、h(e j ) is entity e i 、e j Graph neural network embedding, W is the training parameter, if Match(r k ,r k ') is greater than the set value λ, then it is determined that r k ' is a valid relationship and is added to the knowledge graph E; S44. Use the knowledge fusion strategy based on confidence to merge external data and define the external data credibility score C(D i ): Among them, Trust(e j ) is calculated by weighting the source weight, data consistency and entity matching, filtering the external data with a credibility lower than the threshold θ and retaining only high-confidence knowledge; S45. Perform consistency check on the expanded knowledge graph G'=(V∪V',E∪E'); S46. Store the expanded knowledge graph G' and update the index.

6. The method for intelligent data association analysis and application based on knowledge graph according to claim 1, characterized in that: The S5 comprises the following steps: S51, based on the expanded knowledge graph G', perform data association analysis and define the entity association calculation function Rel(e i ,e j ): Among them, P(r k |e i ,e j ) represents entity e i and e j Through the relationship k The probability of association, based on the association threshold τ r Filter low-correlation entity pairs and retain only high-correlation entity sets V r ; S52. For entity set V r Cluster analysis was performed using the density-based DBSCAN clustering method; Based on the clustering results, calculate the central node c of each entity cluster k : Among them, V k represents the kth entity cluster, c k As the representative entity of the cluster; S53, perform association rule mining, define frequent relational patterns FPM(R), use Apriori algorithm, based on relation support Support(r k ) Calculate the frequent relation set: If Support(r k )>σ s , then determine r k For high-frequency relations, add frequent relation set R f , and further generate association rules R'; S54. Based on the association analysis results, construct the inference graph G inf =(V',E inf ), where E inf It is composed of high confidence relations and defines the inference credibility Conf(r k ,e i ,e j )for: Conf(r k ,and i ,and j )=P(r k |and i ,and j ) Trust(e i ) Trust(e j ); Among them, Trust(e i ) and Trust(e j ) is calculated by the knowledge source confidence. If Conf(r k ,e i ,e j )>γ, then (e i ,r k ,e j ) is added to the inference graph; S55, store the optimized knowledge association analysis results, and provide a query interface Q(V',R f ,G inf ).

7. The method for intelligent data association analysis and application based on knowledge graph according to claim 1 is characterized in that: The S6 comprises the following steps: S61. Based on the results of knowledge association analysis, the knowledge graph G inf =(V',E inf ) to perform anomaly detection and define the anomaly scoring function Anomaly(e i ): Among them, N(e i ) is entity e i The neighbor entity set of Sim(e i ,e j ) is the entity similarity, if Anomaly(e i )>τ a , then e i Classified as abnormal entity set V a ; S62. Relationship E in the knowledge graph inf To detect anomalies, we use a graph-based relationship scoring method to calculate the relationship confidence Conf(r k ,e i ,e j ): Conf(r k ,and i ,and j )=P(r k |and i ,and j ) Trust(e i ) Trust(e j ); Among them, P(r k |e i ,e j ) is the probability of knowledge graph reasoning, Trust(e i )、Trust(e j ) is the entity credibility, if Conf(r k ,e i ,e j )<θ c , then (e i ,r k ,e j ) is classified as the abnormal relation set E a ; S63, based on time series analysis of abnormal change trends, construct time dependency relationship T(e i ,t)=(e i ,r k ,e j ,t), calculate the time series anomaly score TS(e i ,t): Among them, f t (e i ) represents the entity behavior characteristics at time t, μ(f(e i )) is the behavior mean of the entity, σ(f(e i )) is the standard deviation, if TS(e i ,t)>δ t , then determine e i Abnormal behavior occurs at time t; S64. Store the detected abnormal entity V a and abnormal relationship E a , and generate the abnormal knowledge graph G a =(V a ,E a ), providing an exception query interface Q(V a ,E a ,T).

Images