An artificial intelligence-based data analysis method

Abstract

The application provides a kind of data analysis method based on artificial intelligence, comprising: the multi-source data collected is integrated, cleaned and then stored in a relational database, a data model based on entity-relation is designed to construct nodes and relations, and it is stored in the relational database;The knowledge graph is analyzed and calculated by community discovery algorithm, graph neural network and quantitative analysis to capture the complex characteristics and potential abnormal conditions of the knowledge graph;The three results of community discovery algorithm, graph neural network and quantitative analysis are fitted by integrated algorithm, and the final suspicious score is generated in the form of multi-party weighted voting.This application extracts suspected problem clues from hundreds of thousands of data, provides possible clue direction for case investigation, makes problem discovery more timely, makes supervision more accurate and efficient, and improves case handling efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, in particular, and more particularly, to a data analysis method based on artificial intelligence. BACKGROUND

[0002] Data analysis is a systematic and comprehensive work, covering the process from data collection, data cleaning, data arrangement, data analysis, data mining to data visualization. This process not only requires the professional knowledge and experience of the discipline inspection and supervision department, but also requires the use of modern information technology means to deeply mine and analyze the data to discover the information implied in the data.

[0003] The existing methods and defects of data analysis are as follows:

[0004] The Chinese patent "CN115618007 Supervision Data Analysis System Based on Knowledge Graph" provides a supervision data analysis system framework, which mainly includes: knowledge modeling unit, knowledge management unit, knowledge computing unit, knowledge application unit, knowledge storage unit and data warehouse unit. The data warehouse unit stores source data; the knowledge modeling unit obtains the source data, creates a supervision object domain graph, and creates a supervision object portrait; the knowledge management unit accesses the information data of the investigated person or extracts the corresponding source data, forms computer understandable semantic knowledge, and maps and associates with the supervision object portrait to form a supervision graph; the knowledge computing unit obtains the supervision graph, performs normalized fusion processing, and updates the calculation results to the supervision graph; the knowledge application unit searches for entity data that meets the conditions, performs secondary data processing, obtains the relationship graph of the graph entity, and displays it.

[0005] The technical solution described in the Chinese patent "CN115618007 Supervision Data Analysis System Based on Knowledge Graph" performs entity calculation on the built graph through multiple independent computing units when the knowledge graph performs data mining, and then realizes it through the normalized fusion of these entities. However, there is no effective collaboration and resource sharing between these computing units, which may cause idle computing resources and low efficiency. If there is an error or mistake in a computing unit, these errors may have a negative impact on the normalized fusion results.

[0006] The Chinese patent "CN116777634 A financial data analysis system and method based on artificial intelligence" obtains financial data from a pre-configured financial data source, processes the financial data to obtain triple data, and writes the triple data into a pre-constructed knowledge graph database. In the knowledge graph network model, the associated nodes of the target node are determined, the historical data of the target node and the associated nodes are obtained, and the training data set of the multivariate time series model is generated based on the historical data. The multivariate time series model of the target node and the associated nodes is trained through the training data set, and the future data of the target node is predicted using the model, thereby improving the accuracy of financial time series prediction.

[0007] The technical solution of the Chinese patent "CN116777634 A financial data analysis system and method based on artificial intelligence" mainly analyzes and processes data in the financial field, while the data properties of the discipline inspection and supervision commission data are different from those of the financial data. The data comes from multiple aspects and also implies more information, so the model needs to be rebuilt and trained to achieve better performance. SUMMARY

[0008] According to the above technical problems, a data analysis method based on artificial intelligence is provided. The present application extracts suspected problem clues from hundreds of thousands or even hundreds of thousands of data, provides possible clue directions, makes problem discovery more timely, makes supervision more accurate and efficient, and improves efficiency.

[0009] The technical means adopted by the present application are as follows:

[0010] A data analysis method based on artificial intelligence, comprising:

[0011] S1, after integrating and cleaning the collected multi-source data, store them in a relational database, design a data model based on entity-relation to construct nodes and relations, and store them in the relational database;

[0012] S2, analyze and calculate the knowledge graph through community discovery algorithm, graph neural network and quantitative analysis to capture the complex features and potential abnormal situations of the knowledge graph;

[0013] S3, fit the three results of the community discovery algorithm, the graph neural network and the quantitative analysis through the integrated algorithm, and generate the final suspicious score determination in the form of multi-party weighted voting.

[0014] Further, step S1 specifically comprises:

[0015] S11, integrate the collected data of different sources into a consistent data format through data integration and data conversion, etc., and perform data cleaning operation, and store the converted data into a relational database;

[0016] S12, in order to adapt to specific application scenarios, design an entity-relation based data model, define entities (nodes) and relationships (edges), and build a structured data model, which can fully reflect the subject person, transaction and real estate information involved in the relational database;

[0017] S13, on the basis of building the data model, create corresponding nodes and relationships, wherein the nodes represent entities in the data model, and the relationships represent the association and connection between entities;

[0018] S14, store the created nodes and relationships into the Neo4j graph database.

[0019] Further, in step S11:

[0020] The data cleaning operation is used to ensure the accuracy and integrity of the data, including: processing missing values, removing duplicate data, and solving data conflicts.

[0021] Further, in step S11:

[0022] The relational database includes: Neo4j, MySQL, Oracle, PostgreSQL, and Renmin Jinku.

[0023] Further, in step S14:

[0024] The Neo4j graph database is a high-performance graph database management system, which is specially used for storing and processing graph data, and is used to provide powerful data query, graph analysis and graph algorithm functions, so as to support more in-depth data analysis and query requirements; through Neo4j, relevant data can be quickly retrieved and analyzed, and valuable information can be obtained therefrom.

[0025] Further, step S2 includes:

[0026] S21, using Louvain algorithm to discover community structure in the graph;

[0027] S22, using trained graph neural network to classify nodes and edges in the graph to identify suspicious nodes and edges;

[0028] S23, based on requirements and business rules, detect suspicious entities and relationships to discover potential violations or abnormal situations.

[0029] Further, in step S21, the Louvain algorithm is used to discover the community structure in the graph, mainly including two stages:

[0030] Modularity optimization stage:

[0031] In the modularity optimization stage, each node is assigned a respective community label, and then each node iterates through all its neighbor nodes and attempts to update its community label to the community label of the neighbor node; the Louvain algorithm selects the community label with the largest modularity increment as the new community label of the node, which uses a greedy idea, and this process is repeated until all nodes cannot increase the modularity by changing the community label;

[0032] Network condensation stage:

[0033] In the network condensation stage, each community is merged into a new supernode, and the edge weight of the supernode is the sum of the edge weights of all nodes in the original community, and then the nodes in the original network are condensed into larger supernodes to form a new network.

[0034] Further, in step S23, specifically comprising:

[0035] S231, in order to make full use of objective fact data, a statistical-based, distance-based and density-based anomaly detection algorithm is used to detect the degree, aggregation degree and centrality statistical indicators of entities and relationships, and relatively low or high values are considered as abnormal points;

[0036] S232, the statistical-based anomaly detection algorithm identifies data points that are significantly different from normal behavior by statistical characteristics and distribution assumptions of data, and detects entities or relationships that deviate significantly from normal data distribution;

[0037] S233, the distance-based anomaly detection algorithm judges whether it is abnormal by calculating the distance or similarity between entities or relationships;

[0038] S234, if the distance between a certain entity or relationship and other entities or relationships is far or the similarity is low, it is considered to be abnormal;

[0039] S235, the density-based anomaly detection algorithm further judges whether it is abnormal by calculating the density distribution.

[0040] Further, in step S3, the integration algorithm determines the final suspicious score in a multi-party weighted voting manner by combining the results of multiple algorithms, specifically comprising:

[0041] S31, for each entity or relationship, each algorithm will give a suspicious score representing the degree of abnormality;

[0042] S32, obtaining a final suspicious score by performing a weighted summation or an averaging operation on the suspicious scores given by each algorithm.

[0043] Compared with the prior art, the present application has the following advantages:

[0044] 1. The data analysis method based on artificial intelligence provided by the present application uses collected project data to convert and clean the data into structured data, then uses a graph model to establish a visual graph, and through analysis and mining, detailed information such as the social network relationship of suspicious persons is obtained to find clues for handling cases, quickly capture suspicious features, and realize analysis of case-related data.

[0045] 2. The data analysis method based on artificial intelligence provided by the present application provides an interactive page front-end interface, a back-end interface of query and mining functions, and a suspicious score research and judgment model, thereby providing a powerful tool and platform for case handling personnel.

[0046] 3. The data analysis method based on artificial intelligence provided by the present application can strengthen the understanding and in-depth analysis of cases by case handling personnel, assist case handling personnel in finding potential clues, and improve the case solving rate.

[0047] Based on the above reasons, the present application can be widely promoted in the field of data analysis. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0049] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0050] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0052] As Figure 1 shown, the present application provides an artificial intelligence-based data analysis method, comprising:

[0053] S1, after integrating and cleaning the collected multi-source data, store it in a relational database, design an entity-relation-based data model to build nodes and relationships, and store it in the relational database;

[0054] S2, analyze and calculate the knowledge graph through community discovery algorithm, graph neural network and quantitative analysis, to capture the complex features and potential abnormal situations of the knowledge graph;

[0055] S3, fit the three results of community discovery algorithm, graph neural network and quantitative analysis through integrated algorithm, and generate the final suspicious score judgment in the form of multi-party weighted voting.

[0056] In specific implementation, as a preferred embodiment of the present application, step S1 specifically comprises:

[0057] S11, integrate the collected data from different sources into a consistent data format through data integration and data conversion operations, and perform data cleaning operations, and store the converted data in a relational database;

[0058] S12, in order to adapt to specific application scenarios, design an entity-relation-based data model, define entities (nodes) and relationships (edges), and build a structured data model to better organize and manage data. The data model can fully reflect the subject person, transaction and real estate information involved in the relational database;

[0059] S13, on the basis of constructing the data model, create corresponding nodes and relationships, wherein the nodes represent entities in the data model, such as persons, transactions and real estate, etc. The relationship represents the association and connection between entities, such as the relationship between persons, the relationship between transactions and persons, and the relationship between real estate and persons, etc. The definition of such nodes and relationships will help better describe and understand the complex relationship between data.

[0060] S14, store the created nodes and relationships into the Neo4j graph database.

[0061] In this embodiment, the collected raw data is preprocessed, a suitable data model is designed to construct entities and relationships, and it is stored in the Neo4j graph database, which provides a solid foundation for subsequent data query and analysis. This process can help better understand and utilize data information, and provide targeted solutions for specific application scenarios.

[0062] In specific implementation, as a preferred embodiment of the present application, in step S11:

[0063] Data cleaning operation is used to ensure the accuracy and integrity of data, including: processing missing values, removing duplicate data, solving data conflicts.

[0064] In specific implementation, as a preferred embodiment of the present application, in step S11:

[0065] The relational database includes: Neo4j, MySQL, Oracle, PostgreSQL, and Renesas.

[0066] In specific implementation, as a preferred embodiment of the present application, in step S14:

[0067] The Neo4j graph database is a high-performance graph database management system, which is specially used for storing and processing graph data, and is used to provide powerful data query, graph analysis and graph algorithm functions, so as to support more in-depth data analysis and query requirements; through Neo4j, relevant data can be quickly retrieved and analyzed, and valuable information can be obtained therefrom.

[0068] In specific implementation, as a preferred embodiment of the present application, step S2 includes:

[0069] S21, using Louvain algorithm to find the community structure in the graph;

[0070] In this embodiment, in order to extract the local features of the graph, the community discovery algorithm provided by Neo4j is used to find the community related to the suspicious person. Neo4j provides a variety of community discovery algorithms, including Louvain algorithm, LabelPropagation algorithm, Triangle Count algorithm, etc. In this module, the Louvain algorithm is selected to discover the community structure in the graph. The Louvain algorithm is a community discovery algorithm based on modularity, which aims to group nodes into groups with close internal connections and relatively distant groups.

[0071] S22, using the trained graph neural network, classifying the nodes and edges in the graph to identify suspicious nodes and edges;

[0072] In this embodiment, in order to capture the global features of the knowledge graph data, using the graph neural network (GNN) is an effective method. GNN has the ability to learn the complex relationships between nodes and can generate vector representations of nodes. By training the GNN model with graph data as input, the feature representations of nodes and edges can be learned. In the training process, supervised learning can be used with label information of nodes and edges, or unsupervised learning methods such as graph autoencoder can be used. Supervised learning uses known label information of nodes and edges to guide the model to learn the feature representation of nodes and edges, so that the model can better classify. Unsupervised learning methods learn the representation of nodes and edges in the graph through the reconstruction loss of the model itself or other properties of the graph structure without relying on external label information. Through the trained GNN model, the nodes and edges in the graph are classified to identify suspicious nodes and edges. For example, in an investigation, the GNN model can be used to identify potential suspicious persons or abnormal behaviors. The prediction results can be presented through visualization tools to display the nodes and edges in the graph in graphical form, highlighting the nodes and edges identified as suspicious, so as to help users discover hidden patterns, potential associations and abnormal situations.

[0073] S23, based on requirements and business rules, detecting suspicious entities and relationships for discovering potential violations or abnormal situations.

[0074] In specific implementation, as a preferred embodiment of the present application, in step S21, the Louvain algorithm is used to discover the community structure in the graph, mainly including two stages:

[0075] Modularity optimization stage:

[0076] In the modularity optimization phase, each node is tagged as a respective community, and then each node traverses all the neighbor nodes and attempts to update the respective community tag to the community tag of the neighbor node; the Louvain algorithm selects the community tag with the largest modularity increment as the new community tag of the node, which uses a greedy idea, and this process is repeated until all nodes cannot increase the modularity by changing the community tag;

[0077] The network condensation phase:

[0078] In the network condensation phase, the communities are merged into a new supernode, and the edge weight of the supernode is the sum of the edge weights of all nodes in the original community, so that the nodes in the original network are condensed into larger supernodes to form a new network.

[0079] In this embodiment, the Louvain algorithm is used to find the community structure in the graph and gather related nodes together. When applied to the community associated with suspicious personnel, it helps to find potential associated personnel, thereby providing more comprehensive analysis and inspection.

[0080] In specific implementation, as a preferred embodiment of the present application, step S23 specifically includes:

[0081] S231, in order to make full use of objective fact data, a statistical-based, distance-based and density-based anomaly detection algorithm is used to detect the degree, aggregation degree and centrality statistical indicators of entities and relationships, and relatively low or high values are regarded as abnormal points;

[0082] S232, the statistical-based anomaly detection algorithm identifies data points that are significantly different from normal behavior through statistical characteristics and distribution assumptions of data, and detects entities or relationships that deviate significantly from normal data distribution;

[0083] S233, the distance-based anomaly detection algorithm judges whether it is abnormal by calculating the distance or similarity between entities or relationships;

[0084] S234, if the distance between a certain entity or relationship and other entities or relationships is far or the similarity is low, it is considered to be abnormal;

[0085] S235, the density-based anomaly detection algorithm further judges whether it is abnormal by calculating the density distribution;

[0086] In this embodiment, if the density of the surrounding neighbors of a certain entity or relationship is low or lower, it may be an outlier. By formulating a variety of rules to detect suspicious entities and relationships, potential violations or abnormal situations are found. These rules are based on requirements and business rules, focusing on 24 features, extracting these 24 feature data in turn combined with Cypher statements, and generating a feature matrix for each subject person.

[0087] In specific implementation, as a preferred embodiment of the present application, in step S3, the integrated algorithm determines the final suspicious score by combining the results of multiple algorithms in a multi-party weighted voting manner, specifically including:

[0088] S31, for each entity or relationship, each algorithm gives a suspicious score representing the degree of abnormality;

[0089] S32, by weighting and summing or averaging the suspicious scores given by each algorithm, the final suspicious score is obtained.

[0090] In this embodiment, the three algorithms mentioned above play different roles in knowledge graph analysis, and calculate and analyze data from the perspectives of local features, global features and quantitative analysis. However, using a single algorithm alone may not be able to fully capture the complex features and potential abnormal situations of the knowledge graph. Therefore, integrating these algorithms is an effective strategy. The advantage of the integrated method is that it can consider the advantages of different algorithms and alleviate the defects and limitations of a single algorithm. Different algorithms may have different sensitivity and expressiveness for different types of abnormal situations. By integration, the results of these algorithms can be used comprehensively, providing more comprehensive and diverse judgment results. For an entity or relationship, if multiple algorithms give a high suspicious score, it can be more confident that it is an outlier. On the contrary, if only one algorithm gives a high suspicious score, while the scores of other algorithms are low, further analysis is needed to determine its abnormality. By integrating the results of different algorithms, a more powerful integrated model is constructed. This model can consider the information of local features, global features and quantitative analysis, and improve the performance and effect of suspicious entity and relationship identification. At the same time, the integrated model also has good flexibility, which can be adjusted and optimized according to the actual situation to meet the needs of specific application scenarios.

[0091] To sum up, the application provides a data analysis method based on artificial intelligence, first, the collected multi-source data is integrated, cleaned and stored in a relational database, a data model based on entity-relation is designed to construct nodes and relations, and stored in a Neo4j graph database. Then, the knowledge graph is analyzed and calculated by community discovery algorithm, graph neural network and quantitative analysis algorithm to capture the complex features and potential abnormal conditions of the knowledge graph. Finally, the above three results are fitted by an integrated algorithm to generate the final suspicious score judgment in the form of multi-party weighted voting.

[0092] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, but not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application.

Claims

1. A data analysis method based on artificial intelligence, characterized in that: include: S1. Integrate and clean the collected data from multiple sources and store them in a relational database. Design an entity-relationship-based data model to construct nodes and relationships and store them in a relational database. S11. Integrate the collected data from different sources into a consistent data format through data integration and data conversion, perform data cleaning, and store the converted data in a relational database; S12. To adapt to specific application scenarios, design an entity-relationship-based data model. By defining entities as nodes and relationships as edges, a structured data model is constructed. The data model can fully reflect the subject person, transaction, and real estate information involved in the relational database. S13. Based on the data model, corresponding nodes and relationships are created, where nodes represent entities in the data model and relationships represent associations and connections between entities; S14. Store the created nodes and relationships in the Neo4j graph database; S2. Analyze and calculate the knowledge graph through community discovery algorithms, graph neural networks, and quantitative analysis to capture the complex characteristics and potential anomalies of the knowledge graph; S21. Use the Louvain algorithm to discover the community structure in the graph, which consists of two stages: Modularity optimization stage: In the modularity optimization phase, each node is treated as its own community label. Then, each node traverses all its neighboring nodes and attempts to update its own community label to the community label of the neighboring nodes. The Louvain algorithm selects the community label with the largest modularity increment as the node's new community label. This process uses a greedy approach and is repeated until all nodes cannot increase their modularity by changing their community labels. Network cohesion stage: In the network cohesion phase, each community is merged into a new super node. The edge weight of this super node is the sum of the edge weights of all nodes in the original community. The nodes in the original network are condensed into a larger super node, forming a new network. S22. Use the trained graph neural network to classify the nodes and edges in the graph to identify suspicious nodes and edges; S23. Detect suspicious entities and relationships based on requirements and business rules to discover potential violations or anomalies; S231. In order to make full use of objective factual data, use statistical, distance, and density-based anomaly detection algorithms to detect the degree, aggregation, and centrality statistical indicators of entities and relationships, and regard relatively low or high values ​​as outliers; S232. Statistical anomaly detection algorithms use the statistical characteristics and distribution assumptions of data to identify data points that are significantly different from normal behavior and detect entities or relationships that deviate significantly from the normal data distribution; S233, a distance-based anomaly detection algorithm, which determines whether an anomaly exists by calculating the distance or similarity between entities or relationships; S234. If an entity or relationship is far away from or has low similarity to other entities or relationships, it is considered to be potentially abnormal; S235, density-based anomaly detection algorithm, further determines whether it is abnormal by calculating density distribution; S3. The three results of community discovery algorithm, graph neural network and quantitative analysis are fitted through an integrated algorithm, and the final suspicious score judgment is generated by multi-party weighted voting.

2. The data analysis method based on artificial intelligence according to claim 1, characterized in that: In step S11: Data cleaning operations are used to ensure data accuracy and completeness, including processing missing values, removing duplicate data, and resolving data conflicts.

3. The data analysis method based on artificial intelligence according to claim 1, characterized in that: In step S11: The relational databases include: Neo4j, MySQL, Oracle, PostgreSQL, and Renmin University of China Golden Warehouse.

4. The data analysis method based on artificial intelligence according to claim 1, characterized in that: In step S14: The Neo4j graph database is a high-performance graph database management system specifically used to store and process graph data, and is used to provide powerful data query, graph analysis, and graph algorithm functions, thereby supporting more in-depth data analysis and query needs; Neo4j can be used to quickly retrieve and analyze relevant data and obtain valuable information from it.

5. The data analysis method based on artificial intelligence according to claim 1, characterized in that: In step S3, the integrated algorithm combines the results of multiple algorithms and uses a multi-party weighted voting method to make the final suspicious score determination, specifically including: S31. For each entity or relationship, each algorithm will give a suspicious score, which represents the degree of abnormality; S32. Obtain a final suspicion score by performing a weighted sum or averaging operation on the suspicion scores given by each algorithm.

Images