Enterprise information integration and data visualization analysis method

Abstract

The invention discloses an enterprise information integration and data visualization analysis method, and particularly relates to the technical field of enterprise information management, and the method comprises the steps: S1, data collection and preprocessing, S2, dynamic knowledge graph construction, S3, intelligent data integration, S4, dynamic association analysis, S5, intelligent visualization display, and S6, adaptive learning and optimization. According to the method, automatic integration, dynamic updating and intelligent display of multi-source heterogeneous data are realized through technologies of dynamic knowledge graph, intelligent data integration, real-time association analysis, intelligent visual display and the like, so that the high efficiency of data integration, the intelligence of visual display and the self-adaptive optimization capability of the system are greatly improved; and the data analysis and decision support capability of an enterprise can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of enterprise information management, and more specifically, to an enterprise information integration and data visualization analysis method. Background Art

[0002] With the rapid development of information technology, enterprises are facing the challenges of explosive growth in data size and diversification of data sources. Internal enterprise systems (such as ERP, CRM, SCM, etc.) and external data sources (such as social media, public datasets, industry reports, etc.) generate a large amount of multi-source heterogeneous data. This data often exists in different formats and structures, leading to data silos and making it difficult to achieve efficient integration and comprehensive utilization. In addition, traditional data analysis methods often rely on static data models and predefined analysis rules, making it difficult to capture dynamic associations and implicit relationships between data, and unable to meet enterprises' needs for real-time decision support. Specifically:

[0003] 1. Traditional data integration methods mainly rely on ETL (Extract, Transform, Load) tools and data warehouse technology. These methods convert heterogeneous data into a unified format based on predefined rules and load it into the data warehouse. However, these methods have the following limitations:

[0004] Data silo problem: It is difficult to achieve seamless integration of data between different systems, resulting in low data utilization;

[0005] Static association: Traditional methods often use predefined static relationship models, which cannot dynamically capture the implicit relationships between data;

[0006] Lack of real-time performance: Traditional methods are mostly used for offline data processing and cannot meet the needs of enterprises for real-time data analysis.

[0007] 2. Traditional data visualization tools (such as Tableau and PowerBI) are mostly based on static dashboards and predefined visualization templates. These tools have the following limitations:

[0008] Visualization limitations: Traditional tools are mostly based on fixed templates, lacking flexibility and intelligent support, and are difficult to adapt to complex business needs;

[0009] Insufficient interactivity: Traditional tools have limited user interaction capabilities and are unable to support user-defined views and in-depth analysis;

[0010] Lack of immersive experience: Traditional tools cannot provide an immersive data display experience and find it difficult to intuitively present complex data relationships.

[0011] 3. Knowledge graph technology uses graph structures to represent entities and their relationships and is widely used in fields such as enterprise knowledge management, intelligent search, and recommendation systems. However, existing knowledge graph technologies often use static construction methods, making real-time updates and dynamic analysis difficult. In addition, knowledge graph construction often relies on manual annotation and predefined ontology models, resulting in high construction costs and poor flexibility.

[0012] 4. Augmented reality technology, by overlaying virtual information onto real-world scenes, is widely used in fields such as industrial monitoring, medical diagnosis, and education and training. However, existing AR technology is mostly used to display data in a single scene and lacks integration with complex data analysis technologies, making it difficult to support multi-dimensional data visualization.

[0013] 5. Real-time stream processing technologies (such as Apache Flink and Apache Kafka) process and analyze data streams in real time and are widely used in fields such as financial risk control, network monitoring, and the Internet of Things. However, existing real-time stream processing technologies are mostly used for single-type data analysis and lack integration with knowledge graphs and visualization technologies, making it difficult to support complex data correlation analysis and visualization.

[0014] In view of the limitations of the above-mentioned prior art, the present invention proposes an enterprise information integration and data visualization analysis method. Summary of the Invention

[0015] In order to overcome the above-mentioned defects of the prior art, the present invention provides an enterprise information integration and data visualization analysis method to solve the problems raised in the above-mentioned background technology.

[0016] To achieve the above objectives, the present invention provides the following technical solution: a method for enterprise information integration and data visualization analysis, comprising the following steps:

[0017] S1. Data collection and preprocessing: collecting multi-source heterogeneous data from internal enterprise systems and external data sources, and generating structured data through data cleaning and standardization;

[0018] S2. Dynamic knowledge graph construction: Utilize natural language processing technology to extract entities and relationships from structured data, and combine it with graph computing technology to construct a dynamic knowledge graph. The dynamic knowledge graph includes nodes, edges, and attributes, where nodes represent entities, edges represent relationships between entities, and attributes represent characteristics of entities or relationships.

[0019] S3, intelligent data integration, uses semantic mapping technology to automatically map heterogeneous data to nodes and edges in a dynamic knowledge graph, achieving seamless integration of multi-source data;

[0020] S4. Dynamic association analysis: Based on graph neural networks and real-time stream processing technology, it performs real-time analysis on nodes and edges in dynamic knowledge graphs, captures implicit relationships between data, and generates association analysis results.

[0021] S5, intelligent visualization, uses augmented reality (AR) and interactive visualization technology to present dynamic knowledge graphs and association analysis results in a multi-dimensional form, supporting user-defined views and interactive operations;

[0022] S6. Adaptive learning and optimization: Analyze user behavior and data changes through machine learning algorithms, dynamically optimize the knowledge graph structure and visualization effects, and improve the intelligence level of the system.

[0023] Preferably, in step S2, entities and relationships are extracted using the following formula:

[0024] Entity recognition: Entities are extracted from text data using a named entity recognition model. The loss function of the named entity recognition model is: Among them, x i represents the input text sequence, yi represents the entity label sequence, and N is the sequence length;

[0025] Relationship extraction: Extract the relationship between entities from text data through a relationship classification model. The loss function of the relationship classification model is: Among them, e1 and e2 represent entity pairs, rj represents the relationship between entities, M is the number of relationship categories.

[0026] Preferably, in step S4, the node representation update formula of the graph neural network is: in, Representation node v In the feature representation of layer l, N(v) represents the node v The set of neighbor nodes, W (l) and b(l) are the weight matrix and bias term of the lth layer respectively, and σ is the activation function.

[0027] Preferably, in step S5, the implementation of augmented reality technology includes the following steps:

[0028] S5.1. Capture the real scene through the camera and generate the real scene image;

[0029] S5.2. Superimpose the dynamic knowledge graph and association analysis results onto the real scene image in the form of a virtual image;

[0030] S5.3. Display the superimposed image to the user via a display device.

[0031] Preferably, in step S6, the system optimizes the knowledge graph structure by the following formula: OPT =L NER +L REL +λL REG , where L NER is the entity recognition loss, L REL is the relation extraction loss, L REG is the regularization term, λ is the regularization coefficient, and its value range is [0.01, 0.1].

[0032] The present invention also provides a system for realizing the above-mentioned enterprise information integration and data visualization analysis, which specifically includes the following modules: a data acquisition and preprocessing module for acquiring data from multi-source heterogeneous data and performing cleaning and standardization processing;

[0033] Dynamic knowledge graph construction module, used to build dynamic knowledge graphs through natural language processing technology and graph computing technology;

[0034] Intelligent data integration module, used to achieve seamless integration of multi-source data through semantic mapping technology;

[0035] Dynamic association analysis module, used for real-time association analysis based on graph neural network and real-time stream processing technology;

[0036] Intelligent visualization display module, used to display analysis results through AR and interactive visualization technology;

[0037] Adaptive learning and optimization module, used to optimize the knowledge graph structure and visualization effects through machine learning algorithms.

[0038] Preferably, the dynamic knowledge graph construction module includes: an entity recognition unit, used to extract entities from text data through a named entity recognition model; a relationship extraction unit, used to extract relationships between entities from text data through a relationship classification model; and a graph update unit, used to update nodes and edges in the knowledge graph in real time.

[0039] Preferably, the intelligent visualization display module includes: an AR overlay unit for overlaying a virtual image onto a real scene image; and an interactive operation unit for supporting users to customize views through operations such as dragging, zooming, etc.

[0040] Preferably, the adaptive learning and optimization module includes: a user behavior analysis unit for analyzing user operation behavior and generating a user preference model; and a data-driven optimization unit for dynamically optimizing system parameters according to data changes.

[0041] Technical effects and advantages of the present invention:

[0042] 1. Through semantic mapping technology and dynamic knowledge graph construction, it can automatically integrate multi-source heterogeneous data into a unified knowledge graph and support real-time dynamic updates, solving the problems of data silos and static associations in traditional data integration methods. Through dynamic knowledge graphs, it can capture data changes in real time, ensuring the timeliness and accuracy of data. It is suitable for scenarios such as supply chain management and market competition analysis, helping enterprises quickly integrate internal and external data and support real-time decision-making.

[0043] 2. Through augmented reality and interactive visualization technology, dynamic knowledge graphs and association analysis results can be displayed in a multi-dimensional form, supporting user-defined views and interactive operations. This breaks through the static display limitations of traditional visualization tools and provides an immersive dynamic visualization experience. Users can customize views through operations such as dragging and zooming, improving the user experience. It is suitable for scenarios such as customer relationship management and risk management, helping enterprises to intuitively understand complex data relationships and improve decision-making efficiency.

[0044] 3. By analyzing user behavior and data changes through machine learning algorithms, the knowledge graph structure and visualization effects can be dynamically optimized to improve the intelligence level of the system. Using the adaptive learning mechanism, the system can automatically optimize the analysis results and display effects according to user preferences and data characteristics, support real-time stream processing and dynamic correlation analysis, ensure the real-time and accuracy of the system, and is suitable for scenarios such as personalized recommendations and intelligent early warnings, helping enterprises achieve intelligent operations and accurate decision-making.

[0045] In summary, the present invention realizes the automatic integration, dynamic update and intelligent display of multi-source heterogeneous data through technologies such as dynamic knowledge graph, intelligent data integration, real-time correlation analysis and intelligent visualization display, which has significant technological progress and practical value. Its beneficial effects are mainly reflected in the high efficiency of data integration, the intelligence of visualization display and the adaptive optimization ability of the system, which can effectively enhance the data analysis and decision support capabilities of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is the overall flow chart of the present invention.

[0047] Figure 2 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Example 1

[0050] The present invention provides a method for enterprise information integration and data visualization analysis, which specifically includes the following steps:

[0051] S1. Data collection and preprocessing: Collect multi-source heterogeneous data from the enterprise's internal systems and external data sources, and generate structured data through data cleaning and standardization.

[0052] Specifically, in this step, data collection interface programs are used to collect data from internal systems such as the enterprise's ERP system and CRM system, as well as external data sources such as industry databases and government open data platforms. The collected data is in various formats, such as structured database table data, semi-structured XML files, and unstructured text files. Data cleaning tools are used to remove duplicate data based on pre-set cleaning rules.

[0053] For example, by comparing the unique identifier fields of the data, duplicate records are deleted if found. Data statistical analysis methods are used to identify and address outliers. For example, based on the 3σ principle, data that deviates from the mean by more than three standard deviations is considered an outlier and corrected or deleted. Data is processed using normalization algorithms. For example, for numerical data, the Z-score normalization method is used to uniformly quantify and distribute data, ultimately generating structured data.

[0054] S2. Dynamic knowledge graph construction: Utilize natural language processing technology to extract entities and relationships from structured data, and combine it with graph computing technology to construct a dynamic knowledge graph. The dynamic knowledge graph includes nodes, edges, and attributes, where nodes represent entities, edges represent relationships between entities, and attributes represent characteristics of entities or relationships.

[0055] The extraction of entities and relations uses the following formula:

[0056] Entity recognition: Entities are extracted from text data using a named entity recognition model. The loss function of the named entity recognition model is: Among them, xi represents the input text sequence, y i Represents the entity tag sequence, N is the sequence length;

[0057] Relationship extraction: Extract the relationship between entities from text data through a relationship classification model. The loss function of the relationship classification model is: Among them, e1 and e2 represent entity pairs, r j Represents the relationship between entities, M is the number of relationship categories.

[0058] Specifically, in this step, with the help of natural language processing toolkits (such as NLTK, spaCy, etc.), named entity recognition models are used to extract entities from the text content in structured data;

[0059] Taking the order description text in the enterprise sales data as an example, entities such as customer name, product name, order amount, etc. are identified, and the relationship between entities, such as the purchase relationship between customers and products, is extracted from the text using the relational classification model. Combined with the graph computing framework (such as the graph computing function provided by the Neo4j graph database), a dynamic knowledge graph is constructed, with the identified entities as nodes and the relationships between entities as edges. Corresponding attributes are added to the nodes and edges. For example, attributes such as the customer's industry and region are added to the customer node, and attributes such as purchase time and purchase quantity are added to the purchase relationship edge.

[0060] S3, intelligent data integration, automatically maps heterogeneous data to nodes and edges in the dynamic knowledge graph through semantic mapping technology, achieving seamless integration of multi-source data.

[0061] Specifically, in this step, a semantic mapping rule base is pre-built, and based on semantic mapping technology, newly collected heterogeneous data is matched and mapped with nodes and edges in the dynamic knowledge graph;

[0062] For example, when new customer transaction data is obtained, it is accurately mapped to the corresponding customer nodes and product nodes in the knowledge graph based on the customer name, product name and other information in the data and according to the semantic mapping rules, and a new purchase relationship edge is established between the two to achieve seamless integration of multi-source data.

[0063] S4. Dynamic association analysis: Based on graph neural networks and real-time stream processing technology, it performs real-time analysis on nodes and edges in dynamic knowledge graphs, captures implicit relationships between data, and generates association analysis results.

[0064] The node representation update formula of the graph neural network is: in, represents the feature representation of node v at layer l, N(v) represents the set of neighbor nodes of node v, and W (l) and b(l) are the weight matrix and bias term of the lth layer respectively, and σ is the activation function.

[0065] Specifically, in this step, based on the graph neural network framework (such as GraphSAGE) and the real-time stream processing engine (such as Apache Flink), the nodes and edges in the dynamic knowledge graph are analyzed in real time;

[0066] Taking e-commerce companies as an example, they monitor users' purchasing behavior data in real time, use users, products, etc. as nodes, and purchasing behavior as edges. Through the node representation update formula of the graph neural network, they continuously update the feature representation of the nodes to capture the implicit relationship between users and products, and between users. For example, if a user is found to have similar purchasing patterns with other users who have purchased a specific product, association analysis results can be generated to provide a basis for precision marketing.

[0067] S5, intelligent visualization, uses augmented reality (AR) and interactive visualization technology to present dynamic knowledge graphs and association analysis results in a multi-dimensional form, supporting user-defined views and interactive operations;

[0068] The implementation of augmented reality technology includes the following steps:

[0069] S5.1. Capture the real scene through the camera and generate the real scene image;

[0070] S5.2. Superimpose the dynamic knowledge graph and association analysis results onto the real scene image in the form of a virtual image;

[0071] S5.3. Display the superimposed image to the user via a display device.

[0072] Specifically, in this step, a mobile device equipped with a high-definition camera (such as a smartphone, tablet) or a professional AR device is used to capture the real scene and generate a real scene image. The dynamic knowledge graph and association analysis results are converted into a virtual image. Through AR overlay technology, the virtual image is superimposed on the real scene image according to certain spatial layout and display rules.

[0073] For example, in a company exhibition hall, when users view the product display area through AR devices, the device can overlay the product's relevant knowledge graph information in the real scene, such as the product's R&D history, technical parameters, user reviews and other related analysis results. Users can use the device's touch screen or gesture recognition function to drag, zoom, and other operations to customize the view and easily view the information of interest.

[0074] S6, Adaptive Learning and Optimization: Analyze user behavior and data changes through machine learning algorithms, dynamically optimize the knowledge graph structure and visualization effects, and improve the intelligence level of the system;

[0075] The system optimizes the knowledge graph structure through the following formula: L OPT =L NER +L REL +λL REG , where L NER

[0076] is the entity recognition loss, L RELis the relation extraction loss, L REG is the regularization term, λ is the regularization coefficient, and its value range is [0.01, 0.1].

[0077] Specifically, in this step, by embedding user behavior monitoring code in the system interface, collecting user operation behavior data, such as click location, browsing time, operation sequence, etc., using machine learning algorithms (such as clustering algorithms, decision tree algorithms, etc.) to analyze user behavior data and generate user preference models. At the same time, real-time monitoring of data changes, such as the increase or decrease of data volume, changes in data distribution, etc., according to data changes, using the optimization formula L OPT =L NER +L REL +λL REG , (where the value range of λ is [0.01, 0.1])) dynamically adjust the structure of the knowledge graph, such as adding or deleting nodes and edges, updating the properties of nodes and edges, and optimizing the visualization effect, such as adjusting the display style and layout of virtual images, to improve the intelligence level of the system.

[0078] In summary, the method of the present invention realizes a complete process from the collection and processing of multi-source heterogeneous data to the construction of knowledge graphs, data integration, association analysis, visualization, and adaptive learning and optimization of the system. Each step cooperates with each other, and the data is processed and analyzed at different stages, providing enterprises with comprehensive and in-depth data insights, helping enterprises make more informed decisions.

[0079] Example 2

[0080] The embodiment of the present invention further provides a system for implementing the above-mentioned enterprise information integration and data visualization analysis, which specifically includes the following modules:

[0081] Data acquisition and preprocessing module, used to collect data from multi-source heterogeneous data and perform cleaning and standardization processing;

[0082] Dynamic knowledge graph construction module, used to build dynamic knowledge graphs through natural language processing technology and graph computing technology;

[0083] Intelligent data integration module, used to achieve seamless integration of multi-source data through semantic mapping technology;

[0084] Dynamic association analysis module, used for real-time association analysis based on graph neural network and real-time stream processing technology;

[0085] Intelligent visualization display module, used to display analysis results through AR and interactive visualization technology;

[0086] Adaptive learning and optimization module, used to optimize the knowledge graph structure and visualization effects through machine learning algorithms.

[0087] In this embodiment, the dynamic knowledge graph construction module includes: an entity recognition unit, used to extract entities from text data through a named entity recognition model; a relationship extraction unit, used to extract relationships between entities from text data through a relationship classification model; and a graph update unit, used to update nodes and edges in the knowledge graph in real time.

[0088] In this embodiment, the intelligent visualization display module includes: an AR overlay unit, which is used to overlay a virtual image onto a real scene image; and an interactive operation unit, which is used to support users to customize views through operations such as dragging, zooming, etc.

[0089] In this embodiment, the adaptive learning and optimization module includes: a user behavior analysis unit for analyzing user operation behavior and generating a user preference model; and a data-driven optimization unit for dynamically optimizing system parameters according to data changes.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for enterprise information integration and data visualization analysis, characterized by: The following steps are involved: S1. Data collection and preprocessing: collecting multi-source heterogeneous data from internal enterprise systems and external data sources, and generating structured data through data cleaning and standardization; S2. Dynamic knowledge graph construction: Utilize natural language processing technology to extract entities and relationships from structured data, and combine it with graph computing technology to construct a dynamic knowledge graph. The dynamic knowledge graph includes nodes, edges, and attributes, where nodes represent entities, edges represent relationships between entities, and attributes represent characteristics of entities or relationships. S3, intelligent data integration, uses semantic mapping technology to automatically map heterogeneous data to nodes and edges in a dynamic knowledge graph, achieving seamless integration of multi-source data; S4. Dynamic association analysis: Based on graph neural networks and real-time stream processing technology, it performs real-time analysis on nodes and edges in dynamic knowledge graphs, captures implicit relationships between data, and generates association analysis results. S5, intelligent visualization, uses augmented reality (AR) and interactive visualization technology to present dynamic knowledge graphs and association analysis results in a multi-dimensional form, supporting user-defined views and interactive operations; S6. Adaptive learning and optimization: Analyze user behavior and data changes through machine learning algorithms, dynamically optimize the knowledge graph structure and visualization effects, and improve the intelligence level of the system.

2. The enterprise information integration and data visualization analysis method according to claim 1, characterized in that: In step S2, entities and relations are extracted using the following formula: Entity recognition: Entities are extracted from text data using a named entity recognition model. The loss function of the named entity recognition model is: Among them, x i Represents the input text sequence, y i Represents the entity tag sequence, N is the sequence length; Relationship extraction: Extract the relationship between entities from text data through a relationship classification model. The loss function of the relationship classification model is: Among them, e1 and e2 represent entity pairs, r j Represents the relationship between entities, and M is the number of relationship categories.

3. The enterprise information integration and data visualization analysis method according to claim 1, characterized in that: In step S4, the node representation update formula of the graph neural network is: in, represents the feature representation of node v at layer l, N(v) represents the set of neighbor nodes of node v, and W (l) and b(l) are the weight matrix and bias term of the lth layer respectively, and σ is the activation function.

4. The enterprise information integration and data visualization analysis method according to claim 1, characterized in that: In step S5, the implementation of augmented reality technology includes the following steps: S5.

1. Capture the real scene through the camera and generate the real scene image; S5.

2. Superimpose the dynamic knowledge graph and association analysis results onto the real scene image in the form of a virtual image; S5.

3. Display the superimposed image to the user via a display device.

5. The enterprise information integration and data visualization analysis method according to claim 1, characterized in that: In step S6, the system optimizes the knowledge graph structure through the following formula: L OPT =L NER +L REL +λL REG , where L NER is the entity recognition loss, L REL is the relation extraction loss, L REG is the regularization term, λ is the regularization coefficient, and its value range is [0.01, 0.1].

6. The enterprise information integration and data visualization analysis method according to claim 1, characterized in that: Also included is a system, specifically comprising the following modules: Data acquisition and preprocessing module, used to collect data from multi-source heterogeneous data and perform cleaning and standardization processing; Dynamic knowledge graph construction module, used to build dynamic knowledge graphs through natural language processing technology and graph computing technology; Intelligent data integration module, used to achieve seamless integration of multi-source data through semantic mapping technology; Dynamic association analysis module, used for real-time association analysis based on graph neural network and real-time stream processing technology; Intelligent visualization display module, used to display analysis results through AR and interactive visualization technology; Adaptive learning and optimization module, used to optimize the knowledge graph structure and visualization effects through machine learning algorithms.

7. The enterprise information integration and data visualization analysis method according to claim 6, characterized in that: The dynamic knowledge graph construction module includes: Entity recognition unit, used to extract entities from text data through named entity recognition model; Relation extraction unit, used to extract the relationship between entities from text data through the relation classification model; The graph update unit is used to update the nodes and edges in the knowledge graph in real time.

8. The enterprise information integration and data visualization analysis method according to claim 6, characterized in that: The intelligent visual display module includes: An AR overlay unit, used to overlay a virtual image onto a real scene image; Interactive operation unit, used to support users to customize views through operations such as dragging, zooming, etc.

9. The enterprise information integration and data visualization analysis method according to claim 6, characterized in that: The adaptive learning and optimization module includes: User behavior analysis unit, used to analyze user operation behavior and generate user preference models; Data-driven optimization unit, used to dynamically optimize system parameters according to data changes.

Images