Data asset management method based on knowledge graph

Abstract

The invention discloses a data asset management method based on a knowledge graph, and relates to the technical field of data processing and data management.The data asset management method based on the knowledge graph comprises the steps of data collection and preprocessing, knowledge graph construction, data source integration, data asset management and intelligent recommending.The data asset management method based on the knowledge graph has remarkable advantages; comprising the steps of improving the utilization rate and safety of data assets, simplifying the management process of the data assets, reducing the management cost, and supporting intelligent decision and good expandability and customizability. The method is suitable for various scenes needing to manage a large amount of data, and an efficient and intelligent data asset management solution is provided for enterprises.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and data management, and specifically to a data asset management method based on a knowledge graph. Background Art

[0002] With the rapid development of Internet technology and the popularization of mobile intelligent terminal devices, the amount of data has shown explosive growth. Big data has many data types, including text, images, audio, video and other forms, and there are correlations and intersections between data, forming a complex and multidimensional data network. Traditional data asset management methods are difficult to effectively deal with these problems, and have shortcomings such as low data utilization, difficult management, and poor security. Therefore, a method that can uniformly model, integrate, manage and intelligently recommend data assets is needed to improve the utilization and security of data assets. Summary of the invention

[0003] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a data asset management method based on knowledge graph.

[0004] In order to achieve the above object, the present invention adopts the following technical solution: a data asset management method based on knowledge graph, characterized in that it includes the following steps:

[0005] a. Data collection and preprocessing steps: Collect data assets from multiple data sources, including structured data, semi-structured data, and unstructured data, and preprocess the collected data, including data cleaning, deduplication, format conversion, etc.;

[0006] b. Knowledge graph construction steps: Based on the characteristics and needs of data assets, a knowledge graph is constructed to structurally describe the entities, attributes, and relationships in data assets to form a knowledge network with logic and reasoning capabilities;

[0007] c. Data source integration step: Integrate data assets from different data sources to eliminate data redundancy and conflicts, improve data quality and consistency, and form a unified data view;

[0008] d. Data asset management step: Based on the knowledge graph, the data asset management platform manages and monitors data assets in a unified manner, including functions such as query, modification, deletion, and backup of data assets, as well as permission management and access control of data assets;

[0009] e. Intelligent recommendation step: Use machine learning algorithms to analyze and mine the data in the knowledge graph, discover the connections and patterns between the data, and intelligently recommend related data assets and information based on user behavior and needs.

[0010] The data collection and preprocessing steps also include classification and labeling of data to facilitate subsequent knowledge graph construction and data management.

[0011] The steps of building a knowledge graph include defining the types and structures of entities, attributes, and relationships, and establishing a relationship network between entities.

[0012] The data source integration step also includes standardizing the integrated data to ensure data consistency and comparability.

[0013] The data asset management platform also provides statistical analysis functions for data assets, including statistical information such as the distribution, usage, and access frequency of data assets.

[0014] The intelligent recommendation step also includes continuously optimizing the recommendation algorithm based on user feedback and behavioral data to improve the accuracy and personalization of recommendations.

[0015] The system includes a data collection and preprocessing module, a knowledge graph construction module, a data source integration module, a data asset management module and an intelligent recommendation module, and each module executes any corresponding step described above.

[0016] The system also includes a user interaction interface for displaying the management status of data assets, query results and recommended information, and receiving user operation instructions and feedback data.

[0017] The system also includes a computer program that, when executed by a processor, implements any of the above-mentioned data asset management methods based on knowledge graphs.

[0018] The system also includes a processor and a memory, wherein the memory stores a computer program, and when the program is executed by the processor, any of the above-mentioned data asset management methods based on knowledge graphs is implemented.

[0019] The present invention has the following beneficial effects:

[0020] Improve the utilization of data assets: By constructing a knowledge graph, this method can clearly display the associations and dependencies between data assets, allowing enterprises to more accurately understand the value of data assets and thus more efficiently utilize these data assets for business decision-making and operational optimization.

[0021] Enhance the security of data assets: This method uses a unified data asset management platform to ensure the security of data assets through permission management and access control. At the same time, through data backup and recovery mechanisms, the risk of data loss or damage can be effectively dealt with.

[0022] Simplify the data asset management process: Through the modeling and integration of knowledge graphs, this method can eliminate data redundancy and conflict, form a unified data view, and simplify the data asset management process. This enables enterprises to respond to business needs more quickly and improve management efficiency.

[0023] Reduce data asset management costs: Since this method can achieve unified management and monitoring of data assets, it reduces the need for manual intervention, thereby reducing the management costs of data assets. At the same time, through the intelligent recommendation function, enterprises can obtain the required data more accurately, reducing unnecessary data acquisition and processing costs.

[0024] Supporting intelligent decision-making: Using machine learning algorithms to analyze and mine data in knowledge graphs, this method can discover the associations and patterns between data and provide intelligent recommendations and decision-making support for enterprises. This helps enterprises grasp market trends and business opportunities more accurately and improve the scientificity and accuracy of decision-making.

[0025] Good scalability and customizability: This method adopts a modular design concept and has good scalability and customizability. Enterprises can flexibly adjust and optimize data asset management solutions according to their own business needs and data characteristics to meet the needs of different industries and enterprises.

[0026] In summary, the data asset management method based on knowledge graph proposed in this invention has significant beneficial effects, can effectively improve the utilization and security of data assets, reduce management costs, support intelligent decision-making, and has good scalability and customizability. These advantages make this method an ideal choice in the field of enterprise data asset management. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the composition block diagram of a data asset management method based on knowledge graph proposed in the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Reference Figure 1 , an embodiment of the present invention provides: a data asset management method based on knowledge graph, characterized in that it includes the following steps:

[0030] a. Data collection and preprocessing steps: Collect data assets from multiple data sources, including structured data, semi-structured data, and unstructured data, and preprocess the collected data, including data cleaning, deduplication, format conversion, etc.;

[0031] b. Knowledge graph construction steps: Based on the characteristics and needs of data assets, a knowledge graph is constructed to structurally describe the entities, attributes, and relationships in data assets to form a knowledge network with logic and reasoning capabilities;

[0032] c. Data source integration step: Integrate data assets from different data sources to eliminate data redundancy and conflicts, improve data quality and consistency, and form a unified data view;

[0033] d. Data asset management step: Based on the knowledge graph, the data asset management platform manages and monitors data assets in a unified manner, including functions such as query, modification, deletion, and backup of data assets, as well as permission management and access control of data assets;

[0034] e. Intelligent recommendation step: Use machine learning algorithms to analyze and mine the data in the knowledge graph, discover the connections and patterns between the data, and intelligently recommend related data assets and information based on user behavior and needs.

[0035] The data collection and preprocessing steps also include classification and labeling of data to facilitate subsequent knowledge graph construction and data management.

[0036] The steps of building a knowledge graph include defining the types and structures of entities, attributes, and relationships, and establishing a relationship network between entities.

[0037] The data source integration step also includes standardizing the integrated data to ensure data consistency and comparability.

[0038] The data asset management platform also provides statistical analysis functions for data assets, including statistical information such as the distribution, usage, and access frequency of data assets.

[0039] The intelligent recommendation step also includes continuously optimizing the recommendation algorithm based on user feedback and behavioral data to improve the accuracy and personalization of recommendations.

[0040] The system includes a data collection and preprocessing module, a knowledge graph construction module, a data source integration module, a data asset management module and an intelligent recommendation module, and each module executes corresponding steps as described above.

[0041] The system also includes a user interaction interface for displaying the management status of data assets, query results and recommended information, and receiving user operation instructions and feedback data.

[0042] The system also includes a computer program that, when executed by a processor, implements any of the above-mentioned data asset management methods based on knowledge graphs.

[0043] The system also includes a processor and a memory, wherein the memory stores a computer program, and when the program is executed by the processor, any of the above-mentioned data asset management methods based on knowledge graphs is implemented.

[0044] Working Principle: Data Collection and Preprocessing:

[0045] Collect data assets from multiple data sources (such as databases, file servers, API interfaces, etc.). These data assets may include structured data (such as data in relational databases), semi-structured data (such as XML, JSON files), and unstructured data (such as text, images, audio, video, etc.).

[0046] The collected data is preprocessed, including data cleaning (removing duplicate, erroneous or invalid data), deduplication, format conversion (converting data into a unified format for subsequent processing) and other steps to ensure the quality and consistency of the data.

[0047] Knowledge graph construction:

[0048] Define the types and structures of entities, attributes, and relationships based on the characteristics and needs of data assets. An entity can be any object with a unique identifier, such as a user, product, event, etc.; an attribute describes the characteristics or status of an entity; and a relationship expresses the connection between entities.

[0049] Establish a relationship network between entities to form a knowledge graph. This process usually involves natural language processing, information extraction, graph database and other technologies, which are used to extract entities and relationships from unstructured data such as text and images, and store them in a graph database.

[0050] Data source integration:

[0051] Integrate data assets from different data sources to form a unified data view. This includes steps such as data mapping (mapping entities and attributes in different data sources into a unified knowledge graph), data conversion (converting data into a unified format), and data merging (merging data from multiple data sources together).

[0052] During the integration process, the data will be standardized to ensure data consistency and comparability. Standardization may include steps such as data cleaning, data conversion, and data verification.

[0053] Data asset management:

[0054] The data asset management platform based on knowledge graph can manage and monitor data assets in a unified manner. The platform provides functions such as query, modification, deletion, and backup of data assets, and users can operate through a graphical interface or API interface.

[0055] The platform also provides data asset rights management and access control functions to ensure the security of data assets. By setting different permission levels and access policies, users' access to and operations on data assets can be controlled.

[0056] The platform also provides statistical analysis functions for data assets, including statistical information such as the distribution, usage, and access frequency of data assets. These statistical information can help users better understand the status and usage of data assets.

[0057] Smart Recommendation:

[0058] Machine learning algorithms are used to analyze and mine the data in the knowledge graph to discover the associations and patterns between the data. These associations and patterns may be hidden in the complex relationships between the data and need to be mined and extracted through algorithms.

[0059] Intelligently recommend related data assets and information based on user behavior and needs. For example, when a user queries a product, the system can recommend other products related to the product, user reviews, sales data, and other information. The intelligent recommendation function can help users find the required data assets more quickly and improve the utilization rate of data assets.

[0060] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data asset management method based on knowledge graph, characterized in that: The following steps are involved: a. Data collection and preprocessing steps: Collect data assets from multiple data sources, including structured data, semi-structured data, and unstructured data, and preprocess the collected data, including data cleaning, deduplication, and format conversion; b. Knowledge graph construction steps: Based on the characteristics and needs of data assets, a knowledge graph is constructed to structurally describe the entities, attributes, and relationships in data assets to form a knowledge network with logic and reasoning capabilities; c. Data source integration step: Integrate data assets from different data sources to eliminate data redundancy and conflicts, improve data quality and consistency, and form a unified data view; d. Data asset management step: Based on the knowledge graph, the data asset management platform manages and monitors data assets in a unified manner, including query, modification, deletion, and backup functions of data assets, as well as permission management and access control of data assets; e. Intelligent recommendation step: Use machine learning algorithms to analyze and mine the data in the knowledge graph, discover the connections and patterns between the data, and intelligently recommend related data assets and information based on user behavior and needs.

2. According to claim 1, a data asset management method based on knowledge graph is characterized in that: The data collection and preprocessing steps also include classification and labeling of data to facilitate subsequent knowledge graph construction and data management.

3. The data asset management method based on knowledge graph according to claim 1 is characterized in that: The knowledge graph construction steps include defining the types and structures of entities, attributes and relationships, and establishing a relationship network between entities.

4. The data asset management method based on knowledge graph according to claim 1 is characterized in that: The data source integration step also includes standardizing the integrated data to ensure the consistency and comparability of the data.

5. The data asset management method based on knowledge graph according to claim 1 is characterized in that: The data asset management platform also provides statistical analysis functions for data assets, including statistical information on the distribution, usage, and access frequency of data assets.

6. The data asset management method based on knowledge graph according to claim 1 is characterized in that: The intelligent recommendation step also includes continuously optimizing the recommendation algorithm based on user feedback and behavior data to improve the accuracy and personalization of the recommendation.

7. The data asset management method based on knowledge graph according to claim 1 is characterized in that: The system includes a data collection and preprocessing module, a knowledge graph construction module, a data source integration module, a data asset management module and an intelligent recommendation module, and each module executes the corresponding steps as described in any one of claims 1 to 6.

8. The data asset management method based on knowledge graph according to claim 1 is characterized in that: The system also includes a user interaction interface for displaying the management status of data assets, query results and recommended information, and receiving user operation instructions and feedback data.

9. The data asset management method based on knowledge graph according to claim 1 is characterized in that: The system also includes a computer program, which, when executed by a processor, implements the data asset management method based on knowledge graph as described in any one of claims 1 to 6.

10. The data asset management method based on knowledge graph according to claim 1, characterized in that: The system also includes a processor and a memory, wherein a computer program is stored in the memory, and when the program is executed by the processor, the data asset management method based on the knowledge graph as described in any one of claims 1 to 6 is implemented.

Images