A Knowledge Graph Construction Method Based on Naming Rules and Caching Mechanisms
Through the naming rules and caching mechanism methods, the speed and management problems of knowledge graph databases during large-scale data updates are solved, and rapid construction, full-process management and data rollback are achieved, which improves the speed and accuracy of knowledge graph construction, and simplifies the management process.
Patent Information
- Application Number
- CN202111369404.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-07-06
- Filing Date
- 2021-11-18
- Publication Date
- 2025-08-01
- Estimated Expiration
- 2041-11-18
AI Technical Summary
The existing knowledge graph database is slow when updating large-scale data, lacking log and rollback functions, resulting in difficulty in data update and management, and difficulty in traceability of problems and data verification.
Using a method based on naming rules and caching mechanism, we will add a cache layer and develop a graph data management module to achieve rapid construction, full process management, data proofreading, problem traceability and rollback operations by formulating rich knowledge graph schema, reasonable naming specifications and detailed data caching strategies.
It improves the speed and accuracy of knowledge graph construction, reduces the difficulty of data proofreading and management, supports data rollback function, simplifies the entire life cycle management of knowledge graphs, and provides effective data support.
Smart Images

Figure CN113918663B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a knowledge graph construction method based on naming rules and caching mechanisms, which belongs to the technical field of knowledge graph construction and data storage management and utilization, and specifically relates to a knowledge graph construction update rollback and knowledge graph data management based on naming rules and caching mechanisms. Background Art
[0002] With the continuous advancement of computer technology, information and communications technology, and the internet, electronic data has experienced explosive growth, driving the development of a range of fields and related technologies, including big data and artificial intelligence. The skills and capabilities to mine and analyze valuable information from massive amounts of data are becoming increasingly important. Although major advances in big data mining and analysis, such as machine learning and deep learning, have been made, mining and analyzing massive amounts of data still faces challenges such as high repetitive workloads, high resource consumption due to frequent processing of large amounts of data, and poor interpretability of some analytical processes. To address these challenges, knowledge graph-related theories and technologies are experiencing new opportunities and developments.
[0003] Knowledge graphs can represent massive amounts of data and knowledge from different fields through data mining and analysis, information processing, data fusion, knowledge extraction and representation, knowledge fusion reasoning and graphics drawing, revealing the dynamic development laws of the knowledge field in a more concise and intuitive way, supporting higher-level data analysis and mining based on knowledge, and providing practical and valuable references, data and technical support for subject research.
[0004] The construction of a knowledge graph is a process of continuous iteration and improvement. As human experience and data volumes continue to grow, the scale of the knowledge graph will continue to grow, and the complexity of the entity relationship network will increase exponentially. Consequently, data updates, verification, and problem discovery will become increasingly difficult. Furthermore, due to the optimization strategies for most existing knowledge graph database technologies and hardware conditions, the data update speed for small, high-frequency, large-scale knowledge graphs is slow, not as fast as large, low-frequency data updates. Furthermore, many knowledge graph databases lack the process logging and rollback capabilities of traditional relational databases. Once an error occurs, it is difficult to trace the problem and data, making knowledge graph data updates and management quite difficult.
[0005] To ensure the availability, timeliness, accuracy, and stability of computer data, the design of caching is used in many scenarios such as computer storage and web browsers. Therefore, drawing on the design of caching and combining it with the design of intermediate layer data in large-scale data mining and analysis calculations, a transition layer is designed between massive data and manual experience analysis and processing and knowledge graph construction and management through reasonable and standardized naming rules, data caching strategies, and data backup strategies. This is to improve the automation level of knowledge graph construction, the meticulousness of data proofreading, reduce the difficulty of knowledge graph construction and use, support the rapid splitting, fusion, and backup of knowledge graph database data, support the realization of requirements such as data rollback and problem tracing during the knowledge graph construction and management process, and then achieve the effective management of the entire process of knowledge graph construction and use, so as to better support the research and application realization of knowledge graph-related technologies. Summary of the Invention
[0006] To solve the above technical problems, the present invention provides a knowledge graph construction method based on naming rules and caching mechanisms. This knowledge graph construction method based on naming rules and caching mechanisms realizes the rapid construction, full-process management, data proofreading, problem tracing, rollback operations, etc. of the knowledge graph by formulating a more comprehensive knowledge graph schema, reasonable naming specifications, detailed data caching strategies, and developing and integrating a graph data management module and a log management module that combine multiple functions, and adding a caching layer between the knowledge graph database and the graph composition data.
[0007] The present invention is achieved through the following technical solutions.
[0008] A knowledge graph construction method based on naming rules and caching mechanisms provided by the present invention includes the following steps:
[0009] ① Data collection: Obtain multi-modal data for constructing the knowledge graph through interfaces and crawlers;
[0010] ② Establish a knowledge graph schema: Establish a graph schema for guiding the mining and storage of knowledge graph entity data and associated data;
[0011] ③ Determine the caching strategy: Determine the caching data storage location, data storage method, and data backup strategy, determine the data range to be cached, and then construct a caching folder and a caching file naming rule;
[0012] ④ Development of the graph data management module: Complete the development and testing of the cache file automatic reading, comparison, and recognition module, the graph data import, update, deletion, and rollback module, and the cache file management module;
[0013] ⑤Entity relationship data extraction: Extract the necessary entity attribute data and associated relationship attribute data of the knowledge graph from the collected data according to the content planned by the graph schema;
[0014] ⑥Data caching and update verification: Store the extracted entity relationship data according to the requirements of the caching policy to obtain cached data. When the accumulation of cached data meets certain conditions, use the graph data management module to perform preliminary data verification and update, and process the problematic data;
[0015] ⑦Graph generation and secondary verification: Use the graph data management module to automatically import the data that has completed preliminary verification and update into the knowledge graph database, and perform secondary data verification on the updated knowledge graph. Determine the data adjustment strategy according to the data verification situation, and then generate the graph;
[0016] ⑧Backup and management of cached data: Complete the backup and management of cached data according to the cached data backup strategy.
[0017] The above step ① is divided into the following steps:
[0018] (1.1) Obtain the necessary conventional numerical data, text data, image data, video data, and voice data for constructing the knowledge graph through interfaces and crawlers to form multi-modal data;
[0019] (1.2) Perform preliminary data cleaning and processing on the obtained multi-modal data, and classify and store them according to the data format.
[0020] The above step ② is divided into the following steps:
[0021] (2.1) Define the basic principles and conventions for constructing the knowledge graph, including background description, graph usage description, data requirements and graph requirements description, data confidentiality agreement, data interpretation, professional vocabulary interpretation, data field naming convention, data source, data form;
[0022] (2.2) Define the concepts, uniqueness constraints, categories, domain definitions, attribute naming, attribute interpretation, data association methods, attribute constraints, and association constraints of the knowledge graph entity data and associated relationship data;
[0023] (2.3) Define the technology selection and graph architecture of the knowledge graph, including graph data storage technology, graph data retrieval and application technology, graph architecture composition, graph and data development environment;
[0024] (2.4) Define the note information of the knowledge graph, including other tools, data used, and existing problems.
[0025] The above step ③ is divided into the following steps:
[0026] (3.1) Determine the scope, storage location, and storage method of cached data;
[0027] (3.2) Determine the backup strategy for cached data, including backup data naming rules, backup locations, backup quantities, backup data management, and usage strategies;
[0028] (3.3) Define the naming rules for the storage folders of entity and associated relationship data;
[0029] (3.4) Define the naming rules for cached data corresponding to entity and associated relationship data.
[0030] In the said step (3.1), the scope of cached data includes all data caching and partial data caching; the storage locations include the local file system, local server, and cloud server, and the cached data exists in the same path or multiple path file systems; the storage methods include structured data storage, unstructured data storage, and semi-structured data storage;
[0031] In the said step (3.2), the data backup locations include local and server;
[0032] In the said step (3.4), the cached data storage name contains entity or relationship keyword terms or codes, data uniqueness field names or codes, names or codes of newly created entity or relationship types, data update times or codes, data processing methods or codes, and other data-related descriptions or codes. The order of each piece of information in the name is not restricted, and each piece of information is separated and identified by specific characters to ensure that the naming meets the system file naming requirements.
[0033] The said step ④ is divided into the following steps:
[0034] (4.1) Complete the development and testing of the automatic reading, comparison, and recognition module for cached data files and the module for importing, updating, deleting, and rolling back graph data;
[0035] (4.2) Complete the development and testing of the management module for cached data files and the update management module for the log system;
[0036] (4.3) Complete the testing and optimization of the stability, availability, timeliness, and accuracy of the above-mentioned modules.
[0037] In step (4.1), by reading and identifying the key information in the cache data name and comparing it with the log content, duplicate-named or similar-named data is found; the repeatability and validity of the data are judged by judging the similarity of data fields and data content in the cache data; the graph data import, update, deletion, and rollback module includes the import, creation, update, deletion, rollback, automatic identification and processing of duplicate data of single and multiple graph data, and this module supports automated, semi-automated, and manual call operations; the rollback of graph data is to revoke all the recent knowledge graph data update operations, supporting manual rollback and automatic rollback. The difference between manual rollback and automatic rollback lies in whether the parameter input is manual or automatic, and the precise rollback operation of the data is realized by judging through the name and content of the cache file.
[0038] In step (4.2), the management of cache data files includes the management of creating, copying, deleting, and renaming data files, and the log system needs to record the modified content, modified target, and modification time of the data files.
[0039] Step ⑤ is divided into the following steps:
[0040] (5.1) Extract the entity attribute data defined by the fusion graph schema from the collected data, and perform data caching and naming according to the data caching strategy regulations.
[0041] (5.2) Extract the relationship attribute data defined by the fusion graph schema from the collected data, and perform data caching and naming according to the data caching strategy regulations.
[0042] Step ⑥ is divided into the following steps:
[0043] (6.1) According to the requirements of the caching strategy, cache and accumulate the entity attribute data and relationship attribute data extracted in step ⑤, and record the number and volume of the cached data in real time.
[0044] (6.2) When the accumulation of cached data meets the limit conditions, perform proofreading and updating of the cached data, and record and process the problems existing in the data.
[0045] The limit conditions include the limit of the number of cached data, the limit of the size of cached data, the limit of processing time, the limit of the volume of processed data, and the limit conditions formulated manually.
[0046] The proofreading of the cached data is realized through the data caching strategy, naming rules, and graph data management module.
[0047] The proofreading content includes the proofreading of file name similarity and correctness, and the proofreading of data content repeatability and correctness. The proofreading method supports automatic proofreading and manual proofreading.
[0048] The data problems include duplicate data, duplicate naming, incorrect naming, incorrect data, data missing, and data anomalies.
[0049] In step ⑦, the knowledge graph data management module generates and updates knowledge graph data automatically and batch by identifying the name content of the cache file;
[0050] In step ⑦, the secondary data verification and adjustment work: Manually or through an automated script, judge the rationality, validity, and correctness of the data in the knowledge graph after the data update is completed, and determine whether to enter step ⑧, whether data adjustment is required, and whether data rollback is required according to the judgment result;
[0051] In step ⑧, data backup and management: According to the data situation and hardware conditions, perform secondary backup on all or part of the data in the cache data, uniformly name the backup folder and backup file according to the naming rule, and note the backup time, backup person, and backup content.
[0052] The beneficial effects of the present invention are as follows: It can use the cache mechanism to improve the construction and generation speed and accuracy of the knowledge graph under the background of massive data, reduce the difficulty of data verification and log generation management in the process of knowledge graph construction, support the implementation of the knowledge graph data rollback function, facilitate data backup encryption and transmission, and can use reasonable naming rules and functional modules to reduce the difficulty of knowledge graph data management and realize automatic processing, comparison, knowledge graph data import and update, and rollback of cache files, thereby reducing the difficulty of graph construction and management, improving the graph construction speed, supporting the realization of the full life cycle management requirements of graph construction and use, etc., and providing effective data and technical support for subsequent knowledge graph application research. Brief Description of the Drawings
[0053] Figure 1 It is the structural diagram of the present invention. Detailed Embodiment
[0054] The technical solution of the present invention will be further described below, but the scope of protection is not limited thereto.
[0055] As Figure 1 shown, a method for constructing a knowledge graph based on a naming rule and a cache mechanism includes the following steps:
[0056] ① Data collection: Obtain the necessary data for constructing the knowledge graph through interfaces and crawlers;
[0057] Specifically, it is divided into the following steps:
[0058] (1.1) Obtain the data necessary for constructing the knowledge graph through interfaces and crawlers, including conventional numerical data, text data, image data, video data, and voice data;
[0059] (1.2) Conduct preliminary data cleaning and processing on the acquired multimodal data, and classify and store it in a reasonable manner according to the data format.
[0060] ② Knowledge graph schema design: Design the graph schema used to guide the mining and storage of knowledge graph entity data and associated data;
[0061] Specifically, it is divided into the following steps:
[0062] (2.1) Define the basic principles and conventions for constructing the knowledge graph, including background description, graph usage description, data requirements and graph requirements description, data confidentiality agreement, data interpretation, professional vocabulary interpretation, data field naming convention, data source, and data form;
[0063] (2.2) Define the concepts, uniqueness constraints, categories, domain definitions, attribute naming, attribute interpretation, data association methods, attribute constraints, and association constraints of the knowledge graph entity data and associated relationship data;
[0064] (2.3) Define the knowledge graph technology selection and graph architecture, including graph data storage technology, graph data retrieval and application technology, graph architecture composition, and graph and data development environment;
[0065] (2.4) Define the knowledge graph remarks information, including other tools, data used, and existing problems.
[0066] Furthermore, all information that affects the construction, use, management, and expansion of the knowledge graph needs to fill in the remarks information to ensure the smooth progress of the work and handover process.
[0067] ③ Develop a caching strategy: Determine the cache data storage location, data storage method, and data backup strategy, and determine the data range to be cached;
[0068] Specifically, it is divided into the following steps:
[0069] (3.1) Determine the data range, storage location, and storage method of the cache data;
[0070] Preferably, the data range of the cache data includes all data caching and partial data caching, the storage location includes the local file system, local server, and cloud server, the cache data exists in the same path or multiple path file systems, and the storage method includes structured data storage, unstructured data storage, and semi-structured data storage.
[0071] (3.2) Determine the backup strategy for cached data, including backup data naming rules, backup locations, backup quantities, backup data management, and usage policies;
[0072] Preferably, the data backup locations include local and server;
[0073] ④ Naming rule formulation: Formulate naming rules for cache folders and cache files;
[0074] Specifically, it is divided into the following steps:
[0075] (4.1) Define the naming rules for the data storage folder of entities and associated relationships;
[0076] (4.2) Define the naming rules for the cached data corresponding to the entities and associated relationships.
[0077] Preferably, the cached data storage name contains entity or relationship keyword terms or codes, data uniqueness field names or codes, names or codes of newly created entity or relationship types, data update times or codes, data processing methods or codes, other data - related descriptions or codes. The order of each piece of information in the name is not restricted, and each piece of information is separated by a specific character for identification and ensures that the naming meets the system file naming requirements.
[0078] ⑤ Development of the graph data management module: Complete the development and testing of the cache file automatic reading, comparison, and identification module, the graph data import, update, deletion, and rollback module, and the cache file management module;
[0079] Specifically, it is divided into the following steps:
[0080] (5.1) Complete the cache data file automatic reading, comparison, and identification module, and the graph data import, update, deletion, and rollback module;
[0081] Preferably, by reading and identifying the key information in the cached data name and comparing it with the log content, duplicate - named or similar - named data is found; by judging the similarity of data fields and data content in the cached data, data repeatability and validity are judged; the graph data import, update, deletion, and rollback module includes the import, creation, update, deletion, rollback, automatic identification and processing of duplicate data for single - and multiple - graph data. The module supports automated, semi - automated, and manual call operations, and the input parameters of the module are designed to be concise and clear; the rollback of graph data refers to completely canceling the most recent knowledge graph data update operation, supporting manual rollback and automatic rollback. The difference between manual rollback and automatic rollback lies in whether the parameters are input manually or automatically, and the precise rollback operation of data is realized by judging through the name and content of the cache file.
[0082] (5.2) Complete the cache data file management and log system update management module;
[0083] Preferably, the management of the cached data files includes the management of creating, copying, deleting, and renaming data files. The logging system needs to record the modified content, modified target, and modification time of the data files.
[0084] (5.3) Complete the stability, availability, timeliness, and accuracy testing and optimization of the above-mentioned modules;
[0085] ⑥ Entity relationship data extraction: According to the content planned by the graph schema, extract the necessary entity attribute data and associated relationship attribute data of the knowledge graph from the collected data;
[0086] Specifically, it is divided into the following steps:
[0087] (6.1) Extract the entity attribute data defined by the fusion graph schema from the collected data, and perform data caching and naming according to the data caching strategy regulations to ensure the validity and uniqueness of the data;
[0088] (6.2) Extract the associated relationship attribute data defined by the fusion graph schema from the collected data, and perform data caching and naming according to the data caching strategy regulations to ensure the validity, uniqueness, and corresponding relationship with the entity attribute data.
[0089] ⑦ Data caching and update verification: According to the requirements of the caching strategy, store the extracted entity relationship data to obtain cached data. When the accumulation of cached data meets certain conditions, use the graph data management module to perform preliminary verification and update of the data, and process the problematic data;
[0090] Specifically, it is divided into the following steps:
[0091] (7.1) According to the requirements of the caching strategy, cache and accumulate the entity and relationship data extracted in step ⑥, and record the number and volume of the cached data in real time;
[0092] (7.2) When the accumulation of cached data meets certain conditions, perform verification and update of the cached data, and record and process the problems existing in the data;
[0093] Furthermore, the cached data accumulation limit conditions include the limit of the number of cached data, the limit of the size of cached data, the limit of processing time, the limit of the volume of processed data, and the limit conditions formulated manually; the verification of cached data is achieved through the data caching strategy, naming rules, and graph data management module. The verification content includes the similarity and correctness verification of file names and the repeatability and correctness verification of data content. The verification methods support automatic verification and manual verification; data problems include duplicate data, duplicate naming, incorrect naming, incorrect data, data loss, and data anomalies.
[0094] ⑧Spectrum generation and secondary verification: Use the spectrum data management module to automatically import the data that has completed the preliminary verification and update into the knowledge graph database, and perform secondary data verification on the updated knowledge graph, and determine the data adjustment strategy according to the data verification situation;
[0095] Specifically, it is divided into the following steps:
[0096] (8.1) Use the spectrum data management module to automatically or semi-automatically realize the import, automatic generation and update of single or batch cached data into the knowledge graph database;
[0097] Furthermore, the spectrum data management module can generate and update knowledge graph data automatically and in batches by identifying the name and content of the cache file, improving the spectrum generation speed.
[0098] (8.2) Perform secondary verification of the knowledge graph data in the form of manual or script operation. The verification content includes the validity and correctness of the data.
[0099] Furthermore, judge the rationality, validity and correctness of the data in the knowledge graph after the data update through manual or automated scripts, and determine whether to enter the next link, whether data adjustment is required, and whether data rollback is required according to the judgment results.
[0100] It also includes step ⑨ Cache data backup and management: According to the data situation and hardware conditions, perform secondary backup on all or part of the data in the cache data, uniformly name the backup folder and backup file according to the naming rules, and note the backup time, backup person, and backup content;
[0101] Specifically, it is divided into the following steps:
[0102] (9.1) According to the cache policy requirements, back up the data in the cache data that meets the backup requirements;
[0103] (9.2) Manage the backup data of the cache data;
[0104] The present invention is a method for constructing, updating and rolling back a knowledge graph based on naming rules and a caching mechanism. Data necessary for constructing the knowledge graph is obtained through interfaces and crawlers; a knowledge graph schema is designed to guide the mining, storage of entity data and association data in the knowledge graph; the storage location, storage method and data backup strategy of cached data are determined, and the range of data to be cached is determined; naming rules for cache folders and cache files are formulated; the development and testing of a cache file automatic reading, comparison and recognition module, a knowledge graph data import, update, deletion and rollback module, and a cache file management module are completed; according to the content planned by the knowledge graph schema, the necessary entity attribute data and association relationship attribute data of the knowledge graph are extracted from the collected data; according to the requirements of the caching strategy, the extracted entity relationship data is stored to obtain cached data. When the accumulation of cached data meets certain conditions, the knowledge graph data management module is used to perform preliminary data verification and update, and process the problematic data; the knowledge graph data management module is used to automatically import the data that has completed preliminary verification and update into the knowledge graph database, and perform secondary data verification on the updated knowledge graph, and determine the data adjustment strategy according to the data verification situation; the backup and management of cached data are completed according to the cached data backup strategy.
[0105] Embodiment
[0106] As described above, the implementation process of the present invention is as follows:
[0107] 1) Data necessary for constructing the knowledge graph is obtained through interfaces and crawlers, including conventional numerical data, text data, image data, video data, and voice data;
[0108] 2) The obtained multi-modal data is subjected to preliminary data cleaning and data processing, and is classified and stored in a reasonable manner according to the data format;
[0109] 3) Define the basic principles and conventions for constructing the knowledge graph, including background description, description of the purpose of the knowledge graph, data requirements and knowledge graph requirements description, data confidentiality agreement, data interpretation, professional vocabulary interpretation, data field naming convention, data source, data form; define the concepts, uniqueness constraints, categories, domain definitions, attribute naming, attribute interpretation, data association method, attribute constraints, and association constraints of knowledge graph entity data and association relationship data; define the knowledge graph technology selection and knowledge graph architecture, including knowledge graph data storage technology, knowledge graph data retrieval and application technology, knowledge graph architecture composition, knowledge graph and data development environment; define the knowledge graph note information, including other tools, data used, and existing problems;
[0110] 4) Determine the range, storage location, and storage method of the cached data;
[0111] 5) Determine the backup strategy for cached data, including backup data naming rules, backup locations, backup quantities, backup data management, and usage policies;
[0112] 6) Define the entity and associated relationship data storage folders and cached data naming rules;
[0113] 7) Complete the automatic reading, comparison, and recognition module for cached data files, the graph data import, update, deletion, and rollback module; complete the cached data file management and log system update management module;
[0114] 8) Complete the stability, availability, timeliness, and accuracy testing and optimization of each of the above modules;
[0115] 9) Extract the entity attribute data defined by the fusion graph schema from the collected data, cache and name the data according to the data caching strategy, and ensure the validity and uniqueness of the data;
[0116] 10) Extract the associated relationship attribute data defined by the fusion graph schema from the collected data, cache and name the data according to the data caching strategy, and ensure the validity, uniqueness of the data, and the corresponding relationship with the entity attribute data;
[0117] 11) According to the requirements of the caching strategy, cache and accumulate the extracted entity and relationship data, and record the quantity and volume of the cached data in real time; when the accumulation of cached data meets certain conditions, conduct proofreading and updating of the cached data, record and handle the problems existing in the data;
[0118] 12) Use the graph data management module to automatically or semi-automatically implement the import and automatic generation and update of single or batch cached data into the knowledge graph database; conduct secondary proofreading of the knowledge graph data in the form of manual or script operations, and the proofreading content includes the validity and correctness of the data;
[0119] 13) According to the requirements of the caching strategy, back up the data in the cached data that meets the backup requirements, and manage the backup data of the cached data.
[0120] In summary, the present invention is a set of knowledge graph construction and management systems. All changes during the knowledge graph construction and management process can be recorded and saved, and operations such as adding, updating, and rolling back knowledge graph data can be achieved through cached data and historical records. It is applicable to the implementation of the full life cycle management system of the knowledge graph, the establishment of high-quality knowledge graphs, the automated knowledge graph construction system, and the effective backup and management of knowledge graph data of all sizes.
Claims
1. An operation method for constructing a knowledge graph based on naming rules and caching mechanisms, characterized in that: It includes the following steps: ① Data collection: Obtain multi-modal data for constructing a knowledge graph through interfaces and crawlers; ② Establish a knowledge graph schema: Establish a graph schema for guiding the mining, storage of knowledge graph entity data and associated data; ③ Determine the caching strategy: Determine the storage location, storage method and data backup strategy of cached data, determine the scope of data to be cached, and then construct the naming rules for the cache folder and cache files; ④ Develop the graph data management module: Complete the development and testing of the automatic reading, comparison and recognition module for cache files, the graph data import, update, deletion and rollback module, and the cache file management module; ⑤ Extract entity relationship data: According to the content planned by the graph schema, extract the necessary entity attribute data and associated relationship attribute data of the knowledge graph from the collected data; ⑥ Data caching and update verification: Store the extracted entity relationship data according to the requirements of the caching strategy to obtain cached data. When the accumulation of cached data meets certain conditions, use the graph data management module to perform preliminary data verification and update, and process the problematic data; ⑦ Graph generation and secondary verification: Use the graph data management module to automatically import the data that has completed the preliminary verification and update into the knowledge graph database, and perform secondary data verification on the updated knowledge graph. Determine the data adjustment strategy according to the data verification situation, and then generate the graph; ⑧ Backup and management of cached data: Complete the backup and management of cached data according to the cached data backup strategy.
2. The operation method of the knowledge graph construction based on the naming rule and the caching mechanism according to claim 1, wherein: The step ① is divided into the following steps: (1.1) Obtain the necessary conventional numerical data, text data, image data, video data, and voice data for constructing the knowledge graph through interfaces and crawlers to form multi-modal data; (1.2) Perform preliminary data cleaning and data processing on the obtained multi-modal data, and store them in categories according to the data format.
3. The operation method of the knowledge graph construction based on the naming rule and the caching mechanism according to claim 1, characterized in that: The step ② is divided into the following steps: (2.1) Define the basic principles and conventions for constructing the knowledge graph, including background description, graph usage description, data requirements and graph requirements description, data confidentiality agreement, data interpretation, professional vocabulary interpretation, data field naming convention, data source, data form; (2.2) Define the concepts, uniqueness constraints, categories, domain definitions, attribute naming, attribute interpretation, data association methods, attribute constraints, and association constraints of the knowledge graph entity data and associated relationship data; (2.3) Define the knowledge graph technology selection and graph architecture, including graph data storage technology, graph data retrieval and application technology, graph architecture composition, graph and data development environment; (2.4) Define the knowledge graph note information, including other tools, data used, and existing problems.
4. The operation method of the knowledge graph construction based on the naming rule and the cache mechanism according to claim 1, characterized in that: The step ③ is divided into the following steps: (3.1) Determine the scope, storage location, and storage method of the cached data; (3.2) Determine the backup strategy of the cached data, including the naming rule of the backup data, backup location, backup quantity, backup data management and usage strategy; (3.3) Define the naming rule for the storage folder of the entity and associated relationship data; (3.4) Define the naming rule for the cached data corresponding to the entity and associated relationship data.
5. The operation method of the knowledge graph construction based on the naming rule and the caching mechanism according to claim 4, wherein: In step (3.1), the scope of cached data includes all data caching and partial data caching; the storage locations include the local file system, local server, and cloud server, and the cached data exists in the same path or multiple path file systems; the storage methods include structured data storage, unstructured data storage, and semi-structured data storage; in step (3.2), the data backup locations include local and server. In step (3.4), the name of the cached data storage contains entity or relationship keyword terms or codes, data uniqueness field name or codes, name or codes of newly created entities or relationship types, data update time or codes, data processing method or codes, and other data-related descriptions or codes. The order of the information in the name is not restricted, and the information is separated by specific characters for identification and the naming meets the system file naming requirements.
6. The operation method of the knowledge graph construction based on the naming rule and the caching mechanism according to claim 1, characterized in that: Step ④ is divided into the following steps: (4.1) Complete the development and testing of the automatic reading, comparison, and identification module for cached data files and the graph data import, update, deletion, and rollback module; (4.2) Complete the development and testing of the cached data file management and log system update management module; (4.3) Complete the testing and optimization of the stability, availability, timeliness, and accuracy of the above modules.
7. The operation method of the knowledge graph construction based on the naming rule and the caching mechanism according to claim 6, wherein: In step (4.1), by reading and identifying the key information in the name of the cached data and comparing it with the log content, duplicate-named or similar-named data is found; the repetition and validity of the data are judged by judging the similarity of the data fields and data content in the cached data; the graph data import, update, deletion, and rollback module includes the import, creation, update, deletion, rollback, automatic identification and processing of duplicate data of single and multiple graph data. This module supports automated, semi-automated, and manual call operations; the rollback of graph data is to revoke all the latest knowledge graph data update operations, supporting manual rollback and automatic rollback. The difference between manual rollback and automatic rollback lies in whether the parameter input is manual or automatic, and the precise rollback operation of the data is realized by judging the name and content of the cached file; in step (4.2), the management of cached data files includes the management of data file creation, copying, deletion, and renaming, and the log system needs to record the modified content, modification target, and modification time of the data file.
8. The operation method of the knowledge graph construction based on the naming rule and the caching mechanism according to claim 1, characterized in that: Step ⑤ is divided into the following steps: (5.1) Extract the entity attribute data defined by the fusion graph schema from the collected data and perform data caching and naming according to the data caching strategy regulations; (5.2) Extract the associated relationship attribute data defined by the fusion graph schema from the collected data and perform data caching and naming according to the data caching strategy regulations.
9. The operation method of the knowledge graph construction based on the naming rule and the cache mechanism according to claim 1, characterized in that: Step ⑥ is divided into the following steps: (6.1) According to the requirements of the caching policy, cache and accumulate the entity attribute data and associated relationship attribute data extracted in step ⑤, and record the number and volume of the cached data in real time; (6.2) When the accumulation of the cached data meets the limit conditions, perform proofreading and update of the cached data, and record and process the problems existing in the data; the limit conditions include the limit on the number of cached data, the limit on the size of the cached data, the limit on the processing time, the limit on the volume of the processed data, and the conditions restricted by humans; the proofreading of the cached data is implemented through the data caching policy, naming rules, and graph data management module; the content of the proofreading of the cached data includes the similarity and correctness proofreading of the file names and the repeatability and correctness proofreading of the data content, and the proofreading methods support automatic proofreading and manual proofreading; the repeatability and correctness proofreading of the data content include duplicate data, duplicate naming, incorrect naming, incorrect data, data loss, and data anomalies.
10. The operation method of the knowledge graph construction based on the naming rule and the caching mechanism according to claim 1, wherein: In step ⑦, the graph data management module generates and updates the knowledge graph data automatically and in batches by identifying the name content of the cached files; in step ⑦, the secondary data proofreading and adjustment work: judge the rationality, effectiveness, and correctness of the data in the knowledge graph after the data update through manual or automated scripts, and determine whether to enter step ⑧, whether data adjustment is required, and whether data rollback is required according to the judgment results; in step ⑧, store data backup and management: according to the data situation and hardware conditions, perform secondary backup on all or part of the data in the cached data, and uniformly name the backup folders and backup files according to the naming rules, and note the backup time, backup person, and backup content.
Citation Information
Patent Citations
Operation method of knowledge graph structure based on naming rule and cache mechanism
CN113434610A