A metadata-based knowledge graph processing method

Abstract

The application discloses a metadata-based knowledge graph processing method, comprising the following steps: S1) constructing a data unit taking metadata and attributes thereof as a unit; S2) classifying metadata by dividing metadata types according to enumeration types; S3) establishing corresponding documents according to different types and indexes of the metadata, wherein the documents contain unique identifiers of the metadata and metadata attributes; S4) forming a relationship node for each metadata and attribute thereof and storing the relationship node in a mesh mode; and S5) performing quick retrieval based on the relationship node of the metadata. The metadata-based knowledge graph processing method can reasonably store a metadata mesh structure and better perform retrieval and analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to a data generation and retrieval analysis method, in particular to a knowledge graph processing method based on metadata. BACKGROUND

[0002] Metadata is also called intermediate data and relay data, which is data about data and mainly describes the information of data properties to support functions such as indicating storage location, historical data, resource search, file record, etc.

[0003] The core basic unit of the knowledge graph is a triple composed of entity, relationship and entity, which displays the knowledge development process and structural relationship through the knowledge graph, describes knowledge resources and their carriers by using visualization technology, and mines, analyzes, constructs, draws and displays knowledge and their mutual relationships.

[0004] With the passage of time, the amount of data is increasing, the data properties are changing, and the relationship of data is becoming more and more complex. Metadata describes data, including the properties of metadata and the relationship between metadata, which effectively abstracts the data model and manages the data. However, with the continuous expansion of data types, metadata and the relationship between metadata will inevitably expand, and gradually form a mesh structure of metadata. It is impossible to quickly locate metadata and the influence range of metadata from the mesh data, so that it is impossible to quickly locate, analyze and display data properties and data relationships. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a knowledge graph processing method based on metadata, which can reasonably store the mesh relationship of metadata and better perform retrieval and analysis, so as to quickly find the properties of data corresponding to metadata and the relationship network of corresponding data through the metadata knowledge graph.

[0006] The technical solution adopted by the present application to solve the above technical problem is to provide a knowledge graph processing method based on metadata, comprising the following steps: S1) constructing a data unit taking metadata and its properties as a unit; S2) dividing metadata types according to enumeration types, and classifying metadata; S3) establishing corresponding documents according to different types and indexes of metadata, wherein the documents contain unique identifiers of metadata and metadata properties; S4) forming a relationship node for each metadata and its properties and storing in a mesh manner; and S5) performing quick retrieval based on the relationship node of metadata.

[0007] The metadata-based knowledge graph processing method, wherein each metadata in the step S1 has one or more attributes, and the storage unit of each metadata has a unique identifier and is column-extensible for addition, deletion, modification and query operations.

[0008] The metadata-based knowledge graph processing method, wherein the metadata in the step S4 is stored in a Map set at a bottom layer, including the following processes: judging whether each relationship node exists in the Map set; if not, it is the first time to store the node relationship, and a 1-to-1 preprocessing is performed on the metadata relationship to form a key-value pair; if yes, it is a 1-to-N node relationship, and a 1-to-1 mapping relationship is stored first, and then an ordered array is created to store the corresponding N-1 nodes, forming a key-List <key>Storage structure of the metadata.

[0009] The metadata-based knowledge graph processing method, wherein, in the step S4, a MapSize<key, size> and a MapLink<sizeKey, NextKey> storage mode are adopted, when the size=1, the MapLink<sizeKey, NextKey> is a MapLink<Key, NextKey>; when storing the node relationship for the first time, the size is 1, and the MapSize(key, 1) is stored; for a 1-to-N node relationship, after storing a 1-to-1 mapping relationship, a size value is taken as a seed, and the size and the key are used to generate a unique sizeKey corresponding to the key, and the NextKey is put into the MapLink(sizeKey, NextKey) together, thereby storing the corresponding N-1 nodes in the ordered array, and forming a storage structure of the MapLink(sizeKey, NextKey); wherein the sizeKey is the original key plus the size sequence number, the generated sizeKey and the NextKey are unique identification codes of metadata of a node corresponding to a next node of the original key corresponding link.

[0010] The metadata-based knowledge graph processing method, wherein, in the step S5, a quick search is performed based on adjacent nodes pointed by each relationship node, so that the metadata search complexity is O(1), and the relationship search complexity is O(n).

[0011] The metadata-based knowledge graph processing method provided by the present application can effectively store a metadata mesh structure, and can better perform search and analysis; each metadata and attribute thereof forms a relationship node and is stored in a mesh manner, each relationship node has a pointing adjacent node; and the worst case of the metadata relationship search complexity is an ordered array search, and the time complexity is O(n), and the optimal case is O(1). BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 The metadata-based knowledge graph processing flowchart of the present application. DETAILED DESCRIPTION

[0013] The present application will be further described below in combination with the drawings and embodiments.

[0014] Figure 1 The metadata-based knowledge graph processing flowchart of the present application.

[0015] Please refer to Figure 1 The metadata-based knowledge graph processing method provided by the application comprises two elements of nodes and relations, and forms a mesh data storage. The node in the application is metadata and its attribute; and the relation in the application is the relation between the metadata.

[0016] 1. Metadata and its attribute. A data unit in units of metadata and its attribute is constructed, each metadata has one or more attributes, the attributes can be different, and has column uncertainty. Therefore, the metadata storage unit needs to have unique identification, which is used as the unique declaration of the metadata, and the metadata storage unit also needs to have column expansion, and the attribute can be conveniently added, deleted, modified and inquired for storing the metadata attribute.

[0017] 2. Metadata relation. The relation between the metadata has directional relation, non-directional relation, and multiple different relations between two metadata; therefore, the metadata relation storage layer constructed can store the metadata unique identification and its attribute, and the relation between the metadata storage units, meet the directionality, non-directionality and diversity of the relation.

[0018] 3. Metadata retrieval analysis. The mesh relation type data has the characteristics of relation depth and relation complexity. In the case of large data volume and high relation complexity, the data relation mesh and the metadata attribute of the relation mesh node with a specified metadata as the data center can be quickly retrieved through the application, so as to realize the quick retrieval analysis from point to mesh.

[0019] Embodiment:

[0020] 1. Metadata and its attribute storage unit. The metadata node information and the node attribute information are stored.

[0021] 2. Different types (type) are established according to the metadata type. The metadata type can be divided according to the enumeration type, and the metadata is classified.

[0022] 3. The metadata relation storage is constructed, and the application adopts MapSize<key,size> and MapLink<sizeKey,NextKey> storage methods. When size=1, MapLink<sizeKey,NextKey> is MapLink<Key,NextKey>. A new unique key based on the original key is generated by adding the size serial number to the original key, and then the key is ensured not to be repeated in the link Map, so as to solve the 1-to-many link problem.

[0023] The specific storage process is as follows: whether each relation exists in the Map set is judged first:

[0024] If it does not exist, it is the first time to store the node relationship. Store a Map collection, preprocess the simple data of the one-to-one metadata relationship, set it as the key value according to the unique identifier of each metadata, and store MapSize(key, 1) with a size of 1; MapLink stores the link, and constructs a key-value pair Key-NextKey, where NextKey is the unique identifier of the metadata node corresponding to the next node in the key link;

[0025] If it exists, it represents a one-to-N node relationship. First, the one-to-one mapping is stored. Then, based on the size, the size is iterated over, using the size value as a seed. The size is added to the key to generate a unique sizeKey corresponding to the key. After generating the sizeKey, the NextKey is added to MapLink(sizeKey, NextKey) to create an ordered array storing the corresponding N-1 nodes, forming the MapLink(sizeKey, NextKey) storage structure. Here, sizeKey is the original key plus the size index. NextKey is the unique identifier of the metadata of the node corresponding to the next node in the link corresponding to the original key.

[0026] 4. Metadata Relation Retrieval. Retrieval can be performed on any metadata node; Search_Node(Key) has a time complexity of O(1). Relationship retrieval can also be performed on any metadata node; Search_Relation(Key) has a time complexity of O(n), with an optimal time complexity of O(1).

[0027] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be defined by the claims.< / key>

Claims

1. A knowledge graph processing method based on metadata, characterized in that, Includes the following steps: S1) Construct data units based on metadata and its attributes; S2) Classify metadata by enumeration type; S3) Create corresponding documents for metadata according to different types and their respective indexes. Each document contains a unique identifier for the metadata and metadata attributes. S4) Each metadata and its attributes form a relation node and are stored in a mesh manner; S5) Fast retrieval based on metadata relationship nodes; In step S4, the metadata is stored in a Map collection at the underlying level, including the following process: For each relation node, first check if it exists in the Map collection; If it does not exist, then this is the first time the node relationship is stored, and the metadata relationship is preprocessed one-to-one to form key-value pairs; If it exists, it represents a one-to-N node relationship. First, store the one-to-one mapping relationship, then create an ordered array to store the corresponding N-1 nodes, forming a key-list. <key> Storage structure;< / key> Step S4 uses MapSize<key,size> and MapLink<sizeKey,NextKey> Storage method: When size=1, then MapLink<sizeKey,NextKey> For MapLink<Key,NextKey> ; When storing the node relationship for the first time, the size is 1, and the storage is MapSize(key, 1). For a 1-to-N node relationship, after storing the 1-to-1 mapping relationship, the size value is used as the seed, and the size is added to the key to generate a unique sizeKey based on the key. After generating the sizeKey, the NextKey is put into MapLink(sizeKey, NextKey) to create an ordered array to store the corresponding N-1 nodes, forming the MapLink(sizeKey, NextKey) storage structure; where sizeKey is the original key plus the size index, and NextKey is the unique identifier of the metadata of the node corresponding to the next node in the link of the original key; Step S5 performs a fast retrieval based on the adjacent nodes pointed to by each relation node, making the metadata lookup complexity O(1) and the relation lookup complexity O(n).

2. The knowledge graph processing method based on metadata as described in claim 1, characterized in that, In step S1, each metadata has one or more attributes, each metadata storage unit has a unique identifier, and has column extensibility to perform add, delete, modify, and query operations.

Images