Data importing method and device for graph database

Abstract

The invention discloses a data importing method and device for a graph database, and the method comprises the steps: registering a self-defined Spark udf function in a graph database program, and enabling the graph database to be connected with a Spark resource through the Spark udf function; creating a node attribute index in the graph database; inquiring a hive database by utilizing the spark resources to obtain inquired data; re-partitioning the queried data, and registering the re-partitioned data as a temporary data table; and importing the temporary data table into the graph database through the Spark udf function and the node attribute index. According to the method, the spark is combined with the graph database, real-time data import can be realized, the data does not need to be exported into a csv format. The Spark performance can be conveniently adjusted and the data import speed can be conveniently adjusted by utilizing the Spark technology, and by utilizing the concurrencycharacteristic of the Spark, the data import can be accelerated, and meanwhile, data loss is avoided.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a data import method and device for a graph database. Background technique [0002] Spark is a cluster and memory-based data processing technology. It can process a large amount of data through the combination of many machines, and can also be integrated with the graph computing framework to perform data computing operations. Spark can not only integrate in different ways, but also preprocess the data (including aggregation, filtering, transformation, etc.), and then import the preprocessed data into the graph database. [0003] In the existing technology center, there are usually the following ways to import data into the graph database: write create statement, Load CSV statement, Batch inserter, Batch import, Neo4j-import. Except for the create statement, the common feature of these methods is that the file needs to be exported to csv format first, which is very troubl...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problems of the prior art, an embodiment of the present invention provides a data import method and device for a graph database to overcome the need to first import the data into a csv format and import the data into the graph database in real time in the prior art. The speed cannot be adjusted, etc.

Images