Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data import method and device for graph database

A data import and database technology, which is applied in the field of data processing, can solve the problems of unadjustable data import speed and data import, and achieve the effect of facilitating performance tuning and speeding up data import.

Active Publication Date: 2021-03-30
南京星云数字技术有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problems of the prior art, an embodiment of the present invention provides a data import method and device for a graph database to overcome the need to first import the data into a csv format and import the data into the graph database in real time in the prior art. The speed cannot be adjusted, etc.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data import method and device for graph database
  • Data import method and device for graph database
  • Data import method and device for graph database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Some, but not all, embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0043] figure 1 is a flow chart of a data import method for a graph database shown according to an exemplary embodiment, refer to figure 1 As shown, the method includes the following steps:

[0044] S1: Register a custom spark udf function with the graph database program, so that the graph database establishes a connection with the spark resource through the spark ud...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data import method and device for a graph database, the method comprising: registering a self-defined spark udf function to a graph database program, so that the graph database establishes a connection with a spark resource through the spark udf function; Create a node attribute index in the graph database; use the spark resource to query the hive database to obtain the queried data; re-partition the queried data and register it as a temporary data table; use the spark udf function and the node attribute index, and import the temporary data table into the graph database. The present invention can realize real-time data import by using the combination of spark and graph database, and does not need to export data into csv format. By using spark technology, it can facilitate spark performance tuning and adjust the speed of data import. By using the concurrent characteristics of spark , while speeding up the import of data, there will be no loss of data.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a data import method and device for a graph database. Background technique [0002] Spark is a cluster and memory-based data processing technology. It can process a large amount of data through the combination of many machines, and can also be integrated with the graph computing framework to perform data computing operations. Spark can not only integrate in different ways, but also preprocess the data (including aggregation, filtering, transformation, etc.), and then import the preprocessed data into the graph database. [0003] In the existing technology center, there are usually the following ways to import data into the graph database: write create statement, Load CSV statement, Batch inserter, Batch import, Neo4j-import. Except for the create statement, the common feature of these methods is that the file needs to be exported to csv format first, which is very troubl...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/51G06F16/53
CPCG06F16/51G06F16/53
Inventor 王波
Owner 南京星云数字技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com