Data cleaning method, graph database device and computer readable storage medium

Abstract

The invention discloses a data cleaning method, a graph database device and a computer readable storage medium, the method is applied to the graph database device, the graph database device comprises a plurality of storage units, the storage units are used for storing row data, and the row data comprises information of a plurality of vertexes and information of edges connected with the vertexes. The method comprises the following steps: acquiring a life cycle of a vertex; recording a storage unit for storing the information of the vertex as a first storage unit, and setting a life cycle and a timestamp for the first storage unit; recording the storage unit for storing the information of the edge as a second storage unit, acquiring information of a vertex connected with the edge, recording the information as connection point information, and setting a life cycle and a timestamp for the second storage unit based on the connection point information; and after the life cycle of the storage unit is ended, cleaning the data in the storage unit. Through the mode, the expired data can be cleaned, and the implementation is simple.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular to a data cleaning method, a graph database device, and a computer-readable storage medium. Background technique [0002] At present, data cleaning can be realized through the characteristics of the life cycle (Time To Live, TTL) of the open source database HBase, but because it is a data cleaning solution at the column cluster level, it cannot handle the life cycle of multiple data and data. Different situations, and the life cycle needs to be set in advance when the table is created. After the data is updated, due to inconsistent timestamps of some columns, a row of data cannot expire at the same time, resulting in dirty data. In addition, there is also a plan to delete data according to the query results (Delete By Query), create an index for the time attribute field of the object that needs to be cleaned up in the graph data, and set the data expiration tim...

AI Technical Summary

Problems solved by technology

[0002] At present, data cleaning can be realized through the characteristics of the life cycle (Time To Live, TTL) of the open source database HBase, but because it is a data cleaning solution at the column cluster level, it cannot handle the life cycle of multiple data and data. Different situations, and the life cycle needs to be set in advance when the table is created. When the data is updated, due to inconsistent timestamps of some columns, a row of data cannot expire at the same time and dirty data will be generated.

In addition, there is also a plan to delete data according to the query results (Delete By Query), create an index for the time attribute field of the object that needs to be cleaned up in the graph data, and set the data expiration time at the same time, regularly query whether there is an expiration time every day, if there is The data identification (Identity document, ID) corresponding to the expiration time is obtained from the index, and the data is deleted according to the ID. The performance of this data cleaning method is very poor.

Images