Data life cycle management method and device, medium and electronic equipment

A technology of life cycle and management method, applied in the fields of data life cycle management, data life cycle management device, computer readable storage medium and electronic equipment, can solve the problems of downtime, HDFS cluster jitter, waste and so on

Pending Publication Date: 2021-01-26
NETEASE (HANGZHOU) NETWORK CO LTD
View PDF5 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] To this end, an improved data life cycle management method is very much needed to avoid waste of storage resources and reduce the burden on the name node; at the same time, it solves the risk that the e

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data life cycle management method and device, medium and electronic equipment
  • Data life cycle management method and device, medium and electronic equipment
  • Data life cycle management method and device, medium and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043]The principle and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are only provided to enable those skilled in the art to better understand and then implement the present invention, but not to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0044]Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, device, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0045]According to the embodiments of the presen...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a data life cycle management method and device, a medium and electronic equipment, and relates to the technical field of data storage. The method comprises thesteps of obtaining to-be-processed metadata from a cluster database, and determining a life cycle state of the to-be-processed metadata; determining initial metadata to be deleted in a target life state from the to-be-processed metadata ; packaging the initial metadata to be deleted to generate an initial logic deletion task, and performing logic deletion processing on the initial logic deletiontask; determining the initial logic deletion task subjected to logic deletion processing as a logic deletion task, and generating a physical deletion task according to the logic deletion task; and acquiring state parameters of the distributed file storage system, and performing physical deletion processing on the physical deletion task according to the state parameters. The invention provides an implementation scheme of Hive life cycle governance by means of isolation, asynchronization, fine grit and self-adaption.

Description

Technical field[0001]The embodiments of the present invention relate to the field of data storage technology. More specifically, the embodiments of the present invention relate to a data life cycle management method, a data life cycle management device, a computer-readable storage medium, and an electronic device.Background technique[0002]This section is intended to provide background or context for the embodiments of the invention stated in the claims. The description here is not admitted to be prior art just because it is included in this section.[0003]As a Hadoop-based data warehouse tool, Apache Hive's greatest charm lies in the ability to map structured data in the Hadoop Distributed File System (HDFS) into a table, and then provide a similar structured query language The grammar of (Structured Query Language, SQL) allows users to analyze data, and it has a very wide range of applications in the field of big data.[0004]With the continuous growth of business scale, more and more...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/215G06F16/22G06F16/23G06F16/2457G06F16/27G06F16/182
CPCG06F16/215G06F16/2282G06F16/23G06F16/24578G06F16/27G06F16/182Y02D10/00
Inventor 余利华郭忆王金杰柯超
Owner NETEASE (HANGZHOU) NETWORK CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap