Unlock instant, AI-driven research and patent intelligence for your innovation.

Data cleaning method and device

A data cleaning and data technology, applied in the field of data processing, can solve problems such as narrow input voltage range of the power supply, and achieve the effect of improving performance

Active Publication Date: 2020-08-07
深圳华为云计算技术有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problem that the input voltage range of the power supply is relatively narrow in the related art, the embodiment of the present application provides a data cleaning method and device

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data cleaning method and device
  • Data cleaning method and device
  • Data cleaning method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] "First", "second" and similar words mentioned herein do not indicate any order, quantity or importance, but are only used to distinguish different components. Likewise, words like "a" or "one" do not denote a limitation in quantity, but indicate that there is at least one. Words such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0046] The "module" mentioned in this article usually refers to the program or instruction that can realize certain functions stored in the memory; the "unit" mentioned in this article usually refers to a functional structure divided according to logic, and the "unit" It can be implemented by pure hardware, or a combination of software and hardware.

[0047] The "plurality" mentioned herein means two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three types of relationships,...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data cleaning method and device, and belongs to the technical field of data processing. The method comprises the following steps of: obtaining a data cleaning time, wherein the data cleaning time is a time recorded when a history data cleaning request is received, and the data cleaning request is used for cleaning data, satisfying a cleaning condition, in a distributed database; obtaining data, not satisfying the cleaning condition, in the distributed database, and data, satisfying the cleaning condition and stored after the data cleaning time, in the distributed database; and combining the obtained data in the distributed database. According to the method and device, the problem that the data cleaning in the prior art is low in performance and is likely to influence the service performance is solved, and the effect of improving the data cleaning performance is achieved.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a data cleaning method and device. Background technique [0002] HBase (Hadoop Database, Hadoop Database) has the characteristics of distributed, high reliability, high performance, and KeyValue-based storage. Therefore, more and more enterprises and users use HBase to store and build data tables. [0003] After storing data in HBase, users can delete some of it. Therefore, in order to release the storage space of HBase, the data in HBase can be cleaned up. A data cleaning method provided by related technologies includes: associating a Map with each data partition of the distributed storage, reading each piece of data in the data partition, generating a deletion mark corresponding to the data satisfying the deletion condition according to the deletion condition, and deleting The mark is output to the Reducer, and then in the Reducer stage, all deletion marks a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/215G06F16/22G06F16/27
Inventor 毕杰山钟超强
Owner 深圳华为云计算技术有限公司