Unlock instant, AI-driven research and patent intelligence for your innovation.

Data cleaning method and device

A data cleaning and data technology, applied in the field of data processing, can solve the problems of manpower consumption and low data cleaning efficiency.

Pending Publication Date: 2020-09-08
HANGZHOU DT DREAM TECH
View PDF13 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the prior art, data cleaning rules can be manually established by operators, such as establishing corresponding cleaning rules for different data tables. However, in actual situations, the number of data tables that need to be cleaned may reach tens of thousands or hundreds of thousands. Establishing cleaning rules for each data table will obviously consume a lot of manpower, resulting in low efficiency of data cleaning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data cleaning method and device
  • Data cleaning method and device
  • Data cleaning method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

[0032] The terminology used in this application is for the purpose of describing particular embodiments only, and is not intended to limit the application. As used in this application and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data cleaning method and device. The invention discloses a data cleaning method, which comprises a preset corresponding relationship between a standard data element and a cleaning rule, and comprises the following steps of: receiving a cleaning task which comprises to-be-cleaned data; obtaining the attribute of each field in the to-be-cleaned data; judging whether the attribute of the field is matched with a standard data element in the corresponding relationship; if yes, obtaining a cleaning rule corresponding to the standard data element; and cleaning the field matched with the standard data element by adopting the cleaning rule. Compared with the prior art, the data cleaning method provided by the invention has the advantage that the data cleaning efficiency canbe improved.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a method and device for data cleaning. Background technique [0002] As big data gradually penetrates various industries, the types and quantities of data are constantly increasing. High-quality data plays a key role in enterprise decision-making and business support, while low-quality data may affect business or lead to project failure. Based on this, more and more enterprises begin to clean massive data to mine Valuable data. [0003] Data cleaning can detect data consistency, handle invalid and missing values, remove duplicate information, correct errors, and more. In the prior art, data cleaning rules can be manually established by operators, such as establishing corresponding cleaning rules for different data tables. However, in actual situations, the number of data tables that need to be cleaned may reach tens of thousands or hundreds of thousands. Establishing c...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215
CPCG06F16/215
Inventor 方薇荀志
Owner HANGZHOU DT DREAM TECH