Dirty data identification method, device and system

An identification method and identification device technology, applied in the computer field, can solve problems such as inability to accurately identify dirty data, and achieve the effect of high concurrent data volume and large data volume

Pending Publication Date: 2021-05-11
BEIJING WODONG TIANJUN INFORMATION TECH CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] In view of this, the embodiments of the present invention provide a dirty data identification method, device and system to solve the technical problem that dirty data cannot be accurately identified

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dirty data identification method, device and system
  • Dirty data identification method, device and system
  • Dirty data identification method, device and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0073] Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0074] figure 1 It is a system structure diagram for realizing the dirty data identification method of the embodiment of the present invention. Such as figure 1 As shown, the system includes a generator and a consumer, and the producer divides the data according to the primary key to ensure that the data of the same business primary key (key) enters the same operator (su...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a dirty data identification method, device and system, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: receiving or producing data, and distributing the data with the same service primary key to the same operator; in the same operator, when the operator processes the data, generating a label in a self-increasing mode, adding the label into the data, and the label into the data; and issuing the data added with the label to a consumer. According to the embodiment, the technical problem that dirty data cannot be accurately identified can be solved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method, device and system for identifying dirty data. Background technique [0002] In the process of distributed real-time data processing, the order in which data arrives at computing operators may be different from the order in which events occur. Correctly handling the dirty data problems caused by these differences is crucial in real-time applications. [0003] In order to be able to distinguish the sequence of data occurrence, the main method currently used is to add timestamp tags or global count tags to the data, and to distinguish the sequence of data by comparing the size of the tags. Using time tags to distinguish the order of messages requires adding a time stamp tag to the data. When processing data, the order of messages is determined by comparing the time stamps. Use the global counting tag to distinguish the order of messages. By designing a global co...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2455G06F16/2458
CPCG06F16/24553G06F16/2471
Inventor 吴帅袁建军刘业辉张志刚
Owner BEIJING WODONG TIANJUN INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap