Data detection method and device

A data detection and data technology, applied in the field of data processing, can solve the problems of low detection accuracy, inability to detect, and data duplication of missing data, and achieve the effect of simple and easy detection process, improved detection accuracy, and high detection efficiency.

Active Publication Date: 2017-05-10
ALIBABA GRP HLDG LTD
View PDF5 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, for sampling inspection, due to the existence of a certain sampling rate, it is extremely prone to data loss but cannot be detected
For the final consistency check, in the process of data transmission, data duplication often occurs. If there is no deduplication, the final number of records counted at the destination is inaccurate in many cases.
It can be seen that the detection accuracy of the above detection method for missing data is low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data detection method and device
  • Data detection method and device
  • Data detection method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] In order to make the above objects, features and advantages of the present application more obvious and comprehensible, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0042] refer to figure 1 , which shows a flow chart of the steps of an embodiment of a data detection method of the present application, which may specifically include the following steps:

[0043] Step 101, when synchronizing data of a specified file, obtain parameter information of the synchronous data, and the parameter information is associated with the synchronous data.

[0044] In the embodiment of the present application, the device for detecting whether data is lost (hereinafter referred to as the device) can be the target server itself when performing data synchronization, or a module set in the target server, or it can be independent of the target server. A device other than a server that can ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a data detection method and device. The data detection method comprises the following steps of when the data of an appointed file is synchronized, obtaining the parameter information of the synchronized data, wherein the parameter information and the synchronized data are related; determining whether each piece of obtained parameter information corresponding to the appointed file has a vacancy or not; and if each piece of obtained parameter information corresponding to the appointed file has the vacancy, determining that the appointed file is subjected to data loss. By use of the detection method, the detection accuracy of lost data can be improved, a detection process is simple and easy in implementation, and detection efficiency is high.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a data detection method and a data detection device. Background technique [0002] In various web applications, there will be many log files continuously generated, such as buried logs used to record user behavior on the page. These logs are distributed on many servers, up to 10W servers, and there are multiple log files on each machine. In business, these log files need to be synchronized to the data warehouse system for analysis. Then, it is necessary to ensure that the data is not lost during the synchronization process, so as to provide real protection for the business. [0003] At present, during the synchronization process of massive data, there are two schemes for detecting whether data is lost. One is sampling inspection, which regularly outputs regular logs in the log, and then detects whether these regular logs exist at the destination (data warehous...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/34
CPCG06F11/34
Inventor 梁永锋
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap