A data sampling method, device and electronic equipment based on real-time data flow

A technology of data sampling and data flow, which is applied in the field of big data analysis and can solve problems such as fluctuations and large differences in data volume

Active Publication Date: 2021-09-10
BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] For data streams whose traffic may change over time, if sampling is performed according to a fixed sampling ratio, the amount of data sampled per unit time will fluctuate with time, which may be quite different from the expected amount of sampled data. Far

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data sampling method, device and electronic equipment based on real-time data flow
  • A data sampling method, device and electronic equipment based on real-time data flow
  • A data sampling method, device and electronic equipment based on real-time data flow

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.

[0049] see Figure 1a , Figure 1a Shown is a schematic flowchart of a data sampling method based on a real-time data stream provided by an embodiment of the present invention, which may include the following steps:

[0050] S110. When the preset sampling period arrives, acquire the data volume of the data to be sampled received within the latest preset sampling period, and use the data volume of the data to be sampled as a reference data volume.

[0051] Wherein, when the preset sampling period arrives, it refers to the elapse of an integer number of preset sampling periods from a specific moment. In this embodiment, the specified time may be the time when the data to be sampled starts to be received. Exemplarily, it is recorded that the moment of receiving data is t=0s, and the preset sampling period...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Embodiments of the present invention provide a data sampling method, device and electronic equipment based on real-time data streams. The method includes: when the preset sampling period arrives, acquiring the data volume of the data to be sampled received in the latest preset sampling period, and using the data volume of the data to be sampled as a reference data volume; determining the preset expected sampling The ratio of the data amount to the reference data amount, and the ratio is used as a sampling ratio; according to the sampling ratio, the data to be sampled received in the current preset sampling period is sampled. The amount of data to be sampled received in the latest preset sampling period can be used as a reference for the amount of data to be sampled to be received in the current preset sampling period to determine the sampling ratio of the current preset sampling period, so that each The sample data collected in the sampling period is kept close to the sample data amount expected to be collected, that is, the data amount of the sampled data per unit time is relatively stable.

Description

technical field [0001] The present invention relates to the technical field of big data analysis, in particular to a data sampling method, device and electronic equipment based on real-time data flow. Background technique [0002] When faced with a huge amount of data to be processed, if all these data are processed, it will bring a lot of resource overhead. In the prior art, in order to reduce this resource overhead, the data to be processed can be sampled according to a preset sampling ratio to obtain a part of the data, and only this part of the data is processed. [0003] However, the inventor found in the process of realizing the present invention that the prior art has at least the following problems: [0004] For data streams whose traffic may change over time, if sampling is performed according to a fixed sampling ratio, the amount of data sampled per unit time will fluctuate with time, which may be quite different from the expected amount of sampled data. Far. C...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/05
CPCG06F3/05
Inventor 郑培凝
Owner BEIJING QIYI CENTURY SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap