Real-time data processing method and system

A technology for real-time data processing and data processing, which is applied in the field of data processing and can solve problems such as large data processing volume and inability to guarantee calculation results

Active Publication Date: 2019-09-06
DATACANVAS LTD
View PDF8 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present invention provides a real-time data processing method and system to solve the problem

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time data processing method and system
  • Real-time data processing method and system
  • Real-time data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0240] Example 1. Set maxOutOfOrder to 5 seconds, and the system reference time is updated every 3 seconds. For the update of the system reference time, see Table 2 below.

[0241] Table 2 System reference time update table

[0242]

[0243] It should be further noted that maxOutOfOrder implements the function of waiting for delayed arrival data for a specific time, and can handle some out-of-order data, making the calculation more accurate.

[0244] 2. It can only be applied to the update process of updating the system reference time according to the preset time interval

[0245] It should be noted that, in this case, the specific implementation manner of updating the system reference time is as follows:

[0246] Update the system reference time according to the first data time, the first time increase of the operating system and the waiting time;

[0247] Wherein, the first data time is the data time corresponding to the data with the largest data time value among the d...

example 2

[0259] Example 2. Set maxOutOfOrder to 5 seconds, and the system reference time interval is updated at 3 seconds. For the update of the system reference time, see Table 3 below.

[0260] Table 3 System reference time update table

[0261]

[0262] 3. It can only be applied to the update process of updating the system reference time every time the system receives a piece of new data.

[0263] It should be noted that, in this case, the specific implementation manner of updating the system reference time is as follows:

[0264] Update the system reference time according to the data time of the new data received by the system, the second time increase of the operating system and the waiting time;

[0265] Wherein, the second time increase of the operating system is the operating system time interval between the time when the RT system receives new data and the time when the last piece of data is received.

[0266] Further, according to the data time of the new data received b...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a real-time data processing method and system, and relates to the technical field of data processing. The data real-time processing method comprises the steps of dividing data to be processed in a system based on a data processing window; judging the data processing window based on the system reference time; and when the system reference time is greater than or equal to theend time of the data processing window, calculating and outputting a calculation result of the data in the current data processing window. According to the scheme, the calculation accuracy can be ensured, and a large amount of data aggregation calculation can be realized.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a method and system for real-time data processing. Background technique [0002] With the rapid expansion of data, big data processing technology has developed rapidly. At present, the big data processing platform can be divided into offline computing and real-time computing according to the computing mode. As the degree of real-time economic and social information continues to increase, people's demand for real-time data calculations is also increasing. For example: a risk control system (risk control system, compliance inspection, etc.) for anti-fraud and judging whether funds flow into illegal channels, a data processing system for data extraction, conversion, and loading (Extract Transform Load, ETL for short) and other scenarios require used for real-time calculations. [0003] However, the existing real-time computing systems may have the problem of inaccurate da...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/23G06F16/25
CPCG06F16/23G06F16/25
Inventor 郭璐胡宗星
Owner DATACANVAS LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap