Unlock instant, AI-driven research and patent intelligence for your innovation.

Real-time data stream statistical method and device

A technology of data flow and statistical results, applied in the computer field, can solve problems such as complex processing, and achieve the effect of overcoming complex processing

Pending Publication Date: 2020-12-29
西安京迅递供应链科技有限公司
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of this, the embodiments of the present invention provide a statistical method and device for real-time data streams, which can solve the problem of complex processing when the current stream data computing framework processes business logic for multiple data streams based on various API operators. question

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time data stream statistical method and device
  • Real-time data stream statistical method and device
  • Real-time data stream statistical method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030]The following describes exemplary embodiments of the present invention with reference to the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should realize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Likewise, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031]Spark Streaming is a stream data computing framework. Its combination with the open source stream processing platform Kafka is widely used in the real-time processing field. In actual scenarios, we often face the need to connect 2 or 3 Kafka data streams to the Spark Streaming framework. According to different business scenarios, filter the data streams, convert field...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a real-time data stream statistical method and device, and relates to the technical field of computers. One specific embodiment of the method comprises the following steps: registering a virtual table of a data stream according to the read data stream; loading and running a preset SQL statement file on a virtual table of the data stream; and storing a real-time statisticalresult obtained after operation. According to the embodiment, the technical problem that the processing process is complex when the data stream calculation framework processes the multiple data streams based on various API operators is solved, and then the technical effect that the complex conversion calculation function can be replaced by writing SQL statements to achieve statistics of the real-time data streams in a service scene is achieved.

Description

Technical field[0001]The present invention relates to the field of computer technology, in particular to a method and device for real-time data flow statistics.Background technique[0002]The current big data real-time processing technology is developing rapidly, and the use of real-time processing engines in production environments with rich user data is becoming more and more common. It not only adapts to the trend of refined business operations, but also helps businesses improve the accuracy of decision-making. . In business processing, it is often necessary to connect multiple data streams to the stream data calculation framework, and filter the data streams, convert fields, aggregate statistics in different dimensions, and output statistical results according to different business logic scenarios. In order to ensure the non-loss and non-duplication of data in the real-time statistics process, the checkpoint mechanism of the streaming data computing framework itself can be restore...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2458G06F16/2455G06F16/22
CPCG06F16/24568G06F16/2462G06F16/22
Inventor 韩路
Owner 西安京迅递供应链科技有限公司