Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data stream processing method and related device

A data stream and data node technology, applied in the computer field, can solve problems such as difficult to achieve real-time detection, long data time consumption, affecting data aggregation efficiency and stability, etc., to improve real-time performance and efficiency, save statistical time, and improve accuracy Effects on Sex and Stability

Active Publication Date: 2020-05-19
TENCENT TECH (SHENZHEN) CO LTD
View PDF0 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, in the case of a large amount of data, it takes a long time to call all the data, and the whole process will be affected by network delay, it is difficult to achieve the purpose of real-time detection, affecting the efficiency and stability of data aggregation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data stream processing method and related device
  • Data stream processing method and related device
  • Data stream processing method and related device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] The embodiments of the present application provide a data stream processing method and related devices, which can be applied to a system or program process for business security detection and statistics, specifically by acquiring data streams within a preset time period, and comparing data streams according to preset fields. The data streams are aggregated, and then the statistical indicators are counted according to preset statistical rules according to the changes in the timestamps of the aggregated data streams to obtain the statistics of the statistical indicators. Due to the credibility of the timestamp And uniqueness can improve the accuracy and stability of the statistical process; and due to the setting of the preset time period, different statistical methods are adapted to the scenes of different aggregation durations, which makes the statistical process more flexible and can achieve real-time statistics. Further, because the timestamp is easy to read, it saves s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data stream processing method and a related device. The method comprises the following steps: acquiring a data stream in a preset time period; generating a corresponding keyword according to a preset field pair so as to aggregate the data stream into a hash table; according to the change condition of the timestamp of the aggregated data flow, counting the statistical indexes according to a preset statistical rule to obtain the statistical amount of the statistical indexes. According to the method, different statistical methods are adapted to scenes with different aggregation durations due to the setting of the preset time period, so that the statistical process is more flexible, and a real-time statistical effect can be achieved; through the combined use of the hash table and the chain table, various statistical modes can be configured, the statistical time is saved in the statistical process, and the real-time performance and efficiency of the statistical process are improved.

Description

Technical field [0001] This application relates to the field of computer technology, and in particular to a method and related devices for data stream processing. Background technique [0002] In many business security scenarios, the aggregation characteristics of a certain type of specific group are often used to determine whether the group is malicious. Generally speaking, it is mainly used to describe a certain entity a. In a sliding time window, another entity b The number of meeting specific conditions. For example, in the account security scenario, count the number of uin users who have logged in abnormally at an address in the past 24 hours. [0003] Judging the characteristics of a group for a specific aggregation feature generally requires storing all the data to be detected in the database, and then querying and analyzing the specific aggregation feature through a data query statement. [0004] However, in the case of a large amount of data, it takes a long time to call a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2457G06F16/22
CPCG06F16/2457G06F16/2255
Inventor 张少愚李韬赖俊凡尤飞
Owner TENCENT TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com