Data stream processing method and related device

A data stream and data node technology, applied in the computer field, can solve problems such as difficult to achieve real-time detection, long data time consumption, affecting data aggregation efficiency and stability, etc., to improve real-time performance and efficiency, save statistical time, and improve accuracy Effects on Sex and Stability
CN111177201AActive Publication Date: 2020-05-19TENCENT TECH (SHENZHEN) CO LTD
0 Cites 9 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
TENCENT TECH (SHENZHEN) CO LTD
Publication Date
2020-05-19

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a data stream processing method and a related device. The method comprises the following steps: acquiring a data stream in a preset time period; generating a corresponding keyword according to a preset field pair so as to aggregate the data stream into a hash table; according to the change condition of the timestamp of the aggregated data flow, counting the statistical indexes according to a preset statistical rule to obtain the statistical amount of the statistical indexes. According to the method, different statistical methods are adapted to scenes with different aggregation durations due to the setting of the preset time period, so that the statistical process is more flexible, and a real-time statistical effect can be achieved; through the combined use of the hash table and the chain table, various statistical modes can be configured, the statistical time is saved in the statistical process, and the real-time performance and efficiency of the statistical process are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical field

[0001] This application relates to the field of computer technology, and in particular to a method and related devices for data stream processing. Background technique

[0002] In many business security scenarios, the aggregation characteristics of a certain type of specific group are often used to determine whether the group is malicious. Generally speaking, it is mainly used to describe a certain entity a. In a sliding time window, another entity b The number of meeting specific conditions. For example, in the account security scenario, count the number of uin users who have logged in abnormally at an address in the past 24 hours.

[0003] Judging the characteristics of a group for a specific aggregation feature generally requires storing all the data to be detected in the database, and then querying and analyzing the specific aggregation feature through a data query statement.

[0004] However, in the case of a large amount of data, it takes a long time to call a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More