Stream processing method and computing device

The stream processing method optimizes memory access by using a candidate generation sketch and verification sketch to efficiently track frequent items in data streams, supporting retractions and outputting frequent keys while reducing memory consumption and improving processing efficiency.

WO2025226169A1 Publication Date: 2025-10-30HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD +1
0 Cites 0 Cited by

Patent Information

Application Number
PCT/RU2024/000141
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-04-22
Publication Date
2025-10-30

AI Technical Summary

Technical Problem

Existing methods for tracking frequent items in data streams are memory-intensive, do not support retractions effectively, or are not cache-efficient, and cannot output a list of most frequent items in the presence of retractions.

Method used

A stream processing method using a candidate generation sketch with k blocks of majority testers and a verification sketch with k' blocks of counters, where updates and queries are optimized to minimize memory access by loading only two blocks into faster memory, allowing efficient output of frequent keys.

Benefits of technology

Reduces memory consumption and improves processing efficiency by utilizing cache memory effectively, enabling the output of a list of most frequent keys while supporting retractions and maintaining a smooth trade-off between accuracy and memory usage.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

A stream processing method includes: obtaining a candidate generation sketch including k blocks; obtaining a verification sketch including k´ blocks; receiving records in a stream with insertions and / or retractions, each of the records including a key and a count that are in correspondence; updating the sketches by transferring each record to one of the k blocks of the candidate generation sketch and one of the k´ blocks of the verification sketch; querying the k blocks of the candidate generation sketch to determine a set of candidate keys; dividing the set of candidate keys into k´ sub-sets of candidate keys; testing each sub-set of candidate keys by using one of the k´ blocks of the verification sketch to obtain a verified current list of frequent keys; and outputting a current list of frequent keys in the insertion stream and / or retraction stream.
Need to check novelty before this filing date? Find Prior Art