RoaringBitmap-based massive user efficient circling method and RoaringBitmap-based massive user efficient circling device

Abstract

The invention discloses a RoaringBitmap-based massive user efficient circling method and device, and the method comprises the steps: establishing a label library of user behaviors and a label libraryof business classification according to user behavior data, building a data model according to the label libraries, establishing a bitmap partition table according to the data model, and storing userhistorical data; and finally, calculating a result through a custom function, synchronizing the data in the data warehouse into the data model, then supplementing the data in the bitmap partition table from the data model according to a rule, and finally performing cube acceleration on the bitmap partition table according to needs to generate a cube_bitmap table; and according to the dimension selected by the user, selecting to directly look up the result table or distinguish the accelerated cube query from the bitmap partition table through the storage process query. According to the method and the device, the information to be counted can be accurately and efficiently circled and selected from a large number of users.

Description

technical field [0001] The present invention relates to the field of mobile communication Internet of Things, in particular to a method and device for efficient circle selection of massive users based on RoaringBitmap (efficient bitmap calculation). Background technique [0002] With the widespread use of mobile communication devices and the daily increase of users' mobile Internet data, the circle selection of users based on different latitudes is usually solved by using OLAP-Druid's pre-summarization method and SPARK's distributed and efficient memory computing method. However, OLAP-Druid will lose accuracy in deduplication business scenarios, and the statistical results are inaccurate, while the calculation method of SPAKR is to pull detailed data into memory for calculation. Although it is distributed, the memory resources occupied by detailed data It is quite large, and the operation efficiency of doing count (distinct) on massive data is also low. For the business sce...

AI Technical Summary

Problems solved by technology

[0003] In order to solve the business scenario of efficient circle selection of massive users, the problem of poor user experience in conventional technical solutions, the present invention provides a method and device for efficient circle selection of massive users based on RoaringBitmap, and an efficient circle selection business scenario based on massive users , using the RoaringBitmap plug-in to develop a solution that can efficiently circle a large number of users on PostGreSql, enabling users to accurately and efficiently circle the information to be counted among a large number of users

Images