Data processing method, intelligent terminal and storage medium

Abstract

The invention discloses a data processing method, an intelligent terminal and a storage medium. The method comprises the following steps: receiving heartbeat data sent by a client application; importing the heartbeat data into a big data cluster; storing the heartbeat data in a bit encoding mode; and according to a used data warehouse tool, developing a user-defined bit code calculation function,and calling the user-defined function to perform bit code related calculation on the heartbeat data according to actual data analysis requirements. According to the method, the product use state of the user in a certain time period of the current day is recorded in a bit coding mode, the product use time distribution of the user in one day can be recorded in one field, the use condition of the user in the current day is recorded in a single field, and the use condition of the user in the current year can be stored in several fields in the current day; only a plurality of pieces of data of users need to be stored in one year, the storage space is greatly reduced, deduplication of heartbeat data in the same time period is achieved, related calculation is conducted on data based on bit coding, and the calculation efficiency can be improved.

Description

technical field [0001] The present invention relates to the technical field of computer information processing, in particular to a data processing method, an intelligent terminal and a storage medium. Background technique [0002] In order to record the user's application usage, generally the started application sends a heartbeat data consisting of a unique identifier and time to the server at regular intervals, and the server receives the data and stores it in Kafka (an open source stream processing platform, Kafka is a high-throughput A large amount of distributed publish-subscribe message system, which can handle all action flow data of consumers in the website), and then consume data from Kafka. [0003] For the massive heartbeat data consumed from Kafka, the traditional method is to store all the data in the source layer (ODS) of the cluster data warehouse, and directly perform conventional ETL (extract extraction, transform conversion, load loading) processing, because...

AI Technical Summary

Problems solved by technology

[0005] The main purpose of the present invention is to provide a data processing method, an intelligent terminal and a storage medium, aiming at solving the problem that in the prior art, the related calculation of massive heartbeat data consumes a lot of computing resources and time, and the heartbeat data is repeatedly sent in the same period of time. Problems that also affect the calculation results

Images