Data processing method, system and device and storage medium

Abstract

The invention provides a data processing method, system and device and a storage medium, and the method comprises the steps: a gateway obtains a plurality of pieces of data, classifies the plurality of pieces of data according to a data format and / or a compilation language of the data, obtains M groups of data of different types, and sends the M groups of data to a cache module; the cache module comprises M storage pipelines, and the M storage pipelines receive and cache M groups of data sent by the gateway; the calculation module obtains target data from the M groups of data, wherein the target data is one or more groups of data in the M groups of data; and the calculation module preprocesses the target data to obtain to-be-stored data, and stores the to-be-stored data in a database. According to the method, before the data are stored in the database, the data are preprocessed and then stored in the database in a classified mode, the pressure of the database is relieved, and the problems that when the data are directly stored in the database, time delay occurs, and the database bears large pressure are solved.

Description

technical field [0001] The present application relates to the field of data processing, and more specifically, to a data processing method, system, device and storage medium in the field of data processing. Background technique [0002] Against the background of the rapid development of Internet technology, the level of science and technology is also constantly improving. Researchers are committed to creating and inventing more smart devices, such as robots, vacuum cleaners, smart home devices, etc. The emergence of these products can be used in daily life Serving people more intelligently saves manpower, thus changing people's daily life. [0003] During the working process of the smart device, different data are mainly collected through various sensors inside the device, and the smart device processes and analyzes the collected data. Based on the processing and analysis results, the user can further control the working process of the smart device, so that Smart devices ca...

AI Technical Summary

Problems solved by technology

[0005] In the above process, different sensors collect data at different frequencies. For sensors with high data collection frequency, the amount of data collected by these sensors is relatively large. When sending and storing to the database, there will be delayed storage, which will cause the gateway to wait for the database to store for a long time, and when the data flood peak occurs, the adaptability of the above process is poor, which will easily lead to the failure of the data processing system

[0006] In addition, in the above process, when the gateway stores data in the database, the calculation and processing of the data mainly depends on the computing power of the database itself. If the amount of data stored in the database is large, the calculation of the database will be slow and slow. time delay

Images