2results about How to "Guaranteed access performance" patented technology

The invention belongs to the technical field of water conservancy projects, and provides a dam mass data distributed processing method based on a cloud computing architecture, which comprises the following steps: S1, performing unified access and standardized description on multi-source heterogeneous dam monitoring data, and generating a structured data object with a spatio-temporal index; s2, performing adaptive data fragmentation and distributed storage optimization based on space-time attributes and access features of the data objects; s3, according to the characteristics and resource requirements of the computing tasks, constructing a dynamic extensible parallel computing workflow; s4, in the process of executing the parallel computing workflow, implementing resource flexible scheduling and task migration based on load feedback; and S5, performing multi-dimensional aggregation on the processing result, and feeding back to the data storage layer.

The embodiment of the invention provides a data processing method and device, electronic equipment and a storage medium, the method is applied to a database system, and a hot storage layer and a cold storage layer are mounted on the database system; the method comprises the steps of obtaining access logs and metadata change logs of N pieces of first data in a first time period; through a pre-trained prediction model, a first probability of each piece of first data is obtained according to the access log and the metadata change log, and the first probability represents a probability that the first data belongs to hot data or cold data; determining hot data and / or cold data in the N pieces of first data according to the first probability of the N pieces of first data; and performing a migration operation on the hot data and / or the cold data to enable the hot data to be stored in the hot storage layer and the cold data to be stored in the cold storage layer. Therefore, according to the embodiment of the invention, the problem that the architecture of the existing database storage system cannot consider both the storage cost and the data query performance can be solved.