A real-time storage method, device and medium for big data

Through the virtual machine integrated environment and multi-software deployment, the distributed storage architecture and stream processing technology are adopted to solve the problems of high latency and slow speed in massive data storage, realize efficient and real-time data processing and storage, and meet the data needs in diverse scenarios.

CN119620948BActive Publication Date: 2025-09-16浪潮智慧科技有限公司 +1
3 Cites 0 Cited by

Patent Information

Application Number
CN202411763291.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-03
Publication Date
2025-09-16
Estimated Expiration
2044-12-03

AI Technical Summary

Technical Problem

The storage process of massive data in existing technologies has problems of high latency and slow speed, which affects data processing efficiency, increases costs, and makes it difficult to meet data storage and processing needs in diverse scenarios.

Method used

Through the deployment of a virtual machine integrated environment and multiple software, a distributed storage architecture is adopted, combined with the integration of stream processing software and storage software, CDC technology is used to capture relational database change data, and Flink task scripts are written to achieve real-time processing and storage of streaming data.

Benefits of technology

It improves data storage efficiency and security, enhances the scalability and fault tolerance of the system, achieves real-time and accuracy of data, and meets data processing needs in different scenarios.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The present application discloses a method, device and medium for real-time storage of big data. The method includes: determining a plurality of pre-set virtual machines, building an environment based on the plurality of virtual machines to determine an integrated environment, and deploying the integrated environment based on a plurality of pre-set software; determining a plurality of nodes in the deployed integrated environment, and distributing and storing data through the plurality of nodes; converting and processing the distributed stored data to determine streaming data, connecting the streaming data to a pre-set script, and executing the script to determine a streaming task, thereby determining the change of the data through the streaming task. The present application realizes the real-time storage and processing of big data through optimization measures such as virtual machine integration, integration of stream processing and storage software, processing of multiple data types and execution of flinksql task scripts, thereby improving the storage efficiency, real-time performance and scalability of the system and providing strong support for big data applications.
Need to check novelty before this filing date? Find Prior Art