Bucket log-based data circulation method and device, electronic equipment and storage medium
Through the data flow method based on bucket logs, the problems of data redundancy and high storage costs in unstructured data storage are solved, and the rapid access and efficient storage of data are achieved.
CN119646265APending Publication Date: 2025-03-18AGRICULTURAL BANK OF CHINA
0 Cites 0 Cited by
Patent Information
- Application Number
- CN202411738448.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-18
AI Technical Summary
Technical Problem
In the prior art, the storage of unstructured data leads to data redundancy, increases the pressure of accessing storage areas, and has high storage costs and low access efficiency.
Method used
The data flow method based on bucket log is adopted. By acquiring the bucket log, analyzing and processing, writing it to the first message queue, batch processing and writing it to the second message queue, and finally multi-level data flow is performed based on the aggregated data in the second message queue.
Benefits of technology
It realizes fast access to data, reduces storage costs, reduces data redundancy, and improves the level of data service.
✦ Generated by Eureka AI based on patent content.
Abstract
The invention provides a bucket log-based data circulation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a bucket log of a transaction; analyzing and processing the bucket log to obtain log analysis data written into the first message queue; performing batch processing based on the log analysis data in the first message queue to obtain aggregated data, and writing the aggregated data into a second message queue; and carrying out multi-level data circulation based on the aggregated data in the second message queue. The method comprises the following steps: filtering and analyzing a bucket log to determine a corresponding log analysis result; performing batch processing on batch log analysis results, and determining corresponding aggregated data; determining the hierarchy of data of each piece of data in the aggregated data at the current time; and determining whether data circulation is needed or not according to the hierarchy change of the current data and the hierarchy change of the previous time, and if so, carrying out data circulation. Therefore, the user can quickly access the corresponding data, and the access efficiency is improved.
Need to check novelty before this filing date? Find Prior Art