Data governance method based on double message queues
A message queue and dual message technology, which is applied in the field of distributed computing and data processing, can solve the problems of untraceability and low security, and achieve the effects of easy code refactoring, reliable data, and strong scalability
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Example Embodiment
[0034] Example
[0035] See attached figure 1 .
[0036] A data management method based on dual message queues, the method includes the following steps:
[0037] 1) Input data from user data sources through data access tools;
[0038] 2) Store the accessed data resources in the message queue kafka cluster;
[0039] 3) The data management program extracts data from the message queue for preprocessing operations such as cleaning;
[0040] 4) The data management program stores the preprocessed data into the original Hbase database, and at the same time submits the data to the Kafka message queue again;
[0041] 5) Various data management programs extract data from the message queue for governance, and then store the governance results in the message queue kafka cluster again;
[0042] 6) The last governance program extracts data from the message queue, and after the governance is completed, the governance results are stored in the result databases HBase, ElasticSearch, and RabbitMQ for use i...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap