Big data full life cycle quality control and efficient retrieval system

By employing a quality-driven adaptive hierarchical index and a virtual reference data repair mechanism, the problem of the disconnect between data governance and retrieval systems has been resolved. This has enabled efficient data resource management and automatic repair of hidden dirty data, thereby improving the retrieval efficiency and resource utilization of big data systems.

CN122045178APending Publication Date: 2026-05-15HENAN ZHONGYUAN CLOUD BIG DATA GROUP CO LTD
1 Cites 0 Cited by

Patent Information

Application Number
CN202610165017.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-02-05
Publication Date
2026-05-15

AI Technical Summary

Technical Problem

In existing technologies, data governance and retrieval systems are fragmented, failing to effectively identify data activity and timeliness. This results in limited retrieval of high-value data, resource consumption by low-quality data, and a lack of intelligent data repair mechanisms.

Method used

It employs a quality-driven adaptive hierarchical index and a retrieval feedback repair mechanism based on virtual reference data. Through dynamic quality scoring, hierarchical storage, and adaptive index construction, it actively detects and repairs low-quality data by combining semantic intent.

Benefits of technology

It significantly improves the retrieval efficiency and data availability of big data systems, enables on-demand resource allocation and dynamic optimization of hidden dirty data, and improves data recall and resource utilization.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of big data processing and information retrieval, and discloses a big data full life cycle quality control and efficient retrieval system, which comprises a data acquisition processing module, a hierarchical index construction module and a retrieval repair module, and is characterized in that the acquisition processing module generates a dynamic quality score based on static verification, dynamic activeness and data conflict degree; the high-quality data is stored in a hot storage area to construct a high-precision vector index, the data to be treated is stored in a cold storage area, and a low-precision vector index is constructed by using a self-adaptive product quantization algorithm; and when the retrieval of the hot area fails, the retrieval and restoration module generates virtual reference data based on the semantic intention, performs reverse detection and search on the cold area, discovers and restores potential to-be-treated data, and promotes the potential to-be-treated data back to the hot storage area. According to the method, on-demand allocation of resources and automatic discovery and dynamic optimization of hidden dirty data are realized through a quality-driven hierarchical index and reverse detection mechanism, and the data availability is remarkably improved.
Need to check novelty before this filing date? Find Prior Art