A big data-based data cleaning system

By standardizing multi-source heterogeneous data and constructing a weighted directed graph, and using dynamic graph neural networks and variational energy models for data cleaning, the problem of data quality control in dynamic spatiotemporal coupling environments is solved, and automated data repair and adaptive maintenance are achieved.

CN122241033BActive Publication Date: 2026-08-18BEIJING UNIV OF TECH +1
2 Cites 0 Cited by

Patent Information

Application Number
CN202610696712.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-05-20
Publication Date
2026-08-18
Estimated Expiration
2046-05-20

AI Technical Summary

Technical Problem

Existing technologies struggle to eliminate dimensional differences in multi-source heterogeneous data under dynamic spatiotemporal coupling environments, neglect topological connections and temporal evolution characteristics between nodes, and lack closed-loop control mechanisms for automatically correcting anomalous data, making it difficult for data quality control to meet the needs of large-scale dynamic systems.

Method used

The data preprocessing center standardizes multi-source heterogeneous data, constructs a weighted directed graph, generates state space vectors using dynamic graph neural networks, calculates global consistency energy using variational energy models, repairs data using energy gradient scalars, and sets up a feedback control unit to achieve automated closed-loop control.

Benefits of technology

It effectively eliminates dimensional differences, accurately captures features between nodes, realizes the verification and automatic repair of deep logical conflicts, improves the accuracy of data processing and the convergence speed of repair, and has adaptive learning capabilities to adapt to environmental changes.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application relates to the technical field of big data processing and data quality control, in particular to a data cleaning system based on big data, which comprises a data preprocessing center used for collecting multi-source heterogeneous data, generating dimensionless statistical distribution values, and constructing a weighted directed graph; a state mapping unit used for generating a state space vector; an energy verification unit used for comparing a global consistent energy with a preset safety threshold; if the global consistent energy is greater than the preset safety threshold, a logic conflict signal is generated; if the global consistent energy is less than or equal to the preset safety threshold, a logic consistency signal is generated; a gradient repair unit used for generating a dimensionless repair value; and a feedback control unit used for generating a final cleaning result; the multi-source heterogeneous data is set as the final cleaning result; the application solves the problem that the prior art lacks an automatic correction means for system constraints, and improves the convergence speed and precision of repair.
Need to check novelty before this filing date? Find Prior Art