Edge Data Model Updating Through Drift and Outlier Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data drift and outliers degrade the performance of data models deployed at edge devices due to dynamic behaviors in real-world applications, leading to inaccurate predictions.
Innovation Solution
Detect data drift and outliers at the edge system by processing input data to generate representative points, cluster them, identify outlier clusters, and transmit relevant information to the cloud server for updating the data model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data models are continuously updated with all incoming data to maintain prediction accuracy, then prediction accuracy is improved, but bandwidth consumption and processing time increase significantly
Solution Approach 1:
The patent extracts only the essential information about data drift and outliers from the incoming data streams, rather than transmitting all raw data. By identifying and extracting only the anomalous patterns and drift characteristics, the system maintains prediction accuracy while significantly reducing bandwidth consumption during model updates
Solution Approach 2:
The patent segments the incoming data into representative points that capture the essential characteristics of data drift and outliers. By dividing the continuous data stream into discrete, meaningful segments, the system can transmit only the necessary update information to cloud servers, reducing overall data transmission requirements
2Measurement precision
If data drift and outliers are detected and transmitted to cloud servers for model updates, then prediction accuracy is maintained, but processing time at edge devices increases
Solution Approach 1:
The patent performs preliminary detection and identification of data drift and outliers at the edge device before transmission. By pre-processing the data and identifying anomalies in advance, the system reduces the processing burden on cloud servers and enables faster model updates, as the cloud receives pre-processed, targeted information rather than raw data requiring extensive analysis
3Measurement precision
If comprehensive data analysis is performed at cloud servers to update data models, then model accuracy is improved, but bandwidth consumption increases due to transmitting all raw data
Solution Approach 1:
The patent extracts only the essential information about data drift and outliers from the incoming data streams, rather than transmitting all raw data. By identifying and extracting only the anomalous patterns and drift characteristics, the system maintains prediction accuracy while significantly reducing bandwidth consumption during model updates
Data Source
AI summary
The present invention relates to a system and a method for updating data models. Input data received from a data source and/or prediction data obtained from a data model is reduced based on baseline reference data to obtain a plurality of representative points. The plurality of representative points are clustered to generate a plurality of clusters. An outlier cluster is detected from the plurality of clusters based on a maximum distance of the plurality of clusters from a highest density cluster and/or comparison of quantity and values of the plurality of representative points with predefined rules. Data drift is identified based on changes in densities of the plurality of clusters. The data model is updated using information corresponding to the outlier cluster and the data drift.