Siamese Autoencoder Change Detection With Adaptive Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models in edge devices suffer from degradation due to dynamic behaviors such as data drift and outliers, leading to inaccurate predictions over time, which are not effectively addressed by conventional similarity detection techniques like Siamese AutoEncoders with regular contrastive loss functions.
Innovation Solution
Implementing a Siamese AutoEncoder with adaptive loss functions and affinity propagation for clustering reconstruction errors to detect granular changes, outliers, and data drift, using edge devices to refine ML models by transmitting relevant data packages to cloud servers for updating.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional Siamese AutoEncoders with regular contrastive loss functions are used for similarity detection, then the system can detect changes between images, but it fails to effectively address data drift and outliers leading to inaccurate predictions
Solution Approach 1:
The patent applies parameter changes by modifying the loss function from a regular contrastive loss to an adaptive loss function that dynamically adjusts parameters based on the input data characteristics. This allows the system to adapt to data drift and outliers by changing the optimization parameters in response to varying data conditions, thereby maintaining prediction accuracy.
Solution Approach 2:
The patent implements dynamics by introducing an adaptive loss function that dynamically adjusts its behavior based on the input data. The loss function can change its parameters and optimization strategy in real-time to accommodate data drift and outliers, making the system flexible and responsive to changing conditions rather than static.
2Productivity
If ML models are pre-trained using labelled data, then the models can make initial predictions, but outliers and data stream drifts affect prediction behavior over time
Solution Approach 1:
The patent implements feedback by continuously monitoring prediction performance and using the adaptive loss function to adjust model parameters based on ongoing performance metrics. The system incorporates feedback loops where prediction errors and data drift detection trigger retraining or parameter adjustments, ensuring long-term prediction consistency.
Solution Approach 2:
The patent applies preliminary action by pre-training the Siamese AutoEncoder on representative data that includes examples of potential data drift and outliers. This preliminary exposure allows the model to learn robust features and the adaptive loss function to be prepared for handling various data conditions before actual deployment.
3Ease of operation
If edge devices perform limited analytics locally, then processing capability is preserved, but model refinement requires transmission to cloud servers
Solution Approach 1:
The patent applies segmentation by dividing the analytics system into edge and cloud components with distinct responsibilities. The edge device performs local change detection using the Siamese AutoEncoder and only transmits relevant results to the cloud, while the cloud handles comprehensive model retraining. This segmentation allows local processing to maintain speed while cloud resources provide thorough model refinement.
Solution Approach 2:
The patent implements partial action by having the edge device perform only the necessary change detection and feature extraction locally, rather than complete model training. This partial local action preserves processing capability while the cloud performs the excessive/comprehensive model refinement, optimizing the balance between local autonomy and centralized intelligence.
Data Source
AI summary
Systems and methods are provided for implementing a Siamese neural network using improved “sub” neural networks and loss function. For example, the system can detect a granular change in images using a Siamese Neural Network with Convolutional Autoencoders as the twin sub networks (e.g., Siamese AutoEncoder or “SAE”). In some examples, the loss function may be an adaptive loss function to the SAE network rather than a contrastive loss function, which can help enable smooth control of granularity of change detection across the images. In some examples, an image separation distance value may be calculated to determine the value of change between the image pairs. The image separation distance value may be determined using an Euclidean distance associated with a latent space of an encoder portion of the autoencoder of the neural networks.