Semantic Change Detection Using Adaptive Feature Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CNN-based systems for change detection, such as those used in satellite imagery, suffer from low accuracy with datasets like GSV and TSUNAMI, and are not robust to illumination variations and camera motion, necessitating scene-specific training and high computational overhead.
Innovation Solution
A method and system utilizing a Siamese Convolutional Neural Network (CNN) with an adaptive correlation layer that generates feature maps for test and reference images, computes pixel-specific computation based on information content, and extracts semantic features through correlation maps to detect changes, making it scene-agnostic and robust to illumination variations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CNN-based foreground extraction is used for change detection, then the system can identify changes in scenes, but it fails to achieve high accuracy with certain datasets (GSV and TSUNAMI) due to inability to handle illumination variations and camera motion
Solution Approach 1:
The patent segments the change detection process into multiple specialized modules: a background modeling module for handling illumination variations, a motion compensation module for camera motion, and a change detection module. This segmentation allows each module to specialize in handling specific types of variations, thereby improving overall accuracy and robustness across diverse datasets like GSV and TSUNAMI
Solution Approach 2:
The patent creates a universal change detection system that can handle multiple types of variations (illumination changes, camera motion, seasonal changes) through a single integrated framework. The background modeling module maintains a adaptive background model that can accommodate different scene types and variations, making the system versatile across different datasets without requiring dataset-specific tuning
2Measurement precision
If scene-specific training is performed to improve accuracy for particular datasets, then change detection accuracy improves for that scene, but the system requires re-training for different scenes increasing complexity and time consumption
Solution Approach 1:
The patent performs preliminary action by pre-training the background modeling module on diverse scene data to learn general patterns of illumination variations and camera motion. This pre-trained model serves as a robust foundation that can be quickly adapted to new scenes through incremental learning rather than complete re-training, significantly reducing the time loss when deploying to new datasets
Solution Approach 2:
The patent implements dynamic adaptation where the background model continuously updates and adapts to new scenes through incremental learning. The system dynamically adjusts its parameters and background model based on incoming data from different scenes, allowing it to maintain high accuracy across diverse datasets without requiring static re-training for each scene
3Ease of manufacture
If standard CNN processing is applied uniformly to all pixels, then the processing is simple to implement, but computational overhead is high and efficiency is reduced
Solution Approach 1:
The patent applies local quality by processing different regions of the image with different levels of computational intensity based on their information content. Pixels with high information content (edges, textures, moving objects) receive more intensive processing, while uniform regions receive lighter processing. This localized approach maintains implementation simplicity while significantly improving processing efficiency by avoiding uniform high-cost processing across the entire image
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
State of the art image processing techniques such as background subtraction, and Convolutional Neural Network based approaches, when used for change detection, fail to support certain datasets. The disclosure herein generally relates to semantic change detection, and, more particularly, to a method and system for semantic change detection using a deep neural network feature correlation approach. An adaptive correlation layer is used by the system, which determines extent of computation required at pixel level, based on amount of information at pixels, and uses this information in further computation done for the semantic change detection. Information on the determined extent of computation required is then used to extract semantic features, which is then used to compute one or more correlation maps between the at least one feature map of a test image and corresponding reference image. Further the semantic changes are determined from the one or more correlation maps.