CNN Sensor Compensation Using Backpropagation on Edge Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CNN classifiers require extensive computational resources and time for training and image analysis, especially on edge devices with limited capacity, and sensor variations introduce errors in camera movement detection, complicating efficient image processing.
Innovation Solution
A process that trains a first convolutional neural network (CNN) and adds an interface layer to a second trained CNN, using sensor data and backpropagation to generate compensation values that modify sensor inputs, reducing computational load and compensating for sensor inaccuracies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a CNN is trained to process images sequentially with sensor data, then image processing accuracy is improved, but computational resources and training time increase
Solution Approach 1:
The patent pre-trains a first CNN on a large dataset of images to establish a robust base model. Then, it creates a second CNN by copying the first CNN's weights and adds an interface layer for sensor data compensation. This preliminary training approach allows the system to achieve high accuracy without retraining the entire network from scratch, thus reducing overall training time while maintaining image processing accuracy.
Solution Approach 2:
The patent modifies the CNN architecture by adding an interface layer that accepts sensor data as additional input parameters. This allows the network to incorporate compensation values from sensors (such as accelerometer or gyroscope data) without changing the core image processing parameters of the existing CNN, thereby improving accuracy while avoiding complete retraining.
2Measurement precision
If sensor data is used to compensate for camera movement, then image processing accuracy is improved, but device complexity increases
Solution Approach 1:
The patent introduces an interface layer as an intermediary component between the existing CNN and the sensor data compensation mechanism. This interface layer receives sensor data, processes it through the compensation values, and integrates it with the image data flow. By using this intermediary structure, the system achieves accurate camera movement compensation while maintaining modular architecture that reduces overall system complexity.
3Measurement precision
If compensation values are calculated using backpropagation, then sensor error correction is improved, but computational load increases
Solution Approach 1:
The patent creates a second CNN by copying the weights and architecture of the first pre-trained CNN. This copying approach allows the system to use the same computational framework for compensation calculations without requiring a completely new training process. The backpropagation is applied only to the additional interface layer and compensation values, not the entire network, thus reducing overall computational load while maintaining sensor error correction accuracy.
Data Source
AI summary
An embodiment includes training a first convolutional neural network (CNN) using a plurality of training images to generate first and second trained CNNs, and then adding an interface layer to the second trained CNN. The embodiment processes a first and second images in a sequence of images using the first trained CNN to generate a first and second result vectors. The embodiment also processes the second image using the second trained CNN and sensor data input to the interface layer to generate a third result vector. The embodiment modifies the sensor data using a compensation value. The embodiment compares the third result vector to the second result vector to generate an error value, and then calculates a modified compensation value using the error value. The embodiment then generates a sensor-compensated trained CNN based on the second trained CNN with the modified compensation value.