Shallow CNN Infrared Detection Two-Phase Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based solutions for infrared target detection are inefficient on performance-limited systems due to high hardware requirements and data shortages, and shallow networks face challenges in converging to optimization values and preventing overfitting.
Innovation Solution
A two-phase learning strategy for shallow convolutional neural networks is proposed, using L2-norm for initial convolutional layer training and L1-norm for decision layer training, with a focus on optimizing the number of filters and fully connected layer dimensions to achieve scale-invariance and efficient target detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based solutions are used for infrared target detection, then detection performance is improved, but hardware requirements and computing load increase
Solution Approach 1:
The training process is segmented into two distinct phases: first phase trains convolutional layers using L2-norm loss function to extract features, while the second phase trains decision layers using L1-norm loss function for classification. This segmentation allows each phase to optimize specific aspects independently, achieving good detection performance with a shallow network structure that reduces hardware requirements.
Solution Approach 2:
The patent changes the loss function parameter from L2-norm in the first phase to L1-norm in the second phase. This parameter change enables the network to first learn robust feature representations and then focus on accurate classification, achieving satisfactory detection performance with fewer layers and reduced computational load.
2Power
If shallow networks are used to reduce hardware requirements, then power consumption is reduced, but convergence to optimization values becomes difficult
Solution Approach 1:
The first phase performs preliminary action by training the convolutional layers to extract meaningful features using L2-norm loss function before the second phase trains the decision layers. This preliminary feature extraction prepares the network in advance, enabling the shallow structure to converge effectively in the subsequent phase without requiring deep architectures.
Solution Approach 2:
The two-phase training strategy acts as an intermediary mechanism that bridges the gap between shallow network structure and effective convergence. By introducing separate training phases with different loss functions, the patent enables gradual optimization that helps shallow networks reach optimization values despite their limited depth.
3Device complexity
If shallow networks are used for target detection, then device complexity is reduced, but detection performance and scale-invariance capability are compromised
Solution Approach 1:
The patent introduces dynamic training strategies with two distinct phases that adaptively optimize different parts of the network. The first phase dynamically focuses on feature extraction in convolutional layers, while the second phase dynamically focuses on classification in decision layers. This dynamic approach enables shallow networks to achieve scale-invariance capability and satisfactory detection performance without increasing structural complexity.
Solution Approach 2:
By changing the loss function parameter from L2-norm to L1-norm between phases, the patent enables the shallow network to achieve different optimization objectives at different stages. This parameter change allows the network to develop both scale-invariance capability and detection performance while maintaining a simple structure.
4Loss of information
If L2-norm loss function is used for training, then feature extraction is improved, but classification accuracy may be compromised
Solution Approach 1:
The training process is segmented into two phases with different loss functions: L2-norm in the first phase optimizes feature extraction by penalizing large errors quadratically, while L1-norm in the second phase optimizes classification accuracy by being more robust to outliers. This segmentation allows each loss function to excel at its specific task without compromising the other.
Solution Approach 2:
The patent changes the loss function parameter from L2-norm to L1-norm between phases. L2-norm's quadratic penalty is better for continuous feature optimization, while L1-norm's absolute penalty is more suitable for discrete classification decisions. This parameter change enables optimal performance at each stage.
Data Source
AI summary
Disclosed is a method for training shallow convolutional neural networks for infrared target detection using a two-phase learning strategy that can converge to satisfactory detection performance, even with scale-invariance capability. In the first step, the aim is to ensure that only filters in the convolutional layer produce semantic features that serve the problem of target detection. L2-norm (Euclidian norm) is used as loss function for the stable training of semantic filters obtained from the convolutional layers. In the next step, only the decision layers are trained by transferring the weight values in the convolutional layers completely and freezing the learning rate. In this step, unlike the first, the L1-norm (mean-absolute-deviation) loss function is used.