Sparse Coding Layer for Convolutional Neural Network Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Typical convolutional neural networks face high time complexity and low robustness when processing voice and music signals, necessitating a method to reduce execution time complexity while increasing robustness.
Innovation Solution
Integration of a sparse coding layer with dictionary atoms into the convolutional neural network, which performs sparse non-negative coding and dictionary training, and uses a membership function for channel-wise voting to classify dictionary atoms, thereby reducing time complexity and enhancing robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a typical convolutional neural network is used to process voice and music signals, then the network can perform signal classification, but the time complexity is high and robustness is low
Solution Approach 1:
The patent divides the traditional convolutional neural network into separate functional modules: a convolutional layer for feature extraction and a sparse coding layer for signal reconstruction. This segmentation allows each layer to specialize in specific tasks, improving overall robustness while reducing computational redundancy and time complexity.
Solution Approach 2:
The patent introduces dictionary atoms as an intermediary component between the convolutional layer and the output. These dictionary atoms serve as a bridge that enables efficient signal reconstruction through sparse coding, enhancing robustness by providing a structured representation while reducing time complexity through sparse computation.
2Loss of time
If the network structure is simplified to reduce time complexity, then execution time decreases, but robustness and ability to comprehend abstract concepts deteriorates
Solution Approach 1:
The patent changes the computational parameters by introducing sparsity constraints in the sparse coding layer. Instead of dense computations throughout the network, the model uses sparse representations with limited non-zero coefficients, significantly reducing time complexity while maintaining robustness through the structured sparse coding process.
Solution Approach 2:
The patent applies different computational qualities to different parts of the network: the convolutional layer uses dense computations for comprehensive feature extraction, while the sparse coding layer uses sparse computations for efficient signal reconstruction. This local differentiation optimizes both time complexity and robustness in their respective domains.
Data Source
AI summary
A machine learning method and a machine learning device are provided. The machine learning method includes: receiving an input signal and performing normalization on the input signal; transmitting the normalized input signal to a convolutional layer; and adding a sparse coding layer after the convolutional layer, wherein the sparse coding layer uses dictionary atoms to reconstruct signals on a projection of the normalized input signal passing through the convolutional layer, and the sparse coding layer receives a mini-batch input to refresh the dictionary atoms.