Multi-Task Neural Network Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for feature extraction in data processing, particularly in AI for Big Data, rely heavily on handcrafted features and classification frameworks, which are not always reliable and struggle with new data due to the scarcity of labeled training data.
Innovation Solution
A computer-implemented method using a deep neural network with a hidden layer connected to both a supervised classification network and an autoencoder-based reconstruction network for multi-task learning, allowing the neural network to be trained as a whole for feature extraction, guided by classification and reconstruction tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If handcrafted features are used with classification frameworks, then feature extraction can be performed, but reliability is reduced and performance on new data deteriorates due to scarcity of labeled training data
Solution Approach 1:
The patent merges supervised classification learning with unsupervised autoencoder reconstruction learning into a unified multi-task learning framework. The shared neural network backbone learns features that simultaneously satisfy both classification objectives (with limited labeled data) and reconstruction objectives (with abundant unlabeled data), thereby improving reliability and adaptability together rather than trading one off for the other.
Solution Approach 2:
The shared neural network in the multi-task learning framework serves multiple functions: it performs feature extraction for classification tasks and simultaneously performs feature extraction for reconstruction tasks. This universal feature extractor learns robust representations that generalize well to new data while maintaining high reliability across different task types.
2Measurement precision
If only supervised classification framework is used, then classification tasks can be performed, but performance deteriorates when sufficient labeled training data is not available
Solution Approach 1:
The patent introduces unsupervised autoencoder reconstruction as an intermediary learning task that bridges the gap between limited labeled data and sufficient feature learning. The reconstruction task acts as a mediator that provides additional learning signals from unlabeled data, enabling the supervised classification task to achieve high accuracy even with limited labeled training data.
Solution Approach 2:
The patent performs preliminary unsupervised pre-training using autoencoder reconstruction on abundant unlabeled data before fine-tuning with supervised classification on limited labeled data. This preliminary action of learning robust feature representations from unlabeled data prepares the model to achieve high classification accuracy with minimal labeled training data.
3Measurement precision
If multi-task learning with shared layers is used, then feature extraction accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the neural network into a shared backbone that performs feature extraction for both tasks, and task-specific heads that perform classification and reconstruction respectively. This segmentation allows the complex multi-task learning to be organized into manageable components, where the shared backbone learns general features and the task-specific heads specialize in their respective objectives, making the overall system more tractable despite its complexity.
Data Source
AI summary
Systems and methods for training a neural network model are disclosed. In the method, training data is obtained by a deep neural network (DNN) first, the deep neural network comprising at least one hidden layer. Then features of the training data are obtained from a specified hidden layer of the at least one hidden layer, the specified hidden layer being connected respectively to a supervised classification network for classification tasks and an autoencoder based reconstruction network for reconstruction tasks. And at last the DNN, the supervised classification network and the reconstruction network are trained as a whole based on the obtained features, the training being guided by the classification tasks and the reconstruction tasks.