Transfer-Learning Deep Neural Network for Journey Action Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Constructing a new deep neural network for each journey is time-consuming and resource-intensive, especially when data for the journey is limited, as existing approaches require rebuilding the network from scratch.
Innovation Solution
Utilizing transfer-learning to construct a new deep neural network based on a similar existing network by incorporating nodes and weights from the pre-final layer, adding an embedding layer, and applying a softmax function to generate probability distributions for next actions, even when new features are not included in structured data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a new deep neural network is constructed from scratch for each journey, then the network can be specifically tailored to the journey requirements, but the construction time and resource consumption increase substantially
Solution Approach 1:
The patent applies preliminary action by pre-training a source deep neural network on source structured data before it is needed for the target journey. This pre-trained network contains learned representations and patterns that can be transferred to the target journey, eliminating the need to start from scratch and significantly reducing construction time while maintaining journey-specific adaptability through the transfer learning process.
2Adaptability or versatility
If a new deep neural network is constructed from scratch for each journey, then the network architecture can be optimized for the specific journey, but the resource consumption increases substantially
Solution Approach 1:
The patent applies copying by replicating the architecture and weights of a pre-trained source deep neural network to create the target network for the new journey. Instead of consuming substantial computational resources to train a new network from scratch, the system copies the existing network structure and transfers relevant weights, significantly reducing energy consumption while maintaining the ability to optimize for journey-specific requirements through subsequent fine-tuning on target data.
3Loss of time
If transfer-learning is used to construct the target deep neural network based on the source network, then the construction time and resources are reduced, but the network must handle new features not included in the source structured data
Solution Approach 1:
The patent applies segmentation by dividing the deep neural network into distinct layers and components, specifically identifying which layers contain journey-specific features and which contain transferable patterns. This segmentation allows the system to selectively transfer weights from source to target networks while preserving and adapting to new features specific to the target journey, thus reducing construction time without sacrificing feature compatibility or adaptability.
Data Source
AI summary
Techniques are described herein that are capable of performing transfer-learning for structured data with regard to journeys defined by sets of actions. A first deep neural network (DNN) for a first journey is trained using structured data. Weights of nodes in the first DNN are transferred to nodes in a second DNN for a second journey using transfer-learning. An embedding layer replaces a final layer of the first DNN in the second DNN to provide an output with a same number of nodes as a pre-final layer of the first DNN. Weights of the nodes in the embedding layer are initialized based at least on a probability that a new feature of the second journey co-occurs with each feature in the structured data. A softmax function is applied on a final layer of the second DNN to indicate possible next actions of the second journey.