Extending Neural Network Output Layers for Broader Intent Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing syntactic analysis models struggle to classify intents and slots for utterance sentences outside their specified fields, limiting their functionality.
Innovation Solution
The syntactic analysis apparatus extends the output layers by adding neurons and constructing a new learning corpus, allowing for the classification of intents and slots in a broader context, with a faster learning speed for the added neurons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the syntactic analysis model is trained only on a pre-generated learning corpus for a specified field, then the model achieves good classification accuracy for that field, but the model cannot classify intents and slots for utterance sentences outside the specified field
Solution Approach 1:
The output layer of the neural network is designed to be dynamically extendable. New neurons can be added to the output layer to accommodate new intents and slots without retraining the entire model. The system dynamically adapts its structure by adding neurons based on new learning requirements, allowing the model to handle out-of-field utterances while maintaining performance on original tasks.
Solution Approach 2:
The learning corpus is segmented into a pre-generated learning corpus (for original specified fields) and a new learning corpus (for extended fields). This segmentation allows the system to preserve the original training data while adding new data for extended functionality. The model learns from both segments, maintaining reliability on original tasks while gaining adaptability to new fields.
2Adaptability or versatility
If the output layer is extended by adding neurons for new intents and slots, then the model can classify a broader range of utterances, but the learning time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-generating a learning corpus for the original specified field and training the model on it before extension is needed. When extension is required, only the necessary new neurons are added and the model is fine-tuned on a new learning corpus rather than retraining from scratch. This preliminary preparation reduces the time and computational resources needed for future extensions.
Solution Approach 2:
The system changes parameters by adjusting the learning rate and other training parameters when extending the model. The learning rate is optimized for the extension phase to enable faster convergence. By changing training parameters appropriately, the system can efficiently learn new intents and slots without requiring excessive time and computational resources.
3Reliability
If the entire syntactic analysis model is retrained when new fields are added, then the model achieves optimal performance for the extended fields, but the training process becomes time-consuming and computationally expensive
Solution Approach 1:
The model structure is designed to be dynamically extendable by adding neurons to the output layer. When new fields are added, only the necessary new neurons are instantiated and trained, while the existing neurons and their connections are preserved. This dynamic extension approach maintains optimal performance for new fields while avoiding the time and computational cost of retraining the entire model.
Solution Approach 2:
The extended output layer with additional neurons provides universal functionality, allowing the model to handle both original specified fields and new extended fields through a single unified structure. The model achieves multi-functionality by seamlessly integrating new intent and slot classification capabilities alongside existing ones, eliminating the need for separate models or complete retraining.
Data Source
AI summary
A syntactic analysis apparatus according to an embodiment of the present disclosure may include an input device receiving a phrase uttered from a user, and a learning device performing at least one or more of extension of an intent output layer for classifying an utterance intent of the user from the uttered phrase and extension of a slot output layer for classifying a slot including information of the phrase and extending a pre-generated utterance syntactic analysis model, such that the uttered phrase is classified into the extended intent output layer and the extended slot output layer, thereby broadly classifying an intent and a slot for the phrase uttered from a user.