Domain-Adaptive Chatbot Intent Classification With Structured LLM Prompts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for intent classification in chatbots face limitations such as the need for extensive data collection, high development costs, and inefficient model deployment, especially when adapting to new domains, and existing methods fail to address these challenges effectively.
Innovation Solution
A method and system for providing a domain-adaptive chatbot service based on a large language model, which can pre-train the LLM based on a predetermined format of a domain-adaptive chatbot service, which can pre-train the LLM based on a domain-adaptive chatbot service, which can provide a domain-adaptive chatbot service.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a small-scale classification model is fine-tuned with domain-specific data for each domain, then intent classification accuracy is improved, but development cost and time consumption increase significantly
Solution Approach 1:
The patent applies universality by using a single pre-trained LLM that can perform intent classification across multiple domains without requiring separate models for each domain. The model is trained on diverse domain-specific data during pre-training, enabling it to generalize its intent classification capability to new domains, thus eliminating the need for domain-specific model customization and reducing development complexity.
Solution Approach 2:
The patent applies preliminary action by performing extensive pre-training of the LLM on diverse domain-specific data before deployment. This pre-training phase prepares the model to handle various domains, so that when the model is deployed, it can directly perform intent classification without requiring additional fine-tuning or adaptation for each new domain, thereby reducing development time and cost.
2Measurement precision
If commercial closed LLMs are used for intent classification, then classification performance is improved, but cost burden and licensing restrictions increase
Solution Approach 1:
The patent applies this principle by replacing expensive commercial closed LLMs with open-source LLMs that can be freely used and modified. While open-source models may require more computational resources, they eliminate licensing costs and provide greater flexibility for customization and deployment, making the solution more economically viable for long-term implementation.
Solution Approach 2:
The patent applies self-service by enabling organizations to train and deploy their own LLMs using their own domain-specific data and infrastructure. This eliminates dependence on external commercial LLM services, allowing organizations to maintain full control over their intent classification system, customize it to their specific needs, and avoid ongoing licensing fees and API call restrictions.
3Adaptability or versatility
If open-source LLMs are used instead of commercial models, then implementation flexibility is improved, but classification performance in unseen domains decreases
Solution Approach 1:
The patent applies parameter changes by adjusting the training parameters and data composition during the pre-training phase. Specifically, the LLM is trained on diverse domain-specific data with carefully controlled parameters such as data distribution, training duration, and model architecture configurations. This optimization ensures that open-source LLMs achieve classification performance comparable to or exceeding commercial models while maintaining deployment flexibility.
4Ease of operation
If internal customer data is transmitted to external servers for LLM processing, then service functionality is improved, but data security risks increase
Solution Approach 1:
The patent applies taking out by extracting the LLM from external commercial services and deploying it locally within the organization's infrastructure. This allows the system to maintain full service functionality for intent classification while keeping all customer data within the organization's secure environment, eliminating the security risks associated with transmitting sensitive data to external servers.
Data Source
AI summary
A method for providing a domain-adaptive chatbot service based on a large language model (LLM) is performed by a computing device including a memory and a processor, and includes generating a plurality of domain-specific structured training prompts, each of which corresponds to a respective one of a plurality of different domains and includes a plurality of labeled intent sample sentences corresponding to the respective one domain, training the LLM based on the plurality of training prompts, receiving a user input query, generating a structured inference prompt including a plurality of labeled intent sample sentences corresponding to a domain of the user input query and the user input query, inputting the inference prompt into the trained LLM, and providing a response to the user input query according to the intent of the user input query determined by the trained LLM based on the inference prompt.