Dialogue System Model Training Using Named Entity Abstraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Dialogue systems face challenges in accurately understanding utterances containing named entities, as existing techniques do not adequately address the variations and context-dependent nuances of these entities during learning, leading to reduced model accuracy.
Innovation Solution
An information processing method that acquires text information, identifies named entities, replaces them with abstract expressions based on a predetermined rule, and adds context-dependent numeral values, generating improved training data for learning a dialogue system model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If named entities are directly used in training data, then the training data reflects real-world diversity, but the model accuracy deteriorates due to variations and context-dependent nuances
Solution Approach 1:
The patent applies parameter changes by transforming named entities into structured representations with multiple attributes (concept type, appearance, location, etc.) rather than using raw text strings. This allows the system to vary entity parameters across different contexts while maintaining consistent conceptual understanding, thereby improving model accuracy without losing real-world diversity.
Solution Approach 2:
The patent segments named entities into distinct attribute components (concept type, appearance, location, etc.) that can be independently processed. This segmentation allows the model to handle each attribute separately, improving accuracy by focusing on relevant features while ignoring irrelevant variations, thus resolving the contradiction between handling entity diversity and maintaining accuracy.
2Reliability
If named entities are replaced with abstract expressions, then model accuracy improves, but information about specific entities is lost
Solution Approach 1:
Instead of abstracting away entity-specific information, the patent transforms it into structured parameters. Each named entity is represented by multiple attributes (concept type, appearance, location, etc.) that preserve specific information while enabling consistent processing. This maintains information integrity while improving model accuracy through structured representation.
Solution Approach 2:
The patent introduces an intermediary representation layer between raw named entities and the model processing. This intermediary structure (with concept types and attributes) acts as a mediator that preserves entity-specific information in a standardized format, allowing the model to access detailed information without being affected by surface-level variations in entity names.
3Measurement precision
If context-dependent numeral values are added, then entity distinction within dialogues improves, but data processing complexity increases
Solution Approach 1:
The patent segments entity representation into fixed attribute components, including numeral values as one of several standardized attributes. This segmentation allows the system to add context-dependent information in a structured way without overwhelming complexity, as each attribute is processed independently through dedicated processing pathways.
Solution Approach 2:
The patent performs preliminary processing to extract and structure entity attributes before main model processing. Context-dependent numeral values are computed and attached to entity representations in advance, organizing complex information into predefined structures that simplify subsequent processing rather than increasing overall system complexity.
Data Source
AI summary
An information processing method includes acquiring first text information from a storage apparatus in which the first text information representing one or more utterance sentences is stored as a learning data set, identifying one or more named entities included in the acquired first text information, replacing each of the one or more identified named entities with an abstract expression abstracted based on a predetermined rule thereby generating second text information from the first text information, and learning a model of a dialogue system using, as training data, the second text information generated in the replacing.