Topic-Dependent Convolutional Neural Network for Dialog Subject Estimation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing convolutional neural networks struggle to accurately estimate the subject of a dialog when the amount of learning data is insufficient, leading to poor performance and inability to handle multi-label outputs and context-dependent subject variations.

Innovation Solution

A subject estimation system utilizing a convolutional neural network with topic-dependent and topic-independent layers, where topic-dependent layers learn weights specific to each topic and topic-independent layers learn weights not dependent on topics, allowing for accurate estimation even with limited data by integrating results at higher tiers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a conventional convolutional neural network is used for subject estimation, then the system can process dialog text, but the estimation accuracy deteriorates when the amount of learning data is insufficient

Engineering Contradiction:

Improvesubject estimation accuracyVSAvoidamount of learning data

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent segments the convolutional neural network into topic-dependent convolutional layers and topic-independent convolutional layers. The topic-independent layers learn from general dialog patterns across all topics, while topic-dependent layers specialize in topic-specific patterns. This segmentation allows the model to effectively utilize limited topic-specific data by leveraging knowledge from the topic-independent layers, thereby maintaining high estimation accuracy even when learning data is insufficient.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by making different parts of the neural network have different learning characteristics. The topic-independent convolutional layers use one set of weights for general patterns, while topic-dependent convolutional layers use separate weights for topic-specific patterns. This allows the system to adaptively apply the appropriate level of specialization based on the available data, improving accuracy when data is limited.

Inventive Principle:

Principle #3Local quality

2Adaptability or versatility

If topic-specific convolutional layers are added to handle different topics, then the system can handle multi-domain dialog subjects, but the device complexity increases

Engineering Contradiction:

Improvemulti-domain dialog handling capabilityVSAvoidneural network structure complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements universality by designing the topic-independent convolutional layers to serve all topics simultaneously. These layers learn general dialog patterns that are applicable across multiple domains, providing a universal foundation that reduces the need for extensive topic-specific components while maintaining multi-domain handling capability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent segments the network into modular topic-dependent and topic-independent layers, where each segment has a specific function. This modular segmentation allows the system to handle multiple topics by activating appropriate layers while maintaining a manageable overall structure, balancing versatility with complexity.

Inventive Principle:

Principle #1Segmentation

3Measurement precision

If conventional sentence classification methods are used, then the system can classify sentences, but it cannot accurately estimate dialog subjects when learning data is limited

Engineering Contradiction:

Improvedialog subject estimation accuracyVSAvoidinformation loss due to insufficient learning data

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent merges the outputs of multiple convolutional layers (both topic-dependent and topic-independent) through pooling layers and fully connected layers. This merging process integrates information from different sources, allowing the system to compensate for information loss in any single topic by leveraging patterns from other topics, thereby maintaining high estimation accuracy with limited data.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The pooling layers and fully connected layers act as intermediaries that aggregate and synthesize information from the convolutional layers. These intermediary components help preserve important dialog subject information by combining features across different topics and layers, preventing information loss even when learning data is insufficient.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS9870768B2Subject estimation system for estimating subject of dialog

Publication Date: 2018.01.16 PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD

AI summary

A subject estimation system includes a convolutional neural network to estimate a subject label of a dialog. The convolution neural network includes: one or more topic-dependent convolutional layers and one topic-independent convolutional layer, each of the one or more topic-dependent convolutional layers performing, on an input of a word-string vector sequence corresponding to dialog text transcribed from a dialog, a convolution operation dependent on a topic, and the topic-independent convolutional layer performing, on the input of the word-string vector sequence, a convolution operation not dependent on the topic; a pooling layer performing pooling process on outputs of the convolutional layer; and a fully connected layer performing full connection process on outputs of the pooling layer.