Dictionary Matrix Feature Identification in Deep Neural Networks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional deep neural network (DNN) learning technologies face challenges in identifying and predicting which features have been learned, leading to difficulties in ensuring predictability and efficiency in the learning process, as they rely on user-defined criteria and arbitrary data sets that may not accurately represent the intended features.

Innovation Solution

A learning apparatus and method that generates a model with an encoder, vector generating unit, and decoder, where the model learns to output information corresponding to input information using a dictionary matrix, allowing for the identification of learned features through a dictionary matrix that represents the aggregation of base vectors without relying on user-defined criteria.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional DNN learning is performed using arbitrary data sets and user-defined criteria, then the learning process can be completed, but the predictability and accuracy of identified features deteriorate

Engineering Contradiction:

Improvepredictability of learned featuresVSAvoidcomplexity of feature identification

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The DNN automatically generates and identifies learned features through its own internal mechanisms (encoder, vector generating unit, decoder) without requiring external user-defined criteria or complex feature analysis systems. The model serves itself by using its own weights and structure to produce meaningful feature representations that are directly interpretable.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The invention changes the parameter representation from arbitrary user-defined criteria to objective dictionary matrix representations. The dictionary matrix contains base vectors that are systematically derived from the data, transforming the feature identification process from subjective to objective, thereby improving predictability without increasing complexity.

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If user-defined criteria are used for feature learning, then the learning process can be controlled, but the accuracy of feature identification deteriorates

Engineering Contradiction:

Improveaccuracy of feature identificationVSAvoidease of defining learning criteria

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The system eliminates the need for users to define learning criteria by having the DNN automatically determine features through its internal encoder-decoder architecture. The model uses its own learned representations to identify features, making the process both accurate and operationally simple.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The dictionary matrix serves as an intermediary between the raw input data and the feature identification process. It translates arbitrary data into structured base vector representations that are both easy to work with and accurate in representing learned features, eliminating the need for complex user-defined criteria.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If arbitrary data sets are used for learning, then the learning process can be simplified, but the representativeness of learned features deteriorates

Engineering Contradiction:

Improveversatility of feature representationVSAvoidrepresentativeness of learned features

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The invention transforms the representation of learned features from arbitrary data-specific parameters to standardized dictionary matrix parameters. The dictionary matrix contains base vectors that systematically represent features across different data types, achieving both versatility and reliability in feature representation.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The dictionary matrix serves multiple functions: it stores learned features, enables feature identification, and provides a standardized representation that can be applied across different data sets and tasks. This universal structure achieves versatility without sacrificing representativeness.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11521110B2Learning apparatus, learning method, and non-transitory computer readable storage medium

Publication Date: 2022.12.06 YAHOO JAPAN CORP

AI summary

According to one aspect of an embodiment a learning apparatus includes a generating unit that generates a model. The model includes an encoder that encodes input information. The model includes a vector generating unit that generates a vector by applying a predetermined matrix to the information encoded by the encoder. The model includes a decoder that generates information corresponding to the information from the vector. The learning apparatus includes a learning unit that, when predetermined input information is input to the model, learns the model such that the model outputs output information corresponding to the input information and the predetermined matrix serves as a dictionary matrix of the input information.