Permutation Pointwise Mutual Information Model for Efficient NLP Inference

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional deep learning methods for natural language inference require significant computational effort due to context-dependent distributed representations and dimensional compression, making them inefficient.

Innovation Solution

A learning device and inference device that perform morphological analysis to identify word classes and calculate pointwise mutual information to generate permutation and combination pointwise-mutual-information models, reducing computational effort by specifying target terms and learning models based on these calculations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If deep learning methods are used for natural language inference, then inference accuracy is improved, but computational effort increases enormously

Engineering Contradiction:

Improveinference accuracyVSAvoidcomputational effort

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent extracts only the essential linguistic features (word classes, permutations, and pointwise mutual information) from the complete text, rather than processing all contextual information. This selective extraction of critical elements maintains inference accuracy while dramatically reducing computational requirements.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments the natural language processing task into distinct components: morphological analysis to identify word classes, permutation specification to determine word order relationships, and pointwise mutual information calculation to measure associations. This segmentation allows each component to be processed efficiently independently.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If conventional deep learning processing is applied, then model learning capability is improved, but processing time increases

Engineering Contradiction:

Improvemodel learning capabilityVSAvoidprocessing time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent performs preliminary morphological analysis to identify word classes and specifies permutations before conducting the main inference task. By pre-processing and organizing the linguistic data in advance, the actual inference process requires minimal processing time while maintaining strong model learning capability.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent changes the parameter representation from continuous distributed vectors to discrete linguistic features (word classes, permutations, and pointwise mutual information values). This parameter transformation enables faster processing while preserving the essential learning capabilities needed for accurate inference.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20240086768A1Learning device, inference device, non-transitory computer-readable medium, learning method, and inference method

Publication Date: 2024.03.14 MITSUBISHI ELECTRIC CORP

AI summary

An information processing device includes a morphological-analysis performing unit that performs morphological analysis on character strings to identify the word classes of words included in the character strings; a specifying unit that specifies a permutation of target terms, which are words selected from the character strings, on the basis of the identified word classes; and a generating unit that calculates permutation pointwise mutual information, which is pointwise mutual information of the permutation in a corpus, and learns the permutation and the permutation pointwise mutual information, to generate a permutation pointwise-mutual-information model, which is a trained model.