Language Characteristic Extraction for Multilingual Named Entity Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Named entity extraction systems face challenges when processing multiple languages, as the types of characteristics to be considered differ significantly between languages, and existing methods struggle to unify these differences within a single processing system.

Innovation Solution

A language characteristic extraction device that selects abstract rules common to multiple languages and sets language-specific characteristic extraction rules to extract features suited to each language, using a machine learning algorithm to perform named entity extraction, allowing for unified processing across languages.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If a unified processing system is used for multiple languages, then the system complexity is reduced and ease of operation is improved, but the ability to handle language-specific characteristics is insufficient

Engineering Contradiction:

Improveease of operationVSAvoidadaptability

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system segments language processing into two distinct components: (1) language-independent named entity extraction that handles common processing for all languages, and (2) language-specific characteristic extraction that captures language-particular features. This segmentation allows the system to maintain a simple unified architecture while adapting to different language characteristics through modular language-specific modules.

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If language-specific features are extracted for each language, then the extraction accuracy is improved, but the system complexity increases

Engineering Contradiction:

Improveextraction accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The invention extracts only the necessary language-specific characteristics from each language's morphological analysis results, rather than processing all possible features. For example, it extracts case information for German and verb conjugation patterns for Russian, but only when relevant to named entity extraction. This selective extraction maintains high accuracy while controlling system complexity by focusing on essential language-specific features.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system employs a universal named entity extraction framework that can process multiple languages through a common architecture. The language-specific characteristic extraction modules are designed to output standardized formats that can be uniformly processed by the named entity extraction component, allowing one system to serve multiple languages without requiring separate processing pipelines for each language.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If morphological analysis results are used directly as features, then the processing speed is maintained, but the extraction precision is insufficient for languages with different characteristics

Engineering Contradiction:

Improveprocessing speedVSAvoidextraction precision

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system performs preliminary language-specific characteristic extraction from morphological analysis results before the named entity extraction process. This preliminary action prepares language-appropriate features in advance, such as extracting noun case information for German or verb tense information for Russian, so that the subsequent named entity extraction can proceed efficiently with already-prepared language-specific features, maintaining processing speed while improving precision.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11809820B2Language characteristic extraction device, named entity extraction device, extraction method, and program

Publication Date: 2023.11.07 NIPPON TELEGRAPH & TELEPHONE CORP

AI summary

It is an object to successfully absorb a difference in characteristics to be taken into consideration between languages and implement common named entity extraction in a processing system.A language characteristic extraction device 11 includes a language characteristic extraction unit that selects an abstract rule corresponding to a characteristic of a target language from among a collection of abstract rules common to a plurality of languages and sets, as a language-specific characteristic extraction rule, a concrete feature extraction method suited to the target language and a definition given as an output condition, defines, for each of a plurality of target languages, the language-specific characteristic extraction rule, and extracts a feature corresponding to a language of an input sentence from a morphological analysis result of the input sentence while referring to the language-specific characteristic extraction rule defined for the language and aimed at extracting a feature related to a representation or a part of speech included in the morphological analysis result and outputs the feature as a language characteristic extraction result.