Multilingual Document Filtering via Semantic Vector Space

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Multilingual document filtering and search processes are resource-intensive and costly due to the need for syntactic comparisons across multiple languages, which can be inefficient and require separate filtering models for each language.

Innovation Solution

The use of semantic vector representations and translation matrices allows for the transformation of native language semantic vectors into a base language semantic vector space, enabling semantic filtering and search across languages without the need for separate models for each language, using mechanisms like Word2Vec and cosine distance calculations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If syntactic searching is performed across multiple languages, then document filtering capability is improved, but resource consumption and processing time increase

Engineering Contradiction:

Improvemultilingual filtering capabilityVSAvoidresource consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent merges multiple language-specific filtering models into a single unified filtering model that operates on a shared semantic vector space. Documents and search queries are represented as semantic vectors that can be compared across languages without requiring separate syntactic analysis for each language, thereby reducing computational resources while maintaining multilingual capability.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent replaces traditional syntactic search mechanisms with semantic vector representations. Instead of performing word-by-word syntactic comparison across multiple languages, the system uses semantic embeddings to capture meaning and relationships, enabling cross-lingual filtering through mathematical vector operations rather than language-specific parsing rules.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If separate filtering models are created for each language, then filtering accuracy is improved, but device complexity increases

Engineering Contradiction:

Improvefiltering accuracyVSAvoidnumber of filtering models

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent creates a universal filtering model that serves multiple languages simultaneously. The model uses a shared semantic vector space where words from different languages are represented and can be compared using the same filtering logic, eliminating the need for separate language-specific models while maintaining accurate filtering across all supported languages.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent changes the representation parameters from language-specific syntactic features to language-independent semantic vectors. By transforming text into semantic vector representations that capture meaning rather than surface form, the system can filter accurately across languages using a single model with consistent parameter settings.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If syntactic comparison is performed for each language, then search precision is improved, but processing speed decreases

Engineering Contradiction:

Improvesearch precisionVSAvoidprocessing speed

Core Design Contradiction:

Measurement precisionVSSpeed

Solution Approach 1:

The patent substitutes mechanical syntactic comparison operations with semantic vector similarity calculations. Instead of performing detailed grammatical analysis and word-by-word matching for each language, the system uses pre-computed semantic vectors to quickly determine relevance through mathematical operations like cosine similarity, significantly increasing processing speed while maintaining search precision.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent performs preliminary semantic vector computation during the indexing phase, where documents are pre-processed and converted into semantic vector representations. This preliminary action allows for rapid retrieval and filtering during actual search operations, as the heavy computational work is completed in advance rather than in real-time during querying.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9984068B2Systems and methods for multilingual document filtering

Publication Date: 2018.05.29 MCAFEE LLC

AI summary

Systems, apparatus, computer-readable media, and methods to provide filtering and/or search based at least in part on semantic representations of words in a document subject to the filtering and/or search are disclosed. Furthermore key words for conducting the filtering and/or search, such as taboo words and/or search terms, may be semantically compared to the semantic representation of the words in the document. A common semantic vector space, such as a base language semantic vector space, may be used to compare the key word semantic vectors and the semantic vectors of the words of the document, regardless of the native language in which the document is written or the language in which the key words are provided.