Multi-Modal AI Recommender Using Knowledge Graph Embeddings

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing recommender systems often focus on single modalities, struggle with zero-shot (cold start) issues, and fail to provide accurate recommendations for diverse categories like food recipes due to varying user tastes, lacking a holistic approach that handles multi-modalities and multi-purposes.

Innovation Solution

A multi-modal, multi-purpose recommender device that integrates behavior-based, review-based, and image-based recommenders, utilizing knowledge graph embeddings to generate accurate recommendations by analyzing user behavior, text queries, and image features, respectively.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single-modality recommender is used, then the system complexity is reduced, but the recommendation accuracy and versatility deteriorate

Engineering Contradiction:

Improvesystem complexityVSAvoidrecommendation versatility

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent implements a universal recommender system that can handle multiple recommendation tasks (behavior-based, review-based, image-based) within a single framework. The system uses a unified knowledge graph embedding architecture that serves all three recommendation types, allowing one system to perform multiple functions that would traditionally require separate systems.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system segments the recommendation process into distinct modules (behavior-based recommender, review-based recommender, image-based recommender) that can operate independently but share common infrastructure. Each module processes specific input types through dedicated pathways while converging on the same knowledge graph representation layer, enabling modular complexity management.

Inventive Principle:

Principle #1Segmentation

2Ease of operation

If a single-modality recommender is used, then the ease of operation is improved, but the measurement precision of user preferences deteriorates

Engineering Contradiction:

Improveease of operationVSAvoidpreference measurement precision

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The patent merges multiple data sources (user behavior logs, text reviews, and image data) into a unified recommendation framework. By combining these diverse data types through knowledge graph embeddings, the system achieves more precise preference measurement while maintaining ease of operation through a single interface that accepts different input modalities.

Inventive Principle:

Principle #5Merging (Combining)

3Device complexity

If traditional recommender systems are used, then the device complexity is low, but the reliability for diverse categories deteriorates

Engineering Contradiction:

Improvedevice complexityVSAvoidrecommendation reliability

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The system uses composite knowledge graph embeddings that integrate multiple data types (behavior, review, image) into a unified representation. This composite approach allows the system to handle diverse categories reliably by combining the strengths of different data sources, achieving high reliability without proportionally increasing system complexity.

Inventive Principle:

Principle #40Composite materials

4Adaptability or versatility

If zero-shot recommendation is implemented, then the adaptability to new users is improved, but the measurement precision of user preferences deteriorates

Engineering Contradiction:

Improveadaptability to new usersVSAvoidpreference measurement precision

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system performs preliminary actions by pre-processing and embedding knowledge graph data before actual recommendation. This pre-computed knowledge representation allows new users to receive immediate recommendations based on their profile without requiring extensive historical data, achieving adaptability while maintaining precision through the pre-established knowledge framework.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12541786B2Artificial intelligence device for multi-modal recommender and control method thereof

Publication Date: 2026.02.03 LG ELECTRONICS INC

AI summary

A method for controlling an artificial intelligence (AI) device can include receiving, via a processor in the AI device, a request for a recommendation, determining, via the processor, a selected recommender from among a behavior-based recommender, a review-based recommender and an image-based recommender based on at least one of a modality of the request, a format of the request and a content of the request, and transmitting, via the processor, the request to the selected recommender. The method can further include generating, via the processor, recommendation results based on knowledge graph embeddings corresponding to the selected recommender, and outputting, via an output unit in the AI device, the recommendation results.