xLSTM Computer Vision for Linear-Complexity Image Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing neural network architectures for computer vision, such as Transformers and State Space Models, face inefficiencies in processing high-resolution images due to quadratic computational complexity, making them costly for tasks like image classification and segmentation.

Innovation Solution

The use of an autoregressive neural network architecture based on extended Long Short-Term Memory (xLSTM) blocks, which process image patches in alternating directions, providing a linear computational and memory complexity, and incorporating a matrix memory cell with a covariance update rule, to efficiently handle non-sequential inputs like images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If Transformers or State Space Models are used for image processing, then powerful modeling capabilities are achieved, but quadratic computational complexity makes processing costly

Engineering Contradiction:

Improvemodeling capabilityVSAvoidcomputational complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The image is divided into non-overlapping patches that are linearly projected into a sequence of patch tokens. This segmentation transforms the continuous image into discrete manageable units that can be processed sequentially by the autoregressive model, reducing the computational burden while preserving essential visual information

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent replaces the quadratic complexity mechanism of Transformers with a linear complexity autoregressive model. By substituting the self-attention mechanism with an autoregressive approach that processes tokens sequentially, the computational complexity is reduced from O(n²) to O(n), while maintaining powerful modeling capabilities through the autoregressive neural network

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If image resolution is increased for better processing quality, then accuracy improves, but computational cost increases quadratically

Engineering Contradiction:

Improveimage processing accuracyVSAvoidcomputational cost

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

High-resolution images are segmented into smaller patches, allowing the model to process them sequentially. This segmentation enables the autoregressive model to handle high-resolution inputs without incurring quadratic computational costs, as each patch is processed independently in sequence rather than requiring pairwise comparisons across all pixels

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the parameter of computational complexity from quadratic to linear by adopting an autoregressive architecture. This parameter change allows the system to maintain high processing accuracy on high-resolution images while significantly reducing the computational energy required, as the linear complexity scales more efficiently with input size

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP4660875A1Computer vision with extended long short-term memory neural networks

Publication Date: 2025.12.10 NXAI GMBH

Figure 1
Figure 2~3
Figure 4

AI summary

Disclosed is a method (200) of performing computer vision on a data processing apparatus (1302), comprising: receiving (202) an input image (104); processing (204) the input image (104) to generate a plurality of patch tokens (106); and processing (206) the plurality of patch tokens (106) by an autoregressive neural network.