Weight-Shared Bi-Directional RNNs for Resource-Constrained Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision models for processing still images, such as CNNs and transformers, are resource-intensive and not suitable for devices with limited resources like embedded systems, due to their large model sizes and parameter requirements.
Innovation Solution
The use of Recurrent Neural Networks (RNNs) to process still images by interpreting pixels as sequences, with a custom variant called Weight-Shared Bi-Directional RNN (WS-BiRNN) that shares weights for forward and backward directions, and a combination with Separable RNN (SRNN) to reduce model size and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNN or transformer models are used for processing still images, then processing accuracy and capability are improved, but model size and memory requirements increase significantly
Solution Approach 1:
The patent segments the image processing task into sequential steps using multiple RNN layers, where each layer processes the image data in a specific pass (forward, backward, diagonal directions). This segmentation allows the model to achieve complex processing capabilities through repeated application of a smaller, more memory-efficient RNN unit rather than requiring a single large CNN or transformer model.
Solution Approach 2:
The patent changes the fundamental parameters of the neural network architecture by using RNNs with shared weights across different passes and directions. Instead of using millions of parameters in CNN filters or transformer attention mechanisms, the RNN approach uses a fixed set of weights that are reused and updated across sequential processing steps, dramatically reducing the total parameter count while maintaining processing effectiveness.
2Adaptability or versatility
If BiRNN is used to process images in both forward and backward directions, then processing capability is improved, but the number of model weights doubles compared to single-direction RNN
Solution Approach 1:
The patent merges the weight sets of forward and backward RNN layers by enforcing weight sharing. The same RNN weights are used for processing images in forward, backward, and diagonal directions across multiple passes. This merging strategy allows the model to maintain bidirectional and multi-directional processing capabilities while using a single set of weights that is updated collectively, preventing the doubling of parameters.
Solution Approach 2:
The RNN weights serve multiple functions by being applied across different processing directions (forward, backward, diagonal) and multiple passes. Each weight set is universal and adapts to handle various directional processing tasks, eliminating the need for separate dedicated weight sets for each direction and making the model more memory-efficient.
3Measurement precision
If multiple CNN layers are used to process still images, then processing depth and accuracy are improved, but inference time and computational complexity increase
Solution Approach 1:
The patent implements continuous processing through sequential RNN passes where each pass builds upon the previous one. The RNN maintains a hidden state that carries information across passes, allowing the processing to be continuous and iterative rather than requiring separate independent CNN layers. This continuity enables the model to progressively refine its understanding of the image through multiple passes while using the same computational units, reducing inference time compared to stacking multiple full CNN layers.
Data Source
AI summary
Methods, systems, and apparatus are disclosed for processing still images using recurrent neural networks (RNNs). The method can include: generating, by a first forward RNN layer module, first RNN output data from still image data; generating, by a first reverse layer module, first reverse layer data from the first RNN output data; generating, by a first backward RNN layer module, second RNN output data from the first reverse layer data, wherein machine learning model weights are shared between the first forward RNN layer module and the first backward RNN layer module; generating, by a second reverse layer module, output data from the second RNN output data; and processing, by a machine learning backbone module, the still image data using the output data, wherein the generating and the processing are performed by at least one data processor on a resource-constrained hardware device.