Machine Learning Inference Using Intermediate Output Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for image processing, such as those used for noise removal and super-resolution, require multiple stages of processing, leading to inefficiencies when applied to sequential frames, as previously described techniques perform the same first noise removal processing on subsequent frames without utilizing intermediate outputs effectively.
Innovation Solution
An information processing apparatus and method that utilize a machine learning model with a first processing layer and a second processing layer, where intermediate outputs from the first layer are stored and used as inputs to the second layer, allowing for accelerated inference processing by carrying over previous intermediate outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If multi-stage machine learning models are used for image processing, then processing quality is improved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by pre-processing input images through the first processing layer before they are input to the second processing layer. This prepares the data in advance, allowing the second layer to work more efficiently on already-prepared inputs, thereby reducing overall processing time while maintaining quality.
Solution Approach 2:
The patent segments the image processing task into two distinct processing layers. The first layer handles initial processing tasks, and the second layer handles subsequent tasks. This segmentation allows each layer to be optimized independently and enables parallel processing of different frames, improving overall efficiency.
2Stability of the object's composition
If the same first noise removal processing is performed on subsequent frames, then noise removal consistency is improved, but processing efficiency deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-processing input images through the first processing layer before they are input to the second processing layer. This prepares the data in advance, allowing the second layer to work more efficiently on already-prepared inputs, thereby reducing overall processing time while maintaining quality.
Solution Approach 2:
The patent implements continuity of useful action by maintaining a buffer of pre-processed intermediate outputs from the first processing layer. When new frames are input, the system continuously feeds both new and previously stored intermediate outputs to the second processing layer, ensuring uninterrupted processing and maintaining consistent noise removal across all frames without reprocessing.
3Speed
If intermediate outputs are stored and reused, then processing speed is improved, but memory usage increases
Solution Approach 1:
The patent applies partial action by storing only a limited number of intermediate outputs in the buffer - specifically, enough to maintain processing continuity but not excessive amounts. This selective storage achieves the benefit of faster processing through reuse while constraining memory consumption to necessary levels only.
Data Source
AI summary
There is provided with an information processing method, Inference processing using a machine learning model having a first processing layer and a second processing layer a storing an intermediate output from the first processing layer is performed. A first intermediate output corresponding to a first input upon the first input being input into the first processing layer is output. An inference result upon (i) the first intermediate output and (ii) a second intermediate output from the first processing layer corresponding to a second input previous to the first input being input into the second processing layer, is output. The first intermediate output as the intermediate output is stored.