Neural Network Weight Updating for Spatially Selective Contraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for reducing computation in machine learning models, such as creating lightweight models, often result in accuracy deterioration and inference errors due to conventional contraction techniques that uniformly reduce spatial information, leading to suboptimal weight retention and inference errors.
Innovation Solution
A computation device and method that updates the weights of a neural network model based on spatial information using an inference result and importance coefficients, allowing selective contraction of the model to focus on areas requiring accuracy while reducing computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional contraction technology is applied after creating a lightweight model with reduced spatial information, then the model size is reduced, but accuracy deteriorates in the area where space has been reduced
Solution Approach 1:
The patent applies local quality by differentiating the treatment of spatial regions. It creates a mask that identifies important regions (where objects are likely to exist) and applies contraction differently to these regions versus unimportant regions. The contraction rate is adjusted based on the mask values, allowing the model to maintain accuracy in critical areas while reducing size in less critical areas.
Solution Approach 2:
The patent segments the spatial information into important and unimportant regions using a mask generated from depth information and object probability predictions. This segmentation allows the contraction process to operate independently on different regions, applying stricter contraction to unimportant regions while being more conservative with important regions.
2Device complexity
If conventional contraction technology is applied to a lightweight model, then the model parameters are reduced, but inference errors occur due to superfluous retention of features
Solution Approach 1:
The patent performs preliminary action by first creating a mask that identifies important spatial regions before applying the contraction process. This pre-computed mask guides the subsequent contraction, ensuring that features in important regions are preserved while allowing more aggressive reduction in unimportant regions, thereby preventing inference errors.
Solution Approach 2:
The patent incorporates feedback mechanisms where the depth information and object probability predictions are used to generate masks that feedback into the contraction process. This feedback loop ensures that the contraction process adapts to the specific characteristics of the input data and maintains accuracy where needed.
3Productivity
If the range of searching for disparities is reduced to decrease computation, then computational load is reduced, but accuracy deteriorates in areas where information is not required
Solution Approach 1:
The patent applies local quality by differentiating the contraction rate based on spatial regions. Regions with higher object probability and specific depth characteristics receive lower contraction rates (preserving more features), while regions with lower object probability receive higher contraction rates (reducing features more aggressively). This allows the system to maintain processing frequency while preserving accuracy where objects are likely to exist.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computation device comprising an inference unit that uses an inference model, which is a neural network model, and outputs an inference result corresponding to input data, and an updating unit that updates a weight of the inference model based on the inference result, and spatial information indicating at least a part of an area in the input data.