Image Processing Apparatus Convolution With Depth Validity Mask
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing apparatuses face challenges in recognizing the surrounding environment with high accuracy, particularly due to difficulties in handling missing depth values in depth image data, which affects the recognition process.
Innovation Solution
The proposed image processing apparatus includes an encoder and a decoder, where the encoder performs a convolution process on captured image data and depth image data, using mask data to validate depth values, and the decoder generates inference results for the environment, thereby reducing the impact of missing depth values and improving recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If depth image data with missing depth values is used for machine learning processing, then recognition capability is improved, but processing accuracy deteriorates due to missing data
Solution Approach 1:
The patent introduces a validity map as an intermediary component that indicates whether depth values are valid or missing. This validity map is processed alongside the depth image data through convolution operations, allowing the system to distinguish between reliable depth information and missing data, thereby maintaining processing accuracy while utilizing depth image data with missing values
Solution Approach 2:
The patent applies different processing strategies to different regions of the image based on the validity of depth values. By using validity maps and conditional convolution operations, the system processes valid depth regions and missing depth regions differently, allowing the model to adapt to local variations in data quality and maintain overall processing accuracy
2Productivity
If convolution process is performed on depth image data with missing values, then feature extraction is enabled, but harmful factors increase due to invalid data processing
Solution Approach 1:
The validity map serves as a mediator that guides the convolution process. By combining the depth image data with the validity map through convolution operations, the system can identify and exclude regions with missing depth values from the feature extraction process, preventing harmful effects of processing invalid data while maintaining productive feature extraction from valid regions
Solution Approach 2:
The patent converts the presence of missing depth values from a harmful factor into a beneficial feature by using validity maps to explicitly mark invalid regions. This allows the system to learn to ignore or handle missing data appropriately, transforming the problem of incomplete data into an opportunity to improve robustness and handle real-world scenarios where perfect data is rare
Data Source
AI summary
An image processing apparatus includes an encoder and a decoder. The encoder includes a first convolution processor. The first convolution processor generates, based on first feature quantity data and first control data, second feature quantity data by performing a first convolution process on the first feature quantity data. The first feature quantity data includes captured image data and depth image data. The depth image data includes map data on a depth value of a subject corresponding to the captured image data. The first control data includes map data on validity of the depth value corresponding to the depth image data. The encoder generates pieces of feature quantity data including feature quantity data corresponding to the second feature quantity data. The decoder generates, based on the pieces of feature quantity data, an inference result of an environment around an imager that has generated the captured image data.