Foreground Extraction Using Dual Learned Models for Specific Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing foreground extraction methods in image processing, particularly for generating virtual viewpoint images, face challenges in maintaining extraction accuracy when dealing with overlapping image regions of moving objects and specific objects with repeated patterns, such as goal nets. These methods often suffer from over-training issues when learning models are trained extensively on images containing specific objects, leading to decreased accuracy in images without these objects and increased computational load due to complex model structures.
Innovation Solution
The proposed solution involves an image processing apparatus that employs two learned models to extract foreground regions from captured images. The apparatus sets a first region not including the image region where a specific object appears and a second region including the image region where the specific object appears. The first learned model extracts the foreground region from the first region, while the second learned model, trained on more images with the specific object, extracts the foreground region from the second region, achieving higher accuracy when the specific object is present.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a learning model performs learning by using many captured images that capture a specific object such as a football goal as learning data, then extraction accuracy of the foreground region in captured images where the specific object shows up is improved, but extraction accuracy of the foreground region in captured images where the specific object does not show up deteriorates due to over-training
Solution Approach 1:
The patent divides the image processing into two distinct segments: one processing path for images containing specific objects (using a first learning model trained on images with specific objects) and another path for images without specific objects (using a second learning model trained on images without specific objects). This segmentation resolves the over-training problem by ensuring each model is specialized for its intended input type, preventing the degradation of performance in images without specific objects while maintaining high accuracy in images with specific objects.
2Measurement precision
If a learning model having a complex model structure performs learning to improve extraction accuracy of the foreground region irrespective of whether the specific object shows up, then extraction accuracy is improved, but computation load increases
Solution Approach 1:
The patent implements a dynamic model selection mechanism that adaptively chooses between two different learning models based on the presence or absence of specific objects in the input image. This dynamic approach allows the system to use a computationally intensive first learning model only when necessary (when specific objects are present), while using a more efficient second learning model for the majority of cases where specific objects are absent, thereby reducing overall computation load while maintaining high extraction accuracy across all image types.
Data Source
AI summary
A first region not including an image region where a specific object shows up, and a second region including an image region where the specific object shows up are set to a captured image. A first foreground region indicating a foreground region included in the first region extracted by a first learned model based on the captured image and the first region, and second foreground information indicating a foreground region included in the second region extracted by a second learned model based on the captured image and the second region are obtained. Here, extraction accuracy of the second foreground region extracted by the second learned model based on the captured image and on the second region is higher than extraction accuracy of the second foreground region extracted by the first learned model based on the captured image and on the second region.