Pitch Color Model for Video Foreground Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video segmentation methods for sporting scenes face challenges in accurately classifying foreground and shadow areas with high memory and computational costs, particularly in distinguishing moving cast shadows from foreground pixels, which is crucial for applications like player segmentation and tracking.
Innovation Solution
A method is introduced that determines a pitch color model and mask for a sporting scene, classifying elements as background if they match the pitch color model, and updates the pitch background model using colors that match, employing a pitch color model that includes shades of the pitch color and shadows, and using a lit-shadow color model to handle chromatic shifts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional background modelling methods are used for sporting scenes, then foreground segmentation can be achieved, but memory and computational requirements become excessively high
Solution Approach 1:
The patent segments the background into distinct models: a pitch background model for the playing surface and a non-pitch background model for other areas. This segmentation allows selective application of computational resources, using the simplified pitch model for the majority of the scene while maintaining detailed non-pitch modeling only where necessary, thereby reducing overall memory requirements while preserving segmentation accuracy.
Solution Approach 2:
The patent applies different modeling qualities to different regions: a simplified pitch color model with limited modes for the pitch area, and a more comprehensive non-pitch background model for other regions. This local differentiation optimizes memory usage by avoiding unnecessary computational complexity in the pitch region while maintaining detailed modeling where needed.
2Reliability
If traditional background modelling methods are used for sporting scenes, then foreground segmentation can be achieved, but computational costs become excessively high
Solution Approach 1:
By dividing the background into pitch and non-pitch segments with different modeling complexities, the patent reduces computational load. The simplified pitch model requires fewer calculations per pixel compared to comprehensive background modeling, significantly reducing overall computational cost while maintaining segmentation reliability.
Solution Approach 2:
The patent changes the number and complexity of background modes dynamically. The pitch background uses a limited set of modes (e.g., 3-5 modes) representing different pitch colors and lighting conditions, while the non-pitch background uses more modes. This parameter adjustment optimizes computational cost by reducing the number of modes processed in the most extensive area (pitch) while maintaining accuracy.
3Ease of operation
If moving cast shadows are treated as foreground pixels, then shadow detection is simplified, but player segmentation and tracking accuracy deteriorate
Solution Approach 1:
The patent applies shadow handling with different qualities to different regions. Within the pitch area, shadows are treated as variations of the pitch background using the simplified pitch color model, avoiding false foreground detection. In non-pitch areas, traditional shadow modeling applies. This localized approach maintains player segmentation accuracy by preventing pitch shadows from being misclassified as players.
Solution Approach 2:
The patent uses a simplified, lightweight pitch background model that is computationally inexpensive to maintain. This disposable-like model is updated frequently with new modes as needed but doesn't require heavy computational resources, allowing real-time processing of shadow variations without affecting player detection accuracy.
Data Source
AI summary
A method of classifying foreground and background in an image of a video by determining a pitch color model of the image, the pitch color model comprising a pitch color, color shades of the pitch color, and the pitch color and the color shades under different shades of shadow. Then determining a pitch mask based on a pitch segmentation of the image of the video, determining a pitch background model based on the pitch mask and the pitch color model. The method may continue by classifying each of the elements of the pitch mask as background if a color of the element of the pitch mask matches the pitch color model and updating the pitch background model and the pitch color model using the colors of the elements that have been classified to match the pitch color model.