Omnidirectional Abnormality Prediction With Snippet-Frame Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing weakly supervised approaches for abnormality prediction in computer vision struggle with identifying sudden abnormalities in rapidly changing situations and often generalize abnormality scores excessively, leading to inaccurate detection.
Innovation Solution
An apparatus and method that utilize an omnidirectional image to predict abnormality by generating a feature map, extracting snippet and frame features, and calculating direction scores using a direction focus loss, with minimal labeling, enabling accurate frame-level prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If weakly supervised approach is used for abnormality prediction, then labeling cost is reduced, but detection accuracy for sudden abnormalities deteriorates
Solution Approach 1:
The patent divides the video into multiple frames and extracts features at different levels (snippet level and frame level). By segmenting the analysis into coarse anomaly detection at snippet level and fine anomaly detection at frame level, the system achieves high detection accuracy without requiring extensive frame-level annotations, thus resolving the contradiction between labeling cost and detection accuracy.
2Device complexity
If abnormality score is allocated to predetermined number of frames, then processing complexity is reduced, but detection precision deteriorates due to excessive generalization
Solution Approach 1:
The patent applies local quality by performing fine anomaly detection at specific frame levels where abnormalities are most likely to occur, rather than uniformly processing all frames. The system calculates fine anomaly scores for specific frames based on frame-level features, allowing precise detection without the need to process every frame with equal complexity, thus balancing processing complexity and detection precision.
3Measurement precision
If omnidirectional image data is processed in detail, then detection accuracy is improved, but data processing time increases
Solution Approach 1:
The patent performs preliminary coarse anomaly detection at the snippet level before conducting detailed frame-level analysis. By first identifying potentially abnormal snippets using computationally efficient operations, the system can then focus detailed processing only on relevant frames, significantly reducing overall data processing time while maintaining high detection accuracy for omnidirectional image data.
Data Source
AI summary
The present invention relates to an apparatus for predicting abnormality and a method thereof, the method comprises generating a feature map based on a previously captured omnidirectional image, generating a masked image based on the feature map, extracting a snippet feature based on the omnidirectional image, extracting a frame feature based on the masked image and predicting a direction of an abnormality using the feature map and a combined feature obtained by combining the snippet feature and the frame feature.