On-Device Object Detection and Masking with Multi-Scale Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image editing systems face inefficiencies, inflexibility, and inaccuracies due to high computational requirements, reliance on resource-rich devices, and limited ability to detect and segment a wide range of object instances, often resulting in unnatural boundary artifacts.
Innovation Solution
An on-device neural network model with multi-scale synchronized batch normalization and class-agnostic object detection, combined with post-processing techniques, enables efficient and flexible object detection and segmentation on various devices, generating accurate object masks without border artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems use group normalization neural network layers or large backbone networks for object detection, then detection accuracy is improved, but computational cost and resource consumption increase excessively
Solution Approach 1:
The patent changes the normalization parameter from group normalization to synchronized batch normalization, which computes normalization statistics across all groups simultaneously rather than separately. This parameter change maintains detection accuracy while reducing computational overhead by enabling better resource utilization and parallel processing efficiency.
2Power
If conventional systems rely on resource-rich devices for object detection and segmentation, then processing power is sufficient, but device flexibility and accessibility are reduced
Solution Approach 1:
The patent creates a universal object detection and segmentation system that can operate on diverse devices from resource-constrained mobile devices to powerful servers. The synchronized batch normalization approach and optimized architecture enable the same model to achieve consistent performance across different hardware platforms, making the system universally deployable without requiring resource-rich devices.
3Productivity
If conventional systems use closed-set training examples with predefined category labels, then training efficiency is improved, but the ability to detect and segment unseen object classes is reduced
Solution Approach 1:
The patent implements dynamic class-agnostic object detection where the system can adapt to detect objects from unseen classes during inference without requiring retraining. The synchronized batch normalization and flexible architecture allow the model to generalize to new object categories while maintaining training efficiency on the original closed-set data.
4Measurement precision
If conventional systems use tight bounding boxes for detected objects, then localization precision is improved, but segmentation quality deteriorates due to unnatural boundary artifacts
Solution Approach 1:
The patent employs separate detection and segmentation heads that work independently. The detection head provides precise bounding boxes for localization, while the segmentation head generates smooth object masks without being constrained by the tight bounding box boundaries. This segmentation into separate functional components resolves the conflict between localization precision and segmentation quality.
Data Source
AI summary
The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.