Instance-Level Semantic Segmentation via Neural Network Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital image processing systems fail to achieve accurate instance-level segmentation of digital visual media, often resulting in multiple overlapping segmentations for a single object instance and inability to distinguish between objects of the same class, leading to mislabeling and incorrect representation.
Innovation Solution
A neural network is trained using a convolutional encoder-decoder architecture to generate instance-level labels by processing bounding boxes and distance maps, allowing for accurate segmentation of target objects regardless of their positional relationship with the bounding box, and applying weights based on class scores to distinguish between objects of the same class.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional detection and segmentation methods are used to identify objects in digital visual medium, then multiple segmentations can be generated for an object instance, but this results in multiple overlapping candidates with different class labels causing mislabeling
Solution Approach 1:
The patent divides the image into multiple candidate segmentations based on bounding boxes, then applies non-maximum suppression to select the most accurate segmentation. This resolves the contradiction by systematically processing multiple segments and eliminating redundant or inaccurate ones through threshold-based filtering and overlap resolution.
Solution Approach 2:
The system uses class labels and confidence scores as feedback to iteratively refine segmentation selection. By evaluating each candidate segmentation's class label consistency and confidence level, the system feedback-drivenly selects the optimal segmentation, resolving mislabeling issues from multiple overlapping candidates.
2Quantity of substance
If semantic segmentation method labels all pixels of a given object class, then complete coverage of the object class is achieved, but the ability to distinguish individual object instances is lost
Solution Approach 1:
The patent generates multiple candidate segmentations for each bounding box, where each candidate represents a potential instance segmentation. By maintaining and evaluating multiple segmentations rather than merging them into a single class-level mask, the system preserves instance discrimination capability while achieving complete pixel coverage through the union of all candidates.
Solution Approach 2:
The system applies different processing and selection criteria to different regions and candidates based on their local characteristics (confidence scores, overlap ratios, class label consistency). This allows precise instance-level discrimination in regions where multiple candidates exist, while maintaining complete coverage across the entire image.
3Area of stationary object
If multiple bounding boxes are generated for an object instance, then more comprehensive coverage is achieved, but redundant and overlapping segmentations increase computational complexity
Solution Approach 1:
The patent extracts and processes only the most relevant candidate segmentations by applying non-maximum suppression and confidence thresholding. This removes redundant and low-quality candidates, reducing computational complexity while preserving comprehensive object coverage through the selected high-confidence segmentations.
Solution Approach 2:
The system discards redundant and low-confidence candidate segmentations through threshold-based filtering and overlap elimination, then recovers complete object coverage by selecting the optimal segmentation from remaining candidates. This process reduces complexity by eliminating unnecessary computations while maintaining comprehensive coverage.
Data Source
AI summary
Certain aspects involve semantic segmentation of objects in a digital visual medium by determining a score for each pixel of the digital visual medium that is representative of a likelihood that each pixel corresponds to the objects associated with bounding boxes within the digital visual medium. An instance-level label that yields a label for each of the pixels of the digital visual medium corresponding to the objects is determined based, in part, on a collective probability map including the score for each pixel of the digital visual medium. In some aspects, the score for each pixel corresponding to each bounding box is determined by a prediction model trained by a neural network.