Instance Segmentation via Multi-Scale Inference and NMS Replacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine-learning models for computer vision in applications like autonomous vehicles face challenges in accurately detecting objects in real-time due to high computation time and flawed identifications, especially when objects are proximate or outside the receptive field, leading to unreliable decision-making.
Innovation Solution
The techniques involve training machine-learning models using improved hard example mining, non-maximum suppression replacement, and multi-scale inference to produce accurate regions of interest (ROIs) and confidence scores, reducing computation time by constraining ROI sizes and using multiple models to cover a broader range of object sizes, thereby enhancing real-time object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If accurate ML models are used for object detection, then detection accuracy is improved, but computation time increases making real-time application difficult
Solution Approach 1:
The patent applies segmentation by dividing the image into multiple scales or resolutions, processing different regions at different levels of detail. This allows the system to focus computational resources on critical areas while maintaining overall detection accuracy, thereby reducing total computation time without sacrificing precision.
Solution Approach 2:
The patent implements partial action by applying full-resolution analysis only to regions of interest or potentially critical areas, while using lower-resolution analysis for other portions of the image. This selective approach maintains accuracy where needed while significantly reducing overall computation time.
2Adaptability or versatility
If ML models detect objects in proximate positions, then detection coverage is improved, but identification accuracy deteriorates due to difficulty in distinguishing between objects
Solution Approach 1:
The patent applies local quality by using higher resolution or more detailed analysis specifically for proximate objects that are difficult to distinguish, while using standard analysis for well-separated objects. This localized enhancement of quality improves identification accuracy for challenging cases without increasing computation time for all detections.
Solution Approach 2:
The patent introduces an additional dimension of analysis by processing images at multiple scales or resolutions. This dimensional approach allows the system to better separate and identify proximate objects by examining them at different levels of magnification, improving discrimination between closely-spaced objects.
3Adaptability or versatility
If multiple ML models are used to cover broader object size ranges, then detection versatility is improved, but system complexity increases
Solution Approach 1:
The patent implements universality by designing a single multi-scale processing framework that can handle objects of various sizes through different resolution levels. This unified approach provides the versatility of multiple specialized models while avoiding the complexity of maintaining separate model systems, as one framework performs multiple functions across different scales.
Data Source
AI summary
Techniques for using instance segmentation with machine learning (ML) models are discussed herein. An image can be provided as input to a ML model, which can generate, as an output from the ML model, a feature map comprising a plurality of features. Each feature of the plurality of features can comprise a confidence score, classification information, and a region of interest (ROI) determined in accordance with a non-maximal suppression (NMS) technique. Individual ROIs that are similar can be associated together for segmentation purposes. That is, instead of requiring a second ML model and/or a second operation to segment the image (e.g., identify which pixels correspond with the detected object, for example, by outputting a mask or set of lines and/or curves), the techniques discussed herein substantially simultaneously detect an object (e.g., determine an ROI) and segment the image.