Interactive Area Extraction Using User-Guided Feature Map Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural network (DNN) models require significant manual annotation and frequent retraining to maintain accuracy, leading to high costs and inefficiencies in image segmentation tasks.
Innovation Solution
An area extraction method and system that integrates user interaction with DNN models to update feature maps using candidate bounding boxes and masks, reducing the need for extensive manual annotation and retraining through an iterative learning process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a DNN model is frequently retrained on newly obtained data to maintain accuracy, then the model accuracy is improved, but the time and cost for model updating increases
Solution Approach 1:
The system performs preliminary actions by pre-processing user input information and generating candidate bounding boxes before the actual model updating is needed. The area estimation unit extracts candidate bounding boxes from user inputs, and the interaction encoding unit prepares updated feature maps in advance, so that when model updating is required, the preparation work is already done, reducing the actual updating time and cost
Solution Approach 2:
The patent introduces an intermediary mechanism between user interaction and model updating. The interaction encoding unit acts as a mediator that translates user inputs into updated feature maps, which then guide the model updating process. This intermediary layer allows the system to incorporate new data without requiring complete retraining, as the intermediary structures (feature maps, candidate boxes) bridge the gap between raw user inputs and model parameters
2Measurement precision
If two DNN models are used for interactive segmentation, then the segmentation capability is improved, but the complexity of model maintenance increases
Solution Approach 1:
The patent merges the functionality of multiple DNN models into a unified architecture. The area extraction unit, segmentation head, and interaction encoding unit work together as an integrated system rather than separate models. This merging reduces maintenance complexity while preserving segmentation capability, as there is only one model structure to maintain instead of multiple independent models
Solution Approach 2:
The unified model structure performs multiple functions: area extraction, segmentation, and interactive refinement. The interaction encoding unit enables the model to handle various types of user inputs (clicks, draws, boxes) universally, making the system adaptable to different interaction modes without requiring separate specialized models for each function
Data Source
AI summary
An area extraction method includes acquiring a captured image, generating a feature map using the captured image, extracting a candidate bounding box indicating an area in which an object exists in the captured image using the feature map, a segmentation step of generating a mask that is a bounding box indicating the area in which the object exists using the feature map and the candidate bounding box, acquiring input information which is input by a user regarding the object in the captured image, extracting the candidate bounding box using the feature map and the input information, and updating the feature map using the candidate bounding box extracted in the area estimation step and the input information. Wherein the candidate bounding box can be extracted further using the mask generated in the segmentation step, and the feature map can be updated using the mask generated in the segmentation step.