Interactive Image Segmentation Using User-Guided Neural Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep-learning neural networks for image processing, such as image segmentation and matting, suffer from complex operations, high costs, high bit error rates, slow transmission speeds, poor security, and inaccurate results, limiting their application in real-world scenarios, and require excessive user supervision leading to increased calculation burdens.
Innovation Solution
An improved deep-learning neural network architecture that incorporates user correction information to optimize segmentation and matting results by using a deep-learning neural network with a region proposal network (RPN) and a fully convolutional network (FCN), allowing for fewer user interactions and reduced calculation loads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a traditional deep-learning neural network is used for image segmentation, then the segmentation capability is provided, but the calculation burden is greatly increased and the segmentation accuracy is insufficient
Solution Approach 1:
The patent divides the deep-learning neural network into multiple stages: a first neural network performs initial segmentation to obtain a first segmentation result, and a second neural network refines this result based on user correction information to produce a second segmentation result. This staged approach reduces the calculation burden on a single network while improving overall segmentation accuracy through progressive refinement.
Solution Approach 2:
The patent applies partial action by using only a portion of the neural network (the second neural network) to process the correction information and refine the segmentation, rather than requiring the entire network to reprocess all data. This reduces the calculation burden while still achieving improved accuracy through targeted refinement.
2Measurement precision
If user supervision is applied to improve segmentation accuracy, then the segmentation result becomes more accurate, but the calculation burden of the neural network is greatly increased
Solution Approach 1:
The first neural network performs preliminary segmentation to generate an initial segmentation result before user correction is applied. This preliminary action reduces the amount of work needed in the refinement stage, as the second neural network only needs to process the correction information rather than performing complete segmentation from scratch.
Solution Approach 2:
The processing is segmented into two distinct neural networks: the first handles initial segmentation and the second handles refinement based on user feedback. This segmentation of processing tasks allows user supervision to be applied efficiently without requiring the entire system to recalculate everything, thus reducing the overall calculation burden.
3Ease of operation
If conventional image processing methods are used, then the processing can be performed, but the steps become complex and the user experience deteriorates
Solution Approach 1:
The patent merges multiple processing functions into an integrated deep-learning system that automatically performs initial segmentation, accepts user correction input, and refines the results through a unified neural network architecture. This combining of functions reduces the number of separate processing steps the user must manually manage, thereby improving ease of operation and user experience.
Data Source
Figure 1A~1B
Figure 2A~2B
Figure 3A(a)~4
AI summary
Proposed by the present disclosure are an image processing method, device, storage medium, and computer program product. In the method, an image is acquired, wherein said image comprises an object to be segmented or extracted from in the image; a deep learning neural network is used to segment or extract the object from in the image so as to obtain a first segmentation or extraction result; correction information inputted by a user for the first segmentation or extraction result is acquired; and according to the correction information, the deep learning neural network is used to modify the first segmentation or extraction result so as to obtain a second segmentation or extraction result.