Video Image Segmentation Using Reused Semantic Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning-based image segmentation methods require significant computational resources, limiting their application in real-time scenarios such as video calls or video conferences due to the high computational demands.
Innovation Solution
An image processing method that reuses the semantic feature of a previous image to predict the semantic feature of subsequent images in a sequence, reducing the need for re-extraction by using a noise reduction network, and employing a similarity threshold to ensure accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based image segmentation is performed on each captured image in real-time, then image segmentation accuracy is improved, but computational resource consumption increases significantly
Solution Approach 1:
The patent extracts semantic features from the first image in advance using a complete deep learning model, and then reuses these extracted features for subsequent images through a noise reduction network. This preliminary extraction and reuse mechanism avoids repeated full-model inference, significantly reducing computational resource consumption while maintaining segmentation accuracy for images with similar content.
2Measurement precision
If semantic feature extraction is performed on every image in the sequence, then segmentation accuracy is maintained, but processing time increases
Solution Approach 1:
The patent creates a copy of the semantic features extracted from the first image and reuses this copied feature representation for subsequent images. By copying and adapting the semantic features through the noise reduction network instead of re-extracting them, the system significantly reduces processing time while maintaining segmentation accuracy for temporally adjacent images.
3Measurement precision
If the complete deep learning model is applied to all images, then segmentation quality is ensured, but device complexity requirements increase
Solution Approach 1:
The patent segments the image processing task into two parts: (1) complete semantic feature extraction using the deep learning model for the first image, and (2) feature refinement using a lightweight noise reduction network for subsequent images. This segmentation allows the complex model to be used only when necessary, reducing overall device complexity requirements while maintaining segmentation quality.
4Productivity
If feature reusing is implemented for subsequent images, then computational load is reduced, but similarity requirement between images must be met
Solution Approach 1:
The patent changes the parameter of image similarity by focusing on temporally adjacent images in a video sequence, which naturally have higher similarity due to continuous capture. This parameter change enables effective feature reuse and computational load reduction, as the noise reduction network can successfully adapt features from the first image to subsequent images that meet the similarity requirement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An image processing method is provided, and is applied to the field of artificial intelligence technologies. In the method, continuously captured images in an image sequence are classified into a reference frame and a non-reference frame. For a non-reference frame in the image sequence, a semantic feature of a reference frame located before the non-reference frame is reused to predict a semantic feature of the non-reference frame, the semantic feature of the non-reference frame is no longer re-extracted, and then an image segmentation result of the non-reference frame is obtained through prediction based on the semantic feature of the non-reference frame. A similarity between the reference frame and the non-reference frame is effectively used in this solution, and a semantic feature extraction process for the non-reference frame is reduced by reusing the semantic feature of the reference frame. Therefore, an amount of computation of an image segmentation process for the non-reference frame can be effectively reduced, and an application scenario of the image processing method can be extended.