Auto-Encoder Neural Network for Noisy Video Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting targets in noisy moving image data, such as ultrasonography or surveillance videos, suffer from low robustness and inaccuracies due to noise and small target movement, leading to incorrect detections and poor segmentation performance.
Innovation Solution
A training program and method that utilizes an auto-encoder and neural network to combine partial images from preceding and succeeding frames with the current frame, updating parameters based on differences between combined output images and annotation images to improve segmentation accuracy and robustness against noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the first method (using optical flow and segmentation network) is used for target detection, then segmentation of moving targets is improved, but robustness against noise deteriorates for targets with small movement
Solution Approach 1:
The patent combines multiple processing approaches by integrating the optical flow-based segmentation network with a temporal consistency module that processes sequences of frame images. This merging allows the system to leverage motion information for accurate segmentation while using temporal consistency to filter noise, thereby resolving the contradiction between segmentation accuracy and robustness against noise for targets with small movement.
Solution Approach 2:
The patent implements dynamic adaptation by adjusting the weighting between motion-based segmentation and temporal consistency based on detected target movement characteristics. When target movement is small, the system dynamically increases reliance on temporal consistency to maintain robustness, while for larger movements, it prioritizes motion-based segmentation for accuracy, thus resolving the contradiction across different operational conditions.
2Reliability
If the second method (using several preceding and succeeding frame images) is used for target detection, then robustness against noise is improved, but detection accuracy deteriorates when target position changes significantly
Solution Approach 1:
The patent employs dynamic weighting mechanisms that adjust the influence of temporal consistency versus motion-based detection based on the magnitude of target position changes. When significant position changes are detected, the system dynamically reduces reliance on temporal consistency to avoid detection errors, while maintaining noise robustness through selective frame utilization, thereby resolving the contradiction between robustness and accuracy.
Solution Approach 2:
The patent changes key parameters such as the number of frames considered and the weighting factors in the detection algorithm based on detected target motion characteristics. By adapting these parameters dynamically, the system maintains robustness against noise while preserving detection accuracy for targets with significant position changes, resolving the contradiction through parameter optimization.
3Device complexity
If training is performed without considering frame image context, then training complexity is reduced, but detection robustness deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network on synthetic training images that encode temporal consistency characteristics before fine-tuning on actual frame sequences. This preliminary training establishes a foundation for noise robustness without requiring complex training procedures, while subsequent adaptation to real data maintains detection accuracy, thereby resolving the contradiction between training complexity and robustness.
Solution Approach 2:
The patent introduces an intermediary temporal consistency module that acts as a bridge between simple training procedures and robust detection performance. This module incorporates temporal information in a computationally efficient manner during training, enabling the system to achieve high robustness without proportionally increasing training complexity, thus resolving the contradiction.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method includes: acquiring data including moving image obtained by photographing a target and annotation images each indicative of a region of the target in each of frame images in the moving image; executing a process using the data. The process includes: detecting the target in the frame images; inputting, to an auto-encoder, an image obtained by combining partial images including the target and peripheral region images of the target detected in a given number of preceding and succeeding second frame images in a time series of the moving image of a first frame image; inputting a partial image corresponding to the first frame image to a neural network performing a segmentation; updating parameter of the auto-encoder and the neural network based on a difference between an image obtained by combining images from the auto-encoder and the neural network and a partial image of the annotation image.