Cross-Domain Image Comparison Using Semantic Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods fail to effectively compare synthesized accident videos generated by computers with real accident videos, as they are cross-domain and differ significantly in pixel level, making it difficult to assess similarity for self-driving simulation tests.
Innovation Solution
A cross-domain image comparison method and system that uses semantic segmentation to identify region of interest pairs, bounding boxes, and central points, followed by similarity analysis through position, angle, size, and contour similarity analyzers to determine the similarity between frames from different devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional pixel-level comparison methods are used to compare synthesized and real accident videos, then the comparison process is simple, but the similarity cannot be accurately obtained due to significant pixel-level differences between cross-domain videos
Solution Approach 1:
The patent segments the video frames into multiple semantic regions (sky, ground, vehicle, pedestrian, etc.) using semantic segmentation technology. Instead of comparing entire frames or individual pixels, the method divides the comparison task into region-level comparisons. This allows for accurate similarity measurement by focusing on semantically meaningful areas while ignoring irrelevant pixel-level differences between synthesized and real videos.
Solution Approach 2:
The patent introduces semantic segmentation maps and region-based features as intermediary representations between the raw video frames and the similarity comparison. These intermediaries transform the pixel-level data into semantically meaningful region descriptors, enabling accurate comparison while filtering out domain-specific pixel variations. The bounding boxes and region features serve as mediators that bridge the gap between cross-domain videos.
2Measurement precision
If semantic segmentation and region-based comparison methods are used to accurately measure similarity between cross-domain videos, then the measurement precision is improved, but the device complexity increases
Solution Approach 1:
The patent employs a universal semantic segmentation approach that can handle multiple video types (synthesized and real) and multiple object categories (sky, ground, vehicles, pedestrians) using the same framework. The region-based comparison method serves multiple purposes: it identifies corresponding regions, extracts features, and computes similarity metrics, reducing the need for separate specialized modules for each comparison task.
Solution Approach 2:
The patent transforms the comparison problem from pixel-space to feature-space by changing the parameters being compared. Instead of comparing raw pixel values, the method extracts region-based parameters such as bounding box coordinates, region areas, and semantic labels. This parameter transformation simplifies the comparison by focusing on invariant features that remain consistent across domains while reducing sensitivity to pixel-level variations.
Data Source
AI summary
A cross-domain image comparison method and a cross-domain image comparison system are provided. The cross-domain image comparison method includes the following steps. Two videos in cross-domain are obtained. The videos are generated by different types of devices. A plurality of semantic segmentation areas are obtained from one frame of each of the videos. A region of interest pair (ROI pair) is obtained according to moving paths of the semantic segmentation areas in the videos. Two bounding boxes and two central points of the ROI pair are obtained. A similarity between the frames is obtained according to the bounding boxes and the central points.