Remote Support Device With Future-View Image Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing systems for remotely monitoring vehicles face delays in processing camera images and object recognition results due to timing discrepancies, leading to inefficiencies in generating synthetic images for remote support.
Innovation Solution
A remote support system that performs projection transformation and delay compensation on camera images and object recognition data to synchronize processing, allowing for the generation of synthetic images before the original images are fully decoded, using communication and processing circuitry to adjust image and object viewpoints based on vehicle movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If camera images and object recognition results are separately transmitted and processed, then data transmission and processing can be performed in parallel, but timing misalignment causes delay in generating synthetic images
Solution Approach 1:
The system performs projection transformation on the camera image to generate a future image that predicts the camera viewpoint at a future timestamp. This preliminary action allows the synthetic image to be generated using predicted future positions rather than waiting for actual future images, thereby reducing the delay caused by timing misalignment between image and object data.
2Reliability
If the system waits for both image and object recognition data to be ready before generating synthetic images, then data accuracy is ensured, but output delay increases
Solution Approach 1:
The system performs projection transformation in advance to predict the camera image at a future timestamp before the actual image data is ready. This allows the synthetic image generation to proceed with predicted data rather than waiting for actual data arrival, maintaining reliability through mathematical prediction while reducing output delay.
Solution Approach 2:
The system creates a predicted copy of the future camera image through projection transformation based on current image data and vehicle movement information. This predicted copy serves as a placeholder that allows synthetic image generation to proceed without waiting for the actual future image data, thus reducing delay while maintaining data consistency.
3Loss of time
If projection transformation is performed to predict future camera viewpoints, then synthetic image delay is reduced, but processing complexity increases
Solution Approach 1:
The system replaces the mechanical approach of waiting for actual future image capture and transmission with a computational approach using projection transformation. By substituting physical image acquisition with mathematical prediction based on current image data and vehicle movement parameters, the system reduces delay without requiring additional hardware complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An image delay compensation time α is set to be equal to or less than an image delay time indicating a timing difference D1 between the timing T1 at which an image IMG1 is acquired by a camera CAM and a timing T3 at which a remote operator terminal 200 decodes the image IMG1. Based on information on a movement of a mobile vehicle, projection transformation is performed in which the image IMG1 is transformed into a future image IMG2 obtained at a camera viewpoint at a timing T2 that is earlier than the timing T1 by the image delay compensation time α. A synthetic image SIMG1 is generated based on the future image IMG2 and recognition information OR on the object in the image IMG1 and is output from a display device.