Vehicle Image Analysis Device Scene-Based Caption Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image caption generation systems fail to create a database of vehicle images associated with natural explanatory sentences matching user input, and do not consider generating sentences that accurately reflect the presence or absence of surrounding objects based on traffic scenes.
Innovation Solution
An image explanation system with a vehicle-mounted camera and analysis device that generates explanatory sentences by specifying scenes, reading recognition necessity information from a table, recognizing required objects, and associating these with natural sentences, storing them in a database for accurate matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If simple object detection and naming is used as explanatory sentence, then the system is simple and fast, but the explanatory sentence does not match user search sentences and lacks natural language quality
Solution Approach 1:
The system performs preliminary actions by pre-defining traffic scenes and their associated necessary objects in a necessity table before actual image processing. When generating explanatory sentences, the system retrieves pre-determined object lists based on detected traffic scenes, avoiding complex real-time reasoning while ensuring natural and contextually appropriate sentence generation that matches user search intent.
2Measurement precision
If all objects in the image are recognized and included in explanatory sentences, then the sentences are comprehensive, but they become lengthy and do not match typical user search sentences
Solution Approach 1:
The system extracts only the necessary objects for each traffic scene from the complete set of detectable objects. By maintaining a necessity table that specifies which objects are relevant for each traffic scene, the system filters out redundant information and generates concise explanatory sentences that contain only the essential elements needed for accurate search matching, eliminating unnecessary details while preserving search accuracy.
3Measurement precision
If contextual information about traffic scenes is added to explanatory sentences, then the sentences become more natural and match user searches better, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary action by pre-establishing the relationship between traffic scenes and necessary objects in a necessity table during system setup or offline processing. During runtime, when a traffic scene is detected, the system simply retrieves the corresponding object list from the pre-defined table, avoiding complex real-time analysis while still generating contextually appropriate and natural explanatory sentences that improve search accuracy without significant processing overhead.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
An explanatory sentence generation unit of a vehicle image analysis device specifies a scene shown in an image received from a camera, reads, from a necessity table, recognition necessity information on an object in the image for each specified scene, recognizes, from the image, an object designated as requiring recognition based on the read recognition necessity information, generates an explanatory sentence for each object based on a recognition result thereof, generates a situation explanatory sentence of the image based on the specified scene and the explanatory sentence for each object, and stores the situation explanatory sentence of the image and the image in an explanatory sentence DB in association with each other.