Image Capture Captioning Using Context from Non-Recording Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating image captions may not achieve high accuracy due to limited information from a single image, particularly when the content is complex or nuanced.
Innovation Solution
An image capturing apparatus that utilizes a neural network to generate captions by incorporating both recording and non-recording images, using conditions such as temporal proximity and priority subjects to enhance caption accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only a single recording image is used for caption generation, then the processing complexity is low, but the caption accuracy is insufficient due to limited information
Solution Approach 1:
The system performs preliminary classification of images into recording images and non-recording images before caption generation. Non-recording images are captured and stored in advance, and when a recording image is captured, the system automatically retrieves relevant non-recording images based on temporal proximity and subject priority to enhance the caption generation process without requiring complex real-time processing
Solution Approach 2:
The patent transitions from using only spatial information (single image features) to incorporating temporal information by utilizing sequences of images. By adding the time dimension and using multiple images captured at different moments, the system enriches the information available for caption generation, allowing for more accurate descriptions of dynamic scenes and contextual relationships
2Loss of information
If multiple images are used for caption generation, then the information available increases, but the processing time and computational load increase
Solution Approach 1:
The system applies different processing priorities to different images based on their classification. Recording images receive high-priority processing with immediate caption generation, while non-recording images are processed with lower priority and used selectively. This local quality approach ensures that computational resources are focused on the most important images, reducing overall processing time while maintaining information completeness
Solution Approach 2:
The system changes the parameter of image selection by introducing temporal proximity thresholds and priority levels. Instead of processing all captured images equally, it dynamically adjusts which images are used for caption generation based on time differences and subject importance, thereby optimizing the balance between information completeness and processing efficiency
Data Source
Figure 1
Figure 2
Figure 3
AI summary
There is provided an image capturing apparatus. Shooting means shoots at least one first image that is not to be recorded in a nonvolatile storage, and a second image that is to be recorded in the nonvolatile storage. Generation means generates verbal information that describes content of the second image based on the second image and on one or more first images that satisfy one or more conditions among the at least one first image.