Multiresolution Image-Text Features for Relative Target Position Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle navigation systems struggle to accurately estimate target positions relative to reference objects in images due to variations in object sizes and complexities in multimodal models.
Innovation Solution
A learning apparatus that utilizes a model comprising a first submodel generating feature amounts with varying resolutions and a second submodel for target position estimation, using a multimodal approach with input images and texts to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single-resolution feature amount is used for target position estimation, then the model structure is simple, but the estimation accuracy deteriorates due to inability to handle reference objects of different sizes
Solution Approach 1:
The patent divides the feature extraction into multiple resolutions by using a CNN with multiple convolutional layers that produce feature amounts at different resolution levels (first resolution, second resolution, third resolution). This segmentation allows the system to handle reference objects of various sizes independently at each resolution level, improving position estimation accuracy without requiring a completely complex redesign of the entire model architecture.
Solution Approach 2:
The patent adds a resolution dimension to the feature representation by generating multiple feature amounts at different resolution levels. Instead of using a single feature representation, the system now operates in an expanded feature space that includes multiple resolution layers, enabling better capture of spatial relationships regardless of object size while maintaining manageable model complexity through hierarchical organization.
2Adaptability or versatility
If multimodal models are used to handle complex relationships, then the adaptability improves, but the model complexity and difficulty of detecting and measuring increase
Solution Approach 1:
The patent segments the multimodal processing into distinct components: image processing through multiple resolution levels, text processing for relative position designation, and a fusion mechanism that combines these inputs. This segmentation makes the complex multimodal model more manageable and easier to detect/measure by breaking down the processing into identifiable stages with clear input-output relationships.
Solution Approach 2:
The patent applies local quality by processing different aspects of the input at different resolution levels - coarse features at lower resolutions for overall context and fine features at higher resolutions for precise positioning. This localized processing strategy improves adaptability to different object sizes while keeping each local processing stage relatively simple and measurable.
Data Source
AI summary
A learning apparatus acquires teaching data including input data. The input data includes an input image and an input text. The input image includes a reference object. The input text relatively designates a target position with reference to the reference object. The apparatus generates output data by inputting the input data to a model. The output data is for specifying the target position. The model includes first and second submodels. The first submodel generates, based on the input image and the input text, a plurality of feature amounts representing the reference object. The plurality of feature amounts have different resolutions from each other. The second submodel generates the output data based on the plurality of feature amounts and the input text. Each of the plurality of feature amounts is input to the second submodel.