Multimodal Learning Model for Reference-Based Target Position Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle control systems using machine learning models struggle to accurately infer target positions based on image and text inputs, particularly in determining the positional relationship between reference and target objects.
Innovation Solution
A learning apparatus and method that utilizes a model trained with input data comprising images and texts to specify target positions by updating parameters based on a loss function that considers errors in target, reference, and positional relationship errors, using a multi-layered neural network structure with self- and cross-attention layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a model is trained to infer target position using image and text inputs, then the system can perform vehicle control functions, but the accuracy of inferring target position and positional relationships deteriorates
Solution Approach 1:
The patent segments the inference task into three separate output components: target position coordinates (tx, ty), reference object position coordinates (rx, ry), and positional relationship category. By dividing the loss function into three separate error terms (L1 for target position, L2 for reference position, L3 for positional relationship), the model can optimize each component independently, thereby improving overall inference accuracy while maintaining vehicle control functionality.
2Productivity
If a simple loss function is used for model training, then training efficiency is improved, but the precision of positional relationship inference deteriorates
Solution Approach 1:
The patent applies local quality by creating specialized loss terms for different aspects of the inference task. The total loss function L = L1 + L2 + L3 assigns specific optimization goals to different parts of the output: L1 optimizes target position accuracy, L2 optimizes reference position accuracy, and L3 optimizes positional relationship classification accuracy. This localized optimization approach improves precision without significantly compromising training efficiency.
Data Source
AI summary
A learning apparatus for performing machine learning includes an acquisition unit configured to acquire teaching data including input data and correct answer data, the input data including an input image that contains a reference object and an input text that relatively designates a target position by referring to the reference object; a generation unit configured to input the input data to a model to generate output data for specifying the target position, a reference position that is a position of the reference object, and a positional relationship of the target position with respect to the reference position; and an update unit configured to update a parameter of the model to reduce a loss obtained by inputting the output data and the correct answer data to a loss function. The loss function is based on at least two errors of a first error between the target position specified by the output data and the target position specified by the correct answer data, a second error between the reference position specified by the output data and the reference position specified by the correct answer data, and a third error between the positional relationship specified by the output data and the positional relationship specified by the correct answer data.