Mobile Object Stop Position Control Using Visual-Language Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies fail to uniquely specify the stop position of a mobile object based on an instruction sentence due to insufficient training data, leading to a lack of learned correspondence between language and image features, which hinders effective driving control.
Innovation Solution
A mobile object control device and method that utilizes a trained model, integrating a pre-trained visual-language model with a fully connected layer to process both image and instruction data, enabling precise stop position detection and navigation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If different models are used to process language and image features separately (late fusion), then model complexity is reduced, but the correspondence relationship between language and image cannot be learned effectively when training data is limited
Solution Approach 1:
The patent merges the language processing model and image processing model into a single integrated model that processes both inputs simultaneously. This unified architecture enables the model to learn the correspondence relationship between language and image features directly from training data, resolving the contradiction by combining previously separate processing streams into one cohesive system that can capture cross-modal relationships.
Solution Approach 2:
The patent introduces a new dimensional approach by projecting image features into the language feature space (or vice versa) to create a unified feature representation. This dimensional transformation allows the model to learn correspondences across modalities by operating in a shared feature space, thereby improving correspondence learning without excessive complexity increase.
2Adaptability or versatility
If a distribution or region prediction method is used for target position, then flexibility in handling various commands is improved, but the stop position cannot be specified uniquely for driving control
Solution Approach 1:
The patent applies local quality by differentiating the output treatment based on the command type. For stop position commands, the model outputs a precise point coordinate rather than a distributed region. This localized precision for specific critical outputs resolves the contradiction by maintaining flexibility in handling various commands while ensuring unique specification for driving control stop positions.
Solution Approach 2:
The patent implements dynamic output adaptation where the model can switch between different output types (distribution, region, or precise point) depending on the input command requirements. This dynamic behavior allows the system to maintain versatility for general commands while providing unique stop position specification when needed for driving control.
Data Source
AI summary
A mobile object control device including a storage medium and a processor connected to the storage medium is presented. The processor acquires a photographed image, which is obtained by photographing surroundings of a mobile object by a camera mounted on the mobile object, and an input instruction sentence, which is input by a user of the mobile object; detects a stop position of the mobile object corresponding to the input instruction sentence in the photographed image by inputting at least the photographed image and the input instruction sentence into a trained model including a pre-trained visual-language model, the trained model being trained so as to receive input of at least an image and an instruction sentence to output a stop position of the mobile object corresponding to the instruction sentence in the image; and causes the mobile object to travel to the stop position.