3D Semantic Image Analysis Using Spatial Context Descriptors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models struggle to provide precise and reliable semantic image data for complex traffic scenarios, particularly in automated vehicle navigation, due to the lack of effective spatial context descriptors.
Innovation Solution
The integration of predefined spatial context descriptors, such as location and quantity descriptors, into machine learning models like Contrastive Language-Image Pretraining (CLIP) enhances the performance of these models by providing explicit spatial information, enabling accurate analysis of 3D-image data for automated vehicle navigation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained on extensive text and image data without spatial context descriptors, then the models can handle general traffic scenarios, but the precision and reliability of semantic image data for complex traffic scenarios remains insufficient
Solution Approach 1:
The patent applies preliminary action by pre-defining spatial context descriptors that encode spatial relationships between objects before they are used in the machine learning model. These descriptors are calculated in advance based on detected object positions and relationships, allowing the model to receive pre-processed spatial information that improves precision without requiring complex real-time processing during inference.
Solution Approach 2:
The patent introduces spatial context descriptors as an intermediary between raw 3D sensor data and the machine learning model. These descriptors act as a mediator that transforms complex spatial relationships into structured, predefined formats that the model can process more effectively, thereby improving semantic data precision while managing input complexity through structured representation.
2Adaptability or versatility
If joint training of ANNs on text and image data is used, then the models can query images based on text prompts, but the models still struggle to cope with the full breadth of real traffic scenarios
Solution Approach 1:
The patent applies parameter changes by modifying the input parameters of the machine learning model to include spatial context descriptors. These descriptors add explicit spatial relationship information (such as relative positions, distances, and orientations between objects) as new input parameters, enabling the model to better adapt to diverse traffic scenarios while improving the reliability of semantic image data through enhanced spatial context.
3Measurement precision
If spatial context descriptors are added to provide explicit spatial information, then the precision of semantic image data improves, but the complexity of data processing increases
Solution Approach 1:
The patent applies segmentation by dividing the spatial context information into distinct, predefined descriptor components. Each descriptor represents a specific aspect of spatial relationships (such as relative position, distance, orientation) between objects. This segmentation allows the complex spatial analysis task to be broken down into manageable, standardized descriptor calculations that improve precision while controlling processing complexity through modular computation.
Data Source
Figure 1~3
Figure 4~5
AI summary
A computer-implemented method for determining semantic image data representing a vicinity of a vehicle comprises the following steps carried out by computer hardware components: determining a 3D-image based on 3D-sensor data representing a vicinity of a vehicle; detecting a plurality of objects in the 3D-image; determining predefined spatial context descriptors for at least one of the objects; and determining, by using a machine learning model, semantic image data for the 3D-image based on the 3D-image data and the spatial context descriptors.