Scene Detection via Iterative Graph-Image Vector Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image analysis methods for scene detection require post-processing to find specific combinations of objects and actions/relations, and they struggle with predicting abnormal objects and computing accurate matching scores between images and scene graphs.
Innovation Solution
A computer-implemented method that generates graph vectors from a scene graph and image vectors from an input image, then performs an update process to iteratively update these vectors based on each other, extracting information about detected objects and computing a matching score indicating the similarity between the image and the scene graph.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If post-processing is used to find specific combinations of objects and actions, then detection completeness is improved, but processing time and complexity increase
Solution Approach 1:
The scene graph is constructed beforehand to encode prior knowledge about object relationships and actions. This preliminary structuring allows the model to directly query specific scene combinations without requiring post-processing filtering, thus improving detection completeness while avoiding additional processing time.
Solution Approach 2:
The scene graph serves as an intermediary representation that bridges image inputs and detection outputs. By encoding scene semantics in the graph structure, the system can efficiently retrieve and match specific object-action combinations without exhaustive post-processing search.
2Device complexity
If traditional methods are used for scene detection, then simplicity is maintained, but accuracy in predicting abnormal objects deteriorates
Solution Approach 1:
The system changes the representation parameters by encoding scene knowledge in graph structure form rather than traditional image processing. This parameter transformation enables the model to capture abnormal object patterns through relational reasoning in the scene graph, improving detection accuracy while maintaining computational efficiency.
3Measurement precision
If detailed scene analysis is performed, then matching score accuracy is improved, but computational complexity increases
Solution Approach 1:
The scene analysis is segmented into distinct components: image encoding, scene graph encoding, and matching score computation. Each component processes specific aspects independently, allowing detailed analysis where needed while avoiding unnecessary computational overhead in other areas. The scene graph structure organizes information hierarchically, enabling selective detailed analysis of relevant scene elements.
Data Source
AI summary
A computer-implemented method comprising: based on an input scene graph, generating a plurality of graph vectors; encoding an input image to generate a plurality of image vectors; performing an update process to generate a plurality of updated graph vectors and at least one updated object query vector, comprising: updating the at least one object query vector based on the plurality of graph vectors; updating the at least one object query vector based on the plurality of image vectors; and updating the plurality of graph vectors based on the at least one object query vector; extracting from the at least one updated object query vector a region and a category of the at least one object; and computing a matching score indicating a similarity between the input image and the input scene graph based on the at least one updated object query vector and the plurality of updated graph vectors.