基于关系感知视觉增强的多模态大模型关系幻觉缓解方法及系统
By constructing action relationship comparison samples and calculating sensitivity, attention heads are selected, key visual regions are located, and masks are constructed to regulate attention distribution. This solves the illusion problem in action relationship understanding of multimodal large models and achieves precise mitigation and efficient regulation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2026-05-11
- Publication Date
- 2026-07-17
AI Technical Summary
Existing multimodal large models are prone to illusions when understanding the action relationships between objects. Existing methods are difficult to accurately capture key visual regions and effectively model action relationships, resulting in frequent illusions in action relationship understanding tasks.
By constructing action relationship comparison samples, calculating action relationship sensitivity, filtering sensitive and non-sensitive attention heads, locating key visual regions and noisy regions, constructing enhancement and denoising masks, and adjusting attention distribution to enhance key regions and suppress noise interference.
It achieves precise mitigation of action relationship illusions, improves the model's understanding of action relationships, enhances attention to visual regions, reduces the probability of illusions, and requires no additional training or modification of model parameters.
Smart Images
Figure CN122176807B_ABST