A target detection data compression method based on a multi-modal large language model

Abstract

A target detection data compression method based on a multi-modal large language model belongs to the field of data processing and comprises the following steps: inputting real images into a pre-trained visual language model to generate basic description texts, and synthesizing all image text pairs into a basic description text pool; a scene-to-object dictionary is constructed, a pre-trained scene classifier is used to assign a scene category to each image, and each scene category is mapped to the top K object categories with the highest frequency of occurrence through the scene-to-object dictionary; scene-guided basic description text synthesis is performed in the image generation stage, and a richened basic description text pool is output; a unified multi-object global prompt word is generated by using a large language model-based merging strategy or a region-aware combination strategy; and a final synthesized data set is generated through a synthetic data rendering and pseudo-label pipeline. The present application realizes scene-aware high-dimensional semantic injection and high-object-density and structured layout control, and greatly improves the generalization ability of the target detector.

Images