The invention discloses a zero-sample food
image detection method based on visual semantic bidirectional guidance, which belongs to the technical field of image
data processing and generation, takes DETR based on Transform as a detection framework, and fuses a food
semantic enhancement network and a cross-
modal bidirectional guidance module. Firstly, an image is input to an image
backbone network branch to extract visual features, a category
label is input to a food
semantic enhancement network
branch to extract semantic features, then the two obtained features are sent to a cross-
modal two-way guide fusion module, the module comprises a two-way guide
encoder, a two-way guide query vector and a two-way guide decoder, and the cross-
modal two-way guide fusion module comprises a two-way guide query vector and a two-way guide decoder. Visual and semantic features are aligned and fused through coding and decoding processes, and finally, regression and classification tasks of zero sample food
image detection are completed in a full connection layer. According to the method, detailed food attributes and visual features can be captured in the detection process, effective interaction of visual semantic modals is achieved, and therefore the detection precision of unseen food categories is improved.