The invention discloses a visual
scene graph generation method based on differentiable
fuzzy logic reasoning, and solves the technical problems of semantic fuzziness, logic common sense deficiency and the like during long
tail relation
processing in the prior art. The method comprises the following steps: inputting a training image into a target detection module, and respectively outputting corresponding high-dimensional geometric embedding vectors to a relation classifier and a
fuzzy mapping layer; outputting a visual prediction
branch by the relation classifier; meanwhile, the
fuzzy mapping layer outputs a fuzzy membership degree vector; the logic
tensor reasoning module is combined with a common sense rule in an external
knowledge base, simulates a logic reasoning process by utilizing a differentiable logic operator, and outputs a logic reasoning
branch of which the relation of each pair of objects meets a preset logic rule in the training image; a gating residual fusion module performs weighted fusion on the visual prediction
branch and the
logical reasoning branch to generate a
scene graph triple of the training image; and finally, training the
network model, inputting a test image into the trained model, and outputting a corresponding visual
scene graph.