This invention relates to a method, apparatus, and medium for recognizing long-tailed
oracle bone script based on hierarchical structure and multimodal fusion. The method includes: constructing an
oracle bone script image dataset; generating a
superclass-subclass
tree structure of
oracle bone script using a self-supervised model combined with the Ward linking criterion; generating visual descriptions of each oracle bone script category using a large
language model; constructing a pure visual hierarchical recognition model, constraining the
superclass-subclass prediction logic through a hierarchical consistency
loss function, and filtering irrelevant fine-grained categories using a winner-takes-all strategy; constructing a multimodal recognition model, freezing the parameters of the pure visual hierarchical recognition model, obtaining the
superclass prediction result and retrieving the corresponding visual description from the multimodal
knowledge base, inputting the input image, task instructions, and retrieval results introduced based on a random corpus injection strategy into the multimodal recognition model, and outputting the subclass
classification result; obtaining the image to be recognized, inputting it into the pure visual hierarchical recognition model to obtain the superclass prediction result, and then inputting it along with the task instructions into the multimodal recognition model to obtain the subclass prediction result. Compared with the prior art, this invention has the advantages of effectively solving the problems of extreme long-tailed distribution, feature space collapse, and misjudgment of visually similar characters in oracle bone script recognition.