This invention relates to the field of
image processing technology, specifically disclosing a zero-shot text attribute graph learning method based on prototype mutual prompting enhancement. The method calculates instance-aware contrast loss and constrains the similarity of the same node's embedding in two views. It performs PCA
dimensionality reduction on the token embedding of the LLM, extracts the first P principal components to form an aligned coordinate
system C, maps the components to this coordinate
system, and calculates the informative-aware contrast loss. It freezes the pre-trained GNN and the LLM
backbone network, extracts node structural embeddings Z from the pre-trained GNN, generates K prototype embeddings through K linear projectors, and injects them as soft prompts into the LLM instructions. The LLM outputs K expert prompts. It calculates the weights of each expert prompt through a
router model. It freezes the LLM training of related projectors and routers, freezes the GNN training of the LLM prompt
adaptation layer, and iterates and optimizes 1-3 times. This method can cover complex cross-
modal information in text and images while reducing the accumulation of zero-shot transfer bias.