A zero-shot 3D model recognition method

Abstract

The application discloses a zero sample three-dimensional model recognition method, comprising the following steps: constructing an auxiliary information embedded feature enhancement module, extracting text features and image features by using a visual basic model, and extracting point cloud features by using a learnable point cloud encoder; embedding the angle and attribute information of the image into the extracted image features as auxiliary information to obtain final image enhanced features; constructing a text guided feature adaptive selection module, calculating the average similarity score of the image enhanced features and the corresponding text features, and dynamically updating by using an exponential moving average strategy to obtain a global similarity score; adaptively selecting the image features by using the global similarity score; designing a cross-modal feature alignment mechanism constrained by a joint contrast loss, calculating the cross-modal contrast loss among the text, image and point cloud features; and realizing zero sample recognition of the three-dimensional model by measuring the distance between the three-dimensional model point cloud features and the text features of the category candidates.

Images