An Image Annotation Method Based on Weak Matching Probability Canonical Correlation Model

Abstract

The invention discloses an image tagging method and system based on a typical correlation model of weak matching probability. The invention relates to the technical field of network cross-media information processing, including acquiring tagged images and unlabeled images in an image database, and extracting the tagged images respectively. With the image features and text features of the unlabeled image, a matched sample set and an unmatched sample set are generated, the matched sample set includes an annotated image feature set and an annotated text feature set, and the unmatched sample set includes An unlabeled image feature set and an unlabeled text feature set; according to the matched sample set and the unmatched sample set, train the weak matching probability typical correlation model; through the weak matching probability typical correlation model, treat Annotate images for annotation. The present invention simultaneously uses marked images and their keywords and unmarked images to learn the association between the visual modality and the text modality, and accurately marks unknown images.

Description

technical field [0001] The invention relates to the technical field of network cross-media information processing, in particular to an image labeling method based on a typical correlation model of weak matching probability. Background technique [0002] The Internet of Things and the Internet have rich multimedia information resources such as text, images, video, and audio. These information resources are heterogeneous, and it is difficult to directly find the correlation between them. Canonical correlation analysis (CCA) is a It is a statistical analysis tool used to analyze the correlation between two groups of random variables. Its correlation preservation feature has been proved in theory, and it is applied in the fields of economics, meteorology and genome data analysis. CCA finds two groups of random variables through statistical methods. The potential relationship between heterogeneous multimodal features uses a unified model to associate different types of multimodal...

AI Technical Summary

Problems solved by technology

[0003] In canonical correlation analysis, two groups of related random variables can come from multiple sources of information (such as a person's voice and image), or they can be different features extracted from the information of the same source (such as image color features and texture features) , but the training data must be strictly matched one-to-one. Many reasons make it difficult to obtain such strictly matched training data. For example, in a multi-sensor acquisition system, the sensor sampling frequency is not synchronized or the sensor failure will cause the data collected by different channels to be out of sync or The data of a certain channel is lost; single-modal data is relatively easy to obtain, but manual matching is very time-consuming and laborious. In practice, the multi-modal data often only has a small amount of one-to-one strict matching, and the rest of the large amount of data is not matched. Weakly matched multimodal data

[0004] There are two basic approaches to canonical correlation analysis for weakly matched multimodal data: discard unmatched data and only use canonical correlation analysis to process strictly matched multimodal data; match multimodal data according to specific criteria, but Neither approach is likely to achieve the desired result

Images