Visual target retrieval method and system based on target detection

Abstract

The invention relates to a visual target retrieval method and system based on target detection. The method comprises the steps that an IDF weighted cross entropy loss function is adopted to train a public target detection dataset, and a preliminary target detection model is generated; a retrieval dataset containing a target type designated by a user is adopted to slightly adjust the preliminary target detection model, and a final target detection model is generated; and feature extraction is performed on a visual target in a to-be-retrieved picture through the final target detection model, multiple convolution feature graphs of the to-be-retrieved picture are generated, the convolution feature graphs are aggregated through a spatial attention matrix, aggregate feature vectors are generated, and a picture matched with the aggregate feature vectors is retrieved in a picture library. According to the method, visual target retrieval and detection are associated, so that a candidate window prediction step is avoided; and the attention matrix is obtained by selectively accumulating the feature graphs, local descriptors of a convolution layer are aggregated into a global feature expression in a weighted mode, the global feature expression is used for visual target retrieval, and retrieval speed and precision are improved.

Description

technical field [0001] The invention relates to the field of multimedia content analysis, in particular to a visual target retrieval method and system based on target detection. Background technique [0002] Visual object retrieval is a kind of image retrieval, which is widely used in commodity search, object recognition, object tracking and other fields. Different from content-based approximate image retrieval, visual object retrieval does not retrieve images similar to the query image, but images with the same visual object as the query image. Such as figure 1 As shown in the content on the left, this visual target only occupies a small part of the image (the target is in the white box), and the query image containing this target and figure 1 The images in the gallery on the right vary widely in terms of camera angle, lighting, shape, and size. Studying visual object retrieval is both important and challenging. [0003] The traditional target retrieval method performs ...

AI Technical Summary

Problems solved by technology

The strategy it adopts is to use the features in multiple candidate regions for exhaustive matching, and then find the similarity scores of the two candidate regions with the highest similarity as the final similarity scores of the two images. This process can be regarded as a query image It is very time-consuming to match the predicted multiple targets with the multiple predicted targets of the images in the library. On average, it takes nearly 2.5 minutes to query a picture, and the practicability is poor.

In addition, the literature uses the candidate area generated by RPN to replace the sliding window in R-MAC, and learns a global feature expression end-to-end on the cleaned Landmark dataset, thus achieving the best result of current visual target retrieval, but It treats the characteristics of each candidate window equally, but in fact only one or a few windows actually contain the target to be retrieved, so it will introduce a lot of noise. In addition, because the data set it learns mainly contains landmarks, it is only obtained in landmark retrieval. After a good result, the results on other data sets are unknown. The above two methods regard the features in the predicted candidate window as the descriptor of the local block of the image to perform feature matching. The neural network generates the candidate window. In fact, it is time-consuming, and the candidate window is a rectangle, which is different from the actual shape of the object.

[0005] Therefore, there are the following problems in the existing method for solving target retrieval by means of target detection technology: one, the time complexity of exhaustive matching using candidate windows is high ( image 3 shown); Second, the equal weight of each candidate window is likely to cause noise interference ( Figure 4 shown); 3. The candidate area of ​​the rectangle does not match the actual shape of the object, such as Figure 2A and Figure 2B shown, containing two different but very similar buildings

Images