A fast image retrieval method based on deep visual features

Abstract

The application provides a fast image retrieval method based on deep visual features, comprising the following steps: S1, pre-processing the input image and extracting a first feature picture; S2, batch pre-processing the pictures of a data set and extracting a second feature picture; S3, calculating the similarity of the first feature picture and the second feature picture to obtain a calculation result; S4, searching the index according to the calculation result to obtain a result, recalling the result and outputting; S5, evaluating and verifying the two types of images of Dog and Cat in the data set; S6, evaluating and verifying through a pre-training model of a convolutional neural network. The application is simple and easy to implement, does not introduce any additional parameters to the model, greatly improves the speed and efficiency, the score calculation of the search engine is fast, the feature vector similarity precision loss is small, and the use of cosine similarity on the reverse search engine can perform a high-speed and high-precision retrieval method while inhibiting the memory usage.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a fast image retrieval method based on depth visual features. Background Technology

[0002] In today's world, fields such as document analysis and medical image analysis require fast and scalable image retrieval using content-based visual features. Convolutional Neural Network (CNN) activations have achieved outstanding performance in this field as features, and deep convolutions using the softmax function in the output layer are a type of visual feature representation.

[0003] Currently, almost all image retrieval systems store their visual feature indexes in main memory, which improves responsiveness but limits their adaptability to big data applications. Using large-scale information retrieval methods incurs high-dimensional computational costs, and scaling up is also a problem.

[0004] However, memory-based methods can address these issues by indexing all image vectors in memory and rapidly calculating cosine similarity, but this requires a computing environment with access to massive amounts of high-speed memory. Another approach, feature dimension compression, does not require a large amount of memory but sacrifices accuracy. Methods using transfer learning, reordering search results, and truncating feature vectors to calculate similarity in memory also suffer from reduced search accuracy and slower search speed. Summary of the Invention

[0005] To address the shortcomings of the aforementioned related technologies, this invention proposes a fast image retrieval method based on depth visual features, which offers high search accuracy and speed.

[0006] To address the aforementioned technical problems, this invention provides a fast image retrieval method based on depth visual features, comprising the following steps:

[0007] S1. Preprocess the input image and extract the first feature image;

[0008] S2. Perform batch preprocessing on the images in the dataset and extract the second feature images;

[0009] S3. Calculate the similarity between the first feature image and the second feature image to obtain the calculation result;

[0010] S4. Based on the calculation result, retrieve the result by searching the index, recall the result, and output it.

[0011] S5. Evaluate and verify the Dog and Cat images in the dataset;

[0012] S6. Evaluation and verification are performed using a pre-trained model of a convolutional neural network.

[0013] Preferably, step S1 specifically includes the following sub-steps:

[0014] S11. Establish a pre-trained model;

[0015] S12, Preset input image;

[0016] S13. Preprocess the input image to obtain a preprocessed image;

[0017] S14. Input the preprocessed image into the pre-trained model to extract the first feature image.

[0018] Preferably, step S2 specifically includes the following sub-steps:

[0019] The images in the training set are input into the pre-trained model for preprocessing and batch extraction of the second feature images.

[0020] Preferably, step S3 specifically includes the following sub-steps:

[0021] We use a fast method for calculating cosine similarity based on L2 norm indexing to calculate the similarity between the input image and images in the dataset.

[0022] Preferably, in S3, the cosine similarity calculation formula is divided into two stages: the L2 norm of the vector is calculated and registered in the index during the indexing stage, and the cosine similarity expressions (1) and (2) are calculated during the search stage.

[0023]

[0024]

[0025] The cosine similarity score has a high affinity with the vector representation, where the feature vector is output by the softmax function, and the cosine similarity is expressed as expressions (3) and (4).

[0026]

[0027]

[0028] Where i is the variable of the feature vector element, n is the dimension of the feature vector, x represents the query vector, y represents the retrieved image vector, and y' represents the image feature vector normalized by the L2 norm.

[0029] Preferably, step S4 specifically includes the following sub-steps:

[0030] The distances obtained from image retrieval are sorted to obtain the retrieval results; among them, the image with the highest similarity is selected and output.

[0031] Preferably, step S5 specifically includes the following sub-steps:

[0032] The evaluation and validation were performed using images of the Dog and Cat classes from the ImageNet-1k dataset.

[0033] Preferably, the VGG-16 pre-trained model is used for evaluation and verification.

[0034] Compared with related technologies, this invention preprocesses the input image and extracts a first feature image; performs batch preprocessing on the images in the dataset and extracts a second feature image; calculates the similarity between the first and second feature images to obtain the calculation result; retrieves the result by searching the index based on the calculation result, recalls the result, and outputs it; evaluates and verifies the Dog and Cat images in the dataset; and evaluates and verifies the result using a pre-trained model of a convolutional neural network. This fast calculation method for cosine similarity using L2 norm indexing can be implemented with just a few lines of code, is simple and easy to implement, and does not introduce any additional parameters to the model, greatly improving speed and efficiency. This invention solves the problems of slow score calculation in search engines and significant loss of similarity accuracy due to feature vector truncation calculations in memory. Furthermore, using cosine similarity on a reverse search engine can execute a high-speed, high-precision retrieval method while suppressing memory usage. Attached Figure Description

[0035] The present invention will now be described in detail with reference to the accompanying drawings. The above and other aspects of the present invention will become clearer and more readily understood through the detailed description following the accompanying drawings. In the drawings:

[0036] Figure 1 This is a flowchart of the fast image retrieval method based on depth visual features according to the present invention;

[0037] Figure 2 This is a flowchart of the method S1 of the present invention. Detailed Implementation

[0038] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] The specific embodiments / examples described herein are specific implementations of the present invention, used to illustrate the concept of the invention, and are illustrative and exemplary, and should not be construed as limiting the implementation methods or scope of the present invention. In addition to the embodiments described herein, those skilled in the art can employ other obvious technical solutions based on the content disclosed in the claims and specification of this application. These technical solutions include those that make any obvious substitutions and modifications to the embodiments described herein, all of which are within the protection scope of the present invention.

[0040] Please refer to Figures 1-2 As shown, where, Figure 1 This is a flowchart of the fast image retrieval method based on depth visual features according to the present invention; Figure 2 This is a flowchart of the method S1 of the present invention.

[0041] This invention provides a fast image retrieval method based on depth visual features, comprising the following steps:

[0042] S1. Preprocess the input image and extract the first feature image.

[0043] In this process, image denoising and enhancement are performed on the input image to achieve high clarity, making it easier to extract the first feature image. This first feature image can be local or global features of the input image.

[0044] Corresponding to the categories of images in the training set, a randomly selected image from the dataset representing a category is fed into the VGG-16 network for feature extraction. This method uses the Dog and Cat categories, and we randomly select a photo of a cat or dog for feature extraction.

[0045] S2. Perform batch preprocessing on the images in the dataset and extract the second feature images.

[0046] S3. Calculate the similarity between the first feature image and the second feature image to obtain the calculation result.

[0047] S4. Based on the calculation result, retrieve the result by searching the index, recall the result, and output it.

[0048] S5. Evaluate and verify the Dog and Cat images in the dataset;

[0049] S6. Evaluation and verification are performed using a pre-trained model of a convolutional neural network.

[0050] Specifically, the method involves preprocessing the input image and extracting a first feature image; batch preprocessing the images in the dataset and extracting a second feature image; calculating the similarity between the first and second feature images to obtain the calculation result; retrieving the result by searching the index based on the calculation result, recalling the result, and outputting it; evaluating and verifying the Dog and Cat images in the dataset; and evaluating and verifying the result using a pre-trained model of a convolutional neural network. This fast calculation method for cosine similarity using L2 norm indexing can be implemented with just a few lines of code, is simple and easy to implement, and does not introduce any additional parameters to the model, thus greatly improving speed and efficiency. This invention solves the problems of slow score calculation in search engines and large loss of similarity accuracy when truncating feature vectors for calculation in memory. At the same time, using cosine similarity on a reverse search engine can perform a high-speed, high-precision retrieval method while suppressing memory usage.

[0051] In this embodiment, step S1 specifically includes the following sub-steps:

[0052] S11. Establish a pre-trained model.

[0053] Specifically, a convolutional neural network image pre-training model is established to train the input image and obtain the trained image data.

[0054] S12. Preset input image. By assuming A is the input image, the input image can be initially acquired image data or an image obtained in real time, etc.

[0055] S13. Preprocess the input image to obtain a preprocessed image.

[0056] S14. Input the preprocessed image into the pre-trained model to extract the first feature image.

[0057] Specifically, by first establishing a pre-trained model, the input image is preprocessed, resulting in a pre-processed image that is denoised, enhances the image quality, and has high clarity. The pre-processed image is then input into the pre-trained model to extract the first feature image, thus achieving high processing efficiency for the first feature image.

[0058] In this embodiment, S2 specifically includes the following sub-steps: inputting the images in the training set into the pre-trained model for preprocessing and batch extracting the second feature images.

[0059] Specifically, the images in the ImageNet-1k training set are input into the VGG-16 pre-trained model for preprocessing and batch extraction of second feature images.

[0060] In this embodiment, step S3 specifically includes the following sub-steps: calculating the similarity between the input image and the images in the dataset using a fast calculation method of elastic search cosine similarity based on L2 norm index.

[0061] In this embodiment, in S3, the cosine similarity calculation formula is divided into two stages. The L2 norm of the vector is calculated and registered in the index during the indexing stage, and the cosine similarity expressions (1) and (2) are calculated during the search stage.

[0062]

[0063]

[0064] The cosine similarity score has a high affinity with the vector representation, where the feature vector is output by the softmax function, and the cosine similarity is expressed as expressions (3) and (4).

[0065]

[0066]

[0067] Where i is the variable of the feature vector element, n is the dimension of the feature vector, x represents the query vector, y represents the retrieved image vector, and y' represents the image feature vector normalized by the L2 norm.

[0068] Specifically, this fast calculation method for cosine similarity using L2 norm indexing for elastic search can be implemented with just a few lines of code. It is simple and easy to implement and does not introduce any additional parameters to the model, thus greatly improving speed and efficiency. This invention solves the problems of slow score calculation in search engines and large loss of similarity accuracy when truncates feature vectors for calculation in memory. At the same time, using cosine similarity on a reverse search engine can perform high-speed and high-precision retrieval methods while suppressing memory usage.

[0069] In this embodiment, step S4 specifically includes the following sub-steps: sorting the distances obtained from image retrieval to obtain retrieval results; wherein, the image with the highest similarity is selected and output.

[0070] The sorting methods can be based on time, spatial location, etc. For example, if two images are loaded into the same document, selecting to move one image to the side of the other will display all images, allowing the images to be sorted.

[0071] Among them, the images with the highest similarity can be those with the largest size, the largest number of pixels, and the largest image vector of the first and second feature images.

[0072] In this embodiment, step S5 specifically includes the following sub-step: evaluation and verification using images of two classes, Dog and Cat, from the ImageNet-1k dataset. Optionally, a total of 250 images are used, with 10 for testing.

[0073] In this embodiment, the VGG-16 pre-trained model was selected for evaluation and verification.

[0074] Compared with existing technologies, this invention proposes a fast image retrieval method based on deep visual features. Specifically, it uses a fast calculation method for cosine similarity using L2 norm indexing, which can be implemented with just a few lines of code. It is simple and easy to implement and does not introduce any additional parameters to the model. The difficulty of image retrieval is reduced, and the speed and efficiency are greatly improved. This invention solves the problems of slow score calculation in search engines and large loss of similarity accuracy when truncating feature vectors for calculation in memory. It is simple and easy to implement and does not introduce any additional parameters to the model. Using cosine similarity on a reverse search engine can perform a high-speed and high-precision retrieval method while suppressing memory usage.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any alterations, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A fast image retrieval method based on depth visual features, characterized in that, Includes the following steps: S1. Preprocess the input image and extract the first feature image; S2. Perform batch preprocessing on the images in the dataset and extract the second feature images; S3. Calculate the similarity between the first feature image and the second feature image to obtain the calculation result; S4. Based on the calculation result, retrieve the result by searching the index, recall the result, and output it. S5. Evaluate and verify the Dog and Cat images in the dataset; S6. Evaluate and validate using a pre-trained convolutional neural network model; S3 specifically includes the following sub-steps: A fast method for calculating cosine similarity based on L2 norm indexing is used to calculate the similarity between the input image and images in the dataset; In S3, the cosine similarity calculation formula is divided into two stages. The L2 norm of the vector is calculated and registered in the index during the indexing stage, and the cosine similarity expressions (1) and (2) are calculated during the search stage. The cosine similarity score has a high affinity with the vector representation, where the feature vector is output by the softmax function, and the cosine similarity is expressed as expressions (3) and (4). Where i is the variable of the feature vector element, n is the dimension of the feature vector, x represents the query vector, y represents the retrieved image vector, and y' represents the image feature vector normalized by the L2 norm.

2. The fast image retrieval method based on depth visual features as described in claim 1, characterized in that, S1 specifically includes the following sub-steps: S11. Establish a pre-trained model; S12, Preset input image; S13. Preprocess the input image to obtain a preprocessed image; S14. Input the preprocessed image into the pre-trained model to extract the first feature image.

3. The fast image retrieval method based on depth visual features as described in claim 2, characterized in that, S2 specifically includes the following sub-steps: The images in the training set are input into the pre-trained model for preprocessing and batch extraction of the second feature images.

4. The fast image retrieval method based on depth visual features as described in claim 1, characterized in that, S4 specifically includes the following sub-steps: The distances obtained from image retrieval are sorted to obtain the retrieval results; among them, the image with the highest similarity is selected and output.

5. The fast image retrieval method based on depth visual features as described in claim 1, characterized in that, S5 specifically includes the following sub-steps: The evaluation and validation were performed using images of the Dog and Cat classes from the ImageNet-1k dataset.

6. The fast image retrieval method based on depth visual features as described in claim 1, characterized in that, The VGG-16 pre-trained model was selected for evaluation and validation.

Images