A super-resolution image retrieval system based on deep hashing

Abstract

The present invention is applicable to the technical field of image retrieval systems, and provides a super-resolution image retrieval system based on deep hashing, including: S1, super-resolution reconstruction processing is performed on a query image (a) and a similar image (a+), that is, the images are input into a VDSR layer; S2, images SR(a), SR(a+) and a-reconstructed from a and a+ are input into a CNN layer together to extract corresponding image features. The super-resolution image retrieval system based on deep hashing, by first super-resolution reconstruction of the image before inputting it into a neural network for feature extraction, improves its resolution to obtain more detailed features, and then can extract more detailed features, improve retrieval accuracy, and when back propagation is performed, the automatic learning update of hyperparameters and weights of high-resolution images is more reasonable than that of low-resolution images, and the network trained in this way can obtain better results than the original network even if a low-resolution image is input.

Description

Technical Field

[0001] The present invention belongs to the technical field of image retrieval systems, and in particular relates to a super-resolution image retrieval system based on deep hashing. Background Art

[0002] As convolutional neural networks shine in the field of images, image retrieval technology has become more and more mature. Image retrieval is a crucial technology in the fields of medical imaging, autonomous driving, etc. Therefore, how to improve the accuracy of image retrieval has become a research hotspot at home and abroad.

[0003] Image retrieval technology is to use text description to describe the characteristics of images, such as the author, age, genre, size, etc. of a painting. Image retrieval technology that analyzes and retrieves the content semantics of an image, such as its color, texture, and layout, is called content-based image retrieval technology. CBIR is a type of content-based retrieval (CBR for short). CBR also includes retrieval technology for other forms of multimedia information, such as dynamic video and audio.

[0004] The hash code of the image queried by the image retrieval system is similar to the hash codes of similar images, but very different from the hash codes of dissimilar images, making it impossible to extract image features more accurately. Therefore, finding a way to extract more image feature details becomes a key issue. Summary of the invention

[0005] The present invention provides a super-resolution image retrieval system based on deep hashing, aiming to solve the problem that the hash code of the query image of the image retrieval system is similar to the hash codes of similar images, but very different from the hash codes of dissimilar images, making it impossible to extract image features more accurately.

[0006] The present invention is implemented as follows: a super-resolution image retrieval system based on deep hashing, comprising:

[0007] S1. Perform super-resolution reconstruction on the query image (a) and the similar image (a+). The image is input into the VDSR layer. The reconstruction process is as follows:

[0008] (1) The input original image is represented by LR, and the image is bicubicly interpolated to obtain the upsampled image UR;

[0009] (2) Input UR into a 20-layer convolutional neural network to obtain high-frequency image information not contained in the original image, and obtain a high-frequency information image HR;

[0010] (3) Integrate LR and HR to obtain the final super-resolution reconstructed image SR;

[0011] S2, input the images SR(a), SR(a+) and a- reconstructed from a and a+ into the CNN layer to extract the corresponding image features;

[0012] S3. The features of the three images extracted from the convolutional neural network are expressed in the form of feature vectors. Each image has a corresponding feature vector. The feature vector is divided into q slices (slice layers). Each slice generates a 1-bit hash code (piecewise layers) through a full connection. The features extracted from each image are represented by a q-bit hash code.

[0013] Preferably, deep hashing is a combination of a convolutional neural network and a hash function.

[0014] Preferably, the hash algorithm represents the sample as a string of two-dimensional codes of fixed length (usually using 0 / 1 or -1 / +1 to represent each bit therein), so that similar samples have similar binary codes.

[0015] Preferably, the image super-resolution reconstruction is to use a group of low-quality, low-resolution images to generate a single high-quality, high-resolution image.

[0016] Preferably, the image retrieval is to find images with high similarity that contain the same instance in the database images given a retrieval image of a specific instance.

[0017] Preferably, a suitable representation vector of the image is extracted from the image, and similar images are found using Euclidean distance or cosine distance nearest neighbor search.

[0018] Preferably, the inspection result image is fine-tuned by post-processing technology.

[0019] Preferably, the image to be queried is captured by an external shooting unit.

[0020] Beneficial Effects

[0021] Compared with the prior art, the beneficial effects of the present invention are: a super-resolution image retrieval system based on deep hashing of the present invention proposes an idea, that is, before the image is input into the neural network for feature extraction, it is first super-resolution reconstructed to improve its resolution in order to obtain more detailed features, thereby improving the retrieval accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Flowchart of super-resolution image retrieval based on deep hashing;

[0023] Figure 2 It is a schematic diagram of the overall structure of the image retrieval system of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] See also Figure 1-2 The present invention provides a technical solution: a super-resolution image retrieval system based on deep hashing, comprising:

[0026] S1. Perform super-resolution reconstruction on the query image (a) and the similar image (a+). The image is input into the VDSR layer. The reconstruction process is as follows:

[0027] (1) The input original image is represented by LR, and the image is bicubicly interpolated to obtain the upsampled image UR;

[0028] (2) Input UR into a 20-layer convolutional neural network to obtain high-frequency image information not contained in the original image, and obtain a high-frequency information image HR;

[0029] (3) Integrate LR and HR to obtain the final super-resolution reconstructed image SR;

[0030] Input a triplet of images, which are the query image (a), similar image (a+) and dissimilar image (a-).

[0031] S2, input the images SR(a), SR(a+) and a- reconstructed from a and a+ into the CNN layer to extract the corresponding image features;

[0032] S3. The features of the three images extracted from the convolutional neural network are expressed in the form of feature vectors. Each image has a corresponding feature vector. The feature vector is divided into q slices (slice layers). Each slice generates a 1-bit hash code (piecewise layers) through a full connection. The features extracted from each image are represented by a q-bit hash code.

[0033] Since a and a+ have higher resolution after VDSR processing, more detailed features can be extracted when extracting features through CNN, so that the Hamming distance between a and a+ will be closer, and the relative distance from a- will be farther, thereby improving the retrieval accuracy and avoiding the situation in which the hash code of the query image of the traditional image retrieval system is similar to the hash code of similar images and very different from the hash code of dissimilar images, thereby extracting image features more accurately.

[0034] When performing back propagation, the automatic learning update of hyperparameters and weights for high-resolution images will be more reasonable than that for low-resolution images. The network trained in this way will get better results than the original network even if the input is a low-resolution image.

[0035] Furthermore, deep hashing is a combination of convolutional neural networks and hash functions.

[0036] Furthermore, the hash algorithm represents the sample as a string of fixed-length two-dimensional codes (usually using 0 / 1 or -1 / +1 to represent each bit), so that similar samples have similar binary codes.

[0037] Furthermore, image super-resolution reconstruction uses a set of low-quality, low-resolution images to generate a single high-quality, high-resolution image.

[0038] Furthermore, image retrieval is to find images with high similarity that contain the same instance in the database images given a retrieval image of a specific instance.

[0039] Furthermore, a suitable image representation vector is extracted from the image, and similar images are found using Euclidean distance or cosine distance nearest neighbor search.

[0040] Furthermore, the inspection result image is fine-tuned through post-processing technology.

[0041] Furthermore, the image to be queried is captured by an external shooting unit.

[0042] Deep hashing: Deep hashing is the combination of convolutional neural networks and hash functions. In hashing algorithms, the usual goal is to represent samples as a string of fixed-length two-dimensional codes (usually using 0 / 1 or -1 / +1 to represent each bit), so that similar samples have similar binary codes. In order to obtain binary codes with shorter code length and better retrieval effect, various optimization methods have been tried.

[0043] Image super-resolution reconstruction: Image super-resolution reconstruction technology uses a group of low-quality, low-resolution images to generate a single high-quality, high-resolution image. In the field of computer vision, image super-resolution reconstruction technology has the potential to transform images from the detection level to the recognition level, or further to the fine-resolution level.

[0044] Image super-resolution reconstruction technology can improve the recognition ability and accuracy of images, achieve focused analysis of targets, and obtain images with higher spatial resolution in areas of interest without having to directly use the configuration of high spatial resolution images with huge data volumes.

[0045] Image retrieval Given a retrieval image of a specific instance, the purpose of image retrieval is to find images with high similarity that contain the same instance in the existing database images.

[0046] The typical process of image retrieval is to try to extract a suitable image representation vector from the image, and then perform nearest neighbor search on these representation vectors using Euclidean distance or cosine distance to find similar images. Finally, some post-processing techniques can be used to fine-tune the test results.

[0047] The working principle and use process of the present invention: When the present invention is used, a and a+ have higher resolution after VDSR processing, so more detailed features can be extracted when extracting features through CNN, so that the Hamming distance between a and a+ will be closer, and the relative distance with a- will be farther, thereby improving the retrieval accuracy. When performing back propagation, the automatic learning and updating of hyperparameters and weights of high-resolution images will be more reasonable than that of low-resolution images. The network trained in this way will obtain better results than the original network even if a low-resolution image is input.

[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A super-resolution image retrieval system based on deep hashing, characterized in that: include: Input a triplet of images, which are the query image a, similar image a+ and dissimilar image a-; S1. Perform super-resolution reconstruction on the query image a and the similar image a+. The images are input into the VDSR layer. The reconstruction process is as follows: (1) The input original image is represented by LR, and the image is bicubicly interpolated to obtain the upsampled image UR; (2) Input UR into a 20-layer convolutional neural network to obtain high-frequency image information not contained in the original image, and obtain a high-frequency information image HR; (3) Integrate LR and HR to obtain the final super-resolution reconstructed image SR; S2, input the images SRa, SRa+ reconstructed by a and a+ and the dissimilar image a- into the CNN layer to extract the corresponding image features; S3. The features of the three images extracted from the convolutional neural network are expressed in the form of feature vectors. Each image has a corresponding feature vector. The feature vector is divided into q slices. Each slice generates a 1-bit hash code through a full connection. The features extracted from each image are represented by a q-bit hash code.

2. A super-resolution image retrieval system based on deep hashing as claimed in claim 1, characterized in that: Deep hashing is a combination of convolutional neural networks and hash functions.

3. A super-resolution image retrieval system based on deep hashing as claimed in claim 2, characterized in that: The hash algorithm represents the sample as a string of fixed-length two-dimensional codes, so that similar samples have similar binary codes.

4. The super-resolution image retrieval system based on deep hashing according to claim 1, characterized in that: Image super-resolution reconstruction uses a set of low-quality, low-resolution images to produce a single high-quality, high-resolution image.

5. The super-resolution image retrieval system based on deep hashing according to claim 1, characterized in that: Image retrieval is to find images with high similarity that contain the same instance in the database image given a retrieval image of a specific instance.

6. A super-resolution image retrieval system based on deep hashing as claimed in claim 5, characterized in that: Extract a suitable image representation vector from the image and use Euclidean distance or cosine distance neighbor search to find similar images.

7. A super-resolution image retrieval system based on deep hashing as claimed in claim 6, characterized in that: The inspection result images are fine-tuned through post-processing technology.

8. The super-resolution image retrieval system based on deep hashing according to claim 1, characterized in that: The image to be queried is captured by an external photographing unit.

Images