A multi-template image depth matching similar image retrieval method and system

Abstract

This invention proposes a method and system for similar image retrieval based on multi-template image depth matching, relating to the field of digital image technology. The method includes the following steps: Step 1, selecting template images; Step 2, performing SIFT feature extraction on the template images and the image to be retrieved, and calculating the similarity between the template images and the image to be retrieved based on the extracted features; Step 3, selecting potential similar images from the template images based on the similarity; Step 4, constructing multi-scale models for both the potential similar images and the template images, and calculating the similarity between the potential similar images and the template images at different scales; Step 5, selecting confirmed similar images based on the similarity. This invention enables the retrieval of images of similar categories, improves the retrieval accuracy of images of similar categories, and reduces the consumption of human and computing resources.

Description

Technical Field

[0001] This invention relates to the field of digital image technology, and more specifically, to a method and system for similar image retrieval using multi-template image depth matching. Background Technology

[0002] With the advent of the digital media and big data era, massive amounts of digital images have been widely used in various fields. As an important information carrier, digital images provide rich semantic information.

[0003] Traditional image retrieval methods can effectively retrieve images of the same category, but they neglect images with semantic and category similarity, thus exhibiting certain limitations. Furthermore, traditional image retrieval methods often require a large number of training samples, consuming significant human and computational resources. In recent years, while some methods have utilized the idea of ​​multi-template matching, their retrieval results remain insufficient. Therefore, there is an urgent need for an image retrieval method that can address these issues. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for retrieving similar images using multi-template image depth matching, which can achieve the retrieval of images of similar categories, improve the retrieval accuracy of images of similar categories, and reduce the consumption of human and computing resources.

[0005] The embodiments of the present invention are implemented as follows:

[0006] In a first aspect, embodiments of this application provide a similar image retrieval method using multi-template image depth matching, which includes the following steps:

[0007] Step 1: Select a template image;

[0008] Step 2: Perform SIFT feature extraction on the template image and the image to be retrieved, and calculate the similarity between the template image and the image to be retrieved based on the extracted features;

[0009] Step 3: Based on similarity, select potentially similar images from the template images;

[0010] Step 4: Construct potential similar images and template images at multiple scales, and calculate the similarity between potential similar images and template images at different scales;

[0011] Step 5: Select the images that are considered similar based on their similarity.

[0012] In some embodiments of the present invention, step one above includes:

[0013] Select images and classify them to obtain images of multiple categories;

[0014] Images of different categories are clustered to create a clustered image set;

[0015] Representative images are selected from the clustered image set to obtain template images.

[0016] In some embodiments of the present invention, the specific steps for selecting representative images include:

[0017] Select target images from the major categories after classification;

[0018] Perform depth autoencoding on the target image;

[0019] Calculate the Euclidean distance between the autoencoded images of different targets, and select representative images based on this Euclidean distance.

[0020] In some embodiments of the present invention, the method for calculating similarity in step two is the Euclidean distance algorithm.

[0021] In some embodiments of the present invention, step three above includes:

[0022] Based on similarity, the template image and the image to be retrieved are matched and counted to obtain the count results. It is then determined whether the count results exceed a predetermined value. If so, the template image is defined as a potentially similar image.

[0023] In some embodiments of the present invention, step five above includes:

[0024] Based on similarity, the potential similar images and template images at each scale are statistically analyzed to obtain statistical data. If the statistical data exceeds a predetermined value, the potential similar image is defined as the final similar image.

[0025] In some embodiments of the present invention, the specific steps for determining the similarity between the aforementioned potentially similar images and the template image at each scale include:

[0026] The latent similar image and the template image are processed separately using smooth convolution kernels to obtain the filtering results. The similarity A is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance.

[0027] The latent similar image and the template image are processed separately using sharpening convolution kernels to obtain the filtering results. The similarity B is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance.

[0028] The latent similar image and the template image are processed separately using a denoising convolution kernel to obtain the filtering results. The similarity C is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance.

[0029] Secondly, embodiments of this application provide a similar image retrieval system based on multi-template image depth matching, comprising:

[0030] The selection module is used to select a template image;

[0031] The calculation module is used to extract SIFT features from the template image and the image to be retrieved, and to calculate the similarity between the template image and the image to be retrieved after feature extraction.

[0032] The image selection module is used to select potentially similar images from template images based on similarity.

[0033] The scale construction module is used to construct potential similar images and template images at multiple scales, and to calculate the similarity between potential similar images and template images at different scales.

[0034] The image determination module is used to select and determine the final similar images based on similarity.

[0035] In some embodiments of the present invention, the above-mentioned selection module includes:

[0036] The classification submodule is used to classify the selected images to obtain images of multiple categories;

[0037] The clustering submodule is used to cluster images of different categories to create a clustered image set;

[0038] The target submodule is used to select target images from the classified major categories;

[0039] The encoding submodule is used to perform depth auto-encoding on the target image;

[0040] The filtering submodule is used to calculate the Euclidean distance after depth autoencoding of different target images, and to filter template images based on the Euclidean distance.

[0041] In some embodiments of the present invention, the above-mentioned scale construction module includes:

[0042] The similarity calculation submodule is used to process the potential similar image and the template image separately using smoothing, sharpening and denoising convolution kernels to obtain the filtering results, and calculate the similarity between the filtered potential similar image and the template image.

[0043] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0044] This invention provides a method for retrieving similar images using multi-template image depth matching, comprising the following steps: Step 1, selecting template images; Step 2, performing SIFT feature extraction on the template images and the image to be retrieved, and calculating the similarity between the template images and the image to be retrieved based on the extracted features; Step 3, selecting potential similar images from the template images based on the similarity; Step 4, constructing multi-scale models for the potential similar images and the template images respectively, and calculating the similarity between the potential similar images and the template images at different scales; Step 5, selecting determined similar images based on the similarity.

[0045] This invention provides a similar image retrieval method using multi-template image depth matching, which has the following advantages:

[0046] (1) Using deep autoencoders to select high-quality template images helps improve the accuracy of similarity image retrieval;

[0047] (2) The retrieval accuracy of similar images is improved by combining classical feature operators with multiple convolution kernels;

[0048] (3) The absence of a large number of training samples and a complex training process reduces the consumption of human and computing resources. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the steps of a multi-template image depth matching similar image retrieval method according to Embodiment 1 of the present invention;

[0051] Figure 2 This is a structural block diagram of a similar image retrieval system based on multi-template image depth matching according to Embodiment 2 of the present invention;

[0052] Figure 3 This is a structural block diagram of the selection module of a similar image retrieval system based on multi-template image depth matching according to Embodiment 2 of the present invention. Detailed Implementation

[0053] Terminology Explanation:

[0054] SIFT: Scale-invariant feature transform, is a term used in the field of image processing.

[0055] Euclidean distance, also known as the Euclidean metric, is a commonly used definition of distance. It is the true distance between two points in m-dimensional space.

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0058] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other.

[0059] Example 1

[0060] Please see Figure 1 , Figure 1 The diagram shows the steps of a similar image retrieval method using multi-template image depth matching provided in Embodiment 1 of this application.

[0061] Embodiment 1 of this application provides a method for similar image retrieval based on multi-template image depth matching, comprising the following steps:

[0062] Step 1: Select a template image;

[0063] Step 2: Perform SIFT feature extraction on the template image and the image to be retrieved, and calculate the similarity between the template image and the image to be retrieved based on the extracted features;

[0064] Step 3: Based on similarity, select potentially similar images from the template images;

[0065] Step 4: Construct potential similar images and template images at multiple scales, and calculate the similarity between potential similar images and template images at different scales;

[0066] Step 5: Select the images that are considered similar based on their similarity.

[0067] The template images are digital images of various categories selected from a wide range of image datasets from commonly used search engines such as Baidu, 360, Sogou, and Google. By repeating steps one through five, each image to be retrieved is judged in a massive image database, thereby retrieving all similar images for each image to be retrieved.

[0068] As a preferred embodiment, step one includes:

[0069] Select images and classify them to obtain images of multiple categories;

[0070] Images of different categories are clustered to create a clustered image set;

[0071] Representative images are selected from the clustered image set to obtain template images.

[0072] Clustering mainly involves grouping images of similar categories into one class (for example, images of grasslands, mountains, rivers, or forests can be clustered into landscape images), thus creating a clustered image set. Multiple representative images can be selected from each class to serve as template images.

[0073] As a preferred embodiment, the specific steps for selecting representative images include:

[0074] Select target images from the major categories after classification;

[0075] Perform depth autoencoding on the target image;

[0076] Calculate the Euclidean distance between the autoencoded images of different targets, and select representative images based on this Euclidean distance.

[0077] When the Euclidean distance values ​​of several images are similar, only one value needs to be retained (when the Euclidean distance values ​​are similar, we consider them to be highly similar). By performing multiple Euclidean distance calculations, a batch of images with greater differences can be selected, which are then used as representative images for the template images.

[0078] As a preferred implementation, the method for calculating similarity in step two is the Euclidean distance algorithm.

[0079] As a preferred embodiment, step three includes:

[0080] Based on similarity, the template image and the image to be retrieved are matched and counted to obtain the count results. It is then determined whether the count results exceed a predetermined value. If so, the template image is defined as a potentially similar image.

[0081] The process involves statistically analyzing the similarity between template images and the images to be retrieved to obtain statistical results. Based on these results, a predetermined value is set to determine whether there are any template images in the total number of statistically analyzed template images that have a high degree of similarity to the images to be retrieved. If so, they can be classified as potentially similar images; otherwise, they can be classified as dissimilar images.

[0082] As a preferred embodiment, step five includes:

[0083] Based on similarity, the potential similar images and template images at each scale are statistically analyzed to obtain statistical data. If the statistical data exceeds a predetermined value, the potential similar image is defined as the final similar image.

[0084] Specifically, by comparing the similarity between potential similar images and template images at each scale, a number of potential similar images with similarity close to that of template images can be obtained. The number of similar images is then used to determine whether potential similar images at each scale maintain a high degree of similarity with template images exceeding a predetermined value. If so, the potential similar image is defined as the final similar image; otherwise, it is determined as a dissimilar image.

[0085] As a preferred implementation, the specific steps for determining the similarity between potentially similar images and template images at each scale include:

[0086] The latent similar image and the template image are processed separately using smooth convolution kernels to obtain the filtering results. The similarity A is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance.

[0087] The potential similar image and the template image are processed separately using sharpening convolution kernels to obtain the filtered results. The similarity B is obtained by calculating the similarity between the filtered potential similar image and the template image using Euclidean distance.

[0088] The latent similar image and the template image are processed separately using a denoising convolution kernel to obtain the filtering results. The similarity C is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance.

[0089] 0 Wherein, the size of the convolution kernel is 3*3; if two or more of the similarities A, B and C are close, it is considered that the potential similar image and the template image are similar, that is, they maintain a high degree of similarity.

[0090] Understandable. Figure 1 The structure shown is for illustrative purposes only. A similar image retrieval method using multi-template image depth matching may also include... Figure 1The number of more or fewer components shown, or having more or fewer components than 5 Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0091] Example 2

[0092] Please see Figure 2 , Figure 2 The diagram shown is a structural block diagram of a similar image retrieval system based on multi-template image depth matching according to Embodiment 2 of the present invention.

[0093] Embodiment 2 of the present invention provides a similar image retrieval system based on multi-template image depth matching.

[0094] include:

[0095] The selection module is used to select a template image;

[0096] The calculation module is used to extract SIFT features from the template image and the image to be retrieved, and to calculate the similarity between the template image and the image to be retrieved after feature extraction.

[0097] The image selection module is used to select potentially similar images from template images based on similarity.

[0098] The scale construction module is used to construct potential similar images and template images at multiple scales, and to calculate the similarity between potential similar images and template images at different scales.

[0099] The image determination module is used to select and determine the final similar images based on similarity.

[0100] The present invention provides a similar image retrieval system based on multi-template image depth matching. After selecting template images by a selection module, the similarity between the template images and the image to be retrieved is calculated by a calculation module. Then, the potential similar images can be selected from the template images by the image selection module based on the similarity. The potential similar images and template images are then constructed at multiple scales by a scale construction module, and the similarity between the potential similar images and template images at different scales is calculated. Finally, the final similar images of the image to be retrieved can be retrieved based on the similarity.

[0101] As a preferred implementation, the scale construction module includes:

[0102] The similarity calculation submodule is used to process the potential similar image and the template image separately using smoothing, sharpening and denoising convolution kernels to obtain the filtering results, and calculate the similarity between the filtered potential similar image and the template image.

[0103] If two or more of the three methods of smoothing, sharpening, and denoising convolution kernels are used to process each potential similar image and each template image respectively, the similarity between the potential similar image and the template image is high, then the potential similar image and the template image are considered to have a high degree of similarity.

[0104] Please see Figure 3 , Figure 3 The diagram shown is a structural block diagram of the selection module of a similar image retrieval system based on multi-template image depth matching according to Embodiment 2 of the present invention.

[0105] As a preferred embodiment, the selected module includes:

[0106] The classification submodule is used to classify the selected images to obtain images of multiple categories;

[0107] The clustering submodule is used to cluster images of different categories to create a clustered image set;

[0108] The target submodule is used to select target images from the classified major categories;

[0109] The encoding submodule is used to perform depth auto-encoding on the target image;

[0110] The filtering submodule is used to calculate the Euclidean distance after depth autoencoding of different target images, and to filter template images based on the Euclidean distance.

[0111] The target images need to be sufficient to cover representative images in the major categories. When calculating the Euclidean distance after depth autoencoding of different target images, if the Euclidean distance of several images is close, they can be considered to be highly similar, and only one needs to be kept. After calculating the Euclidean distance multiple times, a batch of images with greater differences can be selected as representative template images.

[0112] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0113] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0114] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] In summary, the similar image retrieval method and system based on multi-template image depth matching provided in this application selects relatively high-quality template images using a deep autoencoder method, and then achieves similar image retrieval by using the idea of ​​multi-template image depth matching; it improves the retrieval accuracy of similar images by combining classical feature operators and multiple convolution kernels; and it reduces the consumption of human and computing resources by eliminating the need for a large number of training samples and complex training processes.

[0116] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0117] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this application. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for retrieving similar images using multi-template image depth matching, characterized in that, Includes the following steps: Step 1: Select a template image; Step 2: Perform SIFT feature extraction on the template image and the image to be retrieved, and calculate the similarity between the template image and the image to be retrieved based on the extracted features; Step 3: Based on similarity, select potentially similar images from the template images; Step 4: Construct potential similar images and template images at multiple scales, and calculate the similarity between potential similar images and template images at different scales; Step 5: Select similar images based on their similarity. Step one includes: Select images and classify them to obtain images of multiple categories; Images of different categories are clustered to create a clustered image set; Representative images are selected from clustered image sets to obtain template images; The specific steps for selecting a representative image include: Select target images from the major categories after classification; Perform depth autoencoding on the target image; Calculate the Euclidean distance between the autoencoded images of different targets, and select representative images based on the Euclidean distance.

2. The method for retrieving similar images using multi-template image depth matching as described in claim 1, characterized in that, The method for calculating similarity in step two is the Euclidean distance algorithm.

3. The method for similar image retrieval based on multi-template image depth matching as described in claim 1, characterized in that, Step three includes: Based on similarity, the template image and the image to be retrieved are matched and counted to obtain the count results. It is then determined whether the count results exceed a predetermined value. If so, the template image is defined as a potentially similar image.

4. The method for retrieving similar images using multi-template image depth matching as described in claim 1, characterized in that, Step five includes: Based on similarity, the potential similar images and template images at each scale are statistically analyzed to obtain statistical data. If the statistical data exceeds a predetermined value, the potential similar image is defined as the final similar image.

5. The method for similar image retrieval based on multi-template image depth matching as described in claim 4, characterized in that, The specific steps for determining the similarity between the potential similar image and the template image at each scale include: The latent similar image and the template image are processed separately using a smooth convolution kernel to obtain the filtering result. The similarity A is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance. The latent similar image and the template image are processed separately using sharpening convolution kernels to obtain the filtering results. The similarity B is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance. The latent similar image and the template image are processed separately using a denoising convolution kernel to obtain the filtering results. The similarity C is obtained by calculating the similarity between the filtered latent similar image and the template image using Euclidean distance.

6. A similar image retrieval system based on multi-template image depth matching, characterized in that, include: The selection module is used to select a template image; The calculation module is used to extract SIFT features from the template image and the image to be retrieved, and to calculate the similarity between the template image and the image to be retrieved after feature extraction. The image selection module is used to select potentially similar images from template images based on similarity. The scale construction module is used to construct potential similar images and template images at multiple scales, and to calculate the similarity between potential similar images and template images at different scales. The image determination module is used to select and determine the final similar images based on similarity. The selection module includes: The classification submodule is used to classify the selected images to obtain images of multiple categories; The clustering submodule is used to cluster images of different categories to create a clustered image set; The target submodule is used to select target images from the classified major categories; The encoding submodule is used to perform depth auto-encoding on the target image; The filtering submodule is used to calculate the Euclidean distance after depth autoencoding of different target images, and to filter template images based on the Euclidean distance.

7. The similar image retrieval system based on multi-template image depth matching as described in claim 6, characterized in that, The scale construction module includes: The similarity calculation submodule is used to process the potential similar image and the template image separately using smoothing, sharpening and denoising convolution kernels to obtain the filtering results, and calculate the similarity between the filtered potential similar image and the template image.

Images